Evaluating RAG Beyond Plausible Answers
Decompose RAG failures across retrieval, context construction, and generation so each result points to an actionable change.
Read noteAI is redrawing the boundaries of technology. This is a record of models, systems, and the creativity one programmer still believes in.
Meet meDecompose RAG failures across retrieval, context construction, and generation so each result points to an actionable change.
Read noteOnce an agent calls tools, changes state, and replans, conventional request logs are no longer enough to explain success or failure.
Read noteOffline metrics are evidence, not a conclusion; data splits, exposure bias, and user segments determine whether that evidence is trustworthy.
Read noteI want a record of learning, engineering judgement, and long-term thinking that can be revisited and tested.
Read noteA close reading of scaled dot-product attention, multi-head attention, positional encoding, and the engineering conditions that made Transformer consequential.
Read noteA useful learning note is not a tutorial paraphrase. It keeps the question, the experiment path, and the boundary of the conclusion together.
Read noteAn experiment note is not a diary; it is a small contract covering the hypothesis, cost, evidence, and next decision.
Read noteHow shared embeddings connect the FM and deep branches, and why the architecture remains a useful recommender-systems baseline.
Read noteDepth should follow frequency, the cost of being wrong, and whether the technology forms part of your core judgment.
Read noteMore useful than remembering that a metric improved is keeping how the problem was defined, how the experiment was designed, and where the conclusion stops.
Read note