Writing the security packet for an AI feature
Security review does not fail because the system is risky. It fails because nobody could describe the risk. Five artefacts turn a three-month review into a two-week one.
Field Notes
All notes
How to choose a reranker without a benchmark harness
You do not need a public leaderboard to choose a reranker. You need queries with known correct passages, a way to measure position, and the discipline to include unanswerable quer…
Prompt versioning without a platform
You do not need a prompt management product to stop shipping prompt changes blind. A file, a version identifier and one rule about out-of-band edits cover most of what a platform…
An on-call runbook for an LLM service
Classic latency and error alerting does not see the failures that matter here: quality decay, provider degradation, silent context growth. A runbook has to name the signals first.
Moving a prototype onto managed inference
A prototype built against one vendor API carries assumptions that surface on a managed platform. Settle them before the first request so the migration is configuration, not a rewr…
How to scope a thirty-day AI discovery
A discovery phase that produces a slide deck is a cost centre. The one that produces three measured facts — document reality, permission reality, workflow reality — pays for itsel…
How to build an evaluation set from real traffic
Most evaluation sets are written from documentation and measure the documentation. Building one from logs, tickets and abandonments takes a week and is the highest-leverage task a…
A chunking strategy you can defend in review
Chunk size is usually set by a library default and defended by nothing. Choose it from the documents themselves, and be able to explain the choice to someone who will ask why.
How to run a cost review before scaling inference
Before adding capacity or switching models, walk the request once and price every stage. Most cost reviews find the money in context size, retries and caching, not in the model ch…
Six weeks of silent relevance regression
A retrieval change degrades answer quality without breaking a single test, because nothing compares today ranking against yesterday ranking. Users absorb the loss quietly.
The pilot worked and the rollout did not
A pilot succeeds because everyone involved knows it is a pilot. The properties that make it work — a tolerant audience, a small corpus, manual fixes at the edges — are exactly the…
The data owner nobody assigned
AI features fail on the organisational boundary between the team that builds the system and the team that owns the data. That boundary is rarely named in the project plan, and it…
The on-prem box that ran out of VRAM at month two
Capacity planning for self-hosted inference usually counts the weights and forgets everything else that has to live on the same card. The gap between those two numbers is what run…
The GPU cost model nobody writes down before buying hardware
Self-hosting beats the API on cost per token only above a utilisation threshold almost nobody reaches. Below it you are paying for idle silicon and engineer attention — and only o…
Fine-tune or retrieve: the decision is about what changes, not what you prefer
Fine-tuning encodes behaviour; retrieval supplies facts. Asking which is better is asking the wrong question — ask how often the underlying knowledge changes and whether you must…
Multi-tenant isolation: the requirement that arrives after the demo
Every multi-tenant RAG eventually discovers the same thing: filtering results after retrieval is not isolation. The filter has to constrain the search, not trim its output.
Choosing a vector store: four questions that settle it
Published benchmark tables compare recall on public datasets, which is not your dataset, your filters, or your query distribution. Four operational questions decide it instead.
Before an agent touches production data: the guardrail checklist
The risk is not that the model says something wrong. It is that the agent does something wrong — with credentials, in a system that does not have an undo.
Context engineering is the new prompt engineering
For two years the advice was to write better prompts. That advice stopped paying off somewhere around the point where the model became reliably good at following instructions. Wha…
Why most enterprise RAG pilots stall in week three
The demo works on a handful of clean PDFs, the room is impressed, and then the real document set arrives: scans, merged cells, ten-year-old contracts and three languages. This is…
How to run an AI pilot that survives procurement
The demo goes well. The users are happy. Then legal asks where the data went, security asks who can see what, and finance asks what it costs per month. If you have not prepared fo…
Building 50 evaluation questions that predict production quality
Teams that measure tend to measure the easy cases, because those are the ones where someone already knows the answer. The questions that will cost you money are the ones nobody th…
Permission models: the part of enterprise RAG nobody budgets for
Retrieval quality is a solved-enough problem. Permissions are the reason enterprise RAG projects stop: not because they are hard, but because they arrive late and force a rewrite.
Self-hosting vs. managed inference: where the cost curve actually crosses
"Self-hosting is cheaper" is only true above a certain utilisation. Below it, you are paying for an idle GPU and an engineer's attention — and that second line item is the one nob…
Where the milliseconds go: a latency budget for RAG
A RAG request feels slow long before it is slow. Knowing which of the five stages is responsible is the difference between a targeted fix and three weeks of replacing the wrong co…
Model routing: sending easy questions to cheap models
Routing is the highest-leverage cost control in an LLM system, and it is almost always implemented last, after eighteen months of paying premium prices for questions a smaller mod…
Agent evaluations are the new unit tests
Every team that runs an agent in production eventually builds the same thing: a suite of scenarios that must not regress. The only question is whether you build it before or after…
What a forward-deployed engineer actually does all day
The title sounds like research. The job is mostly data plumbing, security questionnaires, and convincing a department head that the pilot will not embarrass them — which is exactl…
Nothing here yet. The next crawl fills this up.