Why most enterprise RAG pilots stall in week three
The demo works on a handful of clean PDFs, the room is impressed, and then the real document set arrives: scans, merged cells, ten-year-old contracts and three languages. This is the point where most pilots quietly stop.
What happened
The pattern is consistent across deployments we have tracked. Week one is a demo on curated documents. Week two is integration with the customer's identity and permission model. Week three is when someone points the system at the actual document store — and the answer quality collapses.
Why it matters for deployment teams
The failure is almost never the model. It is parsing: tables flattened into unreadable text, scanned pages yielding nothing, and chunks that split a clause from its exceptions. Retrieval then returns confidently wrong context, and the model does what it was told.
What to do about it
Budget the pilot for documents, not models. Parse a representative sample of 200 real files before promising anything, measure context precision on that sample, and make chunking a visible, correctable step rather than a library default.
What to do about it
- Parse 200 real documents before committing to a date
- Make chunking inspectable, not a library default
- Measure faithfulness, not vibes
Where this fits in the FDE path
- FDE career path — the levels, the titles that map to them, and what changes at each step
- Interview guide — what the loop tests beyond coding
- Pay data — 223 postings — midpoint median $231,000, employer-published ranges only
- 922 live FDE roles — pulled daily from employers’ own job boards