Participated in public demo 2606 4W with Vault

The problem [00:00:00] Bhavesh initially assumed the hard part of Vault would be reliably classifying documents and extracting fields into a domain model, a familiar extraction problem. He found that reliable classification and extraction were not the hardest part; the harder question was when a plausible model output becomes safe enough to be trusted as fact by the rest of the system. What they built [00:01:32] He redesigned the extraction pipeline so the model only proposes document shape and per-line facts; deterministic code then checks shape, confidence, totals, dates, and price sanity before any fact is written or event emitted. Ambiguous cases (like the multi-cover renewal) are routed to a review queue instead of silently accepted, and the incident became a permanent regression test. He also exposed the system through an MCP server rather than a custom chat UI, letting people query pending-review reasons through their own AI assistant (demoed via Claude Desktop), with the MCP server deliberately scoped to read/investigate only, unable to ingest, confirm, or discard evidence. What happened [00:02:28] The multi-cover renewal failure is now a golden regression test the system must keep refusing as code evolves. The MCP read boundary means an assistant can surface and explain pending-review reasons verbatim without gaining authority to resolve them itself. The ask [00:06:15] The repo (architecture notes, synthetic examples, tests) is now public; Bhavesh is inviting people to connect on LinkedIn to discuss designing AI systems with clear authority boundaries, and specifically wants input on whether self-reported LLM confidence can ever be trusted enough to gate human review, or whether an independent verifier agent is the only honest approach.