The pipeline worked, the delivery didn't
An OCR script that kept crashing, a compliance deadline, and an Ops team that needed something they could actually use.
background.
The sales operations team was reviewing merchant documents and extracting data almost entirely by hand. With a large backlog of legacy documents coming in under a tight deadline, continuing manually would have consumed the team's entire capacity. We decided to build an automated solution.
Problem.
The team had 1 month to process a backlog of 10,000+ merchant documents. The execution risk was high: documents spanning many years, with poor photo quality and complex background patterns that caused standard OCR tools to produce too many errors for compliance use.
Goal.
Clear the 10,000+ document backlog within 1 month by building an AI pipeline that automates data extraction — freeing the team's capacity for other work.
Contribution.
- MVP scoping under a tight 1-month timeline
- Technical evaluation for a hybrid pipeline combining a custom-trained OCR model with an internal LLM to maximize data extraction accuracy
- "Human-in-the-loop" workflow & UI/UX design
- Internal rollout & change management
the decisions.
The Initial Approach: Build an automation script to process batches of merchant document photos and export the extracted data directly to a spreadsheet.
The Reality Check: Two problems surfaced quickly. The script crashed mid-batch whenever it hit a poor-quality or corrupted photo — every crash meant restarting from scratch. And without any interface, the Ops team had to manually open each image and cross-check it against a spreadsheet cell by cell, which was slower than doing it by hand.
The Reframe: The bottleneck wasn't the idea — it was the execution. Automating data extraction alone wasn't enough if the pipeline was fragile and the output was unusable without manual intervention. The real challenge was building something the Ops team could actually rely on.
Our OCR model could read the text, but poor-quality photos of legacy documents with complex backgrounds caused too many errors for compliance use. How do we get the accuracy high enough without burning through our 1-month deadline?
Pick the move you would make — you can change your answer.
The AI pipeline worked, but the delivery was a script — no interface, crashes mid-batch with no recovery, and an Ops team stuck working in a console they weren't built for. How do we ship this reliably?
Pick the move you would make — you can change your answer.
outcome.
The AI pipeline and web tool launched within the 1-month deadline. By replacing a fragile script with a tool the Ops team could actually use, we cleared the 10,000+ document backlog on time — with significantly fewer errors and interruptions than the original script.
Takeaways
Impact.
- ~90% reduction in processing time per document — from minutes of manual typing to a few seconds of visual review.
- Significantly fewer crashes post-launch — corrupted files were isolated automatically instead of halting the entire batch.
- 95% data accuracy on good-quality photos, 80% on poor-quality ones — with the remaining errors flagged for manual review.
- Cleared 10,000+ document backlog on time within 1 month without additional headcount.
What I learned.
- Delivery mechanism dictates product viability. A strong AI pipeline is useless if the people who need it can't actually use it. The interface isn't secondary — it's what determines whether the tool gets adopted at all.
- LLMs are powerful at fixing structure, not just spelling. In environments with messy, inconsistent data, using an LLM as a post-processing layer is significantly faster than retraining a vision model — and more practical under a tight deadline.
Next steps
- 1
Continue optimizing the extraction pipeline with LLM improvements — better prompt design and model selection to push accuracy higher across all document quality levels.
- 2
Integrate the pipeline directly into the company's CRM system so extracted data flows automatically into merchant records without manual export steps.