What does a pilot actually prove?
That a model can produce the right output from the right input. That is useful, and it is also the easy part. A pilot typically runs against an export of data that someone cleaned by hand, inside a notebook or a sandbox, with the engineer who built it watching every run.
None of those conditions exist in production.
What does production demand that a demo never did?
Real data. In production, "the customer" exists in three systems with three slightly different records. The model has to work on what the systems actually hold, not on what the export looked like after cleaning.
Permissions. Access rules were written for people. An AI system is a new kind of actor: it needs its own credentials, scoped to exactly what its task needs, and no more. Most organisations have never set this up.
An audit trail. A risk or compliance team will ask what the AI read, what it inferred, and what it did, for every decision. If the answer is "we didn't log that," the system does not go live.
Someone on call. Models drift, APIs change, a supplier updates a document format. Something has to notice, and someone has to fix it. A pilot has the builder watching. Production needs a rota.
What does a production-first pilot look like?
It builds the boring parts first. In FetchSky's work, a six-week pilot runs against the client's own data and systems from the start, with permissions, logging and a human review step built in from week one. The output is a working version the client can see running under real conditions, not a slide showing accuracy on a test set.
Two examples from production:
- A brokerage received one consolidated PDF ledger at market close and needed every trade confirmed to each client individually. A pipeline of three agents (extraction, identity matching, messaging) now sends every confirmation on WhatsApp within minutes. The result: 98% less manual effort, about 20 person-hours saved a day, every message logged for compliance.
- A monthly report on top mutual fund holdings required analysts to read 34 fund manager reports by hand. Three agents now collect the reports on publication, extract the allocations, and produce the finished PDF. Four days became three minutes.
Neither of these is a sophisticated model. Both are careful engineering around a model.
How do you keep an AI system under control once it's live?
Scope its permissions to the task. Log everything it reads and does. Keep data in your own cloud or country where that matters. Put a person in the loop for anything that affects a customer or moves money. Test it against misuse before launch, monitor it after, and keep a switch that turns it off.
Frequently asked questions
Is the model choice important?
Less than people expect. Accuracy, cost and where the data must stay decide which model fits a use case, and the right answer changes over time. The integration work outlasts any particular model.
Do we need clean data before starting?
No. Finding out whether the data is ready is part of the first stage, and most useful first use cases work on data the business already produces.
How long should a pilot take?
Six weeks is long enough to work on real data and systems and short enough to decide quickly. FetchSky's process starts with a free consultation to pick the use case, then a six-week pilot.
What happens after the pilot?
A decision: build it for production, change it, or stop. Stopping after six weeks is a good outcome compared with stopping after six months.