Adding AI to your software product: partner, hire or both

A working demo of an AI feature takes an engineer an afternoon: send the customer's text to a model and show what comes back. That speed is why AI items now sit on so many roadmaps, and it is also what throws the planning off, because the demo skips every part that decides whether the feature can ship.

The model does not reliably give the same answer twice. OpenAI's own evaluation guide says models sometimes produce different output from the same input, which makes traditional software testing insufficient. NIST's profile for generative AI names the failure directly: confabulation, the production of confidently stated but erroneous or false content. So an AI feature needs four things a normal feature does not. It needs test cases that measure whether the output is good enough, limits on what the model can see and do, a cap on what each customer's usage can cost, and a record that lets support explain an answer a customer disputes.

Those four pieces are where the time goes, and they are what to plan the staffing around, because whoever owns them owns the feature. A feature that takes actions for a user, rather than drafting text for them to read, needs more controls again, and our guide to where AI agents fit in a business sets those out.

All four routes can ship an AI feature. They differ in when the work starts, who carries the risk and what you are left owning. These are typical patterns, not rules, so ask any candidate or supplier where they differ.

If your engineers have capacity and someone senior wants to own evaluation, this is the cheapest route in cash and the most expensive in lessons. The model APIs are well documented, and your team already knows the product, the data and the customers, which is the knowledge an outsider takes longest to pick up.

The catch is that evaluation, prompt injection and cost control are new disciplines for a team that has built deterministic software. Learned on the job, they are learned on live customers. This route works best for a narrow first feature, such as a summary or a drafting aid that a person reads before acting on it, where a weak answer is visible and cheap to fix.

Hiring makes sense when AI will be a lasting part of what customers pay for and you know what the role will spend its first year doing. A permanent engineer with production AI experience keeps the knowledge inside the company, which no partner can do.

Two problems come with it. The hiring round comes before any feature exists, and the roadmap waits through search, offers, notice periods and ramp-up. And the first AI hire is hard to judge when nobody in the company has shipped an AI feature, so you are assessing experience you cannot yet check. My preference is to ship the first feature, then hire into a role the company now understands. 1AYM is one of the partners that could build that first feature, so weigh my preference accordingly.

If you bring in independent contractors rather than employees, check who owns the code. The US Copyright Office's Circular 30 explains that work an employee creates as part of their regular duties is a work made for hire, owned by the employer. Commissioned work is made for hire only if it falls within nine categories the statute lists and both parties sign a written agreement saying so. Software is not named among those categories, so the dependable route is a written assignment of copyright, which 17 U.S.C. § 204(a) says is not valid unless it is in writing and signed by the owner of the rights.

A partner that has shipped AI features before brings the missing disciplines on the first day: the evaluation harness, the guardrails and the cost controls. The work starts when the scope is signed rather than when a hiring round closes, and a fixed-scope contract prices a defined output, which moves the risk of overrun off your side of the table.

The risk is ending up renting a feature. If the code sits in the partner's repositories, the model keys in the partner's accounts and the evaluation set in someone's head, you have a working feature that only the partner can change. Settle where each of those will live before the contract is signed, not at handover. Our guide to agencies and consultancies goes further into what you own when an AI supplier's contract ends.

Ask two more things of any partner: who will actually do the work, and whether you can see one AI feature they have running in production, with the evaluation set behind it.

If AI is going to be part of what customers pay for, this is the route I would pick, with the same caveat as before: 1AYM is one of those partners. The partner builds the first feature with your engineers in the same repository and sets up the evaluation harness, the guardrails and the cost telemetry once. Your engineers build the second feature on those foundations while the partner reviews. By the third, the partner has stepped back and you know exactly which role to hire for.

It is slower than a partner working alone, because teaching takes time, and it only works if your engineers are given the hours. Put the handover in the plan with dates and names, or it becomes the thing that slips.

An evaluation set is a collection of real inputs with agreed good outcomes, graded automatically, that tells you whether a change made the feature better or worse. Without one, every prompt edit and every model upgrade is a guess, and your customers do the testing.

The vendors' own guides agree on its shape. Anthropic's evaluation guide says to mirror the real mix of tasks, include edge cases and automate the grading, and it prefers many automatically graded cases to a few graded by hand. OpenAI's guide advises calibrating any model-based grader against human judgement and running the evaluations on every change.

Whoever builds it, the evaluation set should end up in your repository, run in your pipeline and be understood by your engineers. It is the asset that lets you change supplier, change model or change your mind without starting again.

Model APIs are usually priced per token, so the running cost of a feature is roughly the tokens each request sends and receives, times the vendor's price, times how often customers use it. That number moves with every prompt change, every extra document retrieved and every customer who finds a heavier use than you planned for. OWASP's 2025 list of risks for LLM applications includes unbounded consumption for this reason. Measure the cost of each request from the first release, per customer, and decide in advance what happens when one customer's usage spikes.

Support is the cost that rarely makes the plan. When a customer disputes an answer, someone has to see which model version, prompt and retrieved documents produced it. When the vendor retires a model, someone has to run the evaluation set against its replacement. Those jobs fall to whoever owns the feature after launch, so decide who that is before you choose a route.

Before customer data goes to a model, read two documents you already have: your customer contracts and your privacy policy. In February 2024 staff at the Federal Trade Commission warned that it may be unfair or deceptive for a company to start using consumers' data for AI training and to tell them only through a surreptitious, retroactive change to its terms of service or privacy policy. The law behind that warning is Section 5 of the FTC Act, 15 U.S.C. § 45(a)(1), which declares unfair or deceptive acts or practices in or affecting commerce unlawful. If your customers are businesses, their contracts usually set out what you may do with their data, so read those first.

The same law covers what you say about the feature. Announcing Operation AI Comply in September 2024, the FTC's chair at the time said there is no AI exemption from the laws on the books. Describe what the feature does only as far as your evaluation results support. Where the model runs, and what the vendor may keep, depends on the hosting route; our guide to private LLMs, listed at the end of this page, compares the options.

This summarises US statute and FTC publications as they stood on 29 September 2026. This is not legal advice: have your own counsel review your contracts and your privacy policy.

Handover is easy to promise and hard to do at the end, so it belongs in the contract as a set of named deliverables. These are the ones to ask for.

Start from the feature and the customers waiting for it, not from the staffing model you would prefer.

If you cannot yet say which feature to build first, or for which customers, none of the four fits yet. Settle that before the staffing, because a hiring plan or a statement of work written before it is a guess.

1AYM builds AI features into production software and hands them over. On one government-accredited EdTech's live platform, about 1,170 automated tests cover the AI speaking assessment, among them a regression suite in which switching off any one guardrail breaks specific frozen cases. That is the kind of evaluation I would expect any partner to leave in your repository.

For a software company the usual shapes are a fixed-scope architecture and production build of the first feature, which can start within a day of the scope being signed, or engineers embedded in your team on a contract from three months, directed by you day to day and reviewed weekly by a 1AYM architect. If you already have the work scoped, 1AYM can resource it on contract from the associates who work with it, held to the same standard. The work is delivered in your repositories, copyright is assigned to you where your contract needs it, and US companies can contract through 1AYM's US entity.