“Can we use this with our data?” is usually the first question a risk team asks about an AI project, and the honest answer is that it depends on where the model runs. There are three broad options, and choosing between them is more a business decision than a technical one.
What are the three options?
- A public hosted API. You send text to a provider’s service and get a response. Nothing to run yourself.
- A private deployment in your cloud. The same class of model, offered through your own cloud account, inside your network boundary and under your access controls.
- A self-hosted model. An open-weight model running on infrastructure you operate, in the cloud or on your own hardware.
How do they compare?
| Factor | Public API | Private cloud deployment | Self-hosted model |
|---|---|---|---|
| Data leaves your boundary | Yes, under the provider’s terms | No, stays in your account | No |
| Cost at low volume | Lowest | Moderate | Highest, capacity sits idle |
| Cost at high, steady volume | Grows with every request | Grows with use | Can be lower once capacity is used well |
| Response time | Varies with shared load | More consistent | Yours to tune |
| Control over model version | Provider decides | You choose when to move | Full |
| Effort to run | Minimal | Some cloud work | A team that operates it |
Start with the data, not the model
Before comparing options, sort the information the system will touch into classes. Three is usually enough:
- Public or low-risk: published content, generic drafting.
- Internal: plans, procedures, non-personal business data.
- Restricted: personal information, health or financial records, anything under a contract or regulation.
Then ask what your contracts, your regulators and your own policies say about each class. Many organizations find that the first class can go anywhere, the second needs contractual assurances about retention and training, and the third must stay inside their own boundary.
What should you check with a hosted provider?
- Is our data used to train or improve models? Can that be switched off, and is it off by default?
- How long are prompts and outputs kept, and can we set that period?
- In which region is the data processed?
- Who at the provider can see it, and under what circumstances?
- What notice do we get before a model version is retired?
Get the answers in the agreement, not from a sales call.
When does self-hosting make sense?
Running your own model is justified in a narrow set of cases: data that may not leave an isolated environment, very high and steady volume where your own capacity costs less than paying per request, or a need for response times you cannot get from a shared service.
It also brings work that is easy to underestimate. Someone has to size the hardware, apply updates, monitor quality and keep it available. If you do not already run comparable infrastructure, count that team in the cost.
A practical way to decide
- Classify the data the use case touches.
- Rule out any option your obligations do not allow for that class.
- Estimate realistic volume for the first year, not the pilot.
- Choose the simplest remaining option.
- Put a thin layer between your application and the model so you can change your mind later.
The last step matters most. Models and prices change quickly. A system built so the model can be swapped keeps your options open, and it lets you route simple requests to a small model and hard ones to a larger one.
Where the model runs is one control among several. Access rules, logging, testing and human review are still needed whichever option you choose.
