Your support agent works, but the monthly API invoice is growing and the prompt has reached 2,500 tokens of rules, examples and house style. Someone on the team says the obvious thing: fine-tune a smaller model on our own data, and the prompt shrinks, the bill drops, the answers sound like us. It is a reasonable idea. It is also an idea that most small and mid-sized companies should price carefully before acting on, because the arithmetic does not look the way the pitch does.
The training run is cheap. A few dollars, sometimes a few hundred. The money goes elsewhere: collecting and labelling examples, building the tests that tell you whether the new model is better, retraining when the base model changes, and keeping a person responsible for all of it. Meanwhile the supply side has shifted this autumn, with the largest self-serve fine-tuning platform being wound down.
If your question is where inference runs in the first place, our page on local inference covers the hardware side. This article stays with the decision in front of you now.
Fine-tuning pays only when you have high, stable volume and a working eval loop; otherwise a better prompt, retrieval and a good harness give you most of the benefit at a fraction of the effort. Below you find what fine-tuning actually changes, what it costs once labour is counted, which providers still offer it, a worked break-even, and the case for and against running the loop on your own hardware.
Scope note: the eval prerequisite is covered in AI evals from real tickets, and the routine that turns staff corrections into rules in the AI feedback loop. Both come before anything in this article.
What Fine-Tuning Changes, and What It Leaves Alone
Fine-tuning adjusts a model’s weights using examples you supply. Hamel Husain, an AI consultant who writes widely on evals, puts the division of labour plainly in his evals post: fine-tuning is best for learning syntax, style and rules, while techniques like retrieval supply the model with context or up-to-date facts.
That matches the research. Gekhman and colleagues, in a 2024 paper, found that fine-tuning examples which introduce new knowledge are learned significantly slower than examples consistent with what the model already knows, and that once learned they linearly increase its tendency to hallucinate. Their conclusion is that models mostly acquire factual knowledge in pre-training, and fine-tuning mainly teaches them to use it more efficiently.
For an owner, the practical split looks like this:
| You want the model to… | Best tool | Why |
|---|---|---|
| Know your price list, policies, product data | Retrieval (RAG) | Update a document and the answer changes tomorrow |
| Follow a fixed output format or tone of voice | Prompt first, fine-tune if the prompt grows unwieldy | Format and style are what fine-tuning handles well |
| Apply a stable set of business rules reliably | Harness (checks, tools, validation) first, fine-tune later | Rules in code are testable and auditable |
| Do one narrow task very cheaply at high volume | Fine-tune a small model | The only case where per-call economics can move |
A fine-tuned model that has absorbed last quarter’s prices will quote them confidently next quarter. Facts that change belong outside the weights.
The Cash Cost of Training Is the Small Number
Take the training invoice first, because it is the part people expect to be large. On OpenAI’s pricing page on 7 November 2026, training gpt-4.1-mini was listed at USD 5.00 per million training tokens. A dataset of 2,000 examples of 500 tokens each, run for 3 epochs, is 3 million training tokens, or about USD 15.
Husain writes that 99% of the labour involved in fine-tuning is assembling high-quality data that covers your product’s surface area. That is the opinion of a practitioner, with no measurement behind the percentage, but anyone who has labelled a few hundred support tickets will recognise the proportion. OpenAI’s own fine-tuning guide opens its advice with “Good evals first!” and says to invest in fine-tuning only after setting up evals. It also advises tuning a prompt on a larger model until you get great performance against your eval criteria.
So the budget lines to plan for are:
- Example collection and labelling. Someone who knows the right answer has to write or approve every training example.
- The eval set. A held-out test the model never trains on, built from real cases.
- Retraining. Each new base model, and each shift in your own rules, means another run and another round of evals.
- An owner. A named person who decides what counts as a good answer when two reviewers disagree.
None of these appear on the cloud invoice. All of them appear on your payroll.
Who Still Offers Fine-Tuning in November 2026
The provider landscape has narrowed, and a project that began on the assumption of “just fine-tune GPT” needs checking against the dates.
OpenAI is winding the platform down. According to its deprecations page, job creation was closed to organisations that had not previously run fine-tuning from 7 May 2026. On 6 January 2027, active existing customers can no longer create new fine-tuning jobs. GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini are listed for shutdown on 14 January 2027, and fine-tuned versions follow the same schedule as their base models.
Anthropic’s glossary states that the Claude API does not currently offer fine-tuning and suggests asking your Anthropic contact if you are interested in exploring it.
Mistral marks its fine-tuning documentation as deprecated and no longer actively supported.
Open-weight models remain the live path. You can train LoRA adapters on hosted services such as Together or Fireworks, on rented GPUs, or on your own machine. Our open-weight models guide describes the model families worth considering. Check the licence of any model before commercial use, since terms differ.
A note on pricing. OpenAI’s fine-tuned models cost more to run than the base. On its pricing page on 7 November 2026, fine-tuned gpt-4.1-mini inference was listed at USD 0.80 per million input tokens and USD 3.20 per million output tokens, against USD 0.40 and USD 1.60 for the base model. The same pattern holds across the listed models, at 1.5 to 2 times base. The saving from a fine-tune therefore has to come from a shorter prompt or a smaller model, because the per-token price goes up.
A Worked Break-Even, With the Assumptions Visible
The following is an illustrative example on assumed numbers, so substitute your own. It uses the gpt-4.1-mini prices above purely because they are published, and it should be read as arithmetic, because new OpenAI jobs end on 6 January 2027.
Assumptions: each call produces 300 output tokens. The base model needs a 2,500-token prompt carrying instructions and examples. The fine-tuned model has absorbed those examples and needs a 300-token prompt.
| Base model | Fine-tuned | |
|---|---|---|
| Input per 1,000 calls | 2.5M tokens at USD 0.40 = USD 1.00 | 0.3M tokens at USD 0.80 = USD 0.24 |
| Output per 1,000 calls | 0.3M tokens at USD 1.60 = USD 0.48 | 0.3M tokens at USD 3.20 = USD 0.96 |
| Total per 1,000 calls | USD 1.48 | USD 1.20 |
The saving is USD 0.28 per 1,000 calls. Against a USD 15 training run, break-even arrives at about 54,000 calls. At 1,000 calls a month, the token saving is 28 US cents a month.
That figure excludes labelling, eval work and retraining, which would swamp it. What the arithmetic says is that the token price rarely decides the question at SMB volumes. The case has to rest on something else: answer quality on a task where prompting plateaus, latency from a shorter prompt and a smaller model, a data-protection requirement that rules out sending prompts to a foreign API, or the effort of maintaining a very long prompt.
The trend also works against you. According to Epoch AI, the price of reaching GPT-4-level performance on one science benchmark fell by about 40 times a year, with a range from 9 to 900 times a year across six benchmarks. Epoch notes that the fastest drops occurred in the most recent year, so it is unclear they will persist. Even so, a falling API price narrows whatever advantage a fine-tuned small model holds.
LoRA and QLoRA: Why Training Fits on Modest Hardware
Training does not need the weights of a large model to be rewritten in full. LoRA, introduced by Hu and colleagues, freezes the base model and trains small additional matrices. Against full fine-tuning of GPT-3 175B with Adam, the authors report 10,000 times fewer trainable parameters and 3 times less GPU memory, with no additional inference latency.
QLoRA from Dettmers and colleagues adds quantisation of the frozen model. The authors report fine-tuning a 65-billion-parameter model on a single 48GB GPU while preserving full 16-bit fine-tuning task performance. Their headline benchmark claim for the resulting Guanaco models is the authors’ own measurement, and their abstract cautions that current chatbot benchmarks are not trustworthy for evaluating chatbot performance.
For a Swiss SMB, the consequence is that a single workstation-class card can run adapter training. How large a model fits on which card depends on the model, sequence length and settings, so run a small test job on your own card before committing to a purchase. A Thinking Machines study, LoRA Without Regret, finds that LoRA matches full fine-tuning on small to medium datasets but underperforms when the data exceeds the adapter’s capacity. The company sells LoRA training through its Tinker service, which is worth bearing in mind when weighing its conclusions.
A Monthly Learning Loop: Own Hardware Against a Rented GPU
A continual-learning loop means that corrections and approved outputs from the past month become training data, a new adapter is trained, regression tests run, and the adapter replaces the old one only if it passes. It is the long-run version of the feedback loop described elsewhere, with weights as the last step.
Here is the cost comparison, again as an illustrative example. We assume a training and evaluation run of 10 GPU hours a month, and an exchange rate of USD 1.25 per CHF, which is our assumption for this arithmetic only.
- Rented: on RunPod’s pricing page, with prices as of 4 November 2026, an RTX 6000 Ada with 48GB was listed at USD 0.99 an hour in the secure cloud, billed per second. Ten hours cost about USD 10, or roughly CHF 8 a month.
- Rented around the clock: 730 hours at USD 0.99 is about USD 723, or roughly CHF 578 a month.
- Owned: in our local inference cost model, the reference workstation of CHF 18,000 (within the CHF 15,000 to 30,000 range) amortised over 36 months, plus electricity at CHF 0.237 per kWh for a roughly 1 kW system, comes to about CHF 671 a month all-in, whether the box is busy or idle.
Read honestly, this comparison says rented hardware wins for a monthly retrain. Ten hours rented costs a small fraction of the owned box, and even continuous renting of a 48GB card undercuts the owned figure. Anyone who tells you that buying a GPU is the cheap route for fine-tuning alone has skipped this sum.
Own hardware earns its place on two other grounds.
Data residency. Under Article 16 of the revised Swiss Federal Act on Data Protection, personal data may be disclosed abroad if the Federal Council has found the destination’s law adequate, or otherwise only with listed safeguards such as approved contractual clauses. RunPod is a US provider. Since September 2024 Switzerland lists the United States as adequate for companies certified under the Swiss-US Data Privacy Framework, so check the provider’s certification or choose an EU region first. Without either, sending customer records for training is a disclosure abroad that needs one of the listed safeguards. This is not legal advice, so confirm with counsel. Training on a machine in your own premises removes that question, although you still have to inform people and secure the data.
Utilisation. If the same box also serves inference for your staff during the day, as discussed in the local inference article, the monthly figure is shared across both jobs, and the retrain cycle costs almost nothing extra.
Then there is forgetting. Luo and colleagues, in a 2023 study, observed catastrophic forgetting across models from 1 to 7 billion parameters, affecting domain knowledge, reasoning and reading comprehension, and found that it intensified as model scale increased. Each monthly retrain can quietly degrade abilities you did not train on. The loop therefore needs a regression eval on held-out tasks, run before every adapter swap, and a rule that a failed run keeps last month’s adapter in place.
What Shopify’s Loop Does and Does Not Prove
Shopify’s engineers describe a detailed production loop in a post on Sidekick dated 5 August 2026. One thing in it is worth taking, and several are worth discounting.
What to take: the sequence. They improve the harness first. According to the post, every improvement lands in prompts, tool definitions and harness code, and only once those gains plateau do they begin optimising weights from anonymised production traffic. The weight stage uses supervised fine-tuning followed by GRPO, a reinforcement learning method. That order matches the advice to build the harness before touching weights.
What to discount:
- The cost figures are estimates. Shopify says serving the traffic on a frontier model could easily cost an estimated USD 27 million a year, and that the fine-tuned model could come in closer to USD 1 million. Both are the team’s own estimates with no derivation shown. The 96% reduction is arithmetic on those two numbers.
- Quality is judged by Shopify’s own judge. It is calibrated against human annotators, but it is still the same team marking its own work.
- The latency result is one load test. At 350 requests per minute, time to first token fell about 19% and end-to-end latency about 38%.
- Shopify is not an SMB, and the loop is not on its own hardware. It has annotators, expert reviewers and a platform team.
The transferable lesson is the sequence and the human labour behind it, with the dollar figures set to one side.
A Checklist Before You Commit
Fine-tune when most of the following are true:
- The task is stable. The inputs and the definition of a good answer will not change much for a year.
- The volume is large, in the hundreds of thousands of calls or more, or a data-protection rule forces the work onto your own machines.
- A working eval set exists, built from real cases, with a held-out part the model never trains on.
- A named person owns the labels and decides disputes.
- The prompt-and-harness version has hit a ceiling that you measured, and was not abandoned early.
- You have a plan for the next base-model release, because the adapter will need retraining.
Keep prompting, retrieval and a stronger harness when any of the following is true: the task or the facts change monthly, volume sits in the thousands of calls, or no one has yet written down what a correct answer looks like. If you are unsure, start with prompting and the eval set, because both carry over to whatever you do next.
The Asset That Outlasts the Model
Weights are tied to a base model, and base models expire. OpenAI’s schedule shows the pattern: the fine-tune dies on the same day as the model underneath it. The labelled examples and the eval set have no such expiry. They transfer to the next model, whether you prompt it, tune it or switch vendors, and they are the part that took the labour.
That is the real economics. A team that builds the eval set and a clean corpus of approved answers has bought an option: when volume or a residency requirement makes fine-tuning worthwhile, the training run is a small final step. A team that skips them and goes straight to training has spent the cash on the part that was cheap.
Frequently asked questions
Is fine-tuning cheaper than prompting?
Rarely at SMB volumes. Training itself can cost a few dollars, but fine-tuned inference on OpenAI is priced at 1.5 to 2 times the base model per token. Savings come from a shorter prompt or a smaller model. In our illustrative example the saving is USD 0.28 per 1,000 calls, so it takes tens of thousands of calls to repay even the training run.
Can fine-tuning teach a model our company facts?
Not reliably. Gekhman and colleagues found that examples introducing new knowledge are learned more slowly and, once learned, raise the tendency to hallucinate. Fine-tuning suits style, output format and rules. Facts such as prices, policies and product data belong in retrieval, where you can update a document without retraining anything.
Can I fine-tune Claude or GPT models in 2026?
Claude: no. Anthropic's glossary says the API does not currently offer fine-tuning. OpenAI is winding its platform down. Active existing customers lose job creation on 6 January 2027, and GPT-4o and GPT-4.1 models and their fine-tunes shut down on 14 January 2027. Open-weight models trained with LoRA remain available through hosted providers or your own hardware.
Does fine-tuning on our own hardware help with Swiss data protection?
It can. Under Article 16 of the revised FADP, disclosing personal data abroad needs an adequacy decision or listed safeguards, and a US training service is such a disclosure. Check whether the provider is certified under the Swiss-US Data Privacy Framework, which Switzerland has recognised as adequate since September 2024, or use its EU regions, and fall back on contractual safeguards otherwise. Training on a machine you control avoids that question, though not your duties to inform and secure the data. This is not legal advice, so have counsel confirm your case.
How many examples do I need to fine-tune a model?
OpenAI says the minimum is 10 examples, reports improvements from 50 to 100, and suggests starting with 50 well-crafted demonstrations. Those are vendor starting points rather than measured thresholds. The harder requirement is an eval set that tells you whether the tuned model is better, and a named person who owns the labels.