CLIEncoders · AI & Agents
AI model training and fine-tuning
Fine-tuning is frequently proposed and rarely the right first move. When it is right, the dataset does the work — and the dataset is most of the cost.
Try the cheaper things first
Most problems people bring to fine-tuning are solved better by a clearer prompt, few-shot examples, or retrieval. Those are faster, cheaper, and adjustable in minutes rather than requiring a training run.
Fine-tuning genuinely wins in a narrower set of cases: teaching a consistent output format or house style, compressing a long complex prompt into the weights to cut per-request cost and latency, adapting to specialist vocabulary a general model handles badly, or getting a small cheap model to match a large one on one narrow task. We will check the cheaper options first and tell you if they suffice.
The dataset is the project
Model architecture choices are largely settled; data quality is where outcomes are decided. That means sourcing and cleaning examples, designing a labelling guideline precise enough that two annotators agree, measuring that agreement, and holding back a genuinely representative test set before any training begins.
A thousand carefully labelled examples routinely outperform ten thousand noisy ones. Getting this right is unglamorous and it is where we spend the time.
Beyond language models
Classical machine learning is often the better tool and gets overlooked in the current enthusiasm. Gradient-boosted trees on tabular data, classifiers, and detectors are cheaper to train, faster to run, easier to explain and frequently more accurate on structured problems than anything with a transformer in it.
We also train vision models — detection and classification — and quantise or convert them with ONNX and TensorRT for edge deployment where inference has to happen on the device.
Questions
What people ask before starting
How much data do we need?
Far less than people assume for fine-tuning style and format — often a few hundred good examples. Considerably more for teaching genuine new capability, and often thousands per class for vision. The honest answer requires seeing your task and data, and the first thing we would do is establish a baseline without training at all, so you know what the training has to beat.
Should we fine-tune or use RAG?
They solve different problems and are frequently confused. RAG gives a model access to knowledge it can cite and you can update instantly. Fine-tuning changes behaviour, format and style. If your requirement is 'answer questions about our documents', that is RAG. If it is 'always respond in this exact structure', that is fine-tuning. Some systems want both.
How do you prove the model is accurate?
A held-out test set drawn from real production data, never touched during training, reported with the metrics that matter for your case — precision and recall separately where the two failure modes have different costs, not a single accuracy figure that hides which one you are getting. Any accuracy claim without a described test set is not a claim.
What happens as our data changes?
Performance drifts, quietly. We set up monitoring against a stable reference set so degradation is visible before users report it, and define the retraining trigger and procedure up front so it is routine maintenance rather than an emergency.
Related
Where this usually connects
Tell us what you are building
One technical call is usually enough to tell you whether this is straightforward, genuinely hard, or the wrong approach entirely. We would rather say so early than quote for the wrong thing.
Start the conversation