LLM Fine-Tuning and Customization with PyTorch
Your data, your weights, your infrastructure
LLM Fine-Tuning and Customization with PyTorch
Prompting a hosted API gets you a demo. Fine-tuning gets you a model that speaks your domain, follows your format, runs on your infrastructure, and does not ship your data to a third party. IntelliSensei fine-tunes open-weight large language models in native PyTorch: your data, your weights, your infrastructure.
This is engineering work, not prompt work. We treat a fine-tune like any other model-development project: a measurable objective, a clean dataset, a reproducible training run, and an evaluation you can trust before anything goes near production.
Open-weight model selection
The first decision is which base model to start from, and it is as much a licensing and sizing question as a quality question. We help you choose between the current open-weight families (Llama, Mistral, Qwen, Gemma and others), weigh their licenses against your commercial use, and size the model to the hardware you will actually serve on. A 7-8B model fine-tuned on your data routinely beats a much larger general model on your task, at a fraction of the inference cost.
Parameter-efficient fine-tuning
For most engagements the right tool is LoRA or QLoRA via the Hugging Face PEFT library: adapters that train a small fraction of the parameters, fit on a single GPU, and can be swapped per customer or per task. We also know when full fine-tuning is warranted (large domain shift, new vocabulary, very large datasets) and run it with FSDP2 across multiple GPUs when it is. Every run uses BF16 mixed precision, gradient checkpointing and distributed checkpointing so a preempted spot instance costs minutes, not a day.
Preference tuning and alignment
Instruction tuning teaches a model what to do; preference tuning teaches it how. We apply Direct Preference Optimization (DPO) and related methods from the TRL library to shape tone, format adherence and refusal behaviour from a modest set of ranked examples, without the infrastructure burden of full RLHF. This is usually what turns a "mostly right" model into one a customer-facing team will sign off on.
Data pipelines for instruction tuning
The dataset decides the outcome. We build the pipeline that turns your raw material (tickets, documents, transcripts, code, logs) into clean instruction pairs: deduplication, PII scrubbing, decontamination against your evaluation set, format normalization, and stratified train/eval splits. Synthetic data generation is used where it helps and audited where it is used.
Evaluation harnesses
Before and after every run you get numbers: task-specific metrics on a held-out set, an LLM-as-judge rubric for open-ended outputs, and human spot checks on a sampled slice. We track regressions on general capability as well as gains on your task, so the model does not forget how to do arithmetic while learning your product catalogue.
Distillation into deployable models
When the best-performing model is too large to serve economically, we distill it: the large model labels or ranks, and a small student model learns to match it. The result is a model that fits your latency and cost budget and carries most of the quality of the teacher.
Deliverables
A typical engagement delivers the trained weights or adapters, the data-preparation and training code in your repository, the evaluation harness with baseline and final scores, and a serving-ready artifact (merged weights exported for vLLM, or adapters ready for multi-tenant serving). See our LLM inference and serving optimization service for the production side, and our RAG systems service for cases where retrieval beats fine-tuning.
Want to find out whether fine-tuning is the right move for your use case? Contact us for a short discovery call; we will tell you honestly if a RAG pipeline or a smaller classical model would serve you better.