Off-the-shelf embedding models are trained on web-scale general text. Your corpus is contracts, support tickets, lab protocols, or parts catalogues, and the words that matter in it (product codes, internal jargon, the difference between two near-identical procedures) are exactly what a general model blurs together. Fine-tuning the embedding model on your own data is the single highest-leverage improvement available to a RAG system, and it is a few hours of PyTorch work. This tutorial shows the full loop: mining training pairs, contrastive fine-tuning with sentence-transformers, evaluating with recall@k, and plugging the result into pgvector.
Why generic embeddings underperform
An embedding model maps text to a vector so that similar meaning lands nearby. "Similar" is defined by the training data. On a domain corpus three things go wrong: rare domain terms get poor representations, near-duplicate documents that differ in one critical detail collapse onto the same point, and the model's notion of relevance (news-like topical similarity) does not match yours (answers a specific question). Fine-tuning realigns all three, and because the model is small (typically 100M-600M parameters) it trains quickly on one GPU.
1. Mine training pairs
Contrastive fine-tuning needs (query, positive passage) pairs, and benefits from hard negatives: passages that look relevant but are not. Sources, in order of quality:
- Search and click logs. A query followed by a click-and-dwell on a passage is a positive. This is the best data you can get, and it is free.
- Question-answer records. Support tickets with linked knowledge-base articles; FAQs; internal Q&A.
- Synthetic queries. Have an LLM write three questions each passage answers. Lower quality than real logs but it covers the whole corpus; audit a sample.
Hard negatives come from your current retriever: for each query, take top-ranked passages that are not the positive. Filter out false negatives (passages that actually do answer the query) with a cross-encoder or an LLM judge, or you will train the model to push correct answers away.
Store it as JSONL with query, positive, and negative fields. A few thousand pairs is enough to see a clear improvement; tens of thousands is better.
2. Build a held-out evaluation set first
Before training, split off 10-20% of the queries with their positives as a labelled evaluation set and never train on them. Also keep the full passage corpus (or a large sample) as the retrieval pool, so recall is measured against realistic distractors rather than just the passages in the eval set.
3. Contrastive fine-tuning with sentence-transformers
Install the current stack:
pip install sentence-transformers datasets torch
The sentence-transformers v3+ training API is built on the Hugging Face Trainer. MultipleNegativesRankingLoss treats every other positive in the batch as a negative in addition to the explicit hard negative, which is why a larger batch (limited by memory) helps.
from datasets import load_dataset
from sentence_transformers import (
SentenceTransformer, SentenceTransformerTrainer,
SentenceTransformerTrainingArguments, losses)
model = SentenceTransformer("BAAI/bge-base-en-v1.5") # any strong open base
train = load_dataset("json", data_files="data/train.jsonl", split="train")
train = train.select_columns(["query", "positive", "negative"])
loss = losses.MultipleNegativesRankingLoss(model)
args = SentenceTransformerTrainingArguments(
output_dir="out/embed-ft",
num_train_epochs=1,
per_device_train_batch_size=64,
learning_rate=2e-5,
warmup_ratio=0.1,
bf16=True,
batch_sampler="no_duplicates", # no repeated positives inside a batch
logging_steps=50,
save_strategy="epoch",
)
trainer = SentenceTransformerTrainer(
model=model, args=args, train_dataset=train, loss=loss)
trainer.train()
model.save_pretrained("out/embed-ft/final")
One epoch at a low learning rate is usually right; embedding models overfit quickly and a second epoch often hurts generalization to unseen queries. If memory limits batch size, CachedMultipleNegativesRankingLoss gets the benefit of a large batch (thousands) in chunks at a compute cost.
4. Evaluate with recall@k
Evaluate the base and fine-tuned models identically: embed the whole passage pool, embed the held-out queries, and count how often the positive is in the top k.
import torch
from sentence_transformers import SentenceTransformer
def recall_at_k(model_path, queries, positives, corpus, ks=(1, 5, 10)):
model = SentenceTransformer(model_path)
c = model.encode(corpus, batch_size=256, normalize_embeddings=True,
convert_to_tensor=True)
q = model.encode(queries, batch_size=256, normalize_embeddings=True,
convert_to_tensor=True)
scores = q @ c.T # cosine, since normalized
top = scores.topk(max(ks), dim=1).indices.cpu()
target = torch.tensor([corpus.index(p) for p in positives]).unsqueeze(1)
return {k: (top[:, :k] == target).any(dim=1).float().mean().item() for k in ks}
for path in ("BAAI/bge-base-en-v1.5", "out/embed-ft/final"):
print(path, recall_at_k(path, eval_queries, eval_positives, corpus))
Report both. On domain corpora we regularly see recall@5 move from the 0.5-0.7 range to 0.8-0.9 after fine-tuning; if you see no improvement, the training pairs are probably not representative of the evaluation queries, or hard negatives contained false negatives. Also check a generic benchmark (a slice of MTEB) to make sure the model has not forgotten general English; a small drop there is acceptable, a large one means the learning rate was too high.
5. Plug into pgvector
If you already run Postgres, pgvector is the pragmatic vector store. Create the table with the dimensionality of your model (768 for bge-base), an HNSW index for cosine distance, and embed with the fine-tuned model at both ingestion and query time; mixing models silently breaks retrieval.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE passages (
id bigserial PRIMARY KEY,
doc_id text NOT NULL,
body text NOT NULL,
embedding vector(768) NOT NULL
);
CREATE INDEX ON passages USING hnsw (embedding vector_cosine_ops);
import psycopg
from pgvector.psycopg import register_vector
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("out/embed-ft/final")
conn = psycopg.connect("dbname=rag")
register_vector(conn)
def ingest(doc_id, chunks):
vecs = model.encode(chunks, normalize_embeddings=True)
with conn.cursor() as cur:
cur.executemany(
"INSERT INTO passages (doc_id, body, embedding) VALUES (%s, %s, %s)",
[(doc_id, c, v) for c, v in zip(chunks, vecs)])
conn.commit()
def search(query, k=5):
v = model.encode([query], normalize_embeddings=True)[0]
with conn.cursor() as cur:
cur.execute(
"SELECT doc_id, body FROM passages ORDER BY embedding <=> %s LIMIT %s",
(v, k))
return cur.fetchall()
Tag every row with the embedding model version so a future re-fine-tune can re-embed incrementally. Combine with a lexical (tsvector) query and a reranker for the hybrid setup described on our RAG systems page.
Keep it improving
Retrieval logs from the deployed system are next quarter's training data. Schedule a re-fine-tune when the corpus or query distribution changes materially, re-run the recall@k evaluation every time, and keep the evaluation set growing with real failed queries. If you would like help with any part of this loop, contact us.