Both are cheap ways to get a model you want from a model you have. That is where the similarity ends.
What LoRA does
Low-rank adaptation freezes the base model and trains small additional matrices alongside it. Because those adapters are tiny relative to the model, training is fast, memory requirements are modest, and the resulting artefact is a few megabytes rather than a full checkpoint.
The base model is unchanged. You can keep many adapters for one base and swap between them per request, which is why multi-tenant serving of fine-tuned behaviour is practical at all.
What it does not do
It does not make the model smaller. At inference you are still running the full base model plus a small addition. Memory and latency are essentially unchanged, possibly marginally worse.
If your problem is that serving costs too much, LoRA does not address it.
The distinction
LoRA is a way of doing fine-tuning efficiently. Fine-tuning changes behaviour. Distillation changes cost by producing a new, smaller model. So the comparison that actually matters is the one in Distillation vs Fine-Tuning, and LoRA sits entirely on the fine-tuning side of it.
Put plainly: LoRA gives you a differently-behaved model of the same size. Distillation gives you a similarly-behaved model of a smaller size.
Using both
They compose in a natural order:
- Take a strong base model
- Adapt it with LoRA until it does your task well
- Merge the adapter and use the result as a teacher
- Distil into a small student sized for your serving budget
- Quantise the student
Each step addresses a distinct constraint. Skipping to step four without step two distils general capability you do not need, at proportionately greater cost.
Common questions
Is LoRA a form of distillation?
No. LoRA is a parameter-efficient fine-tuning method: it trains small low-rank adapter matrices while leaving the base model frozen. The model stays the same size and costs the same to serve. Distillation trains a different, smaller model.
Can you use LoRA and distillation together?
Yes, and it is a sensible pattern. Fine-tune a large model with LoRA until it does the task well, then use that adapted model as a teacher and distil it into something smaller for serving.