Ollama no longer supports LoRA adapters: how to run a fine-tune now

I publish my Qwen3-14B STEM fine-tune three ways: as a LoRA adapter, as a merged model and as GGUF files. For a long time, the adapter route in Ollama was two lines in a Modelfile, a FROM for the base model and an ADAPTER for the fine-tune. On a current Ollama, that second line makes ollama create fail before it touches the model files. I tested what still works with the same adapter, end to end.

What changed in Ollama 0.34.1?

Ollama removed LoRA adapter support in PR #14969, merged on 15 September 2026 and shipped in v0.34.1. The PR limits ollama create to "wrapping existing GGUF inputs into Ollama manifests" and removes the built-in safetensors-to-GGUF converter and the server-side quantizer, "so GGUF conversion and quantization stay in llama.cpp tooling". Adapters went with the converter: the CLI and the server both reject a Modelfile that contains ADAPTER.

The release notes give this one line: "GGUF model creation now requires using llama.cpp tooling for safetensor conversion and quantization." The word adapter isn't in them. The ADAPTER section and the "Importing a fine tuned adapter" section have since been removed from the Modelfile and import docs, so the old instructions now live on mostly in blog posts and tutorials. Nothing has changed since: v0.40.7, released 18 October 2026, rejects adapters the same way.

The last row comes from Ollama's own source: when it loads an older model that has an adapter layer, it logs "LoRA adapters are deprecated; the adapter layer is carried over but new adapters cannot be created" (server/model.go). I didn't have such a model to test.

What does "LoRA adapters are no longer supported" mean?

It means the Modelfile has an ADAPTER line, and Ollama 0.34.1 or later refuses to build any model that has one. I ran the variants against Ollama 0.40.7 on Windows. With ADAPTER pointing at the GGUF LoRA, or at the PEFT folder with adapter_model.safetensors, ollama create stops before it reads either file:

$ ollama create stem-qa -f Modelfile
Error: LoRA adapters are no longer supported

Pointing FROM at the GGUF LoRA itself gets one step further. Ollama copies the file, reads its header and refuses it there:

Error: LoRA adapters are no longer supported: qwen3-14b-stem-qa-lora-f16.gguf is a LoRA adapter

Asking Ollama to quantize a GGUF fails too, because quantization moved to llama.cpp as well:

$ ollama create stem-qa --quantize q4_K_M -f Modelfile
Error: create-time quantization is only supported for safetensors imports; quantize GGUF models before importing

Getting the error for a vision projector?

Some Modelfiles for vision models put the multimodal projector, the mmproj GGUF, on the ADAPTER line. Ollama rejects those now too, with the same message, even though a projector isn't a LoRA. The projector goes on a second FROM line:

FROM ./SmolVLM-256M-Instruct-Q8_0.gguf
FROM ./mmproj-SmolVLM-256M-Instruct-Q8_0.gguf

I tested this with SmolVLM-256M because it is small: with ADAPTER the create failed, and with two FROM lines it succeeded and ollama show listed the projector and the vision capability.

How do I use a LoRA fine-tune in Ollama now?

Merge the adapter into the base model, convert the result to GGUF, quantize it with llama.cpp, and point FROM at that file. Ollama still imports any GGUF; it just won't build one for you anymore.

Merge the adapter, then convert and quantize

If you trained with Unsloth, the model object does the merge and the GGUF export itself. These are the calls from my gpt-oss fine-tuning walkthrough, with this model's names:

# merged 16-bit model, for Transformers or vLLM, and the input for a GGUF
model.save_pretrained_merged("qwen3-14b-stem-qa-merged", tokenizer, save_method="merged_16bit")
 
# or merge, convert and quantize in one call
model.save_pretrained_gguf("qwen3-14b-stem-qa-gguf", tokenizer, quantization_method="q4_k_m")

With plain PEFT, load the base model in 16-bit, not 4-bit, so the merged weights aren't rounded to the 4-bit grid:

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
 
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "khadim-hussain/qwen3-14b-stem-qa").merge_and_unload()
model.save_pretrained("qwen3-14b-stem-qa-merged")
AutoTokenizer.from_pretrained("Qwen/Qwen3-14B").save_pretrained("qwen3-14b-stem-qa-merged")

Then convert and quantize with llama.cpp, the two steps Ollama dropped:

python convert_hf_to_gguf.py qwen3-14b-stem-qa-merged --outtype f16 --outfile qwen3-14b-stem-qa-f16.gguf
llama-quantize qwen3-14b-stem-qa-f16.gguf qwen3-14b-stem-qa-q4_k_m.gguf Q4_K_M

My published Qwen3-14B GGUF came out of this path. Its metadata calls the model "Qwen3 14b Finetuned Merged", the name llama.cpp's converter takes from the merged model's folder, and the Q4_K_M file is 9.0 GB. Merging needs the whole 16-bit model in memory, and my F16 GGUF of it is 29.5 GB, so it is a step for the training machine rather than a laptop.

llama.cpp also has a merge tool that works on GGUF files directly, llama-export-lora, which applies a GGUF LoRA to a GGUF base and writes a merged file. Don't look for it in the release download: the Windows CPU zip I used (build b11793) loads its CPU backends dynamically, and llama.cpp's build file skips export-lora in that configuration (tools/CMakeLists.txt). You would have to build it from source with GGML_BACKEND_DL off. I didn't, because the merged GGUF from training already existed.

Import the merged GGUF into Ollama

The Modelfile loses its ADAPTER line, and FROM points at the merged file:

FROM ./qwen3-14b-stem-qa-q4_k_m.gguf
 
TEMPLATE """<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
 
PARAMETER stop "<|im_end|>"
PARAMETER num_ctx 4096

ollama create qwen3-14b-stem -f Modelfile
ollama run qwen3-14b-stem "What is DNA?"

On Ollama 0.40.7 the create took 20 seconds, most of it spent copying the 9.0 GB file into Ollama's model store, and the first request loaded the model in 9.6 seconds. It answered the DNA question directly, in 49 tokens and without a <think> block, which is how the adapter behaves in llama.cpp below. The wording wasn't identical ("two long chains of nucleotides" where the adapter run says "two polynucleotide chains"). That is expected: this file was merged first and quantized afterwards, while llama.cpp adds the 16-bit adapter on top of an already quantized base, so the two setups don't compute exactly the same numbers.

How do I run a LoRA adapter without Ollama?

llama.cpp still loads LoRA adapters, and it can apply one on top of a quantized base at run time. It needs the adapter as a GGUF file, so the first step is a conversion.

Convert the PEFT adapter to a GGUF LoRA

convert_lora_to_gguf.py ships with llama.cpp. It reads adapter_config.json and adapter_model.safetensors and needs only the base model's config, not its weights:

pip install -r requirements/requirements-convert_lora_to_gguf.txt
python convert_lora_to_gguf.py ./qwen3-14b-stem-qa \
  --base-model-id Qwen/Qwen3-14B \
  --outtype f16 \
  --outfile qwen3-14b-stem-qa-lora-f16.gguf

My adapter is rank 32 on all seven attention and MLP projections of the 40 layers, which makes 560 tensors. Each conversion took under a minute on my laptop's CPU:

Serve the base model with the adapter on top

llama-server takes the base GGUF with -m and the adapter with --lora, and serves an OpenAI-compatible API plus its own /completion endpoint:

llama-server -m Qwen3-14B-Q4_K_M.gguf \
  --lora qwen3-14b-stem-qa-lora-f16.gguf \
  -c 4096 --port 8090

On my laptop (Intel Core Ultra 9 285H, 32 GB RAM, no GPU) the server was ready about 15 seconds after start. The log prints one cannot use buft 'CPU_REPACK', fallback to CPU warning per adapter tensor, 240 of them here. They mean the adapter weights sit in a plain CPU buffer rather than a repacked one, and generation still works.

Each request can set the adapter's strength through a lora field, so one server answers as the base model and as the fine-tune:

curl http://127.0.0.1:8090/completion -d '{
  "prompt": "<|im_start|>user\nWhat is DNA?<|im_end|>\n<|im_start|>assistant\n",
  "n_predict": 160, "temperature": 0,
  "lora": [{"id": 0, "scale": 1.0}]
}'

With the scale at 0, Qwen3-14B opened a <think> block and was still reasoning when it reached the 160-token limit. With the scale at 1, the fine-tune answered in 74 tokens and stopped:

Deoxyribonucleic acid is a polymer composed of two polynucleotide chains that coil around each other to form a double helix carrying genetic instructions for the development, functioning, growth and reproduction of all known organisms and many viruses.

So the adapter was applied: the fine-tune skips Qwen3's reasoning block and answers directly. GET /lora-adapters lists the loaded adapters and their ids, and POST /lora-adapters sets the scale for requests that don't send one. Start the server with --lora-init-without-apply if the adapter should be off by default.

The Q8_0 adapter loads the same way. At temperature 0 its answer to the DNA question differed from the F16 adapter's by a few words ("development, survival and reproduction of all known living organisms" instead of "development, functioning, growth and reproduction of all known organisms"). At half the file size, that makes Q8_0 a reasonable default for this adapter.

Is a runtime LoRA slower than a merged model?

A little. I loaded the merged GGUF and the base-plus-adapter setup side by side, then alternated requests between them: 128 tokens each, temperature 0, four rounds, CPU only with 16 threads.

These are medians. With the adapter on, generation was about 13% slower than the merged file, and prompt processing about 7% slower. I measured CPU only. Alternating matters on a laptop: an earlier run, with a large download going in the background, varied more between identical requests than these setups vary from each other.

For one fine-tune that should always be on, the merged file is faster. The adapter route pays off when one base model serves several adapters, or the plain base model as well.

Which route should you use?

For most people who were using ADAPTER in Ollama, the merged GGUF is the shortest path back: the Modelfile loses one line and points FROM at the merged file. Keep the adapter as well, because it is the file you build every other format from.