Fine-tuning gpt-oss-20b with Unsloth and running it in Ollama

I fine-tuned gpt-oss-20b for STEM reasoning and published it three ways: a LoRA adapter, a merged 16-bit model and GGUF files. Below are the settings and versions behind it, the export code, and the Modelfile that makes the GGUF behave in Ollama.

QLoRA settings for gpt-oss-20b

The published run, as recorded on the model card:

The full training config, with target modules and scheduler, is in configs/gpt_oss_20b.yaml in kllm, the small toolkit I use for these runs.

How much VRAM does fine-tuning gpt-oss-20b need?

QLoRA keeps the base weights in 4-bit and trains only a small adapter, so a 20B model fits on one consumer card. Unsloth's gpt-oss guide lists about 14 GB of VRAM for gpt-oss-20b with QLoRA and recommends at least 16 GB for stable runs. LoRA on unquantized BF16 weights needs about 44 GB, which is why QLoRA is the practical choice on one card.

The environment kllm is verified on:

uv pip install -U vllm --torch-backend=cu128   # cu121 for RTX 40 series
uv pip install unsloth unsloth_zoo bitsandbytes
uv pip install --force-reinstall "transformers==4.57.3"

Export to LoRA, merged and GGUF with Unsloth

After training, the same model object gives all three formats. These are the calls kllm makes:

# 1. LoRA adapter (~61 MB): for people who already run the base model
model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
 
# 2. Merged 16-bit model (~41 GB): for Transformers or vLLM, no adapter handling
model.save_pretrained_merged(f"{output_dir}-merged", tokenizer, save_method="merged_16bit")
 
# 3. GGUF for llama.cpp, Ollama and LM Studio, one call per quantization
model.save_pretrained_gguf("gguf/gpt-oss-20b-stem", tokenizer, quantization_method="q4_k_m")

quantization_method takes f16, q8_0, q4_k_m and the other llama.cpp types listed in Unsloth's GGUF docs. The GGUF files for this model came out at:

If the GGUF step runs out of memory, Unsloth's docs suggest lowering maximum_memory_usage from its 0.75 default to around 0.5.

The Harmony chat template is what breaks

gpt-oss doesn't use ChatML. It was trained on OpenAI's Harmony format, where every message is wrapped in special tokens and the assistant writes on separate channels:

• analysis for its chain of thought, which end users shouldn't see
• final for the answer
• commentary for tool calls

A final answer ends with <|return|>, which OpenAI's docs call "a valid stop token indicating that you should stop inference." Messages kept in the conversation history end with <|end|> instead, while supervised training targets should end with <|return|>. OpenAI's model card is blunt about it: the models "should only be used with the harmony format as it will not work correctly otherwise."

So the template you ship has to match the one used in training. Unsloth's docs name a mismatch as the most common reason a model gives gibberish or endless output on other platforms. And <|return|> has to be a stop token, or generation runs past the answer.

This exact question, how to template a fine-tuned gpt-oss for Ollama, is still open on OpenAI's forum.

A working Ollama Modelfile for fine-tuned gpt-oss

This is the Modelfile that ships with the GGUF repo:

FROM ./gpt-oss-20b-finetuned-q8_0.gguf
 
TEMPLATE """<|start|>system<|message|>You are a helpful assistant trained by OpenAI.
 
Reasoning: medium
 
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>user<|message|>{{ .Prompt }}<|end|><|start|>assistant"""
 
PARAMETER stop "<|return|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 4096
PARAMETER num_predict 2048

Setting reasoning effort

Reasoning: medium in the system message sets how much the model thinks in its analysis channel before it answers. Harmony accepts low, medium and high, and medium is the default. To change it, edit that line in the TEMPLATE and run ollama create again. Ollama shows the analysis channel as "Thinking...", so low is the setting for short, fast answers.

ollama run hf.co or a Modelfile?

With the Modelfile, the template is the one you wrote:

ollama create gpt-oss-20b-stem -f Modelfile
ollama run gpt-oss-20b-stem

Ollama can also pull straight from Hugging Face:

ollama run hf.co/khadim-hussain/gpt-oss-20b-stem-reasoning-GGUF:Q4_K_M

That route doesn't read the Modelfile in the repo. Per Hugging Face's Ollama docs, it uses the chat template stored in the GGUF's tokenizer.chat_template metadata, or a file named template in the repo. The quantization tag is case-insensitive, and without a tag Ollama picks Q4_K_M. If you publish a Harmony model, put a template file in the repo or point people to the Modelfile.

Which format people actually download

All-time Hugging Face downloads as of October 2026:

Most people want a file they can run on their own machine. If you only publish one format, make it a GGUF with a working template. The adapter is worth adding for anyone who wants to keep training.

Troubleshooting

The failures documented in kllm's troubleshooting guide:

The full project, with the Qwen3-14B run and the download numbers, is in the STEM reasoning LLMs case study.