A group developer, GnLOLot, has revealed a 1B mannequin that runs absolutely on native {hardware}. The mannequin is MiniCPM5-1B-Claude-Opus-Fable5-Considering, with GGUF builds for llama.cpp-compatible runtimes. It wants no API key and makes no cloud calls.
The Proposed Mannequin
The mannequin is constructed on openbmb/MiniCPM5-1B. That base is an actual, documented launch from OpenBMB. It’s a dense 1.08B-parameter mannequin utilizing a regular LlamaForCausalLM structure. It has 24 layers, grouped-query consideration, and a 131,072-token context size. OpenBMB reviews 1B-class open-source SOTA inside its personal comparability set.
The bottom already ships a local considering template. Reasoning is toggled by means of enable_thinking, giving each a Suppose and a No Suppose mode. The by-product mannequin retains that template and MiniCPM5’s tool-call format.
On high of that base, the developer utilized a fine-tune. The cardboard states the mannequin is ‘additional fine-tuned on Fable 5 knowledge’ to enhance coding and instruction following. The GGUF card repeats this as ‘post-trained on Fable 5 knowledge.’
How it’s truly constructed
The described technique is just not classical distillation. You don’t shrink the unique mannequin. As an alternative you generate many conversations with a trainer mannequin. You seize its replies and reasoning traces as textual content. You then supervised-fine-tune a smaller base mannequin on these traces.
This distinction is vital for accuracy. Classical distillation transfers sign from a trainer’s logits or weights. Nobody has entry to Claude’s weights or logits. So that is supervised fine-tuning on generated outputs, not weight-level distillation. OpenBMB’s personal base mannequin, against this, makes use of a documented On-Coverage Distillation stage between its personal trainer and pupil checkpoints.
The sensible impact is that the 1B mannequin learns to mimic response format and elegance. It doesn’t soak up the trainer’s underlying functionality. A 1B parameter price range can’t maintain frontier-scale reasoning.
The specs that try
The context window is 128K tokens, inherited from the bottom config.json (131,072). The GGUF repository ships 4 quantizations. Q4_K_M is roughly 657MB and is labeled the smallest footprint. Q5_K_M is roughly 751MB. Q8_0 is roughly 1.1GB and is the maintainer’s advisable default. F16 is roughly 2.1GB.
The ‘657MB footprint’ is the smallest quant, not the default construct. The mannequin masses instantly in llama.cpp, Ollama, LM Studio, jan, and KoboldCpp.
Interactive: how the construct works
The explainer under walks the construct pipeline, the footprint tradeoffs, and the trustworthy cut up of what a fine-tune can and can’t carry over.
The right way to run it
The GGUF card offers a one-line path by means of Ollama:
ollama run hf.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Considering-GGUF:Q4_K_MThe identical repository paperwork llama.cpp, LM Studio, jan, and KoboldCpp. Really useful sampling for Suppose mode is temperature=0.9, top_p=0.95. The mannequin might emit reasoning blocks earlier than the ultimate reply, which downstream apps can strip.
Key Takeaways
- The mannequin is a supervised fine-tune of OpenBMB’s MiniCPM5-1B on Claude Fable 5 traces, not a weight-level distillation.
- Actual specs: 128K context, GGUF quants from ~657MB (Q4_K_M) to ~2.1GB (F16), Q8_0 the advisable default.
- High-quality-tuning on outputs transfers format and elegance, not frontier reasoning or broad data.
- No benchmarks or coaching dataset are revealed, so functionality claims are presently unverifiable.
- Apache-2.0 covers the bottom weights solely; coaching on Claude outputs raises a licensing query the cardboard leaves open.
