Run the Mythos Enhanced Coding Mannequin Domestically with llama.cpp and Pi

# Introduction

 
Qwythos-9B-Claude-Mythos-5-1M is a 9B reasoning and coding mannequin based mostly on Qwen3.5, designed for native coding workflows, agentic improvement, and long-context duties. What makes it fascinating is that it’s sufficiently small to run on shopper {hardware}, whereas nonetheless being succesful sufficient to assist with sensible coding duties.

On this information, I’ll present you how you can run the Mythos-enhanced Qwythos mannequin domestically utilizing llama.cpp, then join it to Pi so you need to use it as a neighborhood coding agent. I will likely be utilizing an RTX 4070 Ti Tremendous with 16GB of VRAM, which permits me to run the Q6_K MTP quantization comfortably. In case you have an 8GB GPU, I like to recommend beginning with the Q4_K_M variant as a result of it provides a greater stability between high quality, velocity, and reminiscence utilization.

# Putting in llama.cpp

 
First, set up the official llama.cpp command-line interface (CLI). This will provide you with entry to the llama command, together with llama serve, which we are going to use to run the mannequin domestically.

curl -LsSf https://llama.app/set up.sh | sh

Subsequent, add the set up listing to your shell path so your terminal can discover the llama command:

echo 'export PATH="$HOME/.native/bin:$PATH"' >> ~/.bashrc
supply ~/.bashrc

To substantiate that the set up labored, run:

If every little thing is put in appropriately, it’s best to see the obtainable llama.cpp instructions and choices.

# Setting a Hugging Face Cache Listing

 
Set the Hugging Face cache someplace with sufficient free storage. That is particularly helpful on cloud machines the place the default dwelling listing has restricted disk area.

export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"

To persist it throughout new terminal classes, add it to your shell configuration:

echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
supply ~/.bashrc

# Beginning the Qwythos MTP Mannequin

 
Run the Q6_K MTP GGUF mannequin with all obtainable GPU layers:

llama serve 
  -hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K" 
  --alias "qwythos-9b-mtp" 
  --host 0.0.0.0 
  --port 8910 
  --n-gpu-layers all 
  --ctx-size 100000 
  --parallel 1 
  --batch-size 1024 
  --ubatch-size 512 
  --flash-attn on 
  --cache-type-k q8_0 
  --cache-type-v q8_0 
  --spec-type draft-mtp 
  --spec-draft-n-max 6 
  --threads 12 
  --threads-batch 24 
  --temp 0.6 
  --top-p 0.95 
  --top-k 20 
  --repeat-penalty 1.05 
  --jinja 
  --perf

The primary time you run this command, llama.cpp will obtain the mannequin information from Hugging Face.

After that, it is going to load the mannequin into GPU reminiscence and begin a neighborhood server.

As soon as the mannequin is operating, open the native interface in your browser:

This additionally offers you an OpenAI-compatible native API endpoint at:

The Q6_K variant offers higher output high quality, nevertheless it additionally makes use of extra reminiscence. If you happen to run into VRAM or RAM points, scale back --ctx-size first. If that’s nonetheless not sufficient, strive the Q4_K_M variant as an alternative.

Listed below are an important flags within the command:

On my RTX 4070 Ti Tremendous, I used to be getting round 81.74 tokens per second. In my check, the mannequin was additionally in a position to cause by the duty, name the required instruments, and return the newest gold worth when used contained in the native agent workflow.

# Putting in Pi and the llama.cpp Integration

 
Pi can connect with the native llama.cpp server and use the mannequin as a coding agent. The pi-llama plugin is designed to attach Pi with a operating native llama.cpp server, without having an exterior API key.

Set up Pi:

curl -fsSL https://pi.dev/set up.sh | sh

Set up the llama.cpp plugin for Pi:

pi set up git:github.com/huggingface/pi-llama

Create a small check undertaking:

mkdir new-project
cd new-project

By default, the plugin seems to be for llama.cpp on port 8080. On this information, our native server is operating on port 8910, so we have to set the proper native API deal with earlier than beginning Pi:

export LLAMA_BASE_URL="http://127.0.0.1:8910/v1"

Now launch Pi:

Inside Pi, sort:

Choose the native mannequin alias:

You must see the mannequin listed inside Pi, and as soon as chosen, you can begin utilizing it as a neighborhood coding agent.

# Testing the Mythos-Enhanced Coding Mannequin with Pi

 
Now it’s time to check the native mannequin on actual coding duties inside Pi. I used two easy however sensible duties: a browser sport and a small Python CLI device.

// Constructing a Easy Browser Sport

Begin with a immediate that asks Pi to create a small browser sport known as “Beat the AI.”

Create a easy browser sport known as “Beat the AI”.

The participant has 30 seconds to reply brief pattern-recognition questions. Every right reply will increase the rating by one. Present a countdown timer, rating show, progress bar, and a ultimate outcomes display screen.

Necessities:
– Use HTML, CSS, and vanilla JavaScript solely.
– Generate not less than three forms of questions, akin to quantity patterns, fast math, and phrase logic.
– Add a restart button after the sport ends.
– Make the interface really feel playful and polished.
– Maintain all code simple to grasp and keep away from pointless information.

Check the sport within the browser earlier than ending.

In my check, Pi created the total sport in a single HTML file with embedded CSS and JavaScript, which retains the undertaking easy and simple to examine.

Pi then summarized what it constructed, together with the 30-second countdown, rating monitoring, progress bar, query sorts, and restart button.

After that, I opened the sport within the browser to confirm that it labored appropriately.

The end result was a sophisticated little sport with:

• A countdown timer
• A rating show
• A progress bar
• A number of query sorts
• A restart circulation after the sport ends

It is a good instance of how the mannequin can deal with a whole front-end job domestically without having an exterior API.

// Constructing a CSV-to-Excel Python CLI

For the second check, ask Pi to construct a small Python CLI that converts a CSV file into an Excel .xlsx file.

Construct a easy Python CLI that converts a CSV file into an Excel .xlsx file, accepts enter and output file paths as arguments, preserves column headers, validates lacking information, and prints clear success or error messages. Earlier than ending, create a small dummy CSV file with pattern knowledge, use it to check the CLI, confirm that the Excel file is created appropriately, after which summarize the check end result.

Pi created a Python script named csv2excel.py, generated a pattern CSV file, ran the check command, after which summarized the outcomes.

python csv2excel.py sample_data.csv output.xlsx

In my run, the abstract confirmed that the script:

• Accepted enter and output file paths.
• Preserved the column headers.
• Accurately transformed the pattern knowledge.
• Dealt with lacking information with clear error messages.
• Dealt with lacking output paths correctly.

I additionally opened the generated Excel file to verify that the output was right.

The pattern output preserved the rows and headers appropriately, which confirmed that the CLI labored as anticipated.

# Remaining Ideas

 
I actually like this small native coding mannequin. It’s quick, correct sufficient for on a regular basis coding duties, and doesn’t want an enormous quantity of VRAM to be helpful. You should use it with Pi, Claude Code, OpenCode, or any coding setup that helps native OpenAI-compatible or Anthropic-compatible endpoints.

For primary front-end apps, Python scripts, CLI instruments, and fast prototypes, it really works surprisingly nicely. You may construct helpful initiatives domestically with out relying on exterior APIs or paying for each request.

To get even higher outcomes, I extremely advocate including net search abilities, Context7, and different helpful Pi integrations. You can too try my full information on optimizing your Pi coding agent setup right here: Tips on how to Set Up Kimi K2.7 Code with Pi: The Final AI Coding Surroundings.
 
 

Abid Ali Awan (@1abidaliawan) is a licensed knowledge scientist skilled who loves constructing machine studying fashions. At present, he’s specializing in content material creation and writing technical blogs on machine studying and knowledge science applied sciences. Abid holds a Grasp’s diploma in know-how administration and a bachelor’s diploma in telecommunication engineering. His imaginative and prescient is to construct an AI product utilizing a graph neural community for college students scuffling with psychological sickness.