BitRun wraps BitNet in a single binary so 1-bit models run on a CPU without a compiler
An alpha-stage runner for ternary LLMs: a Go supervisor around a static C++ engine, with pull, run, and serve commands and an OpenAI-compatible endpoint.
Models with 1-bit and 1.58-bit weights are small enough to run on a plain CPU, but getting them running has usually meant cloning submodules and arguing with a compiler. BitRun, by IshekKhal, is an attempt to make that part boring: one binary that pulls a ternary model, runs it, and serves it.
It is very early. The README labels it alpha, the version badge reads v0.1.0-alpha, and the repo had nine commits and no stars when we looked. We are covering it for how it is put together, not for its track record.
How it works
BitRun is two layers.
The outer layer is a supervisor written in Go. According to the README it handles the command line, detects the CPU’s layout and which vector instructions it supports, keeps a content-addressed store of models under ~/.bitrun, and runs an HTTP server.
The inner layer is a statically compiled C++ engine with hand-tuned kernels for ARM NEON and x86 AVX-512. The README says there are no CGO dependencies and claims crash isolation between the two layers, so a fault in the engine should not take the supervisor with it.
The point of that split is distribution. The engine arrives already compiled, and the supervisor works out what the chip supports. The user never chooses SIMD flags.
The commands will look familiar if you have used Ollama: pull, run, serve, and a Modelfile for custom personas with FROM, SYSTEM, and PARAMETER lines. The server exposes an OpenAI-compatible /v1 endpoint on port 11435, which the README says works with Cursor, Open WebUI, and LangChain.
The catalog lists five models: a 70M BitNet, the 2.4B BitNet, Falcon3 at 1B and 3B, and an 8B Llama 3, mostly in the i2_s ternary format.
The author also cites two pull requests to Microsoft’s BitNet repo, one fixing ARM64 and tokenizer issues and one adding native MSVC support. That suggests the Windows build took real work.
The numbers
All of these are the author’s own measurements from the README.
| Machine | Model | Generation speed |
|---|---|---|
| Apple M4 Pro, 12 cores | falcon3-1b | 98.6 tokens/s |
| Apple M4 Pro, 12 cores | bitnet-2b | 39.8 tokens/s |
| Apple M4 Pro, 12 cores | bitnet-70m | 1,355 tokens/s |
| Intel i5-12600K, Windows, AVX2 | falcon3-1b | 40 to 45 tokens/s |
| Intel i5-12600K, Windows, AVX2 | bitnet-70m | 492 tokens/s |
The README reports roughly 1.9 GB of RAM for the 1B and 2B models and no GPU use in any run.
The headline figure of 98 tokens per second is the 1B model on a current Apple laptop chip. The 2.4B model on the same machine is less than half that, and the Intel desktop is slower again. There is no number for the 8B model on any machine. The README’s two tables also disagree about how big falcon3-1b is on disk.
Try it yourself
The repo is MIT licensed. The README offers install scripts for Apple Silicon macOS, x86_64 Linux, and Windows, and says no compiler or toolchain is needed. Those scripts are the pipe-to-shell kind, so read them first, especially for a project this new.
We have not run BitRun ourselves; this is based on the README.