BaseRT is an open-source large language model inference runtime built from the ground up for Apple Silicon. Developed by Base Compute, it promises the fastest local LLM performance on M1 and later Macs by using hand-written Metal kernels and a custom model format called .base. With a single CLI, you can pull models from Hugging Face, chat with them, serve an OpenAI-compatible API, or convert existing GGUF, MLX, and Hugging Face checkpoints into the optimized .base format. Because everything runs locally, BaseRT is a strong choice for privacy-conscious developers who want to build AI-powered applications without sending sensitive data to cloud providers.
Core Features
- Metal-accelerated inference: Hand-written Metal kernels extract maximum performance from Apple Silicon, outpacing generic runtimes on compatible Macs.
- Unified CLI: One command handles pull, chat, serve, convert, bench, and inspect operations.
- OpenAI-compatible server: Supports chat, completions, embeddings, transcription, tool calls, continuous batching, paged-KV, and prefix caching.
- Multimodal support: Runs text, vision, and audio models where the underlying architecture supports them.
- Language bindings: Stable C API plus Python, Node, Rust, and Swift bindings for building native apps.
- .base format: Custom affine quantization from Q2 to Q8, optional AWQ calibration, and signed bundles.
Use Cases
- Private local chatbot and assistant on a Mac
- Self-hosted OpenAI-compatible API for internal tools
- Coding agent backend for pi and other agent frameworks
- On-device AI features in macOS or iOS applications
- Benchmarking and optimizing LLM throughput on Apple Silicon
Pricing
The BaseRT CLI, .base format specification, public headers, and language bindings are open-source under the Apache-2.0 license and free to use. The prebuilt engine binary is distributed separately under its own license, and the project provides a convenient install script at no cost.
How to Access
Visit the BaseRT repository at https://github.com/basecompute/baseRT or install directly with the command at https://basecompute.co/install.sh. Documentation is available at https://docs.basecompute.co.
