Saluki 27B is a 2-bit compressed build by Underdog of Alibaba's Qwen3.8-27B that gives local agents a tool-calling model in a 7.89 GB file, beating the full 54 GB original at function calling. Released on October 7, 2026 by Conway Research's Underdog project, an a16z-backed on-device assistant effort, it is licensed under Apache 2.0 and runs on stock llama.cpp — making it a practical agent brain for developers building on-device agents on laptops and edge devices. The compression was tuned specifically to preserve function-calling rather than blanket accuracy.
Core Features
- Aggressive 2-bit GGUF: 7.89 GB versus 54 GB for the BF16 original, about a 7x size cut with full GPU offload.
- Tool-calling focus: scored 88 of 120 on Underdog's BFCL v4 benchmark versus 84 for the full model, and 42 of 100 on parallel calls.
- Stock llama.cpp support: no custom runtime required; an optional 629 MB (Q8) or 928 MB (F16) vision add-on enables image input.
- 262,144-token context inherited from the base Qwen3.8-27B, with thinking mode switchable per request.
- Reported 96 percent average retention across nine benchmarks, at the cost of weaker math and multi-step reasoning.
Use Cases
- Developers building local agents that select and invoke tools on a laptop GPU.
- Privacy-sensitive deployments where a model must run fully on-device.
- Edge and embedded assistants with tight memory budgets.
Pricing
Saluki 27B is free and open-weight under Apache 2.0; download the GGUF from Hugging Face and run it with llama.cpp. Note the 7.89 GB figure is decimal gigabytes — plan for roughly 9.5 GiB at 32K context, so a 12 GB card is the realistic floor.
Our Take
Best for on-device agent builders who care about tool calling more than math. The trade-off is real: AIME math drops about 17 points versus the full model, so do not use it for heavy reasoning. If your pipeline leans on coding or calculation, keep the full Qwen3.8-27B for those tasks. Compare with other AI engine and model releases.
