PrismML-Eng/Bonsai-demo

原文摘要

Bonsai Demo Bonsai Demo Website | GitHub | Discord HF Collections: Bonsai 27B · Bonsai (1-bit) · Ternary-Bonsai Whitepapers: Bonsai 27B · 1-bit Bonsai 8B · Ternary-Bonsai 8B Using this demo repository you can run Bonsai (1-bit) and Ternary-Bonsai language models locally on Mac (Metal), Linux/Windows (CUDA, Vulkan, ROCm), or CPU. 🌱 New: Bonsai 27B The family's newest and largest generation, and its first vision-language models ( Bonsai 27B collection ): Vision: send photos, screenshots, and PDFs; ask about them (see VISION.md ). Agentic tool calling: native OpenAI-style tool_calls with full round-trips, plus MCP servers in both demo UIs (see TOOLS.md ). Thinking: a reasoning model; pick the reasoning effort per chat in the UI or budget it per request. Long context: 256k+ token conversations. Tiny footprint: the 1-bit Bonsai-27B packs to ~1.125 bits per weight: it fits on a modern iPhone without memory offloading. Ternary-Bonsai-27B (~1.7 bits per weight, packed into 2-bit for fast accelerated kernels) is the higher-quality option and this demo's default. Quick Start below gets you there in two commands: ./setup.sh downloads Ternary-Bonsai-27B by default, then ./scripts/start_llama_…