antirez/ds4

原文摘要

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm DwarfStar is a small native inference engine optimized first for DeepSeek V4 Flash . It also supports GLM 5.2 and, on very high-memory machines, DeepSeek V4 PRO . It is self-contained and deliberately narrow, not a general GGUF runner. Model loading, prompt rendering, tool calls, KV state, the HTTP server, and the coding agent are built and tested together. The repository also includes tools and data for GGUF, imatrix, quality, and speed. Supported backends: Metal , the primary target, on Macs with 96 GB or more. Smaller machines can use SSD streaming. NVIDIA CUDA , including multi-GPU systems and DGX Spark. ROCm on Strix Halo systems such as the Framework Desktop. This project would not exist without llama.cpp and GGML , make sure to read the acknowledgements section, a big thank you to Georgi Gerganov and all the other contributors. Model support is intentionally opportunistic. The project follows the best open weights for useful local machine sizes, especially 128 GB laptops and 512 GB workstations. A model may be removed when a better replacement arrives. So, what can I do with this software? You can run a…