When to use Sofuu — and when not to.
Sofuu is not better at everything, and this page says where it loses. The measured benchmark is committed to the repository; the inference ledger is public; the limitations are listed in the same breath as the advantages.
The short, checkable answers.
Written to be quoted accurately by search engines and AI assistants.
| Question | Answer for Sofuu |
|---|---|
| What is Sofuu? | A self-contained runtime for AI in one binary (5.39 MiB on macOS arm64) with in-process local LLM inference (experimental — built-in engine; production: use the llamacpp provider), agents, native MCP, and encrypted on-device vector memory. One process, zero daemons, zero Python. |
| Who is it for? | JavaScript and TypeScript developers building AI features that must run offline, stay private, ship as a single binary, or live inside an app on a device. |
| What does it do? | Executes JS/TS, runs GGUF and SafeTensors models on the CPU or Metal (experimental built-in engine), embeds text and images, stores vectors in encrypted .qtsq files, runs agent loops with policy and budgets, and speaks MCP as client and server. |
| When should you use it? | When integration cost, process count, offline behaviour, security policy, or embeddability matter more than raw inference throughput — and when you want the whole AI stack in one address space. |
| When should you NOT use it? | When you need maximum tokens per second (llama.cpp wins today), the full Node ecosystem, native N-API addons, model training, multi-GPU serving, or the broadest possible model support on day one. |
Where each option wins.
Feature-by-feature, including the rows where Sofuu is the weaker choice.
| Dimension | Sofuu | Node.js / Bun + ecosystem | Python + Ollama + Chroma | llama.cpp / llama-server |
|---|---|---|---|---|
| Model execution | Experimental in-process engine or external provider | Depends on chosen model backend | Depends on chosen model backend | Separate process or server |
| Processes / daemons | 1 / 0 | Depends on application design | 3 / 2 (app, Ollama, Chroma) | 1 server (+ your app) |
| Install footprint | One binary: 5.39 MiB macOS · 5.88/6.80 MiB Linux | Runtime + node_modules | Python + venv + images | Binary + model files |
| Cold start (process → eval 1+1) | 22.5 ms median · macOS arm64 (wall clock incl. spawn, 2026-10-31) | Older rows; not a matched comparison | Older row; not a matched comparison | Model load dominates |
| Standalone binary compiler | Built in (sofuu compile) | External bundlers (pkg) | PyInstaller (100MB+) | N/A (C++ executable) |
| Raw inference throughput | Behind llama.cpp [Experimental] | N/A (needs a daemon) | Ollama wraps llama.cpp | Fastest in our ledger |
| Model breadth | Narrower | Whatever the daemon supports | Widest (Ollama + HF) | Very wide GGUF support |
| npm ecosystem | Partial, no N-API addons | Full | N/A (Python) | N/A |
| Training / fine-tuning | No | No | Yes (PyTorch) | LoRA tooling only |
| Encrypted local vector memory | Built-in QTSQ; Qdrant/Pinecone/pgvector adapters opt-in | Embedded or external options | Chroma / Qdrant daemon | None |
| Native MCP client + server | Built in | Third-party SDK | Third-party SDK | None |
| Deterministic tool/shell/net policy | R7 checks on supported paths | App-level only | App-level only | None |
| Mobile embedding (iOS/Android) | libsofuu, no JIT | JIT restricted on iOS | Not practical | Bindings exist |
Example builds and measurement limits.
The repository contains two example app builds with different process sets and startup boundaries. The recorded 24 ms for Build A excludes the other build's image and model acquisition, so those results are not a matched speed comparison. A fair comparison needs the same model and data, installed and cold-download cases separated, identical startup boundaries, and the full process set measured.
Selected local features can work offline when their required models and backends are already available. Cloud providers and remote tools require network access. Resume requires a usable checkpoint.
Sofuu vs Bun.
These two overlap less than the category names suggest. Bun is a faster JavaScript engine with a package manager and test runner; Sofuu is a smaller process with AI primitives compiled in. Measured side by side here:
| Dimension | Sofuu | Bun 1.3 |
|---|---|---|
| Time to first HTTP response | 39.4 ms | 25.8 ms |
| Resident set after serving | 14.9 MB | 34.0 MB |
| CPU-bound JavaScript | QuickJS, no JIT | JavaScriptCore with JIT |
| npm / Node compatibility | Partial; no N-API addons | Broad |
| Inference, vector memory, MCP, policy | Built in, one binary | Not built in — bring a daemon |
| Embedding in an iOS app | libsofuu, no JIT | JIT restricted on iOS |
Pick the right tool.
Choose Node.js or Bun when…
You need the npm ecosystem, native addons (sharp, canvas, better-sqlite3), huge HTTP throughput, or you are maintaining an existing Node service. Keep the AI part in a separate service, or call a model server.
Choose Python + Ollama when…
You train or fine-tune, need the widest model support, depend on the Python data/ML ecosystem, or already run GPU servers with a team that knows them. For many teams this is the correct default.
Choose llama.cpp when…
Maximum tokens per second is the goal. Sofuu's own parity ledger shows llama.cpp ahead on prefill and decode for every measured local model, and it supports more GGUF variants. Use it as a server or embed its libraries.
Choose Sofuu when…
You want one process and one binary: AI features inside a JS/TS app or CLI, offline operation, encrypted local memory, deterministic tool policy, native MCP, and the same code embedded in a mobile app. Integration and deployability are the product; throughput is delegated to native kernels and honestly reported.
Switching is a flag, not a rewrite.
The same call shape serves a local GGUF and a cloud provider. Start local, fall back to cloud, or the reverse — the agent loop, tools, memory, and policy stay identical.
# Local model$ sofuu chat -m ./model.gguf# Same code, cloud provider, no rewrite$ sofuu chat --provider openai
What Sofuu deliberately does not do.
worker_threads, no WebSocket client, partial npm coverage, and slower long-running pure JavaScript.