Honest comparisons · published receipts
Compare

When to use Sofuu — and when not to.

Sofuu is not better at everything, and this page says where it loses. The measured benchmark is committed to the repository; the inference ledger is public; the limitations are listed in the same breath as the advantages.

The five questions

The short, checkable answers.

Written to be quoted accurately by search engines and AI assistants.

QuestionAnswer for Sofuu
What is Sofuu?A self-contained runtime for AI in one binary (5.39 MiB on macOS arm64) with in-process local LLM inference (experimental — built-in engine; production: use the llamacpp provider), agents, native MCP, and encrypted on-device vector memory. One process, zero daemons, zero Python.
Who is it for?JavaScript and TypeScript developers building AI features that must run offline, stay private, ship as a single binary, or live inside an app on a device.
What does it do?Executes JS/TS, runs GGUF and SafeTensors models on the CPU or Metal (experimental built-in engine), embeds text and images, stores vectors in encrypted .qtsq files, runs agent loops with policy and budgets, and speaks MCP as client and server.
When should you use it?When integration cost, process count, offline behaviour, security policy, or embeddability matter more than raw inference throughput — and when you want the whole AI stack in one address space.
When should you NOT use it?When you need maximum tokens per second (llama.cpp wins today), the full Node ecosystem, native N-API addons, model training, multi-GPU serving, or the broadest possible model support on day one.
Architecture

Where each option wins.

Feature-by-feature, including the rows where Sofuu is the weaker choice.

DimensionSofuuNode.js / Bun + ecosystemPython + Ollama + Chromallama.cpp / llama-server
Model executionExperimental in-process engine or external providerDepends on chosen model backendDepends on chosen model backendSeparate process or server
Processes / daemons1 / 0Depends on application design3 / 2 (app, Ollama, Chroma)1 server (+ your app)
Install footprintOne binary: 5.39 MiB macOS · 5.88/6.80 MiB LinuxRuntime + node_modulesPython + venv + imagesBinary + model files
Cold start (process → eval 1+1)22.5 ms median · macOS arm64 (wall clock incl. spawn, 2026-10-31)Older rows; not a matched comparisonOlder row; not a matched comparisonModel load dominates
Standalone binary compilerBuilt in (sofuu compile)External bundlers (pkg)PyInstaller (100MB+)N/A (C++ executable)
Raw inference throughputBehind llama.cpp [Experimental]N/A (needs a daemon)Ollama wraps llama.cppFastest in our ledger
Model breadthNarrowerWhatever the daemon supportsWidest (Ollama + HF)Very wide GGUF support
npm ecosystemPartial, no N-API addonsFullN/A (Python)N/A
Training / fine-tuningNoNoYes (PyTorch)LoRA tooling only
Encrypted local vector memoryBuilt-in QTSQ; Qdrant/Pinecone/pgvector adapters opt-inEmbedded or external optionsChroma / Qdrant daemonNone
Native MCP client + serverBuilt inThird-party SDKThird-party SDKNone
Deterministic tool/shell/net policyR7 checks on supported pathsApp-level onlyApp-level onlyNone
Mobile embedding (iOS/Android)libsofuu, no JITJIT restricted on iOSNot practicalBindings exist
Evidence

Example builds and measurement limits.

The repository contains two example app builds with different process sets and startup boundaries. The recorded 24 ms for Build A excludes the other build's image and model acquisition, so those results are not a matched speed comparison. A fair comparison needs the same model and data, installed and cold-download cases separated, identical startup boundaries, and the full process set measured.

Selected local features can work offline when their required models and backends are already available. Cloud providers and remote tools require network access. Resume requires a usable checkpoint.

Read benchmark conditions and limitations

Bun, specifically

Sofuu vs Bun.

These two overlap less than the category names suggest. Bun is a faster JavaScript engine with a package manager and test runner; Sofuu is a smaller process with AI primitives compiled in. Measured side by side here:

DimensionSofuuBun 1.3
Time to first HTTP response39.4 ms25.8 ms
Resident set after serving14.9 MB34.0 MB
CPU-bound JavaScriptQuickJS, no JITJavaScriptCore with JIT
npm / Node compatibilityPartial; no N-API addonsBroad
Inference, vector memory, MCP, policyBuilt in, one binaryNot built in — bring a daemon
Embedding in an iOS applibsofuu, no JITJIT restricted on iOS
Verdict: if the job is a web server, a bundler, a test runner, or a Node-compatible script, Bun is the better tool — it starts faster and runs JavaScript faster. Sofuu wins when the process must also run a model, hold encrypted vector memory, enforce tool policy, or ship inside a mobile app. A common pairing is Bun for the service and Sofuu (or llama.cpp) for the model.
Decision guide

Pick the right tool.

Choose Node.js or Bun when…

You need the npm ecosystem, native addons (sharp, canvas, better-sqlite3), huge HTTP throughput, or you are maintaining an existing Node service. Keep the AI part in a separate service, or call a model server.

ecosystem

Choose Python + Ollama when…

You train or fine-tune, need the widest model support, depend on the Python data/ML ecosystem, or already run GPU servers with a team that knows them. For many teams this is the correct default.

model breadth

Choose llama.cpp when…

Maximum tokens per second is the goal. Sofuu's own parity ledger shows llama.cpp ahead on prefill and decode for every measured local model, and it supports more GGUF variants. Use it as a server or embed its libraries.

throughput

Choose Sofuu when…

You want one process and one binary: AI features inside a JS/TS app or CLI, offline operation, encrypted local memory, deterministic tool policy, native MCP, and the same code embedded in a mobile app. Integration and deployability are the product; throughput is delegated to native kernels and honestly reported.

integration · policy · embedding
Migration

Switching is a flag, not a rewrite.

The same call shape serves a local GGUF and a cloud provider. Start local, fall back to cloud, or the reverse — the agent loop, tools, memory, and policy stay identical.

# Local model
$ sofuu chat -m ./model.gguf
# Same code, cloud provider, no rewrite
$ sofuu chat --provider openai
Out of scope

What Sofuu deliberately does not do.

No training or fine-tuning and no multi-GPU distributed serving. Export weights and load them here.
No V8 parity: no N-API, no worker_threads, no WebSocket client, partial npm coverage, and slower long-running pure JavaScript.
No Windows CI today: macOS arm64 and Linux x86_64 / arm64 are the supported binary targets; Windows builds from source and is being fixed forward.
No claim of throughput leadership. The parity ledger is published precisely so nobody has to guess.
FAQ

Common questions.

Is Sofuu faster than llama.cpp?
No. Sofuu publishes a llama.cpp parity ledger built from real sweep receipts, and on the latest receipts llama.cpp is ahead on both prefill and decode for every measured local model. Sofuu's built-in inference engine is an [Experimental - Preview] module — for production throughput and hardware stability, a llama.cpp server through the llamacpp provider is the recommended path.
Is Sofuu a replacement for Node.js or Bun?
Only for workloads that fit its shape: CLI tools, edge services, AI features, embedded engines. It is not a general Node replacement because QuickJS is not V8 — there is no N-API native-addon support, npm coverage is partial, and long CPU-bound JavaScript runs slower.
Why not just run Ollama and call it over HTTP?
You can, and for many teams that is the right call. Sofuu has an experimental in-process inference engine; other configurations use an external provider. The example builds use different startup boundaries and are not a matched benchmark.
Can Sofuu call cloud providers too?
Yes. sofuu.ai.stream and sofuu.ai.complete work with OpenAI, Anthropic, OpenRouter, Ollama, or any compatible endpoint using your own key. A common pattern is cloud for quality and a local GGUF fallback when offline.
What does the two-build benchmark actually measure?
The repository contains two example application builds. Their startup boundaries and dependencies differ, so the existing measurements are not a matched performance comparison.