Will It Run on Your Computer? Test Your Hardware Fit
Pick a device preset below—or slide VRAM and RAM to match your computer—to see how Cipanion's 3-Tier Memory Planner automatically balances model weights across GPU VRAM, System RAM, and NVMe SSD without crashing your system.
Hot layers and KV-cache stay resident in VRAM while remaining layers sit in host RAM with zero disk streaming.
A Native Desktop AI Studio Built for Everyone—Not Just Developers
Double-click Cipanion Studio (cipanion-studio.exe) for a clean, standalone
desktop window on Windows, macOS, or Linux—with a 1-Click Model Hub (19 curated models),
PDF & Vision (mmproj) attachments, and optional TapirusDB adaptive
long-term memory running 100% locally on your machine. Click the tabs below to tour the interface:
Multimodal text + image understanding (140+ languages including BM & EN). Downloads model +
companion mmproj.gguf automatically in 1 click.
Fast everyday reasoning, coding, and bilingual chat with collapsible <think> trace
support on 4–8 GB laptops.
Lightweight edge vision model for OCR, screenshots, and receipts on everyday laptops without a dedicated GPU.
Powers local semantic search & RAG vectors via POST /v1/embeddings for private
document indexing.
/v1/chat/completions,
/v1/embeddings) stay 100% stateless—just like Ollama and vLLM.
Launches directly as a clean standalone window (no browser clutter or terminal needed). Automatically scores 19 curated models—including Qwen3, Gemma 3 Vision, and SmolVLM2—against your exact RAM & GPU.
Drop PDFs, CSVs, source code, or images directly into chat. Pure-Rust mmproj.gguf vision
encoding processes images locally without Python or external image libraries.
Studio learns your preferred language, tone, and project facts inside a single encrypted-ready
.tapir file using Hybrid BM25 + Vector recall—100% toggleable and never sent to the cloud.
One Engine, Tailored for Every Machine
Whether you are empowering students and office workers on budget laptops or deploying high-density inference nodes in a data center, Cipanion adapts to your hardware:
Designed for ordinary computers without dedicated GPUs. Compresses context memory by 75% (Q8_0
KV-cache), streams layers through a tiny 2-slot SSD ring buffer, reserves 30% RAM so your OS never lags, and
installs with a 1-click Windows installer.
One portable ~2.2 MiB executable that automatically detects your host hardware and accelerates inference
using AVX2+FMA, ARM64 NEON, and dynamic NVIDIA CUDA PTX 7.0 (including
F16/F32 SRAM FlashAttention), plus live Vulkan GPU device discovery
for AMD, Intel, NVIDIA, and Apple hardware.
Built to reduce data-center TCO. Features Resident VRAM Weight Caching (eliminating repeated PCIe
transfers), parallel NVMe I/O workers, hardened systemd service deployment, live Prometheus
/metrics, and a ~5 MiB FROM scratch container image.
Plug directly into Claude Code (POST /v1/messages),
OpenCode, VS Code (Continue / Cline / Roo), Open WebUI,
and n8n via /v1/chat/completions (with native tool_calls &
guaranteed JSON), /api/chat, and optional --api-key Bearer security.
How Cipanion Solves the Memory Bottleneck Under the Hood
Traditional engines try to load entire AI models into RAM or VRAM at once. Cipanion is engineered from scratch as six modular, zero-dependency Rust subsystems that treat your SSD, RAM, and GPU as one coordinated pipeline:
| Subsystem | What Problem It Solves | Key Capabilities |
|---|---|---|
Instant GGUF Inspector (cipanion-core) |
Opens multi-gigabyte model files in milliseconds without loading weights into RAM first. | Zero-weight GGUF v2/v3 parser, Multi-Shard Split GGUF (-00001-of-0000N),
22 quantization formats (Q4_0–Q8_K,
IQ1–IQ4, MXFP4, BF16), per-layer SSD offset map.
|
3-Tier Memory Planner (cipanion-scheduler) |
Prevents out-of-memory crashes and OS freeze-ups on machines with limited RAM or VRAM. | Auto hardware profiler, Full-VRAM / Hybrid-RAM / 3-Tier-NVMe /
MoE-Smart-LRU strategies, double-buffered ring planner.
|
Async NVMe Streamer (cipanion-io-streamer) |
Hides SSD read latency behind active computation so large models stream smoothly. | 64-byte aligned buffer pool, coalesced background prefetching, selective 3D MoE expert slice reads, OS page-cache eviction. |
Zero-Setup GPU & SIMD Compute (cipanion-compute) |
Delivers GPU and multi-core CPU speed without forcing users to install toolkits or drivers at build time. | Dynamic CUDA Driver PTX 7.0 kernels
(F32/Q8_0/Q4_0/Q4_K/Q6_K + SRAM
FlashAttention) + Resident
VRAM Cache, L1-tiled CPU Online-Softmax FlashAttention, Vulkan probe, AVX2 vpmaddubsw int8,
ARM64 NEON. |
Runtime, Hub & Studio Server (cipanion-runtime) |
Provides instant multi-turn chat, structured JSON output, and 1-click model management in one package. | BPE/SentencePiece tokenizer, Lossless Speculative Decoding (single-pass batched NVMe
verification), Prefix KV-Cache reuse,
JsonConstraint, Model Hub, OpenAI + Anthropic + Ollama APIs & Prometheus
/metrics.
|
Unified Executable (cipanion-cli) |
Replaces complex Python/Docker stacks with a single ~2.2 MiB file you can run anywhere. | cipanion ui, serve, chat, run, hw,
plan, inspect, bench-compute, bench-model.
|
Verified Performance on Real Consumer Hardware
Every metric below is measured directly on a standard consumer laptop (Windows x86_64, 12 logical
CPU cores with AVX2+FMA, 15.29 GiB RAM, NVIDIA GeForce RTX 3050 6GB Laptop
GPU)—proving you don't need an expensive server to get fast local AI:
| Real-World Workload | Measured Result | Why It Matters to You |
|---|---|---|
CUDA PTX 7.0 Q4_K GPU Kernel (2048×2048) |
72.33 GFLOP/s · 0.12 ms/op (18.95 GiB/s eff. BW) |
Accelerates standard 4-bit GGUF models directly on NVIDIA GPUs with zero CUDA SDK setup. |
Resident VRAM Weight Cache (Q8_0 GPU) |
59.71 GFLOP/s · 0.14 ms/op (121/126 VRAM hits) |
Keeps hot layers pinned in GPU memory so subsequent tokens skip PCIe transfer overhead. |
Batched Prompt Prefill (Q8_0×16 GEMM) |
84.71 GFLOP/s · 1.58 ms/batch (0.10 ms/tok) |
Processes long prompts and documents rapidly across all CPU cores in parallel. |
CPU-Only SIMD (Q6_K / Q4_0 / Q8xQ8i) |
Q6_K: 71.78 · Q4_0: 59.88 ·
Q8xQ8i: 56.43 GFLOP/s
|
Ensures laptops without a dedicated GPU still achieve high speed via AVX2/NEON instructions. |
Real HuggingFace GGUF Chat (SmolLM2-135M-Q4_K_M) |
Generation: 30.49 tok/s · Prefill: 46.3 tok/s · TTFT:
281 ms
|
Instant, fluent real-time streaming response faster than human reading speed. |
| Multi-Turn Prefix KV-Cache Reuse (Turn 2) | 32 / 34 prompt tok reused in 56.80 ms · 0.00 MiB disk read
|
Follow-up questions in a chat reuse previous context immediately without re-reading the SSD. |
How Cipanion Compares Across the AI Engine Landscape
| What You Care About | Cipanion | llama.cpp / Ollama | vLLM | Candle (HuggingFace) |
|---|---|---|---|---|
| Installation & Footprint | ~2.2 MiB single file · 0 external dependencies | C/C++ build + Go wrapper (~50–500 MB) | Python + PyTorch + CUDA stack (5–15 GB+ image) | Rust library (~80–150 external crates) |
| Low-RAM Laptop Survival | Built-in 3-Tier VRAM/RAM/NVMe streamer + Eco mode |
OS mmap paging + manual layer offload |
Requires high-VRAM data-center GPUs | In-memory loading (OOM if model exceeds RAM) |
| Zero-CLI Experience | Built-in 61.9 KiB Studio UI + 1-Click Model Hub + Hot-Swap | llama-server UI / CLI + third-party web UIs | Headless API server only | Code examples only (no built-in UI) |
| IDE & Agent Protocols | OpenAI + Tool Calling + Anthropic /v1/messages + Ollama /api/* + API
Key Auth |
OpenAI + Anthropic + Ollama native APIs | OpenAI + Anthropic + Tool Calling | Manual HTTP wrapper required |
| Data-Center Efficiency | ~5 MiB FROM scratch container + Prometheus /metrics |
Standard container images + OpenAI API | High-throughput multi-GPU tensor parallelism | Custom microservice integration required |
Download & Start Running Private AI in Under a Minute
Download the ready-to-run Windows portable package below (unzip and double-click
cipanion-studio.exe for the native desktop app), or build from source on Linux and macOS:
Includes cipanion-studio.exe (native desktop app with Multimodal PDF/Vision & embedded
TapirusDB Memory), cipanion.exe (zero-dependency CLI & OpenAI API server), and
Start-Cipanion-Eco-LowRAM.cmd (for 4–8 GB RAM laptops).
SHA-256: A73C3894E6C65C8F4599CE080B37847B7882403D766CC1356B58EC7C8F48B986
1. Unzip & Double-Click: Extract the .zip and double-click
cipanion-studio.exe (or Start-Cipanion-Eco-LowRAM.cmd on 4–8 GB laptops).
2. Pick a Model in 1 Click: Click Open Model Hub inside the desktop window
to download Qwen3 1.7B, Gemma 3 4B Vision, or SmolVLM2—or click
📂 Browse to load any local .gguf.
3. Chat 100% Offline: Attach PDFs or images, enable TapirusDB Memory, switch EN/BM, or
toggle Eco/Balanced/HPC profiles live.
# Build or run the standalone ~2.2 MiB binary
git clone https://github.com/tapiruslab/cipanion.git
cd cipanion
cargo build --release -p cipanion-cli
# Open Cipanion Studio & 1-Click Model Hub directly in your browser
./target/release/cipanion ui
# Running on a 4–8 GB RAM laptop? Start in Eco Low-RAM mode:
./target/release/cipanion ui --profile eco
# Auto-detect CPU SIMD, available RAM, NVIDIA CUDA PTX, and Vulkan GPUs
cipanion hw
# See how Cipanion will split a model across VRAM, RAM, and NVMe SSD
cipanion plan /path/to/model.gguf --vram-gb 6 --ram-gb 16 --ctx 4096
# Run built-in CPU/GPU speed benchmarks on your machine
cipanion bench-compute --rows 2048 --cols 2048 --iters 20
# 1) Claude Code (native Anthropic Messages API on /v1/messages)
ANTHROPIC_BASE_URL=http://127.0.0.1:11434 ANTHROPIC_API_KEY=cipanion claude
# 2) OpenAI / OpenCode / VS Code (Continue, Cline, Roo) with Tool Calling & JSON Mode
curl http://127.0.0.1:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "cipanion",
"messages": [{"role": "user", "content": "Summarize 3 benefits of local AI as JSON."}],
"response_format": {"type": "json_object"},
"temperature": 0.2,
"stream": true
}'
# 3) Ollama Native Clients (/api/tags, /api/chat, /api/generate) + Secure Team Gateway
cipanion serve --host 0.0.0.0 --port 11434 --api-key YOUR_SECRET_KEY
# 4) Frontier 70B+ Multi-Shard GGUF + Lossless Speculative Decoding (3x–5x NVMe Speedup!)
cipanion run Llama-3.3-70B-Q4_K_M-00001-of-00002.gguf \
--draft-model Llama-3.2-1B-Instruct-Q4_K_M.gguf --draft-tokens 5 \
--prompt "Explain FlashAttention online softmax concisely."
The Complete Tapirus Lab Local-to-Cloud AI Stack
Combine Cipanion with TapirusDB and TapisaurusDB to build private AI applications with long-term memory and enterprise scale—without external cloud dependencies:
Run GGUF language models locally on laptops or in data centers with 3-tier NVMe streaming, dynamic GPU acceleration, and a built-in 1-Click Studio UI.
Give your local AI long-term memory in one encrypted .tapir file unifying SQL, JSON Documents,
HNSW Vectors, and Knowledge Graphs in <4 MB RAM.
Scale across clusters speaking PostgreSQL, MySQL, MongoDB, gRPC, and REST natively over Raft consensus, SSD DiskANN vectors, and HTAP analytics.