TapirusDB

Tapirus Tech Lab · TapirusDB · TapisaurusDB · Cipanion

Cipanion · Private Local & Data-Center AI Engine · Zero Python or CUDA Setup

Run powerful AI on any computer. Cut GPU memory costs at scale.

Most AI tools demand expensive GPUs or gigabytes of Python and driver setup. Cipanion lets you run modern GGUF language models privately on an everyday 4–8 GB RAM laptop by streaming weights intelligently across VRAM, System RAM, and NVMe SSD—while giving workstations and data centers native GPU acceleration, an instant 1-Click Studio UI, and a single 2.2 MiB engine that slashes infrastructure costs.

4 GB+ RAM Runs smoothly on everyday laptops—even without a dedicated graphics card
~2.2 MiB Complete standalone engine with built-in Web UI, Universal API Gateway & zero external dependencies
Up to 75% Lower KV-cache memory footprint in Eco mode + zero idle container bloat
100% Private Runs completely offline on your machine with drop-in OpenAI API compatibility
For Everyday PCs & Laptops No Expensive GPU? No Problem.

Stop hitting out-of-memory crashes. Cipanion keeps active layers in fast memory and streams remaining model blocks smoothly from your SSD in the background—so ordinary laptops stay fast and responsive.

For Non-Technical Users & Teams Click, Download & Chat Instantly

Forget terminal commands, Python virtual environments, or manual model hunting. Open Cipanion Studio in your browser, see which AI models fit your PC automatically, and download or switch models in one click.

For Workstations & Data Centers Maximum GPU Speed, Minimal Cloud Bill

Replace 10 GB+ Python container stacks with a ~5 MiB container or native binary. Unlock NVIDIA CUDA PTX 7.0 GPU acceleration, Vulkan GPU auto-detection, Resident VRAM Weight Caching, structured JSON output, and Prometheus monitoring out of the box.

Will It Run on Your Computer? Test Your Hardware Fit

Pick a device preset below—or slide VRAM and RAM to match your computer—to see how Cipanion's 3-Tier Memory Planner automatically balances model weights across GPU VRAM, System RAM, and NVMe SSD without crashing your system.

Interactive Hardware Fit & 3-Tier Memory Simulator
AUTOMATIC EXECUTION PLAN Hybrid-VRAM+RAM
VRAM: 22L RAM: 10L NVMe: 0L
GPU VRAM Used4.1 GiB
System RAM Used1.4 GiB
Context Memory (KV)512 MiB
SSD Stream / Token0.00 GiB

Hot layers and KV-cache stay resident in VRAM while remaining layers sit in host RAM with zero disk streaming.

A Native Desktop AI Studio Built for Everyone—Not Just Developers

Double-click Cipanion Studio (cipanion-studio.exe) for a clean, standalone desktop window on Windows, macOS, or Linux—with a 1-Click Model Hub (19 curated models), PDF & Vision (mmproj) attachments, and optional TapirusDB adaptive long-term memory running 100% locally on your machine. Click the tabs below to tour the interface:

C Cipanion Studio v0.1.0 · Native Desktop App · 100% Offline
+ New Chat
📦 Model Hub & Switcher
🧠 Memory (TapirusDB)6
Recent Chats
Quarterly Report + Diagram
Rust Parser Benchmark
Office Policy Summary
● Gemma 3 4BEN · BM
● Gemma 3 4B Vision Full-VRAM 👁 Vision mmproj
Eco Balanced HPC
📄 Q3-Operations.pdf (412 KiB) 🖼️ architecture-diagram.png
Summarize the bottleneck shown in this diagram and cross-check it against Section 3 of the PDF.
💭 Reasoning (collapsed) · 0.6s · Cross-referenced 3 PDF sections + 256 vision tokens
Both the diagram and Section 3 confirm that PCIe weight reloads occur on layers 18–31 whenever context exceeds 4K tokens. Enabling Balanced (F16 KV-cache) with Resident VRAM Weight Caching eliminates repeat transfers and keeps generation local with zero cloud upload. 30.49 tok/s • TTFT 281 ms • Prefix KV Reused • TapirusDB Memory Active • 100% Offline
📎 Attach PDF, CSV, code, or paste image (Ctrl+V) · Ask anything locally... Send ↵
1-Click Model Hub & Smart Hardware Fit Advisor · 19 Verified GGUF Models
All (19) 👁 Vision (3) Qwen3 / Gemma 3 / R1 📂 Browse .gguf
Gemma 3 4B Vision (+ mmproj)Full-VRAM

Multimodal text + image understanding (140+ languages including BM & EN). Downloads model + companion mmproj.gguf automatically in 1 click.

2.49 GiB + 851 MiB mmprojRec: Balanced
Qwen3 1.7B Instruct (Thinking)Full-VRAM

Fast everyday reasoning, coding, and bilingual chat with collapsible <think> trace support on 4–8 GB laptops.

1.11 GiB · Q4_K_MRec: Eco / Balanced
SmolVLM2 500M Vision (+ mmproj)Ultra-Lite Vision

Lightweight edge vision model for OCR, screenshots, and receipts on everyday laptops without a dedicated GPU.

436 MiB + 199 MiB mmprojRec: Eco
EmbeddingGemma 300MEmbeddings · 100+ Langs

Powers local semantic search & RAG vectors via POST /v1/embeddings for private document indexing.

318 MiB · Q8_0Rec: Eco
🧠 Studio Long-Term Memory · Embedded TapirusDB v1.2.1 (AI Brain)
● Active ✨ Auto-Learn Style: ON ~/.cipanion/studio_memory.tapir
AUTO-LEARNED COMMUNICATION PROFILEpattern
Prefers concise bullet-point answers in Bahasa Melayu or technical English; frequently asks about Rust systems architecture, SQL databases, and local LLM optimization.
HYBRID BM25 + HNSW VECTOR RECALLauto-fact · 0.94
Remembers explicitly stated facts ("Remember that...") and implicit project preferences locally—injecting only top-K relevant context into Studio chats.
Strict API Isolation: Memory is strictly a local Studio Desktop feature. Developer endpoints (/v1/chat/completions, /v1/embeddings) stay 100% stateless—just like Ollama and vLLM.
📊 Live Engine & 3-Tier Hardware Telemetry
NVIDIA CUDA PTX 7.0 + AVX2/FMA Active
Execution StrategyHybrid VRAM + RAM / 3-Tier NVMe
GPU AcceleratorNVIDIA CUDA PTX 7.0 + FlashAttention
Resident VRAM Cache121 / 126 hits (Zero PCIe churn)
CPU SIMD EngineAVX2 + FMA + Online-Softmax Tiles
70B+ Accelerator⚡ Speculative Draft (3x–5x NVMe)
GGUF FormatSingle & Multi-Shard Split GGUF
KV-Cache ModeEco Q8_0 (75% saved) + Prefix Reuse
ObservabilityLive Prometheus /metrics endpoint
Visual UI Showcase: Cipanion Studio runs 100% locally on your CPU/GPU—nothing is uploaded to the web. ⬇ Download Studio (4.13 MiB .zip)
Native Desktop App + 1-Click Hub

Launches directly as a clean standalone window (no browser clutter or terminal needed). Automatically scores 19 curated models—including Qwen3, Gemma 3 Vision, and SmolVLM2—against your exact RAM & GPU.

Multimodal PDF, Code & Vision

Drop PDFs, CSVs, source code, or images directly into chat. Pure-Rust mmproj.gguf vision encoding processes images locally without Python or external image libraries.

Private TapirusDB Adaptive Memory

Studio learns your preferred language, tone, and project facts inside a single encrypted-ready .tapir file using Hybrid BM25 + Vector recall—100% toggleable and never sent to the cloud.

One Engine, Tailored for Every Machine

Whether you are empowering students and office workers on budget laptops or deploying high-density inference nodes in a data center, Cipanion adapts to your hardware:

Everyday Laptops · 4–8 GB RAM 1. Cipanion Lite / Eco Mode

Designed for ordinary computers without dedicated GPUs. Compresses context memory by 75% (Q8_0 KV-cache), streams layers through a tiny 2-slot SSD ring buffer, reserves 30% RAM so your OS never lags, and installs with a 1-click Windows installer.

Windows · macOS · Linux (x64 & ARM64) 2. Cipanion Universal Cross-Platform

One portable ~2.2 MiB executable that automatically detects your host hardware and accelerates inference using AVX2+FMA, ARM64 NEON, and dynamic NVIDIA CUDA PTX 7.0 (including F16/F32 SRAM FlashAttention), plus live Vulkan GPU device discovery for AMD, Intel, NVIDIA, and Apple hardware.

Workstations, Servers & Supercomputers 3. Cipanion Enterprise / HPC Mode

Built to reduce data-center TCO. Features Resident VRAM Weight Caching (eliminating repeated PCIe transfers), parallel NVMe I/O workers, hardened systemd service deployment, live Prometheus /metrics, and a ~5 MiB FROM scratch container image.

Universal AI Gateway · 3 Protocols 4. OpenAI, Anthropic & Ollama APIs + Tool Calling

Plug directly into Claude Code (POST /v1/messages), OpenCode, VS Code (Continue / Cline / Roo), Open WebUI, and n8n via /v1/chat/completions (with native tool_calls & guaranteed JSON), /api/chat, and optional --api-key Bearer security.

How Cipanion Solves the Memory Bottleneck Under the Hood

Traditional engines try to load entire AI models into RAM or VRAM at once. Cipanion is engineered from scratch as six modular, zero-dependency Rust subsystems that treat your SSD, RAM, and GPU as one coordinated pipeline:

Subsystem What Problem It Solves Key Capabilities
Instant GGUF Inspector (cipanion-core) Opens multi-gigabyte model files in milliseconds without loading weights into RAM first. Zero-weight GGUF v2/v3 parser, Multi-Shard Split GGUF (-00001-of-0000N), 22 quantization formats (Q4_0–Q8_K, IQ1–IQ4, MXFP4, BF16), per-layer SSD offset map.
3-Tier Memory Planner (cipanion-scheduler) Prevents out-of-memory crashes and OS freeze-ups on machines with limited RAM or VRAM. Auto hardware profiler, Full-VRAM / Hybrid-RAM / 3-Tier-NVMe / MoE-Smart-LRU strategies, double-buffered ring planner.
Async NVMe Streamer (cipanion-io-streamer) Hides SSD read latency behind active computation so large models stream smoothly. 64-byte aligned buffer pool, coalesced background prefetching, selective 3D MoE expert slice reads, OS page-cache eviction.
Zero-Setup GPU & SIMD Compute (cipanion-compute) Delivers GPU and multi-core CPU speed without forcing users to install toolkits or drivers at build time. Dynamic CUDA Driver PTX 7.0 kernels (F32/Q8_0/Q4_0/Q4_K/Q6_K + SRAM FlashAttention) + Resident VRAM Cache, L1-tiled CPU Online-Softmax FlashAttention, Vulkan probe, AVX2 vpmaddubsw int8, ARM64 NEON.
Runtime, Hub & Studio Server (cipanion-runtime) Provides instant multi-turn chat, structured JSON output, and 1-click model management in one package. BPE/SentencePiece tokenizer, Lossless Speculative Decoding (single-pass batched NVMe verification), Prefix KV-Cache reuse, JsonConstraint, Model Hub, OpenAI + Anthropic + Ollama APIs & Prometheus /metrics.
Unified Executable (cipanion-cli) Replaces complex Python/Docker stacks with a single ~2.2 MiB file you can run anywhere. cipanion ui, serve, chat, run, hw, plan, inspect, bench-compute, bench-model.

Verified Performance on Real Consumer Hardware

Every metric below is measured directly on a standard consumer laptop (Windows x86_64, 12 logical CPU cores with AVX2+FMA, 15.29 GiB RAM, NVIDIA GeForce RTX 3050 6GB Laptop GPU)—proving you don't need an expensive server to get fast local AI:

Real-World Workload Measured Result Why It Matters to You
CUDA PTX 7.0 Q4_K GPU Kernel (2048×2048) 72.33 GFLOP/s · 0.12 ms/op (18.95 GiB/s eff. BW) Accelerates standard 4-bit GGUF models directly on NVIDIA GPUs with zero CUDA SDK setup.
Resident VRAM Weight Cache (Q8_0 GPU) 59.71 GFLOP/s · 0.14 ms/op (121/126 VRAM hits) Keeps hot layers pinned in GPU memory so subsequent tokens skip PCIe transfer overhead.
Batched Prompt Prefill (Q8_0×16 GEMM) 84.71 GFLOP/s · 1.58 ms/batch (0.10 ms/tok) Processes long prompts and documents rapidly across all CPU cores in parallel.
CPU-Only SIMD (Q6_K / Q4_0 / Q8xQ8i) Q6_K: 71.78 · Q4_0: 59.88 · Q8xQ8i: 56.43 GFLOP/s Ensures laptops without a dedicated GPU still achieve high speed via AVX2/NEON instructions.
Real HuggingFace GGUF Chat (SmolLM2-135M-Q4_K_M) Generation: 30.49 tok/s · Prefill: 46.3 tok/s · TTFT: 281 ms Instant, fluent real-time streaming response faster than human reading speed.
Multi-Turn Prefix KV-Cache Reuse (Turn 2) 32 / 34 prompt tok reused in 56.80 ms · 0.00 MiB disk read Follow-up questions in a chat reuse previous context immediately without re-reading the SSD.

How Cipanion Compares Across the AI Engine Landscape

What You Care About Cipanion llama.cpp / Ollama vLLM Candle (HuggingFace)
Installation & Footprint ~2.2 MiB single file · 0 external dependencies C/C++ build + Go wrapper (~50–500 MB) Python + PyTorch + CUDA stack (5–15 GB+ image) Rust library (~80–150 external crates)
Low-RAM Laptop Survival Built-in 3-Tier VRAM/RAM/NVMe streamer + Eco mode OS mmap paging + manual layer offload Requires high-VRAM data-center GPUs In-memory loading (OOM if model exceeds RAM)
Zero-CLI Experience Built-in 61.9 KiB Studio UI + 1-Click Model Hub + Hot-Swap llama-server UI / CLI + third-party web UIs Headless API server only Code examples only (no built-in UI)
IDE & Agent Protocols OpenAI + Tool Calling + Anthropic /v1/messages + Ollama /api/* + API Key Auth OpenAI + Anthropic + Ollama native APIs OpenAI + Anthropic + Tool Calling Manual HTTP wrapper required
Data-Center Efficiency ~5 MiB FROM scratch container + Prometheus /metrics Standard container images + OpenAI API High-throughput multi-GPU tensor parallelism Custom microservice integration required

Download & Start Running Private AI in Under a Minute

Download the ready-to-run Windows portable package below (unzip and double-click cipanion-studio.exe for the native desktop app), or build from source on Linux and macOS:

1-Click Windows x64 Portable Package · 4.13 MiB .zip Download Cipanion Studio v0.1.0 (Windows)

Includes cipanion-studio.exe (native desktop app with Multimodal PDF/Vision & embedded TapirusDB Memory), cipanion.exe (zero-dependency CLI & OpenAI API server), and Start-Cipanion-Eco-LowRAM.cmd (for 4–8 GB RAM laptops).

⬇ Download Windows .zip (4.13 MiB) ✨ Tour Desktop UI

SHA-256: A73C3894E6C65C8F4599CE080B37847B7882403D766CC1356B58EC7C8F48B986

How to Test in 3 Simple Steps Zero Terminal Commands Required

1. Unzip & Double-Click: Extract the .zip and double-click cipanion-studio.exe (or Start-Cipanion-Eco-LowRAM.cmd on 4–8 GB laptops).
2. Pick a Model in 1 Click: Click Open Model Hub inside the desktop window to download Qwen3 1.7B, Gemma 3 4B Vision, or SmolVLM2—or click 📂 Browse to load any local .gguf.
3. Chat 100% Offline: Attach PDFs or images, enable TapirusDB Memory, switch EN/BM, or toggle Eco/Balanced/HPC profiles live.

1. Launch Cipanion Studio Web UI (Zero Terminal Arguments Needed)
# Build or run the standalone ~2.2 MiB binary
git clone https://github.com/tapiruslab/cipanion.git
cd cipanion
cargo build --release -p cipanion-cli

# Open Cipanion Studio & 1-Click Model Hub directly in your browser
./target/release/cipanion ui

# Running on a 4–8 GB RAM laptop? Start in Eco Low-RAM mode:
./target/release/cipanion ui --profile eco
2. Check Your Hardware & Preview Any Model's Memory Plan
# Auto-detect CPU SIMD, available RAM, NVIDIA CUDA PTX, and Vulkan GPUs
cipanion hw

# See how Cipanion will split a model across VRAM, RAM, and NVMe SSD
cipanion plan /path/to/model.gguf --vram-gb 6 --ram-gb 16 --ctx 4096

# Run built-in CPU/GPU speed benchmarks on your machine
cipanion bench-compute --rows 2048 --cols 2048 --iters 20
3. Connect Claude Code, OpenCode, VS Code, Ollama Clients & Secure Team APIs
# 1) Claude Code (native Anthropic Messages API on /v1/messages)
ANTHROPIC_BASE_URL=http://127.0.0.1:11434 ANTHROPIC_API_KEY=cipanion claude

# 2) OpenAI / OpenCode / VS Code (Continue, Cline, Roo) with Tool Calling & JSON Mode
curl http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cipanion",
    "messages": [{"role": "user", "content": "Summarize 3 benefits of local AI as JSON."}],
    "response_format": {"type": "json_object"},
    "temperature": 0.2,
    "stream": true
  }'

# 3) Ollama Native Clients (/api/tags, /api/chat, /api/generate) + Secure Team Gateway
cipanion serve --host 0.0.0.0 --port 11434 --api-key YOUR_SECRET_KEY

# 4) Frontier 70B+ Multi-Shard GGUF + Lossless Speculative Decoding (3x–5x NVMe Speedup!)
cipanion run Llama-3.3-70B-Q4_K_M-00001-of-00002.gguf \
  --draft-model Llama-3.2-1B-Instruct-Q4_K_M.gguf --draft-tokens 5 \
  --prompt "Explain FlashAttention online softmax concisely."

The Complete Tapirus Lab Local-to-Cloud AI Stack

Combine Cipanion with TapirusDB and TapisaurusDB to build private AI applications with long-term memory and enterprise scale—without external cloud dependencies:

1. AI Inference Engine Cipanion (~2.2 MiB)

Run GGUF language models locally on laptops or in data centers with 3-tier NVMe streaming, dynamic GPU acceleration, and a built-in 1-Click Studio UI.

2. Embedded AI Memory TapirusDB (.tapir)

Give your local AI long-term memory in one encrypted .tapir file unifying SQL, JSON Documents, HNSW Vectors, and Knowledge Graphs in <4 MB RAM.

3. Distributed Multi-Model Server TapisaurusDB (7.70 MB)

Scale across clusters speaking PostgreSQL, MySQL, MongoDB, gRPC, and REST natively over Raft consensus, SSD DiskANN vectors, and HTAP analytics.