TapirusDB

Tapirus Lab · Embedded Single-File & Polyglot Distributed Engines

TapisaurusDB · Polyglot Multi-Model Server & Cluster Kernel · v0.1.0 · #![forbid(unsafe_code)]

One 7.70 MB server kernel. Five native wire protocols.

TapisaurusDB is the distributed server sibling to TapirusDB. Engineered in memory-safe Rust by Ahmad Faiz, it decodes PostgreSQL (5432), MySQL (3306), MongoDB (27017), gRPC (50051), and REST/WebSockets (27018) natively on top of a unified LSM-Tree, HNSW Vector, and openCypher GraphRAG storage engine in just 23.05 MB of idle resident RAM.

7.70 MB Self-contained static server binary
23.05 MB Idle Linux VmRSS (all 5 gateways active)
32,305 ops/s Durable WAL writes with OS sync_data()
375 / 375 Verified unit, MVCC SSI, Raft & chaos tests
5-Way Native Wire Comprehension

Connect unmodified psql, DBeaver, Prisma, Drizzle, Laravel Eloquent, PHP PDO, mongosh, Mongoose, PyMongo, and gRPC microservices directly—no external translation proxy required.

Transactional GraphRAG in 1 Query

Combine relational SQL/JSON filters, openCypher multi-hop graph traversal (< 0.91 ms), and SIMD HNSW vector ranking (< 0.88 ms, 32x binary quantization) inside one atomic ACID store.

Raft Quorum (CP) & CRDT Mesh (AP)

Enforce majority-quorum Raft replication and MVCC Serializable Snapshot Isolation (SQLSTATE 40001) for financial ledgers, or use active-active CRDTs for sub-millisecond multi-region edge telemetry.

Tapirus Lab Architecture: TapirusDB vs. TapisaurusDB

Tapirus Tech Lab builds two purpose-designed Rust database engines so engineering teams never have to compromise between embedded single-file simplicity and polyglot network server scalability:

Dimension TapirusDB (v1.2.1) TapisaurusDB (v0.1.0)
Primary Form Factor Embedded single-file .tapir engine, Browser WASM, and no_std tapirus-micro Standalone 5-port multi-protocol network server, embedded library, and distributed cluster
Storage Engine Single-file slotted B+Tree pages + TLA+ verified WAL + XChaCha20-Poly1305 page encryption Lock-free SkipMap MemTable + CRC32 WAL + LZ4 SSTables + 16-shard ARC cache + S3/GCS cold tiering
Network Wire Protocols Native Rust/C-FFI/Python/Node/PHP SDKs + HTTP/TLS server PostgreSQL (5432) + MySQL (3306) + MongoDB (27017) + gRPC (50051) + REST/WSS (27018)
Distributed Consensus Local-first single-node ACID + signed offline IoT capsules Multi-node Raft Quorum (CP) with state-machine applier + Active-Active CRDTs (AP) + 16,384 virtual shards
Ideal Sweet Spot Local AI agent memory, desktop/mobile apps, browser IndexedDB WASM, bare-metal MCUs Polyglot microservices, ORM drop-in replacement, clustered GraphRAG backends, robotics/edge servers

5-Way Native Wire Protocol Gateways

Every incoming connection across all five network ports authenticates against a single unified credential store (Argon2id password verification + Ed25519 JWT role-based access control) and compiles directly into the native TapisaurusQL Abstract Syntax Tree:

🐘 PostgreSQL Wire (Port 5432 / 5433)

Supports Simple Query ('Q') and Extended Query Protocol (Parse / Bind / Describe / Execute / Sync), MVCC SSI transactions (BEGIN / COMMIT / ROLLBACK with undo logs and SQLSTATE 40001 conflict detection), Pratt expression evaluation, joins, and the tapisaurus-catalog relational SQL materializer (pg_class, pg_attribute, pg_attrdef, pg_index, FK pg_constraint, format_type(), information_schema.*) for Prisma, SQLAlchemy, Drizzle, and psql.

🍃 MongoDB Wire (Port 27017)

Decodes binary OP_MSG and OP_QUERY BSON frames natively at 3,390.6 ops/sec (p99 = 526 µs). Supports SASL PLAIN authentication, RBAC guards, CRUD operations, stateful cursor pagination, secondary indexes, and aggregation pipelines ($match, $project, $group, $sort, $limit).

🐬 MySQL / MariaDB Wire (Port 3306)

Implements Protocol v10 capability handshakes, OK_Packet / ERR_Packet framing, standard SQL DML, full SHOW FULL TABLES / SHOW CREATE TABLE (with PK & FK DDL), and dynamic information_schema.{tables,columns,statistics,key_column_usage,referential_constraints} queries for PHP PDO, Laravel Eloquent, SQLAlchemy, and Prisma.

⚡ gRPC (50051) & REST / WebSocket CDC (27018)

Provides strongly typed Protocol Buffers RPCs for low-latency vector search and document ingestion, alongside HTTP REST pagination, Prometheus histograms (/metrics), Kubernetes probes (/v1/health/liveness, /readiness), and real-time Change Data Capture (CDC) streams.

1. PostgreSQL Wire (Port 5432) — MVCC SSI Transaction & SQL Join
psql -h 127.0.0.1 -p 5432 -U admin -d tapisaurus

BEGIN;
INSERT INTO accounts (_id, owner, balance) VALUES ('acc_101', 'Ahmad Faiz', 2500);
UPDATE accounts SET balance = balance + 500 WHERE _id = 'acc_101';
SELECT * FROM orders JOIN accounts ON orders.account_id = accounts.id ORDER BY amount DESC;
COMMIT;
2. Transactional GraphRAG — Unified Vector HNSW + openCypher Graph Traversal
-- Create Knowledge Graph vertices & edges via openCypher
CREATE (a:research {id: 'doc_1', topic: 'consensus', title: 'Raft Quorum State Machine'});
CREATE (b:research {id: 'doc_2', topic: 'consensus', title: 'MVCC Serializable Snapshot Isolation'});
CREATE (a {id: 'doc_1'})-[:CITES]->(b {id: 'doc_2'});

-- Execute multi-hop graph traversal + HNSW vector similarity in a single query
FIND research WHERE topic = 'consensus'
  TRAVERSE FROM 'doc_1' DEPTH 2 VIA 'CITES'
  VECTOR NEAR [0.12, 0.84, 0.33, 0.91] TOP 5 USING INDEX 'research_emb';

Core Storage, Consensus & Next-Gen Leapfrog Innovations

Storage & HTAP LSM-Tree + TPBIN001 Slotted Binary + HTAP

Hot point mutations land in a Clock-Pro managed SkipMap MemTable and persist as TPBIN001 slotted binary records with a sorted 16-byte field offset directory for O(log F) zero-deserialization field projection. Analytical scans execute over 2,048-row HtapColumnarTable stripes (TPHTAP01, LZ4 + CRC32) with ColumnZoneMap pruning—delivering 32.5x faster selective aggregations.

SSD Vector Engine DiskANN / Vamana Paged Vector Index

Alongside in-memory HNSW (956.89 ms build for 2,000 × 128D vectors), DiskAnnIndex (TPDANN01) builds an α-RNG pruned Vamana graph (alpha = 1.2) on SSD while keeping only 1-bit Binary Quantized routing codes in RAM (32x smaller footprint) plus a real-time OLTP hot buffer for instant updates.

Cluster & WASM UDF Calvin Deterministic Tx & WASM Sandbox

DeterministicSequencer sequences multi-shard transactions through Raft epochs in canonical (shard_slot, key) lock order without Two-Phase Commit (2PC) stalls, while WasmSandboxModule executes fuel-metered \0asm bytecode triggers and UDFs safely inside ACID SQL mutations.

Complete Engine Capability Matrix

Every subsystem across storage, analytics, AI vectors, graph traversal, consensus, wire protocols, and cryptography is compiled directly into one self-contained Rust server kernel:

Subsystem Implemented Capabilities & Standards Key Primitives / Formats
Core Storage & OLTP Leveled LSM-Tree (SkipMap MemTable + Bloom-filtered SSTables), CRC32 WAL with PutBatch group commit, MVCC SSI isolation (SQLSTATE 40001), Min-Heap TTL cache, and S3/GCS multipart cold-tier offloader. TPSAUR01, sync_data(), 16-shard ARC cache
HTAP Columnar Analytics 2,048-row morsel stripes with per-column min/max Zone-Map predicate pruning, LZ4 block compression, and vectorized SUM, AVG, MIN, MAX, and GROUP BY. HtapColumnarTable (TPHTAP01), ColumnZoneMap
Vector Search & AI In-memory HNSW with Vec<bool> L1-cache visited bitmap (Cosine, L2, Dot, Manhattan), IdBitset pre-filtering, Scalar/1-bit Binary Quantization (32x compression), and SSD-paged Vamana DiskANN with real-time OLTP hot buffer. TPSHNSW1, TPDANN01
Graph & GraphRAG Native openCypher parser, zero-clone &str BFS shortest path, Dijkstra, PageRank, hybrid TRAVERSE ... VECTOR NEAR, and built-in Semantic Cache. GraphStore, SemanticCache
Distributed Consensus Majority-quorum Raft (leader election, log compaction, BSON disk persistence), Calvin DeterministicSequencer (zero-2PC multi-shard transactions), 16,384 hash slots with {hash_tag} colocation, and active-active multi-region CRDTs. RaftNode, ShardRouter, CrdtDocument
Sandboxed WASM UDFs In-transaction WebAssembly bytecode validators and computed field triggers executed atomically inside SQL INSERT with deterministic fuel limits. WasmUdfRegistry (\0asm)
5-Way Wire Protocols PostgreSQL (5432, Simple + Extended Query), MySQL v10 (3306), MongoDB OP_MSG (27017), gRPC Protobuf (50051), and HTTP REST / WebSocket CDC (27018). psql, Prisma, PDO, mongosh, Tonic
Full-Text & Security Okapi BM25 ranking with Levenshtein fuzzy typo search; AES-256-GCM storage encryption, Argon2id per-payload salted KDF with ZeroizeOnDrop, Ed25519 JWT RBAC, and hash-chained audit logs. #![forbid(unsafe_code)], Rustls TLS 1.3

Same-Host Head-to-Head Benchmark: TapirusDB vs. TapisaurusDB

Measured directly on the same host (Windows x86_64, release profile opt-level = 3, vs_bench harness) to evaluate embedded single-file B+Tree latency alongside polyglot server LSM-Tree & HTAP throughput:

Workload (Release, Same Host) TapirusDB v1.2.1 TapisaurusDB v0.1.0 Engineering Takeaway
SQL Batch Insert (2,000 rows, WAL) 5.22 ms (383,120 rows/s, BEGIN..COMMIT) SQL Batch: 41.25 ms (48,480 rows/s)
KV put_batch: 5.51 ms (363,016 rows/s)
TapirusDB leads on relational SQL batch; raw storage batch fsync is effectively tied (~5.2–5.5 ms).
Warm Point Lookup (2,000 queries) SQL p50: 7.00 µs (p99: 22.60 µs) SQL p50: 4.60 µs (p99: 9.40 µs)
Raw KV get(): 0.60 µs (p99: 1.00 µs)
O(1) _id fast-path gives TapisaurusDB 4.60 µs SQL lookups and 0.60 µs (>1.5M ops/s) direct KV reads.
8-Thread Concurrent Writes (400 WAL ops) Strict fsync (SWMR): 1,236.31 ms (324 tx/s) Strict fsync: 1,198.76 ms (334 ops/s)
Buffered WAL: 4.93 ms (81,141 ops/s)
Lock-free SkipMap MemTable scales concurrent writes to 81k+ ops/s under buffered WAL without SWMR writer serialization.
HNSW Vector Index (2,000 × 128D, Top-10) Build: 1,875.04 ms
Search p50: 160.90 µs
Build: 956.89 ms
Search p50: 421.60 µs (p99: 776.50 µs)
TapisaurusDB builds HNSW graphs 1.96x faster (<1s); TapirusDB searches 2.62x faster via SIMD kernels.
Graph Shortest-Path BFS (500 nodes, 500 q) p50: 6.80 µs (3.66 ms total) 4.60 µs p50 (3.05 ms total) Zero-clone &str BFS traversal delivers sub-5µs multi-hop pathfinding.
Selective Analytical SUM (20,480 rows, >19,456) Row Scan p50: 2,476.20 µs (124.88 ms) HTAP ZoneMap p50: 76.20 µs (3.78 ms, 9/10 stripes pruned) 32.5x faster analytical aggregations on TapisaurusDB via TPHTAP01 Zone-Map pruning.

Industry Architectural Positioning

How does TapisaurusDB compare against CockroachDB, LanceDB, DuckDB, and PostgreSQL across architecture, footprint, and workload specialization?

Evaluation Dimension TapisaurusDB 🚀 CockroachDB LanceDB DuckDB PostgreSQL (+ pgvector / AGE)
Primary Category Polyglot Multi-Model Server & Cluster Kernel (OLTP + HTAP + AI) Distributed NewSQL (Global OLTP) Embedded Multimodal & Disk Vector Store In-Process Vectorized OLAP Engine Traditional RDBMS + C Extensions
Core Language Memory-Safe Rust Go + C++ Rust (Arrow / Lance) C++17 C
Binary Size & Idle RAM 7.70 MB binary · 23.05 MB VmRSS ~120 MB · 512 MB – 2 GB+ RAM ~20 MB lib · 50 – 150 MB RAM ~20 MB · 20 – 100 MB RAM ~150 MB · 128 – 512 MB RAM
Wire Protocols 5-Way: PG (5432), MySQL (3306), Mongo (27017), gRPC (50051), REST (27018) PostgreSQL (26257) only SDK / HTTP REST only In-process library API only PostgreSQL (5432) only
ACID & Transactions MVCC SSI (SQLSTATE 40001) + Raft + Calvin Deterministic Sequencer Distributed Serializable (SSI + 2PC/Raft) Dataset snapshot versioning (Append-focused) Optimistic MVCC (Single writer) Heap MVCC + SSI
Vector & Analytical HNSW + SSD DiskANN (TPDANN01) + HTAP Columnar Zone Maps (TPHTAP01) + openCypher No native HNSW, HTAP columnar, or Graph Disk IVF-PQ / HNSW (No Cypher Graph) Vectorized OLAP (experimental vector/graph ext) External C extensions (pgvector + Apache AGE)
Extensibility & Triggers Sandboxed Fuel-Metered WASM Triggers (\0asm) SQL constraints / external CDC Python/Rust UDF transforms C++ / Python scalar UDFs C shared libs / PL/pgSQL
Why TapisaurusDB over PostgreSQL + Extensions?

Stacking pgvector and Apache AGE inside PostgreSQL introduces C shared-library crash risks, severe WAL write amplification during HNSW graph mutations, and process-per-connection RAM overhead. TapisaurusDB unifies SQL, BSON Documents, HNSW/DiskANN Vectors, HTAP Columnar Stripes, and openCypher GraphRAG natively in one 7.70 MB Rust binary with asynchronous tokio connection governance.

Why TapisaurusDB over LanceDB or DuckDB for Operational AI?

LanceDB and DuckDB excel at columnar analytical scans and bulk dataset appends, but struggle with high-concurrency point mutations, multi-client network wire protocols, and multi-hop graph transactions. TapisaurusDB delivers 81,141 ops/s concurrent multi-thread ingestion (buffered WAL), 4.60 µs SQL point lookups, 76.20 µs Zone-Map OLAP aggregations, and simultaneous PostgreSQL/MySQL/MongoDB wire access.

Production Use Cases: Choosing Between TapirusDB & TapisaurusDB

Together, TapirusDB and TapisaurusDB form a sovereign Edge-to-Cloud data architecture without external database sprawl:

TapirusDB · Embedded .tapir Where TapirusDB Excels (In-Process Edge)

1. Local AI Agents & MCP Memory: Run long-term episodic vector + BM25 + graph memory in <4 MB RAM via tapirus mcp without cloud leaks.
2. Offline-First Desktop, Mobile & Browser: Single encrypted .tapir files with free XChaCha20-Poly1305 AEAD and 480 KB gzipped @tapirus/wasm-lite.
3. Robotics, Automotive & Air-Gapped RegTech: Power-safe TLA+ verified WAL and no_std tapirus-micro flash capsules for smart factories, drones, and bank AML workstations.

TapisaurusDB · Multi-Wire Server Where TapisaurusDB Excels (Distributed Cloud/Server)

1. Polyglot SaaS Consolidation: Connect Laravel (MySQL 3306), Prisma/Drizzle (Postgres 5432), and Node/PyMongo (Mongo 27017) to one RBAC-secured server.
2. Billion-Scale RAG & Live HTAP Dashboards: Combine SSD DiskANN vector retrieval with 32.5x faster TPHTAP01 Zone-Map aggregations without ETL to ClickHouse.
3. Multi-Shard Ledgers & Sandboxed Triggers: Execute deadlock-free Calvin multi-shard transactions over Raft alongside fuel-metered WASM business rules.

TapirSync · TPSYNC01 Binary Wire Bridge Edge-to-Cloud Incremental Synchronization (.tapir ↔ TapisaurusDB)

TapirSync (TPSYNC01) connects offline-first TapirusDB (.tapir) edge databases directly to TapisaurusDB cloud clusters via POST /v1/tapirsync/push and POST /v1/tapirsync/pull. Local SQL rows, JSON documents, Knowledge Graph vertices/edges, and AI Agent Memories (with dense embeddings) are tracked with per-record CRC32 digests and causal Hybrid Logical Clocks (TapirSyncHlc), transmitting only incremental deltas and deletion tombstones with deterministic Last-Write-Wins (LWW) conflict convergence.