Tapirus Lab Architecture: TapirusDB vs. TapisaurusDB
Tapirus Tech Lab builds two purpose-designed Rust database engines so engineering teams never have to compromise between embedded single-file simplicity and polyglot network server scalability:
| Dimension | TapirusDB (v1.2.1) |
TapisaurusDB (v0.1.0) |
|---|---|---|
| Primary Form Factor | Embedded single-file .tapir engine, Browser WASM, and no_std tapirus-micro |
Standalone 5-port multi-protocol network server, embedded library, and distributed cluster |
| Storage Engine | Single-file slotted B+Tree pages + TLA+ verified WAL + XChaCha20-Poly1305 page encryption | Lock-free SkipMap MemTable + CRC32 WAL + LZ4 SSTables + 16-shard ARC cache + S3/GCS cold tiering |
| Network Wire Protocols | Native Rust/C-FFI/Python/Node/PHP SDKs + HTTP/TLS server | PostgreSQL (5432) + MySQL (3306) + MongoDB (27017) + gRPC (50051) + REST/WSS (27018) |
| Distributed Consensus | Local-first single-node ACID + signed offline IoT capsules | Multi-node Raft Quorum (CP) with state-machine applier + Active-Active CRDTs (AP) + 16,384 virtual shards |
| Ideal Sweet Spot | Local AI agent memory, desktop/mobile apps, browser IndexedDB WASM, bare-metal MCUs | Polyglot microservices, ORM drop-in replacement, clustered GraphRAG backends, robotics/edge servers |
5-Way Native Wire Protocol Gateways
Every incoming connection across all five network ports authenticates against a single unified credential store (Argon2id password verification + Ed25519 JWT role-based access control) and compiles directly into the native TapisaurusQL Abstract Syntax Tree:
Supports Simple Query ('Q') and Extended Query Protocol (Parse / Bind / Describe / Execute / Sync), MVCC SSI transactions (BEGIN / COMMIT / ROLLBACK with undo logs and SQLSTATE 40001 conflict detection), Pratt expression evaluation, joins, and the tapisaurus-catalog relational SQL materializer (pg_class, pg_attribute, pg_attrdef, pg_index, FK pg_constraint, format_type(), information_schema.*) for Prisma, SQLAlchemy, Drizzle, and psql.
Decodes binary OP_MSG and OP_QUERY BSON frames natively at 3,390.6 ops/sec (p99 = 526 µs). Supports SASL PLAIN authentication, RBAC guards, CRUD operations, stateful cursor pagination, secondary indexes, and aggregation pipelines ($match, $project, $group, $sort, $limit).
Implements Protocol v10 capability handshakes, OK_Packet / ERR_Packet framing, standard SQL DML, full SHOW FULL TABLES / SHOW CREATE TABLE (with PK & FK DDL), and dynamic information_schema.{tables,columns,statistics,key_column_usage,referential_constraints} queries for PHP PDO, Laravel Eloquent, SQLAlchemy, and Prisma.
Provides strongly typed Protocol Buffers RPCs for low-latency vector search and document ingestion, alongside HTTP REST pagination, Prometheus histograms (/metrics), Kubernetes probes (/v1/health/liveness, /readiness), and real-time Change Data Capture (CDC) streams.
psql -h 127.0.0.1 -p 5432 -U admin -d tapisaurus
BEGIN;
INSERT INTO accounts (_id, owner, balance) VALUES ('acc_101', 'Ahmad Faiz', 2500);
UPDATE accounts SET balance = balance + 500 WHERE _id = 'acc_101';
SELECT * FROM orders JOIN accounts ON orders.account_id = accounts.id ORDER BY amount DESC;
COMMIT;
-- Create Knowledge Graph vertices & edges via openCypher
CREATE (a:research {id: 'doc_1', topic: 'consensus', title: 'Raft Quorum State Machine'});
CREATE (b:research {id: 'doc_2', topic: 'consensus', title: 'MVCC Serializable Snapshot Isolation'});
CREATE (a {id: 'doc_1'})-[:CITES]->(b {id: 'doc_2'});
-- Execute multi-hop graph traversal + HNSW vector similarity in a single query
FIND research WHERE topic = 'consensus'
TRAVERSE FROM 'doc_1' DEPTH 2 VIA 'CITES'
VECTOR NEAR [0.12, 0.84, 0.33, 0.91] TOP 5 USING INDEX 'research_emb';
Core Storage, Consensus & Next-Gen Leapfrog Innovations
Hot point mutations land in a Clock-Pro managed SkipMap MemTable and persist as TPBIN001 slotted binary records with a sorted 16-byte field offset directory for O(log F) zero-deserialization field projection. Analytical scans execute over 2,048-row HtapColumnarTable stripes (TPHTAP01, LZ4 + CRC32) with ColumnZoneMap pruning—delivering 32.5x faster selective aggregations.
Alongside in-memory HNSW (956.89 ms build for 2,000 × 128D vectors), DiskAnnIndex (TPDANN01) builds an α-RNG pruned Vamana graph (alpha = 1.2) on SSD while keeping only 1-bit Binary Quantized routing codes in RAM (32x smaller footprint) plus a real-time OLTP hot buffer for instant updates.
DeterministicSequencer sequences multi-shard transactions through Raft epochs in canonical (shard_slot, key) lock order without Two-Phase Commit (2PC) stalls, while WasmSandboxModule executes fuel-metered \0asm bytecode triggers and UDFs safely inside ACID SQL mutations.
Complete Engine Capability Matrix
Every subsystem across storage, analytics, AI vectors, graph traversal, consensus, wire protocols, and cryptography is compiled directly into one self-contained Rust server kernel:
| Subsystem | Implemented Capabilities & Standards | Key Primitives / Formats |
|---|---|---|
| Core Storage & OLTP | Leveled LSM-Tree (SkipMap MemTable + Bloom-filtered SSTables), CRC32 WAL with PutBatch group commit, MVCC SSI isolation (SQLSTATE 40001), Min-Heap TTL cache, and S3/GCS multipart cold-tier offloader. |
TPSAUR01, sync_data(), 16-shard ARC cache |
| HTAP Columnar Analytics | 2,048-row morsel stripes with per-column min/max Zone-Map predicate pruning, LZ4 block compression, and vectorized SUM, AVG, MIN, MAX, and GROUP BY. |
HtapColumnarTable (TPHTAP01), ColumnZoneMap |
| Vector Search & AI | In-memory HNSW with Vec<bool> L1-cache visited bitmap (Cosine, L2, Dot, Manhattan), IdBitset pre-filtering, Scalar/1-bit Binary Quantization (32x compression), and SSD-paged Vamana DiskANN with real-time OLTP hot buffer. |
TPSHNSW1, TPDANN01 |
| Graph & GraphRAG | Native openCypher parser, zero-clone &str BFS shortest path, Dijkstra, PageRank, hybrid TRAVERSE ... VECTOR NEAR, and built-in Semantic Cache. |
GraphStore, SemanticCache |
| Distributed Consensus | Majority-quorum Raft (leader election, log compaction, BSON disk persistence), Calvin DeterministicSequencer (zero-2PC multi-shard transactions), 16,384 hash slots with {hash_tag} colocation, and active-active multi-region CRDTs. |
RaftNode, ShardRouter, CrdtDocument |
| Sandboxed WASM UDFs | In-transaction WebAssembly bytecode validators and computed field triggers executed atomically inside SQL INSERT with deterministic fuel limits. |
WasmUdfRegistry (\0asm) |
| 5-Way Wire Protocols | PostgreSQL (5432, Simple + Extended Query), MySQL v10 (3306), MongoDB OP_MSG (27017), gRPC Protobuf (50051), and HTTP REST / WebSocket CDC (27018). |
psql, Prisma, PDO, mongosh, Tonic |
| Full-Text & Security | Okapi BM25 ranking with Levenshtein fuzzy typo search; AES-256-GCM storage encryption, Argon2id per-payload salted KDF with ZeroizeOnDrop, Ed25519 JWT RBAC, and hash-chained audit logs. |
#![forbid(unsafe_code)], Rustls TLS 1.3 |
Same-Host Head-to-Head Benchmark: TapirusDB vs. TapisaurusDB
Measured directly on the same host (Windows x86_64, release profile opt-level = 3, vs_bench harness) to evaluate embedded single-file B+Tree latency alongside polyglot server LSM-Tree & HTAP throughput:
| Workload (Release, Same Host) | TapirusDB v1.2.1 |
TapisaurusDB v0.1.0 |
Engineering Takeaway |
|---|---|---|---|
| SQL Batch Insert (2,000 rows, WAL) | 5.22 ms (383,120 rows/s, BEGIN..COMMIT) |
SQL Batch: 41.25 ms (48,480 rows/s)KV put_batch: 5.51 ms (363,016 rows/s) |
TapirusDB leads on relational SQL batch; raw storage batch fsync is effectively tied (~5.2–5.5 ms). |
| Warm Point Lookup (2,000 queries) | SQL p50: 7.00 µs (p99: 22.60 µs) |
SQL p50: 4.60 µs (p99: 9.40 µs)Raw KV get(): 0.60 µs (p99: 1.00 µs) |
O(1) _id fast-path gives TapisaurusDB 4.60 µs SQL lookups and 0.60 µs (>1.5M ops/s) direct KV reads. |
| 8-Thread Concurrent Writes (400 WAL ops) | Strict fsync (SWMR): 1,236.31 ms (324 tx/s) |
Strict fsync: 1,198.76 ms (334 ops/s)Buffered WAL: 4.93 ms (81,141 ops/s) |
Lock-free SkipMap MemTable scales concurrent writes to 81k+ ops/s under buffered WAL without SWMR writer serialization. |
| HNSW Vector Index (2,000 × 128D, Top-10) | Build: 1,875.04 msSearch p50: 160.90 µs |
Build: 956.89 ms Search p50: 421.60 µs (p99: 776.50 µs) |
TapisaurusDB builds HNSW graphs 1.96x faster (<1s); TapirusDB searches 2.62x faster via SIMD kernels. |
| Graph Shortest-Path BFS (500 nodes, 500 q) | p50: 6.80 µs (3.66 ms total) |
4.60 µs p50 (3.05 ms total) |
Zero-clone &str BFS traversal delivers sub-5µs multi-hop pathfinding. |
| Selective Analytical SUM (20,480 rows, >19,456) | Row Scan p50: 2,476.20 µs (124.88 ms) |
HTAP ZoneMap p50: 76.20 µs (3.78 ms, 9/10 stripes pruned) |
32.5x faster analytical aggregations on TapisaurusDB via TPHTAP01 Zone-Map pruning. |
Industry Architectural Positioning
How does TapisaurusDB compare against CockroachDB, LanceDB, DuckDB, and PostgreSQL across architecture, footprint, and workload specialization?
| Evaluation Dimension | TapisaurusDB 🚀 | CockroachDB | LanceDB | DuckDB | PostgreSQL (+ pgvector / AGE) |
|---|---|---|---|---|---|
| Primary Category | Polyglot Multi-Model Server & Cluster Kernel (OLTP + HTAP + AI) | Distributed NewSQL (Global OLTP) | Embedded Multimodal & Disk Vector Store | In-Process Vectorized OLAP Engine | Traditional RDBMS + C Extensions |
| Core Language | Memory-Safe Rust | Go + C++ | Rust (Arrow / Lance) | C++17 | C |
| Binary Size & Idle RAM | 7.70 MB binary · 23.05 MB VmRSS | ~120 MB · 512 MB – 2 GB+ RAM | ~20 MB lib · 50 – 150 MB RAM | ~20 MB · 20 – 100 MB RAM | ~150 MB · 128 – 512 MB RAM |
| Wire Protocols | 5-Way: PG (5432), MySQL (3306), Mongo (27017), gRPC (50051), REST (27018) | PostgreSQL (26257) only | SDK / HTTP REST only | In-process library API only | PostgreSQL (5432) only |
| ACID & Transactions | MVCC SSI (SQLSTATE 40001) + Raft + Calvin Deterministic Sequencer |
Distributed Serializable (SSI + 2PC/Raft) | Dataset snapshot versioning (Append-focused) | Optimistic MVCC (Single writer) | Heap MVCC + SSI |
| Vector & Analytical | HNSW + SSD DiskANN (TPDANN01) + HTAP Columnar Zone Maps (TPHTAP01) + openCypher |
No native HNSW, HTAP columnar, or Graph | Disk IVF-PQ / HNSW (No Cypher Graph) | Vectorized OLAP (experimental vector/graph ext) | External C extensions (pgvector + Apache AGE) |
| Extensibility & Triggers | Sandboxed Fuel-Metered WASM Triggers (\0asm) |
SQL constraints / external CDC | Python/Rust UDF transforms | C++ / Python scalar UDFs | C shared libs / PL/pgSQL |
Stacking pgvector and Apache AGE inside PostgreSQL introduces C shared-library crash risks, severe WAL write amplification during HNSW graph mutations, and process-per-connection RAM overhead. TapisaurusDB unifies SQL, BSON Documents, HNSW/DiskANN Vectors, HTAP Columnar Stripes, and openCypher GraphRAG natively in one 7.70 MB Rust binary with asynchronous tokio connection governance.
LanceDB and DuckDB excel at columnar analytical scans and bulk dataset appends, but struggle with high-concurrency point mutations, multi-client network wire protocols, and multi-hop graph transactions. TapisaurusDB delivers 81,141 ops/s concurrent multi-thread ingestion (buffered WAL), 4.60 µs SQL point lookups, 76.20 µs Zone-Map OLAP aggregations, and simultaneous PostgreSQL/MySQL/MongoDB wire access.
Production Use Cases: Choosing Between TapirusDB & TapisaurusDB
Together, TapirusDB and TapisaurusDB form a sovereign Edge-to-Cloud data architecture without external database sprawl:
1. Local AI Agents & MCP Memory: Run long-term episodic vector + BM25 + graph memory in <4 MB RAM via tapirus mcp without cloud leaks.
2. Offline-First Desktop, Mobile & Browser: Single encrypted .tapir files with free XChaCha20-Poly1305 AEAD and 480 KB gzipped @tapirus/wasm-lite.
3. Robotics, Automotive & Air-Gapped RegTech: Power-safe TLA+ verified WAL and no_std tapirus-micro flash capsules for smart factories, drones, and bank AML workstations.
1. Polyglot SaaS Consolidation: Connect Laravel (MySQL 3306), Prisma/Drizzle (Postgres 5432), and Node/PyMongo (Mongo 27017) to one RBAC-secured server.
2. Billion-Scale RAG & Live HTAP Dashboards: Combine SSD DiskANN vector retrieval with 32.5x faster TPHTAP01 Zone-Map aggregations without ETL to ClickHouse.
3. Multi-Shard Ledgers & Sandboxed Triggers: Execute deadlock-free Calvin multi-shard transactions over Raft alongside fuel-metered WASM business rules.
TapirSync (TPSYNC01) connects offline-first TapirusDB (.tapir) edge databases directly to TapisaurusDB cloud clusters via POST /v1/tapirsync/push and POST /v1/tapirsync/pull. Local SQL rows, JSON documents, Knowledge Graph vertices/edges, and AI Agent Memories (with dense embeddings) are tracked with per-record CRC32 digests and causal Hybrid Logical Clocks (TapirSyncHlc), transmitting only incremental deltas and deletion tombstones with deterministic Last-Write-Wins (LWW) conflict convergence.