A binary format for data that has to last.

Graven is an open specification for a zero-copy serialization format and its schema language. Messages are read in place, straight from the buffer. The format is frozen and fully specified, and its conformance vectors let an implementation in any language prove that it reads every message correctly.

Inspired by FlatBuffers and Cap'n Proto. We started from their ideas, measured their trade-offs in benchmarks, and kept exploring until we reached this format. How we got here

Read the specification Implementations

Format v1 is frozen. Reference implementation in Rust. MIT licensed.

One message, byte by byte

namespace bank;

table Entry {
  account @0: u64;
  amount  @1: i64;
  memo    @2: string?;
}

Built for data that outlives the code that wrote it

Business data is written once and read for years: by the next release, by a service that upgrades later, by an audit long after the original team has moved on. Graven is designed around that reality.

Read in place: Fields are read directly from the bytes. Strings and byte slices borrow from the buffer, so reading a large record costs nothing until you touch a field.
Evolve without guessing: A lockfile records every layout. The compiler refuses changes that would make old and new code disagree, and asks for explicit confirmation before anything breaks an older reader.
Untrusted bytes are safe to open: A single verify pass walks the whole message under fixed depth and work limits. After that, a verified view reads without re-checking, and malformed input can only ever produce an error.
Know what happened to a file: The optional envelope carries a schema id and a CRC32C. A write cut short by a crash is reported as incomplete; a flipped bit is reported as corruption. They are never confused.

How the format is kept honest

Every claim the specification makes is pinned by something that runs. This is what is checked, and how.

Guarantee	How it is checked
The wire format does not drift	A normative specification (RFC 2119 language) and 42 frozen conformance vectors, each a `.bin` with its expected result. The reference writer must reproduce the canonical vectors byte for byte.
Two implementations agree	Within the reference implementation, an independent interpreter driven by the schema as data and the generated code read every vector and every single-byte mutation of it. They must accept, reject and decode identically. Every new implementation is held to the same vectors.
Hostile input stays bounded	`verify` limits depth to 64 and total work to `max(len × 8, 1 MiB)`, so cycles and shared-data amplification are rejected. Five fuzz targets ran more than 80 million inputs with overflow checks enabled, with no findings.
The unsafe fast path is sound	In the Rust implementation, the only `unsafe` code skips UTF-8 revalidation after `verify`. Its invariants are written down, enforced by a mutation gate, and the runtime and generated code run under Miri with no undefined behaviour reported.
Schema changes cannot corrupt silently	The compiler compares every change against the lockfile: 15 rules reject changes that would misread old data, 2 warnings require confirmation in the schema. A corpus of version chains tests each rule.
Crashes and corruption are told apart	The envelope's check order is normative, so a truncated tail and a damaged record produce different, predictable errors on every implementation.

Schema changes are reviewed by the compiler

Change a type or add a required field, and gravenc update refuses, naming the rule and the line. Add the field with a default instead, and the new lock is written. This is real output.

Rejected

table Entry {
  account  @0: u64;
  amount   @1: i32;
  memo     @2: string?;
  currency @3: u16;
}

$ gravenc update ledger.graven
ledger.graven:5:3: field `Entry.amount` (@1): type changed
  from `i64` to `i32` (type-changed)
ledger.graven:7:3: new field `Entry.currency` (@3) is
  required: every new reader would fail on old messages;
  use `u16?` or a default (new-required)

Accepted

table Entry {
  account  @0: u64;
  amount   @1: i64;
  memo     @2: string?;
  currency @3: u16 = 986;
}

$ gravenc update ledger.graven
ledger.graven.lock: written

What it is for

Graven fits wherever records are written once, kept for a long time and read by software that changes.

Ledgers and event logs: Append envelopes to a file and never rewrite them. Each record carries its schema id and checksum, a crash leaves a detectably incomplete tail, and reports can scan the file without deserializing it.
Service and SDK protocols: Route requests by schema id, verify once at the edge and pass the verified view along without copying. The compatibility rules mean an older client keeps working when the server moves ahead.
Archives and audit trails: A frozen format with public conformance vectors is something you can still decode in ten years, with a different implementation if need be.
Devices and offline sync: The format needs no allocator and no alignment, and the Rust runtime is no_std with two dependencies, so the same records can come from point-of-sale terminals and embedded devices that batch them while offline.
Caches and storage values: Store records in a key-value store and read only the field you need, straight from the stored bytes.
Plugin boundaries: Exchange buffers with untrusted extensions, for example WebAssembly modules, and let verify protect the host from whatever they send back.

Implementations

The specification is the product; implementations are measured against it. A conforming implementation reads all 42 vectors with exactly the expected result (spec §10).

Language	Status	What it includes
Rust	Reference implementation. Passes all v1 vectors.	Runtime (`no_std`), the `gravenc` compiler with lockfile and compatibility checks, Rust code generation, and the conformance interpreter. API reference

Writing an implementation

Start with the layout algorithm (spec §3.3), the read rules (§4–§6) and verify (§7), then run the conformance vectors. Each vector is a .bin with a .json stating the decoded value or the exact error, and each invalid one has a single defect, so a failure points at one rule.

Generating code for another language

The schema language and its compatibility rules are language-neutral. The lockfile written by gravenc already records every table's layout, schema id and field presence, which is what a code generator for another language needs to emit readers and writers.

Getting started with Rust

Using the reference implementation needs only the Rust toolchain.

Install the compiler

cargo install --path crates/graven-compiler

Write a schema and create its lockfile

gravenc update schemas/bank.graven --init

Generate code from `build.rs`

graven_compiler::build::Build::new()
    .schema("schemas/bank.graven")
    .compile()
    .unwrap_or_else(|e| panic!("{e}"));

Write and read

let bytes = TransactionW::to_bytes(&mut b, |t| {
    t.id(42).currency(Currency::USD);
})?;
let tx = Transaction::from_bytes(&bytes)?;
println!("{}", tx.id());

The getting started guide and the schema language reference cover the full workflow.