Coming from JSON

If your data is JSON today, you already know its shape. This page translates that shape into a Graven schema, piece by piece, using a small product catalog. Every command and output below is real.

The data

A catalog as most APIs would send it: an object with a list of products.

{
  "products": [
    { "sku": "CF-01", "name": "Coffee beans 1kg", "price": { "cents": 8990, "currency": 986 },
      "tags": ["coffee", "beans"], "stock": 120 },
    { "sku": "MG-02", "name": "Ceramic mug", "price": { "cents": 3500, "currency": 986 },
      "status": "Draft" }
  ],
  "updated_at": 1700000000
}

The schema

namespace shop;

enum Status : u8 { Draft = 0, Active = 1, Archived = 2 }

struct Money { cents: i64, currency: u16 }

table Product {
  sku    @0: string;
  name   @1: string;
  price  @2: Money;
  status @3: Status = Active;
  tags   @4: [string]?;
  stock  @5: u32?;
}

table Catalog {
  products   @0: [Product];
  updated_at @1: u64?;
}

Each decision, against the JSON it came from:

• Product and Catalog are tables. A JSON object whose keys you know becomes a table. Every field gets an ID (@0, @1, …); new fields are added later with new IDs at the end, and old readers keep working.
• price is a struct. cents and currency always travel together and will never change, so they are stored inline, side by side, with no per-field overhead. Use a struct only for groups like this; anything that may grow is a table.
• price.cents is an integer. JSON has one number type; here you choose. Money is an integer number of cents, never a float.
• status is an enum with a default. A fixed set of strings becomes an enum: one byte instead of a string. The first product has no status in the JSON, so it reads as the default, Active.
• tags and stock are optional (?). In JSON these keys are sometimes missing. In Graven, absent is recorded on the wire, so a stock of 0 is different from no stock.
• sku and name are required. A reader rejects a message without them.

JSON concepts and their Graven counterparts

Converting existing JSON

gravenc converts JSON into Graven messages and back, using the schema:

gravenc update catalog.graven --init        # create the lockfile once
gravenc from-json catalog.graven --root Catalog catalog.json -o catalog.bin

catalog.bin: 1 message(s), 185 bytes

The minified JSON is 248 bytes; the Graven message is 185, including its 24-byte envelope with schema id and checksum. To look inside:

$ gravenc inspect catalog.graven catalog.bin
#0  offset 0  shop.Catalog  185 bytes
{
  "products": [
    {
      "sku": "CF-01",
      "name": "Coffee beans 1kg",
      "price": {
        "cents": 8990,
        "currency": 986
      },
      "status": "Active",
      "tags": [
        "coffee",
        "beans"
      ],
      "stock": 120
    },
    {
      "sku": "MG-02",
      "name": "Ceramic mug",
      "price": {
        "cents": 3500,
        "currency": 986
      },
      "status": "Draft"
    }
  ],
  "updated_at": 1700000000
}

The first product shows "status": "Active": it was absent, so the reader applies the default.

A top-level array

Plenty of JSON is just an array: [{"sku": "CF-01", …}, {"sku": "MG-02", …}]. A Graven message always starts at a table, so there are two ways to store it.

Wrap it in a table, as Catalog does above. The whole list is one message: choose this when the list is read and written as a unit, like an API response or a configuration. Binary layout shows such a list byte by byte.

Store one message per element. from-json reads a top-level array (or JSON Lines) as one message per element and writes them one after another, each in its own envelope:

gravenc from-json catalog.graven --root Product products.json -o products.bin

products.bin: 2 message(s), 192 bytes

Choose this for collections that grow: logs, events, exports. New records are appended without rewriting the file, each record is checked on its own, and gravenc scan tells a crash-truncated tail from real corruption.

$ gravenc to-json catalog.graven products.bin
{"sku":"CF-01","name":"Coffee beans 1kg","price":{"cents":8990,"currency":986},"status":"Active","tags":["coffee","beans"],"stock":120}
{"sku":"MG-02","name":"Ceramic mug","price":{"cents":3500,"currency":986},"status":"Draft"}

The same data from Rust

With the code gravenc generates for catalog.graven (see getting started):

use graven::{Builder, SimplePool};
use shop::{Catalog, CatalogW, Money, Status};

fn main() -> Result<(), graven::Error> {
    let pool = SimplePool::new(4096);
    let mut b = Builder::new_enveloped(&pool);
    let bytes = CatalogW::to_bytes(&mut b, |c| {
        c.updated_at(1_700_000_000);
        let mut products = c.products(2);
        products
            .push()
            .sku("CF-01")
            .name("Coffee beans 1kg")
            .price(Money { cents: 8990, currency: 986 })
            .tags(["coffee", "beans"])
            .stock(120);
        products
            .push()
            .sku("MG-02")
            .name("Ceramic mug")
            .price(Money { cents: 3500, currency: 986 })
            .status(Status::Draft);
    })?;

    let catalog = Catalog::from_bytes(&bytes)?;
    for p in catalog.products()?.iter() {
        let p = p?;
        println!("{} {} {:?} stock={:?}", p.sku()?, p.name()?, p.status(), p.stock());
    }
    Ok(())
}

CF-01 Coffee beans 1kg Active stock=Some(120)
MG-02 Ceramic mug Draft stock=None

These bytes are identical to the catalog.bin that gravenc from-json produced from the JSON: the tools and the generated code follow the same canonical layout.

What works differently from JSON

• Names are not in the bytes; IDs are. Renaming a field is free. Reusing an ID, or changing a type, is refused by gravenc update.
• Types are fixed by the schema. A field cannot be a number in one message and a string in the next. Evolution follows the compatibility rules.
• Absent is explicit. Every field knows whether it was written, so 0, "" and "not sent" are three different things.
• Reading is free until you touch a field. There is no parse step: fields are read in place, and one verify makes untrusted input safe to read.