Graven

zidl v1: the zidlc compiler + lockfile (Plan 2 of 3)

Historical document. Written on 2026-11-13, when the project was called zidl (later Graven, with the compiler gravenc). It is the plan that built the compiler and the lockfile. Names and some details changed afterwards (for example, the lockfile became <schema>.graven.lock and namespaces no longer carry versions); the current references are the language guide and the format specification.

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A .zidl → Rust compiler (zidlc + a build.rs API) with a lockfile that refuses incompatible changes. It generates exactly the pattern that is hand-written in zidl-schemas today. At the end, zidl-schemas uses only generated code, and the generated code of conformance.zidl passes every frozen vector.

Architecture: First the generatable pattern is frozen by hand (Task 1): the support moves to the runtime, there is a single kind of view, the header is charged by the caller and every unsafe primitive lives in the runtime. Then the compiler is built in pure layers:

The CLI and the build.rs API only orchestrate those layers. The switch in zidl-schemas only happens with the current tests passing without changing assertions and the benchmark within ±5%.

Tech Stack: Rust 2024; in the compiler: serde + toml (lockfile), quote + proc-macro2 + syn + prettyplease (codegen), proptest (dev). No new dependencies in the runtime.

Spec: design/2026-11-12-zidl-v1-hardening-design.md (part D) and the normative spec docs/spec/zidl-format-v1.md (§3.3 layout, §4 presence, §5–6 types, §7 verify, §8 envelope, §9 compatibility). Decisions inherited from Plan 1 (record: the Plan 1 progress ledger, lines "Final: Ruling" and "CARRY TO PLAN 2").

Global Constraints

Review Focus

  1. A schema with a recursive type (Leaf.next: Leaf?) ⇒ layout, lock and codegen terminate (no infinite recursion) and the generated reader gives DepthExceeded on the self_cycle vector. → Task 6, test conformance_vectors_generated.
  2. The same bytes reached through two edges as different types ⇒ the generated view only exists through the edge that verify walked. → Task 6, compile_fail doctest + review of the invariants.
  3. A lockfile edited by hand in an invalid way (broken TOML, missing field, unknown format) ⇒ a clear error with the file path, never a panic. → Task 4, lock_rejects_malformed.
  4. build.rs without a lockfile ⇒ an error telling you to run zidlc update <schema>, without writing anything into src/. → Task 7, build_missing_lock_errors.
  5. A union with an unknown tag in a verified message ⇒ the view returns Unknown(tag) and does not descend. → Task 6, the unknown_union_tag vector via generated code.

Decisions in this plan (by the assistant)

File Structure

File Responsibility
crates/zidl/src/rt.rs (new) support for generated code, moved from zidl-schemas/src/lib.rs + Req::Union, VStrList, the invariants doc
crates/zidl/src/le.rs (mod) impl Le for bool
crates/zidl-schemas/src/*.rs (mod, Task 1; removed in Task 8) the final pattern, hand-written
crates/zidl-compiler/ (new) lib + bin zidlc
…/src/ast.rs, …/src/parse.rs AST with spans; hand-written lexer + recursive-descent parser
…/src/ir.rs, …/src/check.rs resolved IR + layout (zidl-layout)
…/src/lock.rs TOML lockfile ↔ IR, schema_id
…/src/compat.rs the rules of spec §9
…/src/codegen/{mod,read,write,view}.rs IR → TokenStream → prettyplease
…/src/build.rs (lib), …/src/main.rs build.rs API, CLI
…/tests/{parse,check,lock,compat,codegen}.rs, …/tests/compat/<case>/, …/tests/snapshots/ tests
crates/zidl-schemas/build.rs, schemas/*.zidl.lock, schemas/evo/*.zidl switch to generated code
crates/zidl-conformance/build.rs, schemas/conformance.zidl.lock generated code of conformance.zidl for the vectors gate
docs/zidl-language.md (new) reference for the .zidl language and zidlc

Task 1: Freeze the generatable pattern by hand

Files:

Interfaces:

Task 2: The .zidl parser

Files:

Interfaces:

Task 3: Semantic checking and layout (IR)

Files:

Interfaces:

Task 4: Lockfile

Files:

Interfaces:

Task 5: Compatibility checking (spec §9)

Files:

Interfaces:

Task 6: Codegen

Files:

Interfaces:

Task 7: The zidlc CLI and the build.rs API

Files:

Interfaces:

Task 8: zidl-schemas switches to generated code

Files: