zidl v1 — Format: envelope, Verified, spec and vectors (Plan 1 of 3)
Historical document. Written on 2026-11-12, when the project was called zidl (later Graven). It is the plan that froze format v1. Names and some details changed afterwards (for example, the magic became
GRVNand namespaces no longer carry versions); the normative text is the format specification.
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Freeze format v1: new errors, CRC32C, the to_bytes/from_bytes envelope (with and
without the Builder), an API without public unsafe (verified views), a normative spec and
conformance vectors generated by a dynamic reader/writer independent of the generated code.
Architecture: Everything about the envelope and the CRC lives in the zidl runtime
(no_std + alloc). The verified views are written by hand in zidl-schemas following the
same pattern the compiler (Plan 2) will generate. Conformance lives in a new crate
zidl-conformance with a dynamic interpreter (a schema described as data → read/write via
TableRef/TableW + zidl-layout) that generates and checks the frozen vectors; Plan 2
checks the generated code against the same vectors.
Tech Stack: Rust 2024, bytes, crc32c 0.6.9 (only with the std feature), proptest,
serde + serde_json (only in the conformance crate), criterion.
Spec: design/2026-11-12-zidl-v1-hardening-design.md (parts A, B, C and the review items
of part E). Following plans: 2 = compiler + lockfile (part D); 3 = fuzzing, generated matrix,
Miri, report (part E).
Global Constraints
- No git: do not create commits or branches (the user's preference). Where the template
asks for a commit, the step is to run
cargo test --workspaceand move on. - The wire format of the raw message does NOT change. The envelope is an addition.
- The
zidlruntime keeps building with--no-default-features(no_std + alloc). - Performance: writing/reading without an envelope within ±5% of
benchmarks/2026-11-12-round2.md. - No panic and no UB when reading malformed bytes, including the envelope.
- No
unsafefunction reachable by safe code outside#[doc(hidden)]. - Envelope header = 20 bytes, a 4-byte CRC32C at the end, little-endian, magic
"ZIDL",format_version = 1.
Review Focus
payload_len = u32::MAX(or any value that overflows20 + N + 4on 32 bits) ⇒Incomplete, never a panic/overflow. → Task 3,decode_huge_len_is_incomplete.- Several concatenated envelopes with a corrupt one in the middle ⇒ the iterator stops with
ChecksumMismatchand does not silently "resynchronize". → Task 4,envelopes_stop_at_corruption. from_byteswith an intact envelope of another root type ⇒SchemaMismatchbefore looking at the payload. → Task 5,from_bytes_wrong_schema.- A verified view built from unverified bytes by safe code ⇒ does not compile. → Task 5,
compile_faildoctest. - An envelope with an empty payload (
payload_len = 0) ⇒decodeaccepts it,from_bytesreturnsOutOfBoundsfrom reading the root, no panic. → Task 3,decode_empty_payload.
File Structure
| File | Responsibility |
|---|---|
crates/zidl/src/error.rs (mod) |
new variants, #[non_exhaustive] |
crates/zidl/src/crc.rs (new) |
crc32c(&[u8]) -> u32: software slicing-by-8; accelerated via a crate with std |
crates/zidl/src/envelope.rs (new) |
constants, encode, decode, Envelope, envelopes(), schema_id() |
crates/zidl/src/write.rs (mod) |
envelope mode in the Builder; list overflow |
crates/zidl/src/read.rs (mod) |
str_unchecked becomes #[doc(hidden)] |
crates/zidl/src/lib.rs (mod) |
#![deny(unsafe_code)], re-exports |
crates/zidl-schemas/src/lib.rs (mod) |
FromVerified, VListR |
crates/zidl-schemas/src/{bank,game,iot,stream}.rs (mod) |
SCHEMA_ID, verified(), from_bytes(), *Verified views |
crates/zidl-bench/src/zidl_impl.rs, benches/codec.rs (mod) |
trusted via the view; envelope group |
crates/zidl-conformance/ (new crate, publish = false) |
dynamic schema model, interpreter, vector generator, test |
crates/zidl-conformance/schemas/conformance.zidl (new) |
coverage schema (the textual source of truth; Plan 2 compiles it) |
testvectors/v1/ (new) |
frozen .bin + .json, README.md |
docs/spec/zidl-format-v1.md (new) |
normative spec |
docs/benchmarks/2026-11-12-envelope.md (new) |
cost of the envelope |
Task 1: New errors and list overflow in the writer
Files:
- Modify:
crates/zidl/src/error.rs,crates/zidl/src/write.rs(tables,tables_fixed) - Test:
crates/zidl/tests/writer_limits.rs(new)
Interfaces:
-
Produces:
Error::{BadMagic, UnsupportedVersion(u8), BadHeader, Incomplete { needed: usize }, SchemaMismatch { expected: u64, found: u64 }, ChecksumMismatch};Errorwith#[non_exhaustive];Displayfor each. -
[ ] Step 1: Failing tests in
writer_limits.rs:error_display_new_variants:Error::Incomplete { needed: 24 }.to_string()contains"24";SchemaMismatch { expected: 1, found: 2 }shows both in hex (0x1,0x2).#[should_panic(expected = "list too large")] list_over_2gib_panics:TableW::tables(off, u32::MAX, u16::MAX)on a new builder.#[should_panic(expected = "list too large")] fixed_list_over_2gib_panics:tables_fixed::<65535>(off, u32::MAX).#[should_panic] push_beyond_n_panics:tables(0, 1, 4)and twopush()calls.#[should_panic(expected = "sealed frames pending")] flush_with_pending_frames_panics:seal()and thenflush(). Test root:b.root(4)and slot0.
-
[ ] Step 2:
cargo test -p zidl --test writer_limits→ fails (variants do not exist; the list test currently tries to allocate/overflows). -
[ ] Step 3: Add the variants; in
tables/tables_fixedcomputen as u64 * efl as u64andpanic!("list too large: {n} × {efl} bytes exceeds i32::MAX")if> i32::MAX as u64, before theresize. Document in the doc comment ofTableWthat writer panics are programmer errors (push beyondn, a list > 2 GiB, a builder used afterfinish), never caused by data read. (A conscious deviation from the design, which asked forTooLarge:tables()does not return aResultand the size comes from the application's code, not from external data; on 32 bits the computation no longer overflows silently.) -
[ ] Step 4:
cargo test -p zidl→ passes;cargo build -p zidl --no-default-features→ ok.
Task 2: CRC32C
Files:
- Create:
crates/zidl/src/crc.rs - Modify:
crates/zidl/Cargo.toml(crc32c = { version = "0.6.9", optional = true }, featurestd = ["bytes/std", "dep:crc32c"]),crates/zidl/src/lib.rs(pub use crc::crc32c;)
Interfaces:
-
Produces:
pub fn crc32c(data: &[u8]) -> u32(withstddelegates tocrc32c::crc32c; withoutstdusescrc32c_sw);#[doc(hidden)] pub fn crc32c_sw(data: &[u8]) -> u32(software, reflected polynomial0x82F63B78, slicing-by-8 withconsttables computed in aconst fn). -
[ ] Step 1: Tests (in
crc.rsitself,#[cfg(test)]):check_value:crc32c(b"123456789") == 0xE306_9283andcrc32c_sw(...)too.empty_is_zero:crc32c(b"") == 0.sw_matches_accel(proptest,Vec<u8>of 0..5000 bytes):crc32c_sw(&v) == crc32c(&v).
-
[ ] Step 2:
cargo test -p zidl crc→ fails. -
[ ] Step 3: Implement.
-
[ ] Step 4:
cargo test -p zidl crcandcargo test -p zidl --no-default-features crc→ pass.
Task 3: Envelope encode / decode
Files:
- Create:
crates/zidl/src/envelope.rs - Test:
crates/zidl/tests/envelope.rs
Interfaces:
-
Consumes:
crc32c, the errors from Task 1. -
Produces:
pub const MAGIC: [u8; 4] = *b"ZIDL"; pub const FORMAT_VERSION: u8 = 1; pub const HEADER_LEN: usize = 20; pub const TRAILER_LEN: usize = 4; pub const fn schema_id(qualified_name: &str) -> u64; // FNV-1a 64: offset 0xcbf29ce484222325, prime 0x100000001b3 pub fn encode(schema_id: u64, payload: &[u8], out: &mut BytesMut) -> Result<(), Error>; // TooLarge if payload > i32::MAX pub fn decode(buf: &[u8]) -> Result<Envelope<'_>, Error>; #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Envelope<'a> { pub schema_id: u64, pub payload: &'a [u8], len: usize } impl Envelope<'_> { pub fn len(&self) -> usize /* bytes consumed = 24 + payload */ }Layout and check order: exactly §B of the design (
Incompletefor a short header → magic → version → flags≠0 or reserved≠0 ⇒BadHeader→payload_lenfits ⇒ otherwiseIncomplete { needed }→ CRC over[0, 20+N)).neededcomputed inu64/checked_add; if it does not fit inusize,needed = usize::MAX. -
[ ] Step 1: Failing tests in
tests/envelope.rs(helperenc(id, payload) -> Vec<u8>):schema_id_known:schema_id("") == 0xcbf29ce484222325;schema_id("a") == 0xaf63dc4c8601ec8c.roundtrip: decode(enc(7, b"hello")) →schema_id 7, payloadb"hello",len() == 29.layout_bytes:enc(0x0102030405060708, b"")[..20] == b"ZIDL\x01\0\0\0" ++ 08 07 06 05 04 03 02 01 ++ 00 00 00 00.truncated_everywhere: for everyk in 0..enc.len(),decode(&e[..k])isErr(Incomplete{..})(k < 20:needed == 20; otherwiseneeded == 29). Exception: none.bitflip_everywhere: for each bit ofenc(7, b"hello world"),decodereturnsErr—BadMagic(bytes 0..4),UnsupportedVersion(byte 4),BadHeader(5..8),ChecksumMismatch(8..16, payload, CRC), and in 16..20ChecksumMismatchorIncomplete. NeverOk.trailing_bytes_ok: decode(enc ++ b"xyz") → ok,len() == 24 + payload.decode_huge_len_is_incomplete: a valid header withpayload_len = u32::MAXand a 24-byte buffer ⇒Incomplete { needed }withneeded == 20 + u32::MAX as usize + 4(64 bits).decode_empty_payload:enc(1, b"")decodes withpayload.is_empty().- proptest
decode_never_panics(Vec<u8> 0..256)anddecode_mutated_never_ok: mutate 1..4 bytes of a valid envelope (within the firstlenbytes) ⇒ neverOkwith a payload different from the original.
-
[ ] Step 2:
cargo test -p zidl --test envelope→ fails. -
[ ] Step 3: Implement;
pub mod envelope;inlib.rs. -
[ ] Step 4:
cargo test -p zidl→ passes.
Task 4: Copy-free envelope in the Builder and iteration
Files:
- Modify:
crates/zidl/src/write.rs(Builder,begin_msg),crates/zidl/src/envelope.rs(envelopes()) - Test:
crates/zidl/tests/envelope.rs,crates/zidl/tests/allocs.rs
Interfaces:
-
Produces:
impl<'p> Builder<'p> { pub fn new_enveloped(pool: &'p dyn BufPool) -> Self; // reserves HEADER_LEN before each message (instead of 4) pub fn flush_envelope(&mut self, schema_id: u64) -> Result<Bytes, Error>; // header + payload + crc, without copying the payload pub fn seal_envelope(&mut self, schema_id: u64) -> Result<(), Error>; // batch = concatenated envelopes } pub fn envelopes(buf: &[u8]) -> Envelopes<'_>; // Iterator<Item = Result<Envelope<'_>, Error>>; stops at the first ErrRules: the
Builderkeepsprefix: usize(4 or 20). In envelope modefinished()/flush()still return only the payload;seal()(frames) andseal_envelopedo not mix in the same batch: callingseal()on an enveloped builder orseal_envelope/flush_envelopeon a normal builder ⇒panic!("builder mode mismatch")(a programmer error).flush_envelopewrites the header into the 20 reserved bytes, appends the CRC at the end and doessplit().freeze()+advancelikeflush(). -
[ ] Step 1: Failing tests:
builder_envelope_equals_encode: write a raw table (root(6), au32at slot 1 with bit(0,0)) withnew_enveloped;flush_envelope(9)⇒ bytes equal toencode(9, payload_of_the_same_content).batch_of_envelopes: 3 messages withseal_envelope,take_frames(),envelopes(&batch)returns 3Okwith the right payloads and stops.envelopes_stop_at_corruption: in the batch of 3, flip a bit in the payload of the 2nd ⇒ items:Ok,Err(ChecksumMismatch), end.envelopes_truncated_tail: the batch cut in the middle of the 3rd ⇒Ok, Ok, Err(Incomplete{..}).#[should_panic(expected = "builder mode mismatch")] seal_on_enveloped_panics.flush_on_enveloped_returns_payload:flush()on an enveloped builder returns the raw payload (no header).- In
allocs.rs:enveloped_steady_state_zero_allocs— the same pattern as the existing test, withflush_envelope.
-
[ ] Step 2:
cargo test -p zidl→ fails. -
[ ] Step 3: Implement (parametrize
begin_msgbyprefix;seal()keeps writing theu32atms-4..ms). -
[ ] Step 4:
cargo test --workspace→ passes. -
[ ] Step 5: No regression:
cargo bench -p zidl-bench --bench codec -- writeand comparezidl,zidl_batched,zidl_ownedwith round 2: every number within ±5%. If not, investigate before moving on.
Task 5: Verified views and from_bytes (no public unsafe)
Files:
- Modify:
crates/zidl/src/lib.rs,crates/zidl/src/read.rs,crates/zidl-schemas/src/lib.rs,crates/zidl-schemas/src/{bank,game,iot,stream}.rs,crates/zidl-schemas/src/evo/*.rs(only what breaks),crates/zidl-bench/src/zidl_impl.rs - Test:
crates/zidl-schemas/tests/verified.rs(new)
Decision (a deviation from the design's name): a generic Verified<T> in the runtime does not work: methods with trusted access would have to be impls on a foreign type (the orphan rule), and any public FromTable would allow building it without verifying. So each table gets a view XVerified<'a> with a private field, built only by verified()/from_bytes() of its own module. The only unsafe sits in the string accessors of those views (generated code), with // SAFETY: pointing to the verify() that built them.
Interfaces:
-
Consumes:
envelope::{decode, schema_id},VerifyCtx. -
Produces (the pattern Plan 2 will generate):
// zidl #![deny(unsafe_code)] // allow only in read::str_unchecked #[doc(hidden)] pub unsafe fn TableRef::str_unchecked(...) // same body // zidl-schemas/src/lib.rs pub trait FromVerified<'a>: Sized { #[doc(hidden)] unsafe fn from_verified(t: TableRef<'a>) -> Self; } pub struct VListR<'a, T> { /* private */ } // len, is_empty, get(i) -> Result<T, Error>, iter // per root type X (Transaction, Frame, Telemetry, Chunk): pub const SCHEMA_ID: u64 = zidl::envelope::schema_id("<namespace>.<X>"); // e.g. "bank.v1.Transaction" impl<'a> X<'a> { pub fn verified(buf: &'a [u8]) -> Result<XVerified<'a>, Error>; // read + verify(VerifyCtx::new(len)) pub fn from_bytes(buf: &'a [u8]) -> Result<XVerified<'a>, Error>; // decode → schema_id == SCHEMA_ID otherwise SchemaMismatch → verified(payload) } // per table T: TVerified<'a>(T<'a>) with a private field, impl FromVerified, // impl Deref<Target = T<'a>> when T reaches no string; // when it does (bank::Transaction, bank::Entry): its own accessors, strings via str_unchecked, // table lists as VListR<'a, EntryVerified<'a>>, the other accessors delegate.Remove the public
memo_unchecked/note_unchecked.zidl-schemas/src/lib.rs:#![deny(unsafe_code)],#[allow(unsafe_code)]only on the views' string accessors and onfrom_verified. -
[ ] Step 1: Failing tests in
tests/verified.rs:schema_ids:bank::SCHEMA_ID == zidl::envelope::schema_id("bank.v1.Transaction")and the 4 ids are distinct.verified_reads_same_as_safe: proptest with a randomTransaction(notes/memos of arbitrary UTF-8, 0..20 entries): every accessor of the view == the safe accessor.verified_rejects_bad_utf8: a bank message with byte0xFFinside thenotestring ⇒Transaction::verifiedgivesErr(InvalidUtf8).from_bytes_roundtrip:encode(bank::SCHEMA_ID, msg)⇒from_bytesok andid()matches.from_bytes_wrong_schema:encode(game::SCHEMA_ID, bank_msg)⇒Err(SchemaMismatch { expected: bank::SCHEMA_ID, found: game::SCHEMA_ID }).from_bytes_empty_payload:encode(bank::SCHEMA_ID, b"")⇒Err(OutOfBounds).- a
compile_faildoctest onTransactionVerified:let v = zidl_schemas::bank::TransactionVerified(t);does not compile.
-
[ ] Step 2:
cargo test -p zidl-schemas --test verified→ fails. -
[ ] Step 3: Implement. In the bench:
read_all_bank_trustedreceives&TransactionVerified<'_>and criterion builds the view once, outsideb.iter(it measures only the trusted read, as before);read_all_bank_verified(buf)=Transaction::verified(buf)+ reading through the view (measures verify + read). -
[ ] Step 4:
cargo test --workspace→ passes;grep -rn "unsafe" crates/zidl/src crates/zidl-schemas/srcshows onlystr_uncheckedand the views' accessors/from_verified. -
[ ] Step 5:
cargo bench -p zidl-bench --bench codec -- read_all→zidl_trustedandzidl(verified) for bank within ±5% of round 2.
Task 6: The zidl-conformance crate — dynamic schema and interpreter
Files:
- Create:
crates/zidl-conformance/Cargo.toml(publish = false; depszidl,zidl-layout,bytes,serde(derive),serde_json; devproptest),src/lib.rs,src/model.rs,src/interp.rs,schemas/conformance.zidl - Test:
crates/zidl-conformance/tests/interp.rs
schemas/conformance.zidl (the textual source; model.rs describes it by hand and Plan 2 compiles it):
namespace conformance.v1;
enum Color : u8 { Red = 0, Green = 1, Blue = 2 }
struct Point { x: i32, y: i32 }
table Leaf { n @0: u32; s @1: string?; }
union Choice { Leaf @1: Leaf }
table Root {
a_u8 @0: u8; a_i16 @1: i16?; a_u32 @2: u32 = 7;
a_i64 @3: i64?; a_u128 @4: u128?; a_i128 @5: i128?;
a_f32 @6: f32?; a_f64 @7: f64?;
flag @8: bool?; flag2 @9: bool;
color @10: Color?; point @11: Point?;
name @12: string?; blob @13: bytes?;
nums @14: [u16]?; points @15: [Point]?; leaves @16: [Leaf]?;
names @17: [string]?; blobs @18: [bytes]?;
child @19: Leaf?; choice @20: Choice?;
@21: reserved; // was `old @21: u32?`
}
Plus RootV2 = Root + extra @22: u64? (for the evolution vectors).
Interfaces:
-
Produces:
// model.rs pub enum Ty { U8, U16, U32, U64, U128, I8, I16, I32, I64, I128, F32, F64, Bool, Enum { base: Box<Ty> }, Struct(&'static StructDef), Str, Bytes, List(Box<Ty>), Table(&'static TableDef), Union(&'static UnionDef) } pub struct Field { pub id: u16, pub name: &'static str, pub ty: Ty, pub presence: Presence } // Presence::{Required, Optional, Default(Value), Reserved} pub struct TableDef { pub name: &'static str, pub fields: Vec<Field> } // in ID order pub fn conformance_root() -> &'static TableDef; pub fn conformance_root_v2() -> &'static TableDef; // interp.rs #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[serde(untagged or tagged — the builder's decision, documented in the vectors' README)] pub enum Value { Bool(bool), UInt(u128), Int(i128), F32(f32), F64(f64), Str(String), Bytes(Vec<u8>), List(Vec<Value>), Table(BTreeMap<String, Value>), Union { tag: u16, value: Box<Value> } } pub fn read(buf: &[u8], root: &TableDef) -> Result<Value, zidl::Error>; // full read: required fields, defaults, UTF-8, VerifyCtx::new(len) pub fn write(v: &Value, root: &TableDef, b: &mut zidl::Builder) -> Result<(), zidl::Error>; // fields absent from the map = not written pub fn layout(t: &TableDef) -> zidl_layout::TableLayout; // FieldKind from Ty; reserved keeps the original kindWhen reading, absent optional fields are not in the map; an applied default is in it with the default value; an unknown enum is
UInt(raw); unsigned integers →UInt, signed →Int; an unknown union tag isUnion { tag, value: Table({}) }. Floats in JSON by their bits ("f32:0x3fc00000") for exactness;u128/i128as decimal strings — exact, documented in the README. -
[ ] Step 1: Failing tests in
tests/interp.rs:layout_matches_runtime:layout()ofLeaf==compute_layout(&[Scalar{size:4}, Ptr]).roundtrip_all_fields(proptest generating a validValueforRoot):read(write(v)) == normalize(v)(normalize = apply the default ofa_u32).matches_hand_written: bytes fromzidl_schemas::bank(dev-dep) read through aTableDefofbank.v1.Transactiondescribed inmodel.rsgive the same values as the hand-written accessors (proving the interpreter follows the same format).missing_required: aRootwithouta_u8⇒Err(Missing { field: "a_u8" }).random_bytes_never_panic(proptestVec<u8>0..512).
-
[ ] Step 2:
cargo test -p zidl-conformance→ fails. -
[ ] Step 3: Implement. Reading uses only the public API of
TableRef/lists; writing onlyTableW. -
[ ] Step 4:
cargo test -p zidl-conformance→ passes.
Task 7: Frozen test vectors
Files:
- Create:
crates/zidl-conformance/src/vectors.rs(list of cases),crates/zidl-conformance/src/bin/gen-vectors.rs,testvectors/v1/README.md,testvectors/v1/*.bin|*.json - Test:
crates/zidl-conformance/tests/conformance.rs
Interfaces:
- Produces:
pub fn cases() -> Vec<Case>;pub struct Case { pub name: &'static str, pub schema: &'static str /* "conformance.v1.Root" | "...RootV2" */, pub enveloped: bool, pub bytes: Vec<u8>, pub expect: Expect };pub enum Expect { Value(Value), Error(&'static str /* variant name, e.g. "Truncated" */) }. A.jsonfile:{"schema": "...", "enveloped": bool, "expect": {"value": …} | {"error": "…"}}.
Minimum cases (file name = name):
-
valid:
minimal(only required fields;a_u32comes out as 7 by default),all_fields,zero_is_present(every optional scalar = 0, present),empty_lists,streaming_leaves(leaveswritten bytables_streaming: backward offsets),unknown_enum(color = 9),unknown_union_tag(tag 5 pointing to aLeaf),v2_read_by_v1(written withRootV2, read withRoot:extraignored),v1_read_by_v2(extraabsent),reserved_ignored(bytes with slot 21 filled by an old writer: read without error and without the field),enveloped_all_fields. -
invalid (bytes built by hand from
minimal):truncated_slot→Truncated;bad_offset→BadOffset;string_len_oob→OutOfBounds;bad_utf8→InvalidUtf8;missing_required→Missing;self_cycle(childpointing to the root itself) →DepthExceeded;budget_amplification(a list of 1000Leafs all pointing to the same large string) →BudgetExceeded; envelope:env_bad_magic,env_bad_version→UnsupportedVersion,env_bad_header,env_cut_header/env_cut_payload/env_cut_crc→Incomplete,env_bitflip_payload/env_bitflip_crc→ChecksumMismatch,env_wrong_schema→SchemaMismatch. -
[ ] Step 1: Failing test
conformance.rs:vectors_match_disk: for eachCase,testvectors/v1/<name>.bin==case.bytes(deterministic writer) and the.jsondeserializes equal to theExpect.vectors_read_as_expected: for each.binin the directory (not fromcases()), apply a dynamicfrom_bytesequivalent (envelope → schema id ofschema→interp::read) orinterp::readdirectly, and compare with the.json. For an error, compare the variant name (format!("{e:?}")up to{/().no_orphan_files: every file in the directory has a matchingCaseand vice versa.
-
[ ] Step 2:
cargo test -p zidl-conformance --test conformance→ fails (empty directory). -
[ ] Step 3: Implement
cases()andgen-vectors(writes.bin+ pretty-printed.json; refuses to overwrite an existing file with different content unless given--force). Runcargo run -p zidl-conformance --bin gen-vectors. Write the vectors'README.md: what each file is, the JSON format, the rule "a frozen vector never changes; a change = format v2". -
[ ] Step 4:
cargo test -p zidl-conformance→ passes. Runninggen-vectorsagain ⇒ "0 files changed".
Task 8: Normative v1 spec
Files:
- Create:
docs/spec/zidl-format-v1.md
Mandatory sections (normative, MUST/MUST NOT), describing the implemented behavior (check each rule against read.rs/write.rs/zidl-layout; differences from the original design go in a "Differences from the design" section at the end):
- Scope, conventions (LE, no alignment,
i32offsets relative to the slot,0= absent, thei32::MAXlimit). - Table: the
u16 fixed_lenheader, the fixed part, the layout algorithm in pseudocode (a faithful copy ofcompute_layout), slot kinds. - Presence and the 3-step read rule; required × optional × default; reserved.
- Variable-size data: string, bytes,
[scalar],[struct],[table](withelem_fixed_len),[string]/[bytes]. - Enum, union (tag 0 = absent, unknown =
Unknown), frozen struct. - Verification: what a reader MUST reject (the list = the
Errorvariants except the envelope ones), default limits (depth 64,budget max(len*8, 1 MiB)), unreferenced bytes allowed. - Envelope: layout table,
schema_id(64-bit FNV-1a of the qualified name, fixed by the lockfile), CRC32C (polynomial, check value0xE3069283), check order,Incomplete×ChecksumMismatch, concatenation. - Schema compatibility (§6.1 of the design, normative).
- Conformance: an implementation conforms if it passes
testvectors/v1/.
- [ ] Step 1: Write it.
- [ ] Step 2: Cross-check: for every byte example in the spec, there is a vector that covers it (name the vector next to the example).
Task 9: Cost of the envelope and wrap-up
Files:
-
Modify:
crates/zidl-bench/src/zidl_impl.rs,crates/zidl-bench/benches/codec.rs -
Create:
docs/benchmarks/2026-11-12-envelope.md -
[ ] Step 1: An
envelopegroup in criterion: for telemetry, chunk/64, chunk/65536, bank/10:write_enveloped(flush_envelope),from_bytes+ reading one field. Compare withwrite(zidl_owned) andread_all_verified. -
[ ] Step 2:
cargo bench -p zidl-bench --bench codec -- envelopeand the comparison series. -
[ ] Step 3: Report: a table in ns, absolute and relative overhead, CRC throughput (GB/s), same format and machine as round 2.
-
[ ] Step 4: Wrap-up:
cargo test --workspace,cargo build -p zidl --no-default-features,cargo clippy --workspace --all-targetswith no new warnings.