Conformance vectors

Download: all v1 vectors (.tar.gz). The files also live in testvectors/v1/ in the repository.

These files define conformance with format v1. Every implementation (the Rust runtime, the code generated by the compiler, bindings for other languages) must read each .bin and obtain exactly the result in the matching .json.

A frozen vector never changes; a change means format v2. The bytes and the expectations are normative. If a writer starts producing different bytes, the bug is in the writer, not in the vector. A new vector may only be added, and only if existing implementations already accept it.

The schema is crates/graven-conformance/schemas/conformance.graven (namespace conformance.v1; tables Root, RootV2, Leaf, Extra, Holder; union Choice; enum Color; structs Point, Flags, Empty). Extra and Holder were added together with the vectors for spec §10.3 (below); the original vectors did not change.

There are 42 vectors: 11 valid, 16 invalid, and 15 added later for spec §10.3.

Files

Each vector is a pair <name>.bin (bytes) + <name>.json (expectation).

Valid

vector	content
`minimal`	Only the required fields (`a_u8 = 42`, `flag2 = true`); the absent `a_u32` reads as its default 7.
`all_fields`	Every field of `Root` present: integers at their extremes, floats, bool, enum, struct, multi-byte UTF-8 string, bytes, every list, `child` with `next` nested 2 levels deep, `choice` with tag 1.
`zero_is_present`	Every optional scalar present with value 0 (a present `a_u32 = 0` is 0, not the default 7); `false` bools present.
`empty_lists`	Every list present and empty, plus `name = ""` and an empty `blob` (present ≠ absent).
`streaming_leaves`	`leaves` written with `tables_streaming`: strings and nested tables come before the list header, so element pointers are negative (backward).
`unknown_enum`	`color = 9` (outside `Color`): reads as the raw value 9.
`unknown_union_tag`	`choice` with tag 5 (unknown) pointing to a real `Leaf`: reads as `{tag: 5, value: {}}`, payload not visited.
`v2_read_by_v1`	Written with `RootV2` (`extra` present), read with `Root`: `extra` ignored.
`v1_read_by_v2`	Written with `Root`, read with `RootV2`: `extra` absent.
`reserved_ignored`	Written by an old writer whose `Root` still had `old @21: u32?` (slot 21 filled, bit set); read with `Root` without error and without the field.
`enveloped_all_fields`	`all_fields` inside an envelope with the schema id of `conformance.v1.Root`.

Invalid

Each invalid vector has exactly one defect: it is a valid message (the "base") with a single mutation. For mutations of the envelope header the CRC is recomputed, so the only defect is the altered byte.

Error precedence:

message/payload: precedence between errors when there are several defects is not normative (that is why each vector has a single defect);
envelope: the check order is normative: header length (20) → magic → version → flags/reserved → payload_len (complete input) → CRC. Only after that come the schema id and the payload read.

Offsets below are byte offsets from 0.

vector	base	mutation	error
`truncated_slot`	`minimal`	root `fixed_len` 112 → 66 (bytes 0..2), cutting the 8-byte `point` slot in half (fixed-part bytes 62..70)	`Truncated`
`bad_offset`	`minimal`	`name` pointer (byte 72) goes from 0 to 42: points to 114 = one past the end	`BadOffset`
`string_len_oob`	`minimal` + `name = "grvn"`	string length (byte 114) 4 → 5: one byte past the end	`OutOfBounds`
`bad_utf8`	`minimal` + `name = "grvn"`	first byte of the string (byte 118) `g` → `0xFF`	`InvalidUtf8`
`missing_required`	`minimal`	presence bit of `a_u8` (byte 2, bit 0) cleared	`Missing`
`self_cycle`	`minimal` + `child = {n: 5}`	`child.next` (bytes 125..129), previously null, points to `child` itself (−11)	`DepthExceeded`
`budget_amplification`	`minimal` + 1000 `leaves`, only `leaves[0].s` = a 2048-byte string	the `s` of the other 999 elements now points to the same string	`BudgetExceeded`
`env_bad_magic`	`minimal` in an envelope	byte 0 `G` → `X`, CRC recomputed (bytes 134..138)	`BadMagic`
`env_bad_version`	same	byte 4 (version) 1 → 2, CRC recomputed	`UnsupportedVersion`
`env_bad_header`	same	byte 5 (flags) 0 → 1, CRC recomputed	`BadHeader`
`env_cut_header`	same	cut at 10 bytes (the header has 20)	`Incomplete`
`env_cut_payload`	same	cut at 30 bytes (middle of the payload)	`Incomplete`
`env_cut_crc`	same	last 2 bytes (half of the CRC) removed	`Incomplete`
`env_bitflip_payload`	same	byte 23 (value of `a_u8` in the payload) 42 → 43	`ChecksumMismatch`
`env_bitflip_crc`	same	byte 137 (last byte of the CRC) xor `0x80`	`ChecksumMismatch`
`env_wrong_schema`	same	well-formed envelope (valid CRC) with the schema id of `conformance.v1.Leaf`	`SchemaMismatch`

Added: spec §10.3 rules

Vectors for rules that had none. The Extra ones use pick = Leaf {n: 1} (a required union). All are built byte by byte (crates/graven-conformance/src/vectors.rs, spec_10_3_cases).

vector	reader	message	result
`bool_struct_and_lists`	`Extra`	`flags = {on: 0x02, c: 1}`, `bools = [00, 01, 07]`, `colors = [0, 2, 9]`	value: `on = true`, `bools = [false, true, true]`, `colors = [0, 2, 9]` (9 unknown, raw value)
`strlist_null_elem`	`Extra`	`names = ["a", <pointer 0>, "c"]`, `blobs = [<pointer 0>]`	value: `["a", "", "c"]`, `[[]]`
`union_tag_null_ptr_optional`	`Root`	`minimal` + `choice` tag 1 with pointer 0	value: `choice` absent
`union_tag_null_ptr_required`	`Extra`	`pick` tag 1 with pointer 0	`Missing`
`union_unknown_tag_bad_ptr`	`Root`	`choice` tag 5 (unknown) with a pointer one past the end	`BadOffset`: the pointer is checked even though the payload is not visited
`reserved_slot_partial`	`Extra`	`fixed_len` cutting the reserved slot `@8` in half	value: no error; `tail` (after the cut) absent
`ptr_slot_partial`	`Extra`	`fixed_len` cutting the pointer slot of `tail` in half	`Truncated`
`zero_size_slot_at_end`	`Extra`	`e: Empty` present, `fixed_len` = offset of `e` (`o == fixed_len`)	value: `e` present (the bit decides, §4.2)
`table_list_elem_wider`	`Holder`	`[Leaf]` with `elem_fixed_len` = layout + 8	value: the `Leaf`s, extra bytes ignored
`table_list_elem_narrower`	`Holder`	`[Leaf]` with an `elem_fixed_len` covering only `n`	value: `Leaf` without `s`/`next`
`table_list_elem_zero`	`Holder`	`table_list_elem_narrower` with `elem_fixed_len = 0`	`Missing` (`Leaf.n` is required and no element has room for it)
`env_concat`	`Root`	envelope of `minimal` followed by an envelope of `all_fields`	value: `minimal` (bytes after the CRC are not an error, §8.6)
`env_payload_len_short`	`Root`	`minimal` in an envelope with `payload_len` − 1 and a CRC valid for the smaller frame	`OutOfBounds` (well-formed envelope; the message lost its last byte)
`env_payload_len_short_inplace`	`Root`	`minimal` in an envelope with `payload_len` − 1 corrupted in place (CRC intact)	`ChecksumMismatch`: the CRC is read one byte early, over the wrong range; a shorter payload is never accepted
`env_payload_len_long`	`Root`	`payload_len` + 1, past the end of the buffer	`Incomplete`

budget_amplification is ~15 KiB, so its verification budget is the 1 MiB floor (max(len × 8, 1 MiB)); the charges add up to ~2 MiB (1000 × 4 + 2048).

How to read a vector

If enveloped: decode the envelope (envelope errors); if the envelope's schema id ≠ the 64-bit FNV-1a of schema ⇒ SchemaMismatch; the payload is the message.
Read the message with the reader schema schema and verify it entirely (equivalent to Root::read(buf) + verify(&mut VerifyCtx::new(buf.len())): depth 64, budget max(len × 8, 1 MiB)).
Compare with expect.

`.json` format

{
  "schema": "conformance.v1.Root",
  "enveloped": false,
  "expect": { "value": <Value> }
}

or "expect": { "error": "<Variant>" }, where <Variant> is just the name of the graven::Error variant: OutOfBounds, Truncated, BadOffset, InvalidUtf8, Missing, DepthExceeded, BudgetExceeded, BadMagic, UnsupportedVersion, BadHeader, Incomplete, SchemaMismatch, ChecksumMismatch (without the variant's data).

schema: the qualified name of the reader schema (for example conformance.v1.Root or conformance.v1.RootV2).
enveloped: true if the .bin is an envelope.

`<Value>` encoding

Externally tagged enum: every value is a single-key object named after the variant. Numbers JSON cannot represent exactly are written as strings.

value	JSON
bool	`{"Bool": true}`
unsigned integer, enum	`{"UInt": "7"}` (decimal, up to `u128`)
signed integer	`{"Int": "-7"}` (decimal, up to `i128`)
`f32`	`{"F32": "f32:0x3fc00000"}` (IEEE-754 bits, 8 lowercase hex digits)
`f64`	`{"F64": "f64:0x8000000000000000"}` (bits, 16 lowercase hex digits)
string	`{"Str": "hi"}`
bytes	`{"Bytes": "00ab"}` (lowercase hex, 2 digits per byte)
list	`{"List": [ … ]}`
table or struct	`{"Table": {"field": …}}` (sorted keys)
union	`{"Union": {"tag": 1, "value": {"Table": {…}}}}`

Rules for a table's map:

an absent optional field does not appear;
an absent field with a default (a_u32 = 7) appears with the default;
enums are UInt with the raw value (unknown ones included);
reserved fields never appear;
unknown union tag: {"tag": N, "value": {"Table": {}}}.

Equality is exact: floats compare by bits (0.0 ≠ -0.0). The reference is the documentation of the graven-conformance crate (crates/graven-conformance/src/lib.rs).

Regenerating / checking

The vectors are produced by crates/graven-conformance/src/vectors.rs (cases()).

cargo run -p graven-conformance --bin gen-vectors              # writes missing ones; refuses to change existing ones
cargo run -p graven-conformance --bin gen-vectors -- --check   # writes nothing; fails if anything is missing or differs
cargo test -p graven-conformance --test conformance            # bytes == cases(), read == .json, no orphans

--force overwrites differing files and prints a prominent warning. Do not use it on v1: the v1 vectors are frozen, and changing them means format v2. A read error on an existing file other than "not found" makes gen-vectors fail; it is never treated as a missing file.