Introduction

omnist is one canonical data model for JSON, YAML, TOML, XML, and its own native OML, plus a schema language (OSD) to validate and compare shapes over that model. This book is the Rust port’s documentation (the crate is omnist, CLI is omnist-cli).

Start with the Quickstart, then read the user guide for the full model. The CLI reference covers the omnist binary, and each entry under Formats documents one codec’s specific round-trip behavior.

This is the Rust port of omnist. See Python divergences for where the two implementations differ, and Limitations & stability for the current alpha-status caveats.

Quickstart

[dependencies]
omnist = "0.0.1-alpha"

The shortest possible tour – one round trip, one schema, one validation, one inference. Each snippet below is a trimmed version of a real file under omnist/examples/; run it yourself with cargo run --example <name>.

1. Read and write a document

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};
use omnist::formats::json::{read_json, write_json};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ada".to_string()));
fields.insert("age".to_string(), Value::Int(37));
let doc = Doc::of(&Value::Object(fields)).unwrap();

let text = write_json(&doc, Some(2), true, None).unwrap();
// {
//   "name": "Ada",
//   "age": 37
// }

let doc2 = read_json(&text).unwrap();
assert!(doc.eq_doc(&doc2), "round trip must be lossless");
}

2. Validate against a schema

#![allow(unused)]
fn main() {
use omnist::osd::parse_schema;

let schema = parse_schema(
    r#"record Person { "name": string, "age": integer } root Person"#,
).unwrap();
// schema.validate(&doc.root()).ok() == true for the document above
}

3. Infer a schema from example documents

#![allow(unused)]
fn main() {
use omnist::infer::infer;
use omnist::osd::to_osd;

let schema = infer(&samples, "Person").unwrap();
println!("{}", to_osd(&schema, Some(2)));
// record Person {
//   "name": string,
//   "age": integer,
//   "tags" [0,]: string,
// }
// root Person
}

That’s it – a Doc, a Schema, validate(), and infer(). From here:

User guide – the full practical tour, including formats and the CLI.
CLI reference – the omnist binary’s command surface.
Per-format pages – adjustment/lossy-conversion behavior for JSON, YAML, TOML, XML, and OML.
Limitations & stability – this port’s 0.0.x alpha status and known scoping gaps.

Omnist (Rust) – user guide

Omnist gives you one canonical data model for JSON, YAML, TOML, XML, and its own native OML, and a schema language (OSD) to validate and compare shapes over it. This is the Rust port of omnist; its public types (Doc, Schema, error enums) are not identical to Python’s or TypeScript’s by design – see the workflow playbook’s “architecture freedom” note – but the model and every format’s observable behavior are the same.

The two ideas
Documents
OML – the native format
Schemas – OSD
Validation
Schema algebra
Reading & writing other formats
Inferring a schema
The CLI

The two ideas

A Document ([omnist::document::Doc]) is a tree: a node is either a scalar value or an ordered list of labeled edges. “Many” is a label that repeats, not a field pointing to an array.
A Schema ([omnist::schema::Schema]) is built from named record definitions (a closed set of named fields, each with a cardinality). A field’s type is exactly one of seven fixed scalar kinds (optionally nullable, e.g. string?) or a Ref to a named record – never a composition of the two. Refs are how reuse and recursion work.

Documents

[Doc::of] builds a Doc from a [Value] (this crate’s plain-value input type, analogous to a parsed JSON value). An object becomes an edge list; a key whose value is a Value::Array expands into one edge per item (a repeated label) – there is no separate array node type.

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ann".to_string()));
fields.insert(
    "tag".to_string(),
    Value::Array(vec![Value::Str("x".into()), Value::Str("y".into())]),
);
let doc = Doc::of(&Value::Object(fields)).unwrap();
let root = doc.root();

root.labels();                 // ["name", "tag"]
root.count("tag");              // 2 -- "tag" is a repeated label (an array)
root.get_one("name").unwrap();  // a Cursor whose .value() is Scalar::Str("Ann")
root.get("tag");                 // both "tag" cursors, in order
}

Doc::to_grouped() projects a Doc back to a JSON-shaped [Value] (same- label edges become an array); Doc::to_raw()/Doc::from_raw() go through [RawNode], the interleaving-preserving representation OML and XML need (see formats/oml.md).

OML – the native format

OML is omnist’s own serialization format: every Document round-trips through it exactly, with no adjustment ever needed (unlike JSON/YAML/TOML/ XML, which each have some lossy corner – see the per-format pages).

#![allow(unused)]
fn main() {
use omnist::oml::{read_oml, write_oml};

let text = write_oml(&doc.to_raw(), 2).unwrap();
// name: "Ada"
// age: 37
let doc2 = omnist::document::Doc::from_raw(read_oml(&text).unwrap()).unwrap();
assert!(doc.eq_doc(&doc2));
}

Schemas – OSD

OSD (Omnist Schema Definition) is the text language for [Schema]. A quoted token is always a field label; an unquoted identifier is always a schema name (scalar keyword or Ref).

record Person {
    "name": string,
    "age": integer,
}
root Person

omnist::osd::parse_schema parses this text into a Schema; omnist::osd::to_osd writes one back out.

Validation

Schema::validate checks a document’s [Cursor] against the schema’s root type and collects every problem found, not just the first.

#![allow(unused)]
fn main() {
use omnist::osd::parse_schema;

let schema = parse_schema(
    r#"record Person { "name": string, "age": integer } root Person"#,
).unwrap();
let result = schema.validate(&doc.root());
assert!(result.ok());
}

Schema algebra

omnist::ops implements the schema-algebra operations: prune (drop unreachable/unsatisfiable parts), normalize (canonical minimal form), extract (a subschema over a subset of root fields), lint, is_isomorphic, compatible_with/equivalent (subschema relations), and local_signature.

#![allow(unused)]
fn main() {
use omnist::ops::prune;

// `Broken` requires itself and can never be satisfied; `Dead` is
// unreachable from `Root`. `prune` drops both, keeping `Root`.
let pruned = prune(&schema);
assert!(pruned.env().contains_key("Root"));
assert!(!pruned.env().contains_key("Dead"));
assert!(!pruned.env().contains_key("Broken"));
}

Reading & writing other formats

Every format module (omnist::formats::{json,yaml,toml,xml}, plus omnist::oml for OML) exposes a read_*/write_*/check_* triple. write_*/check_* return a [WriteReport] cataloging every adjustment a lossy write had to make (dropped nulls, stringified NaN, …); strict mode turns any adjustment into an error instead of a silent write. See formats/ for what each format actually adjusts, verified against each codec’s own merged PR.

The format registry

Alongside the per-format functions, [Doc::from_format]/to_format/ check_format dispatch by format name through a runtime registry (omnist::formats/register_format/get_format), so a new format can be registered under an arbitrary name at runtime and used everywhere a format name is accepted – not just the five builtins. register_format takes a [Format] (name + read/write closures, plus an optional check); a plugin with no check still works for from_format/to_format, but check_format on it returns a clean error rather than panicking.

#![allow(unused)]
fn main() {
use omnist::document::Doc;
use omnist::{Format, formats, register_format};

let d = Doc::from_format("json", r#"{"a": 1}"#).unwrap();
assert_eq!(d.to_format("yaml").unwrap(), "a: 1");

register_format(Format::new(
    "kv",
    |text| {
        let edges = text
            .split(',')
            .map(|pair| {
                let (k, v) = pair.split_once('=').unwrap();
                (k.to_string(), omnist::document::RawNode::Leaf(
                    omnist::document::Scalar::Str(v.to_string()),
                ))
            })
            .collect();
        Doc::from_raw(omnist::document::RawNode::Edges(edges)).map_err(Into::into)
    },
    |doc| {
        let omnist::document::RawNode::Edges(edges) = doc.to_raw() else {
            return Ok(String::new());
        };
        Ok(edges.iter().map(|(k, v)| {
            let omnist::document::RawNode::Leaf(omnist::document::Scalar::Str(s)) = v else {
                unreachable!()
            };
            format!("{k}={s}")
        }).collect::<Vec<_>>().join(","))
    },
));
assert!(formats().contains(&"kv".to_string()));
assert_eq!(Doc::from_format("kv", "a=1,b=2").unwrap().to_format("kv").unwrap(), "a=1,b=2");
}

Inferring a schema

omnist::infer::infer drafts a record schema that accepts a set of sample Documents, without an allow_any fallback (see limitations.md for why).

#![allow(unused)]
fn main() {
use omnist::infer::infer;

let schema = infer(&samples, "Person").unwrap();
for doc in &samples {
    assert!(schema.accepts(&doc.root()));
}
}

The CLI

The omnist binary (crate omnist-cli) wraps this library as a command-line tool – format, convert, check, validate, infer, and the schema subcommand family. See cli.md for the full command reference.

CLI reference

The omnist binary (omnist-cli, issue #24) wraps the library crate as a command-line tool. This page mirrors the actual clap command surface in omnist-cli/src/lib.rs; every example below is drawn from a real assertion in omnist-cli/tests/cli.rs.

omnist <COMMAND>

Commands:
  format    Canonicalize an OML document (the only format with no other tool for this)
  convert   Convert a document between formats (one in, one out)
  check     Report what writing as --to would adjust, without ever writing
  validate  Check a document against a schema (no schema-directed upgrading)
  infer     Draft a schema from example documents (all the same format)
  schema    Operate on a Schema (OSD)

Every subcommand accepts - for its input file meaning stdin, --output (-o)/stdout for where the result goes, and --json to wrap CLI-level errors as structured JSON on stderr instead of plain text. Exit codes: 0 success, 1 a conformance/adjustment failure the command itself reports, 2 a usage or parse error.

`format`

Canonicalizes an OML document – the only format with no other canonicalization tool.

$ omnist format doc.oml
a: 1
b: "x"

--compact prints single-line OML (; -separated) instead of one edge per line.

`convert`

Converts one format to another in a single pass (--from/--to are required; json, yaml, toml, xml, oml).

$ omnist convert doc.json --from json --to oml
a: 1
b: "x"

--schema <file> materializes the document against an OSD schema before writing (schema-directed deserialization, e.g. numeric-string coercion); --strict turns any lossy-write adjustment into an error instead of a silent write; --report prints the adjustment report to stderr; --result-format {text,json,oml} controls how a report (or --schema outcome) is rendered.

`check`

Like convert, but never writes – reports what a --to write would adjust.

$ omnist check doc.json --from json --to toml
no adjustments

`validate`

Checks a document against an OSD schema (--schema <file>, required). No schema-directed upgrading happens here – validate only checks, convert --schema is what materializes.

$ omnist validate doc.json --from json --schema person.osd
valid

An invalid document exits 1 and prints every collected ValidationError; --json emits {"ok": false, "errors": [{"path", "message", "code"}, ...]} with the same stable code strings schema::ErrorCode::as_str returns ("type-mismatch", "cardinality", …).

`infer`

Drafts a schema from one or more sample documents (--from, required; all samples must be the same format).

$ omnist infer doc.json --from json
record Root {
    "a": integer,
    "b": string,
}
root Root

--allow-any is accepted by the argument parser (matching Python’s surface) but returns a clear “not supported yet” error – this port’s [omnist::infer::infer] has no any-fallback (see limitations.md).

`schema <subcommand>`

Operates on an OSD schema file (all subcommands take a .osd file, - for stdin):

format – canonicalize OSD text.
normalize – canonical minimal form (structurally identical records collapsed to one).
prune – drop unreachable/unsatisfiable records and fields.
is-empty – exit 0 if the schema’s language is empty, 1 otherwise (and prints true/false).
extract --keep <labels> – a subschema over a subset of root fields.
lint [--severity {info,warning}] – structural lint findings.
compatible-with <a> <b> / equivalent <a> <b> – subschema relations between two schema files.

$ omnist schema prune schema.osd   # Dead (unreachable) record is gone
record Root {
    "a": string,
}
root Root

Known scope gaps (library limitations, not CLI bugs)

--arrays: accepted wherever the Python CLI accepts it (OML output paths), but this port’s omnist::oml::write_oml has no arrays mode yet – passing it where it would apply returns a clear “not supported yet” error (exit 2) rather than being silently ignored or faked. Where --arrays has no effect per spec, it is accepted and silently ignored, matching Python.
schema-directed --schema on convert: implemented via omnist::materialize::materialize on the raw node after a schema-less read, mirroring Python’s own _materialize(node, schema) call inside each reader.

Formats

omnist reads and writes five formats through one canonical Doc model: JSON, YAML, TOML, XML, and omnist’s own native OML. Each format page below documents that format’s specific round-trip behavior and any lossy corners it has.

JSON
YAML
TOML
XML
OML

JSON

omnist::formats::json::{read_json, write_json, check_json}. Ported from ~/dev/omnist/omnist/formats.py’s read_json/write_json/check_json; see omnist/src/formats/json.rs’s module doc for the full detail behind every claim below.

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};
use omnist::formats::json::{read_json, write_json};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ada".to_string()));
fields.insert("age".to_string(), Value::Int(37));
let doc = Doc::of(&Value::Object(fields)).unwrap();

let text = write_json(&doc, Some(2), true, None).unwrap();
let doc2 = read_json(&text).unwrap();
assert!(doc.eq_doc(&doc2));
}

The one lossy JSON adjustment: `NaN`/`Infinity`

JSON’s grammar has no token for NaN/Infinity/-Infinity. Writing one of these floats substitutes null and records a temporal.stringified-style adjustment via WriteReport; strict mode turns this into an error instead of a silent substitution.

No native temporal type

This port’s Scalar has no date/time/datetime variant at all (a Rust-specific architecture consequence, not a JSON limitation) – a temporal value is already a Scalar::Str holding its ISO spelling by the time it reaches this codec, so there is nothing left to adjust or report here, unlike Python (which stringifies a live datetime.date/time object and records a temporal.stringified warning).

Integer digit cap and the `i64` representational limit

A JSON integer literal over 4300 digits is a ParseError (mirrors CPython’s sys.set_int_max_str_digits guard, which fires inside json.loads before Python ever sees the value). Because this port’s Scalar::Int is i64 (max ~19 significant digits), any literal over 19 digits fails first with “out of range for a 64-bit integer” – the 4300-digit cap essentially never fires in practice for this port, but is kept anyway for parity with the same guard oml.rs/yaml.rs/toml.rs share. Python’s reference, by contrast, holds arbitrary-precision ints under the 4300-digit cap with no 64-bit ceiling at all – a disclosed Rust-specific representational gap, not a bug.

YAML

omnist::formats::yaml::{read_yaml, write_yaml, check_yaml}. Ported from ~/dev/omnist/omnist/formats.py’s read_yaml/write_yaml/check_yaml; see omnist/src/formats/yaml.rs’s module doc for the full detail.

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};
use omnist::formats::yaml::{read_yaml, write_yaml};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ada".to_string()));
fields.insert("age".to_string(), Value::Int(37));
let doc = Doc::of(&Value::Object(fields)).unwrap();

let text = write_yaml(&doc, true, None).unwrap();
let doc2 = read_yaml(&text).unwrap();
assert!(doc.eq_doc(&doc2));
}

Scalar-tag resolution: PyYAML’s rules, not `yaml_rust2`’s

yaml_rust2’s own resolver only recognizes true/false for booleans (YAML 1.2 core schema). This module ignores that and re-implements PyYAML’s YAML-1.1 implicit-resolver regexes instead, live-checked against PyYAML (this project’s Python reference): yes/no/on/off (and case variants) are also booleans; bare y/n are not and stay strings. Quoted scalars are never auto-typed, matching PyYAML (implicit resolution only applies to plain-style scalars).

Merge keys (`<<`)

yaml_rust2 has no built-in merge-key support; this module implements the YAML merge-key spec directly (an unquoted << key’s value must be a mapping or sequence of mappings, merged in order, explicit keys taking precedence).

No native temporal type, but a looser input grammar than JSON

Like json.rs, Scalar has no temporal variant – a timestamp becomes a Scalar::Str holding its ISO spelling. Unlike JSON, YAML’s timestamp grammar is looser (space-separated date/time, single-digit month/day, a bare Z suffix, no zero-padding); this module normalizes any such spelling to the same canonical, zero-padded, T-joined ISO shape PyYAML’s own datetime.isoformat() would produce – so 2001-12-14 21:59:43.10 -5 round-trips to 2001-12-14T21:59:43.100000-05:00, not its original spelling. A timestamp-shaped string naming a calendar/clock value that doesn’t exist (2024-13-01) is a ParseError, matching PyYAML’s own construction-time failure.

Native `NaN`/`Infinity` – no lossy adjustment here (unlike JSON)

YAML’s float grammar has native .nan/.inf/-.inf tokens, so unlike write_json, write_yaml never substitutes null for a special float. The only adjustment check_yaml ever records is forcing double-quoted style for a string containing U+0085 (NEL), which PyYAML’s default styles would otherwise normalize away as a line break.

Integer digit cap

Same 4300-digit cap as json.rs/oml.rs/toml.rs, applied to a plain decimal integer scalar’s digit run before parsing; the same i64 representational ceiling applies (see formats/json.md).

TOML

omnist::formats::toml::{read_toml, write_toml, check_toml}. Ported from ~/dev/omnist/omnist/formats.py’s read_toml/write_toml/check_toml; see omnist/src/formats/toml.rs’s module doc for the full detail.

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};
use omnist::formats::toml::{read_toml, write_toml};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ada".to_string()));
fields.insert("age".to_string(), Value::Int(37));
let doc = Doc::of(&Value::Object(fields)).unwrap();

let text = write_toml(&doc, true, None).unwrap();
let doc2 = read_toml(&text).unwrap();
assert!(doc.eq_doc(&doc2));
}

No `null` – the one lossy TOML adjustment

TOML has no null at all. Writing a null-valued field drops the field entirely ({"a": 1, "b": null} writes as a = 1\n, no trace of b); a null inside an array drops just that element, shifting later elements down. Each drop is recorded as a null.omitted/Severity::Warning adjustment, live-confirmed against tomli_w.dumps (Python’s reference TOML writer) to match exactly, path-for-path. strict mode raises even though the severity is only Warning.

Integer digit cap, and a disclosed hex/octal/binary divergence from Python

Live-confirmed against tomllib.loads: a decimal integer literal follows the identical 4300-digit sys.set_int_max_str_digits cap json.rs/yaml.rs already apply. Hex/octal/binary literals are a genuine exception in Python – tomllib.loads("x = 0x" + "f" * 10000) parses successfully with no error at all, because CPython’s digit-limit guard explicitly exempts power-of-two bases.

This port does not replicate that decimal/non-decimal split: every radix goes through the same 4300-digit cap, because the underlying toml_edit crate itself enforces a strict i64 range at parse time regardless of radix – there is no path here for an oversized hex/octal/ binary literal to reach Scalar::Int uncapped the way Python’s arbitrary-precision int does. This is a disclosed divergence from Python, not a parity gap to close: Scalar::Int is i64-backed regardless of a literal’s original radix, so an “uncapped hex” path would still have to fail somewhere past 64 bits either way.

Native temporal types, truncated (not rounded) sub-microsecond precision

TOML has four first-class temporal literal forms (local date, local time, local datetime, offset datetime), stricter-shaped than YAML’s – toml_edit itself fully validates calendar/clock fields at parse time (2024-02-30, 25:00:00, and a +25:00 offset are all parse errors, not accepted-then-rejected-later). Fractional seconds beyond microsecond precision are truncated, not rounded to six digits, live-confirmed against tomllib: 00:32:00.9999999 (7 nines) reads as datetime.time(0, 32, 0, 999999), matching this module’s integer- truncating conversion exactly. A numeric UTC offset is preserved exactly across read+write (no offset-erasure bug).

On write, an ISO-shaped Scalar::Str is written as a native, unquoted TOML temporal literal – this diverges from Python’s own write_toml, whose document model retains a real datetime object end-to-end: a plain Python string that merely looks like a date (tomli_w.dumps({'a': '1979-05-27'})) writes as the quoted string a = "1979-05-27", not a native date literal. This is the same, already-accepted consequence of this port’s Scalar-has-no-temporal- variant architecture decision that yaml.rs documents too, not a new bug.

XML

omnist::formats::xml::{read_xml, write_xml, check_xml}. Ported from ~/dev/omnist/omnist/formats.py’s read_xml/write_xml/check_xml; see omnist/src/formats/xml.rs’s module doc for the full detail.

XML needs exactly one document element, so (unlike JSON/YAML/TOML) an example document must be wrapped under a single top-level element.

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};
use omnist::formats::xml::{read_xml, write_xml};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ada".to_string()));
fields.insert("age".to_string(), Value::Int(37));
let mut root = IndexMap::new();
root.insert("person".to_string(), Value::Object(fields));
let doc = Doc::of(&Value::Object(root)).unwrap();

let text = write_xml(&doc, true, None).unwrap();
let doc2 = read_xml(&text).unwrap();
assert!(doc.eq_doc(&doc2));
}

Structural difference: interleaving-preserving, not grouped

Unlike the other three codecs, this module goes through Doc::from_raw/ Doc::to_raw (RawNode), not Doc::to_grouped – XML element order can interleave distinct labels arbitrarily (<b/><c/><b/>), which a JSON-shaped IndexMap can’t represent.

`quick-xml`: no advisory, no DTD/entity-expansion at all

cargo audit against this crate’s full dependency tree (29 crates, including quick-xml 0.41.0) on 2026-07-26 against the RustSec advisory database found zero matches – unlike TS’s port, which carried an unfixable fast-xml-parser advisory (omnist-ts#38). quick-xml also has no DTD/external-entity expansion support at all (only the five predefined XML entities are recognized; an undefined entity is a parse error) – XXE -safe by construction, not by configuration (Python’s read_xml needs defusedxml instead of the stdlib ElementTree for the same protection).

Namespaces: a disclosed simplification

quick_xml runs in non-namespace-aware mode here; a namespaced tag’s local name is taken by stripping a lexical prefix: up to the last :, which coincides with Python’s ElementTree-based behavior for the common case (a declared, in-scope prefix) but does not resolve prefixes through xmlns declarations. Namespaces are outside this issue’s spec.

Scalar coercion, live-checked against Python’s `_coerce`

Confirmed against a live Python interpreter, not assumed:

The empty string reads as "", never coerced.
A case-insensitive, whole-string, untrimmed match against "true"/"false" reads as a bool – " true " (with surrounding whitespace) stays a string.
Otherwise int(text) is tried, then float(text), both after Python-style trimming and accepting a single _ between two digits as a group separator ("1_0" -> 10); text that matches neither stays a string.
float(text) also accepts inf/infinity/nan (any case, optionally signed).
Disclosed representational gap: this port’s Scalar::Int is i64-backed (max ~19 digits); Python’s int is arbitrary-precision. Text between 20 and 4300 digits that Python holds as an exact int falls through here to Scalar::Float instead (the same int-then-float fallback order Python’s _coerce itself uses, not a new divergence).
Not replicated: Python’s int()/float() accept any Unicode decimal digit (e.g. Arabic-Indic digits); this module’s parsers are ASCII-digit-only – a disclosed, narrower-than-Python simplification.

OML

omnist::oml::{read_oml, write_oml}. Ported from ~/dev/omnist/omnist/oml.py (issue #10); see omnist/src/oml.rs’s module doc for the full detail. OML (Omnist Markup Language) is omnist’s own native format: every Document round-trips through it exactly, with no adjustment ever needed – unlike JSON/YAML/TOML/XML, each of which has at least one lossy corner (see their own pages).

#![allow(unused)]
fn main() {
use indexmap::IndexMap;
use omnist::document::{Doc, Value};
use omnist::oml::{read_oml, write_oml};

let mut fields = IndexMap::new();
fields.insert("name".to_string(), Value::Str("Ada".to_string()));
fields.insert("age".to_string(), Value::Int(37));
let doc = Doc::of(&Value::Object(fields)).unwrap();

let text = write_oml(&doc.to_raw(), 2).unwrap();
let raw2 = read_oml(&text).unwrap();
let doc2 = Doc::from_raw(raw2).unwrap();
assert!(doc.eq_doc(&doc2));
}

Core vs. Extended

read_oml implements the OML-Core grammar in full, plus the OML-Extended raw-string ('...') and triple-quoted multiline-string ("""...""") spellings on read only. write_oml only ever emits OML-Core double-quoted strings, matching the Python reference exactly – reading back a raw-string or triple-quoted literal never round-trips to that same spelling, only to an equivalent Core one.

`RawNode`, not `Value` – the only codec (with XML) that needs it

Value::Object’s IndexMap can only represent “repeated label” as a contiguous run (an array under one key). OML must round-trip arbitrary interleaving of repeated labels losslessly – its whole reason for existing – so read_oml/write_oml work over RawNode’s literal edge list, never Value.

Temporal literals: shape-checked, then validated

date/time/datetime literals are recognized by shape in this module’s own scanner, then validated by the same shared crate::schema::is_iso_date/is_iso_time/is_iso_datetime checks schema.rs and every other codec use. A recognized-but-invalid literal (2024-02-30) is a ParseError, never silently accepted. Like every other codec, Scalar has no native temporal variant, so a valid literal becomes a Scalar::Str holding its exact source spelling.

Integer digit cap and `i64`

Same 4300-digit cap (MAX_INT_DIGITS) as json.rs/yaml.rs/toml.rs, and the same i64-backed representational ceiling – see formats/json.md.

`--arrays` is not yet implemented

Python’s write_oml supports an arrays=True mode that collapses runs of same-label edges into [...] array syntax. This port’s write_oml has no such parameter yet – separate, not-yet-ported library work tracked by the CLI’s own --arrays “not supported yet” error (see cli.md).

Python divergences

A single index of every disclosed, live-checked behavioral divergence from the Python reference (~/dev/omnist) found across this port’s merged PR history. Each entry names what Python does, what this port does, the reasoning from the PR that introduced it, and whether it’s a permanent representational choice or something revisitable later.

This page is an index, not a replacement for docs/limitations.md or the per-format pages under docs/formats/ – those already carry the full detail for the i64/temporal/any gaps. Where an entry below is already documented there, this page states it briefly and links out rather than repeating it.

Integer representation: `i64` vs Python’s arbitrary-precision `int`

Python: int is arbitrary precision; a decimal literal is only rejected past sys.set_int_max_str_digits’s 4300-digit cap. Rust: document::Scalar::Int is i64 (max ~19 significant decimal digits), so any literal over that – even one comfortably under the 4300-digit cap – is a ParseError (“out of range for a 64-bit integer”).

Origin: PR #5 (document.rs) chose i64 for the Document model itself and, as a direct consequence, dropped Python’s _check_int_digits guard entirely rather than ship “permanently-dead code with no reachable branch to test” – the guard exists in Python to stop a superlinear str() conversion of a huge int, which can’t happen when the type tops out at 19 digits. PR #11 (oml.rs) then had to reintroduce a 4300-digit cap anyway, but at the text-scanning layer (MAX_INT_DIGITS in oml.rs) rather than the Document model, because OML source text is “the one place an arbitrarily-long digit run can actually reach the parser” before the i64 conversion fails. PR #17 (json.rs) verified live against CPython’s json.loads that the same 4300-digit cap fires independently of the 64-bit ceiling, then applied the identical text-layer cap. PR #23 (xml.rs) found a genuinely different Python behavior for this same gap: Python’s XML _coerce falls through a 20-4300-digit numeral to a float rather than erroring, so the Rust port matches that specific control flow (falls through to Scalar::Float) instead of raising.

Permanent: yes. It’s a direct, disclosed consequence of #5’s Document-model decision (Scalar has exactly five variants, Int is i64), applied consistently rather than special-cased per format. See limitations.md for the format-by-format table.

TOML’s hex/octal/binary digit-cap (corrected finding, PR #21)

Python: tomllib applies the 4300-digit cap to decimal literals (same sys.set_int_max_str_digits mechanism as JSON), but exempts hex/octal/binary (power-of-two-base) literals from that cap entirely – confirmed live: tomllib.loads("x = 0x" + "f"*10000) parses with no error at all.

Rust: applies the same i64 bound to every radix uniformly, via toml_edit’s parser. Any oversized hex/octal/binary literal is rejected the same way an oversized decimal one is.

An earlier draft of PR #21 (and its module doc) incorrectly claimed hex literals hit the identical ValueError as decimal ones in Python – that claim was wrong and was corrected mid-review once live-checked against tomllib directly. The corrected finding: Python’s radix exemption is real, and this port does not replicate it, because Scalar::Int is i64-backed regardless of the literal’s source radix – there’s no representational path for an “uncapped hex” literal past 64 bits in this port regardless of what the cap logic does.

Permanent: yes, for the same reason as the general i64 gap above – revisiting it would require a non-i64 Scalar::Int.

Date-shaped strings as native temporal literals (TOML/YAML) vs Python’s quoted strings

Python: distinguishes a real datetime.date/datetime/time object from a plain str at runtime. Writing a Python str that merely looks like a date (e.g. "1979-05-27") writes back as a quoted string; only an actual datetime object writes as a native literal.

Rust: Scalar has no temporal variant (see limitations.md), so there is no type-level way to distinguish “a string that happens to look like a date” from “a value that was read as a date.” PR #21 (toml.rs) and PR #19 (yaml.rs) both resolve this the same way: any Scalar::Str whose contents match the temporal shape check (schema::is_iso_date/is_iso_time/is_iso_datetime) writes back as a native temporal literal, full stop – a plain string that happens to look like a date becomes a native date literal on write, unlike Python.

Origin quote (PR #21): “writing a plain string that merely looks like a date… produces a native TOML date literal in this port, whereas Python’s write_toml keeps it a quoted string… This is the same already-accepted trade-off yaml.rs documents for YAML timestamps, applied consistently – not a new upstream bug.”

Permanent: yes, unless/until Scalar grows a native temporal variant – which PR #11 and #17 both treat as an open, not-yet-decided architectural question rather than a near-term plan.

Bare time literal round-tripping (OML, PR #11)

Python: parses a bare time literal like 12:00 into a real datetime.time object, then always writes it back via isoformat(), which normalizes to 12:00:00 (seconds always present).

Rust: Scalar stores the literal’s exact source spelling and round-trips it byte-for-byte, so 12:00 stays 12:00.

PR #11’s live Python cross-check (19 OML source strings, every scalar kind) found this as the only difference between Rust and Python output, and characterized it deliberately as “a stronger round-trip guarantee than Python’s own, not a bug against Python’s docs” – no upstream issue was filed.

Permanent: yes, and arguably a Rust-side improvement rather than a gap to close.

OML’s UTC-offset preservation (PR #11) / TOML’s identical case (PR #21)

Both codecs preserve an explicit UTC offset (-07:00/+07:00) exactly on round-trip, and both normalize a bare Z suffix to +00:00 rather than preserving Z literally – a normalization choice inherited from yaml.rs’s normalize_timestamp, applied consistently across format codecs rather than re-decided per format. Not a divergence from Python’s observable output; noted here because it was explicitly cross-checked (the omnist-ts#51-derived regression class) in both PRs.

`infer()`’s API shape: one Python kwarg vs two Rust functions

Python: infer(samples, root_name, allow_any=False) – a single function, allow_any is a keyword argument.

Rust: infer::infer(samples, root_name) always behaves as Python’s allow_any=False path (any ambiguous field is a SchemaError); infer::infer_with_report(samples, root_name, allow_any) is a second function that also accepts allow_any: true and returns every recorded AnyFallback alongside the schema, mirroring Python’s infer_with_report/AnyFallback pair rather than its single infer(..., allow_any=...) surface.

Origin: PR #15 (infer.rs) scoped any out of inference entirely, leaving infer with no allow_any parameter at all – an unforced scoping choice, not something the deferred any governance question required (see limitations.md). Issue #29 later flagged that scoping-out as a mistake and tracked porting any for real. PR #33 (closing #29) fixed it by porting allow_any as a second function, infer_with_report, rather than adding an optional parameter to infer itself – an intentional API-shape split from Python’s single-function-with-kwarg design, flagged during PR #33’s review as a deliberate divergence worth recording rather than silently carrying forward.

Revisitable: yes, in principle – collapsing back to a single function with a defaulted parameter is a compatible API change if ever desired, but the two-function split is deliberate for now.

XML: scalar coercion narrowing (PR #23)

Python: _coerce’s rules, live-checked directly rather than assumed. Two narrowings this port discloses:

A 20-4300-digit numeral (too big for i64, still under the security cap) falls through to Scalar::Float, matching Python’s own int-then-float control flow – covered under the general i64 gap above, not a separate issue.
Unicode decimal digits (e.g. Arabic-indic digits) are not recognized by this port’s coercion – ASCII-digit-only, a narrowing from whatever Python’s coercion accepts.

Permanent: yes, treated as a disclosed narrowing rather than a bug.

XML: DTD/XXE safety by construction vs Python’s `defusedxml` dependency

Python: read_xml requires defusedxml instead of the stdlib ElementTree, specifically to guard against XXE/DTD-expansion attacks – an explicit dependency choice to close a real vulnerability class.

Rust: uses quick-xml 0.41.0 (tokenization only, no serde feature), which has no DTD/external-entity expansion support at all. XXE safety is a structural property of the crate, not a guard that has to be separately verified or maintained.

Not a behavioral divergence in observable output – both are safe against XXE – but a divergence in how that safety is achieved, worth recording because it removes an entire dependency Python’s implementation needs. Permanent (a consequence of the crate choice in PR #23).

YAML: bool tag spellings and calendar-invalid timestamps (PR #19, bugs found and fixed, not divergences)

Two items from PR #19 are not divergences from Python – they were bugs in this port’s own WIP checkpoint, found via live cross-check against PyYAML and fixed to match Python exactly, before merge:

Explicit !!bool tag construction was initially too narrow (only true/false spellings); PyYAML’s bool_values accepts yes/no/on/off case-insensitively regardless of implicit vs explicit tagging. Fixed to match.
A timestamp-shaped scalar naming an invalid calendar/clock value (2024-13-01, 2024-02-30, hour 25, tz +25:00) initially fell back silently to a plain string; PyYAML actually raises and fails the whole document. Fixed to raise ParseError.

Listed here for completeness (issue #39 asked for both), but neither is a live divergence today – both match Python’s behavior as of PR #19’s merge.

Not a divergence: the `any`-type gap and the `i64`/temporal gaps documented elsewhere

Two structural gaps referenced above are covered in full in limitations.md rather than repeated here:

The any-type scoping gap (deferred pending the sibling omnist project’s openness decision, per issue #29 and PR #33) – limitations.md.
The i64 representational ceiling, per format – limitations.md.
The lack of a native temporal Scalar variant – limitations.md.

Summary table

Divergence	Permanent or revisitable	Origin PR
`i64` vs arbitrary-precision `int`	Permanent	#5, #11, #17, #23
TOML hex/octal/binary uncapped in Python, capped here	Permanent	#21
Date-shaped string becomes native literal on write (TOML/YAML)	Permanent (pending a temporal `Scalar` variant)	#19, #21
Bare time literal round-trips exactly instead of normalizing	Permanent (stronger guarantee, not a gap)	#11
`infer()`/`infer_with_report()` split vs single `allow_any` kwarg	Revisitable	#15, #33
XML coercion: over-`i64` numeral falls to float	Permanent	#23
XML coercion: ASCII-only digit recognition	Permanent	#23
XML XXE-safety by construction (`quick-xml`) vs `defusedxml`	Permanent (crate choice)	#23

Limitations & stability

Alpha status: `0.0.x`, per this project’s versioning rule

This is the Rust port’s first feature-complete milestone – all library modules, all five format codecs, the CLI, a fuzzing/oracle harness, and this documentation are now in place (issue #28, the last port-order item per docs/workflow-playbook.md §4). It still ships as 0.0.x. There is no beta until the project’s maintainer explicitly signs off on the scoping decisions below; accumulating features or fixes alone never moves the version past 0.0.x. Treat every public API in this crate as subject to change without a deprecation cycle until that sign-off happens.

The `any`-type scoping gap (deferred, not forgotten)

Python’s schema model has an AnyType/ANY type and an allow_any option several APIs (osd, schema algebra, inference) use as a fallback when a precise type can’t otherwise be resolved. This Rust port’s omnist::schema module (issue #6) deliberately does not implement any – its inclusion in the public API is an explicitly deferred design question pending user sign-off, not an oversight. The same scoping choice threads through every module that would otherwise need it:

omnist::osd still recognizes "any" as a reserved schema-text keyword (so it can’t be used as a record name), but using it as a field’s type returns a clear SchemaError explaining it isn’t supported yet, rather than silently misparsing.
omnist::infer::infer has no allow_any fallback: a label whose samples mix objects and scalars, or whose scalars disagree on kind outside the integer/number subset relation, returns a SchemaError instead of silently degrading to any.
The CLI’s infer --allow-any flag is accepted by the argument parser (matching Python’s surface) but returns the same “not supported yet” error rather than doing something different from plain infer.

Closing this gap is 1.0-gating work, not a bug to fix incrementally – see the sibling omnist project’s own any-openness decision, which this port’s scoping deliberately mirrors rather than resolves independently.

Representational limits from `Scalar::Int` being `i64`

Every format codec shares one representational ceiling this port’s Python and TypeScript siblings don’t have: omnist::document::Scalar::Int is a plain i64 (max ~19 significant decimal digits), not an arbitrary- precision integer. Each format’s own page documents the specific consequence:

formats/json.md, formats/yaml.md – a decimal integer literal over 19 digits (but under the shared 4300-digit security cap) is a ParseError (“out of range for a 64-bit integer”), where Python holds it as an exact arbitrary-precision int.
formats/toml.md – the same cap is applied uniformly to hex/octal/binary literals too, a disclosed divergence from Python (which exempts those radixes from its digit-limit guard entirely).
formats/xml.md – an over-i64 numeral falls through to Scalar::Float instead of erroring, mirroring Python’s own int-then-float coercion fallback, just with a narrower Scalar::Int.

None of this is a bug: it is the direct, disclosed consequence of issue #4’s Document-model architecture decision (Scalar has exactly five variants, Int is i64), applied consistently across every codec rather than special-cased away.

No native temporal `Scalar` variant

Scalar has no date/time/datetime variant at all – every temporal value, in every format, is represented as a Scalar::Str holding its canonical ISO spelling, validated by the shared crate::schema::is_iso_date/is_iso_time/is_iso_datetime checks. This means:

omnist::infer::infer never infers date/time/datetime from a string-shaped sample – every string sample infers as string. A schema wanting a temporal field must be authored (or edited in after inference), not inferred.
Formats with a native temporal type on the wire (TOML’s four temporal literal forms, YAML’s looser timestamp grammar) still round-trip through a Scalar::Str internally; see each format’s own page for exactly what that means for their write-side behavior (particularly formats/toml.md’s divergence from Python’s live-datetime-object model).

Architecture-freedom disclosures already made per codec

Beyond the two structural gaps above, each format module documents its own disclosed, live-checked divergences from the Python reference (namespace resolution in XML, ASCII-only digit parsing in XML’s coercion, strict vs. non-strict OML-Extended string spellings, and more) – see formats/ for the specifics, all checked against a live Python interpreter or the Python reference’s own merged PR history, not assumed from memory.

Keyboard shortcuts

omnist