Skip to main content

Module toml

Module toml 

Source
Expand description

TOML codec. Ported from ~/dev/omnist/omnist/formats.py’s read_toml/write_toml/check_toml.

§Crate choice

Like yaml.rs (issue #18), this module delegates raw tokenization to a well-tested crate – [toml_edit] – rather than hand-rolling TOML’s grammar (tables, dotted keys, inline tables, array-of-tables headers, four temporal literal forms, three integer radixes). toml_edit is used read-side only: read_toml walks its parsed DocumentMut into this crate’s canonical crate::document::Value. The writer is hand-written (mirroring json.rs/yaml.rs’s writers), producing TOML text directly from a crate::document::Value rather than round- tripping back through toml_edit’s own formatter – see “Writer output shape” below for why.

Everything omnist-specific stays hand-written Rust, not delegated to the crate:

  • Null-adjustment (strip_nulls) – TOML has no null at all; dropping a null-valued field/array-item and recording it via crate::report::WriteReport is this module’s own logic (see “No null” below).
  • Temporal canonicalization (format_datetime) – turning toml_edit’s parsed Date/Time/Datetime structs into this port’s canonical ISO-string spelling (see “Native temporal types” below).
  • Big integers and overflowing floats – toml_edit holds an integer in 64 bits; this module reads the literals it refuses itself (see “Integers beyond 64 bits” below).
  • Depth guard, shape-check reuse – see their own sections below.

§No null (the one unrepresentable-value case TOML has)

Spec Sec8.3.8/Sec8.3.9 (updated 2026-08-24): writing a node containing a null-valued field now fails the write unconditionally (write.unsupported-value, via [crate::report::unsupported_value_error]) regardless of strict, rather than the old “drop the field and record a warning” behavior. TOML has no null token at all – dropping the edge didn’t just alter what’s represented at that position, it erased the edge’s existence entirely, with zero trace on read-back that a labeled edge with that path was ever there (confirmed live; retired the null.omitted code – see strip_nulls’s doc comment). check_toml still reports the same condition as a write.unsupported-value Severity::Error adjustment for preview purposes, since it never produces output to begin with.

If stripping nulls leaves a document whose root isn’t a table (object), write_toml raises WriteError unconditionally – not part of the report, mirrors Python’s write_toml raising WriteError("TOML needs a top-level table (the root must be an object)") outside of finish_write entirely (so it fires even when report is supplied and even though the message never enters the accumulated WriteReport).

§Integers beyond 64 bits (D-2, D-28) and overflowing floats (D-29)

A TOML integer is arbitrary precision here, in every radix, and the digit limit counts the decimal digits of its value. toml_edit holds an integer in i64 and refuses a longer literal with an “overflow” error that carries a byte span but no value. read_toml therefore reads such a literal itself (recover_number): sign, radix prefix, underscores and all. It replaces the literal in a copy of the text by a stand-in of the same width (0, or 0.0 for a float), parses again, and swaps the real value back in by byte offset when the toml_edit tree is converted. The copy keeps every other offset in place, so a syntax error anywhere in the text is still reported at its own position and still wins over a limit.

A decimal literal already longer than the limit in its own spelling is not converted to a number: the shared [IntGuard] placeholder (10^max) goes in instead, so the Document constructor refuses it with document.limit.int-digits at the right path without the reader paying the superlinear digit-string conversion. Every other literal is converted exactly and checked by value (so 0xFFF is four digits, 4095).

A float literal that overflows f64 (1e999) is refused by toml_edit the same way; it reads as an infinity (str::parse::<f64> gives one). This is a disclosed difference from Python only in that Python’s tomllib exempts hex/octal/binary literals from CPython’s 4,300-digit string-conversion guard, while this port applies D-28 to every radix.

§Native temporal types (the opposite direction from JSON’s problem)

Unlike JSON (no temporal type at all) and like YAML (a native but looser timestamp grammar), TOML has four first-class temporal literal forms (local date, local time, local datetime, offset datetime) that are stricter-shaped than YAML’s – toml_edit’s own parser already fully validates calendar/clock fields (leap years, per-month day counts, valid hour/minute/offset ranges: live-confirmed 2024-02-30, 2024-13-01, 25:00:00, 00:60:00, and a +25:00 offset are all parse errors, not accepted-then-rejected-later), so read_toml does not need to re-validate calendar/clock fields the way yaml.rs’s normalize_timestamp must (YAML’s own crate does no such validation).

crate::document::Scalar has real Date/Time/Datetime variants (issue #105) – toml_value_to_value reads toml_edit’s own already fully-validated Datetime struct’s date/time presence to construct the right one directly (real provenance, not a shape guess), and format_datetime renders the canonical ISO spelling either way: zero-padded, T-joined, a bare Z offset normalized to +00:00 (matching yaml.rs’s identical normalization – this port’s canonical temporal strings never contain a literal Z, only a numeric offset, which is what crate::schema::is_iso_datetime’s regex expects). Fractional seconds beyond microsecond precision are truncated, not rounded, to six digits – live-confirmed against tomllib: 00:32:00.9999999 (7 nines) reads as datetime.time(0, 32, 0, 999999) (truncated, not rounded to 1000000 and carried), matching this module’s nanosecond / 1000 integer-truncating conversion exactly.

UTC-offset preservation (the omnist-ts#51-pattern check this issue calls for): an offset datetime’s numeric offset is preserved exactly in the canonical string (-07:00 stays -07:00across read+write), so this module does not repeat the OML writer’s offset-erasure bug – confirmed by this module’s round_trips_offset_datetime_preserving_negative_offset and _positive_offset tests.

On the write side, a genuine Scalar::Date/Datetime writes as a native TOML temporal literal (unquoted) unconditionally – no shape-check, since the variant itself is the provenance signal (issue #105, the same fix issue #99 already applied to OML). An ordinary Scalar::Str always writes quoted, however date-shaped its text – this now matches Python’s write_toml exactly (previously diverged: Python’s document model retains a real datetime.date/time/ datetime object end-to-end, so a plain str that merely looks like a date – live-confirmed: tomli_w.dumps({'a': '1979-05-27'}) – always wrote as the quoted string a = "1979-05-27", never a native literal; this port’s pre-#105 Scalar had no way to make that distinction, so it wrote bare unconditionally whenever the text merely looked temporal-shaped – a real bug, confirmed live and fixed, not a permitted variation). A Scalar::Time carrying a UTC offset is the one remaining case with no native TOML spelling at all (TOML’s local time literal has no offset field) – see write_scalar’s own has_offset fallback.

§Depth guard reuse

read_toml parses TOML text into a crate::document::Value, then builds a Doc via Doc::of – which calls crate::document::check_write_depth internally (see document.rs). write_toml/check_toml/strip_nulls walk an already-built Doc (via Doc::to_grouped), whose every node was depth-checked at construction time – exactly json.rs’s reasoning (not yaml.rs’s, which pre-processes raw structures before Doc::of ever runs) – there is nothing left to re-guard on the way out, so strip_nulls does not take or check a depth parameter at all.

toml_edit additionally enforces its own, separate recursion cap while parsing – empirically found (see this module’s tests) to reject TOML text nested roughly 81 levels deep (inline tables), well below this crate’s own 200-level MAX_DEPTH. This means a read-side test can only ever observe toml_edit’s own ParseError firing first, never this crate’s DocumentError – the depth-guard-reuse obligation is instead demonstrated the way json.rs/yaml.rs already do, by building an over-deep Value directly and confirming Doc::of rejects it (see deeply_nested_document_write_reuses_doc_construction_depth_guard).

§Writer output shape (architecture freedom, per issue #1)

This module always emits nested tables and table-arrays as inline TOML ({ k = v } / [ v, v ]), never [section]/[[section]] headers. This is a deliberate divergence from tomli_w’s (and most hand-written TOML’s) header-based style, chosen because it is unambiguous, needs no header-nesting state machine, and is fully spec-valid TOML – the “one constraint” from issue #1 is observable behavior (what a round trip produces), not byte-for-byte resemblance to tomli_w’s pretty-printing choices, and inline tables/arrays parse back to an identical Doc either way.

Functions§

check_toml
Report what writing TOML would adjust, without producing output.
read_toml
Parse TOML text into a Doc.
read_toml_with
read_toml under explicit Limits.
write_toml
Project a Doc to TOML text. See this module’s doc comment for the null-adjustment, integer-cap, temporal, and output-shape decisions.