Expand description
TOML codec. Ported from ~/dev/omnist/omnist/formats.py’s
read_toml/write_toml/check_toml.
§Crate choice
Like yaml.rs (issue #18), this module delegates raw tokenization to a
well-tested crate – [toml_edit] – rather than hand-rolling TOML’s
grammar (tables, dotted keys, inline tables, array-of-tables headers,
four temporal literal forms, three integer radixes). toml_edit is used
read-side only: read_toml walks its parsed DocumentMut into
this crate’s canonical crate::document::Value. The writer is
hand-written (mirroring json.rs/yaml.rs’s writers), producing TOML
text directly from a crate::document::Value rather than round-
tripping back through toml_edit’s own formatter – see “Writer output
shape” below for why.
Everything omnist-specific stays hand-written Rust, not delegated to the crate:
- Null-adjustment (
strip_nulls) – TOML has nonullat all; dropping a null-valued field/array-item and recording it viacrate::report::WriteReportis this module’s own logic (see “No null” below). - Temporal canonicalization (
format_datetime) – turningtoml_edit’s parsedDate/Time/Datetimestructs into this port’s canonical ISO-string spelling (see “Native temporal types” below). - Big integers and overflowing floats –
toml_editholds an integer in 64 bits; this module reads the literals it refuses itself (see “Integers beyond 64 bits” below). - Depth guard, shape-check reuse – see their own sections below.
§No null (the one unrepresentable-value case TOML has)
Spec Sec8.3.8/Sec8.3.9 (updated 2026-08-24): writing a node containing a
null-valued field now fails the write unconditionally
(write.unsupported-value, via [crate::report::unsupported_value_error])
regardless of strict, rather than the old “drop the field and record a
warning” behavior. TOML has no null token at all – dropping the edge
didn’t just alter what’s represented at that position, it erased the
edge’s existence entirely, with zero trace on read-back that a labeled
edge with that path was ever there (confirmed live; retired the
null.omitted code – see strip_nulls’s doc comment). check_toml
still reports the same condition as a write.unsupported-value
Severity::Error adjustment for preview purposes, since it never
produces output to begin with.
If stripping nulls leaves a document whose root isn’t a table (object),
write_toml raises WriteError unconditionally – not part of
the report, mirrors Python’s write_toml raising
WriteError("TOML needs a top-level table (the root must be an object)") outside of finish_write entirely (so it fires even when
report is supplied and even though the message never enters the
accumulated WriteReport).
§Integers beyond 64 bits (D-2, D-28) and overflowing floats (D-29)
A TOML integer is arbitrary precision here, in every radix, and the digit
limit counts the decimal digits of its value. toml_edit holds an
integer in i64 and refuses a longer literal with an “overflow” error
that carries a byte span but no value. read_toml therefore reads such
a literal itself (recover_number): sign, radix prefix, underscores and
all. It replaces the literal in a copy of the text by a stand-in of the
same width (0, or 0.0 for a float), parses again, and swaps the real
value back in by byte offset when the toml_edit tree is converted. The
copy keeps every other offset in place, so a syntax error anywhere in the
text is still reported at its own position and still wins over a limit.
A decimal literal already longer than the limit in its own spelling is not
converted to a number: the shared [IntGuard] placeholder (10^max) goes
in instead, so the Document constructor refuses it with
document.limit.int-digits at the right path without the reader paying
the superlinear digit-string conversion. Every other literal is converted
exactly and checked by value (so 0xFFF is four digits, 4095).
A float literal that overflows f64 (1e999) is refused by toml_edit
the same way; it reads as an infinity (str::parse::<f64> gives one).
This is a disclosed difference from Python only in that Python’s
tomllib exempts hex/octal/binary literals from CPython’s 4,300-digit
string-conversion guard, while this port applies D-28 to every radix.
§Native temporal types (the opposite direction from JSON’s problem)
Unlike JSON (no temporal type at all) and like YAML (a native but looser
timestamp grammar), TOML has four first-class temporal literal forms
(local date, local time, local datetime, offset datetime) that are
stricter-shaped than YAML’s – toml_edit’s own parser already fully
validates calendar/clock fields (leap years, per-month day counts, valid
hour/minute/offset ranges: live-confirmed 2024-02-30, 2024-13-01,
25:00:00, 00:60:00, and a +25:00 offset are all parse
errors, not accepted-then-rejected-later), so read_toml does not
need to re-validate calendar/clock fields the way yaml.rs’s
normalize_timestamp must (YAML’s own crate does no such validation).
crate::document::Scalar has real Date/Time/Datetime variants
(issue #105) – toml_value_to_value reads toml_edit’s own already
fully-validated Datetime struct’s date/time presence to construct
the right one directly (real provenance, not a shape guess), and
format_datetime renders the canonical ISO spelling either way:
zero-padded, T-joined, a bare Z offset normalized to +00:00
(matching yaml.rs’s identical normalization – this port’s canonical
temporal strings never contain a literal Z, only a numeric offset,
which is what crate::schema::is_iso_datetime’s regex expects).
Fractional seconds beyond microsecond precision are truncated, not
rounded, to six digits – live-confirmed against tomllib:
00:32:00.9999999 (7 nines) reads as datetime.time(0, 32, 0, 999999)
(truncated, not rounded to 1000000 and carried), matching this
module’s nanosecond / 1000 integer-truncating conversion exactly.
UTC-offset preservation (the omnist-ts#51-pattern check this issue
calls for): an offset datetime’s numeric offset is preserved exactly in
the canonical string (-07:00 stays -07:00across read+write), so this
module does not repeat the OML writer’s offset-erasure bug – confirmed
by this module’s round_trips_offset_datetime_preserving_negative_offset
and _positive_offset tests.
On the write side, a genuine Scalar::Date/Datetime writes as a
native TOML temporal literal (unquoted) unconditionally – no
shape-check, since the variant itself is the provenance signal (issue
#105, the same fix issue #99 already applied to OML). An ordinary
Scalar::Str always writes quoted, however date-shaped its text –
this now matches Python’s write_toml exactly (previously diverged:
Python’s document model retains a real datetime.date/time/
datetime object end-to-end, so a plain str that merely looks like a
date – live-confirmed: tomli_w.dumps({'a': '1979-05-27'}) – always
wrote as the quoted string a = "1979-05-27", never a native literal;
this port’s pre-#105 Scalar had no way to make that distinction, so
it wrote bare unconditionally whenever the text merely looked
temporal-shaped – a real bug, confirmed live and fixed, not a
permitted variation). A Scalar::Time carrying a UTC offset is the one
remaining case with no native TOML spelling at all (TOML’s local
time literal has no offset field) – see write_scalar’s own
has_offset fallback.
§Depth guard reuse
read_toml parses TOML text into a crate::document::Value, then
builds a Doc via Doc::of – which calls
crate::document::check_write_depth internally (see document.rs).
write_toml/check_toml/strip_nulls walk an already-built
Doc (via Doc::to_grouped), whose every node was depth-checked at
construction time – exactly json.rs’s reasoning (not yaml.rs’s,
which pre-processes raw structures before Doc::of ever runs) –
there is nothing left to re-guard on the way out, so strip_nulls does
not take or check a depth parameter at all.
toml_edit additionally enforces its own, separate recursion cap
while parsing – empirically found (see this module’s tests) to reject
TOML text nested roughly 81 levels deep (inline tables), well below this
crate’s own 200-level MAX_DEPTH. This means a read-side test can
only ever observe toml_edit’s own ParseError firing first, never
this crate’s DocumentError – the depth-guard-reuse obligation is
instead demonstrated the way json.rs/yaml.rs already do, by building
an over-deep Value directly and confirming Doc::of rejects it
(see deeply_nested_document_write_reuses_doc_construction_depth_guard).
§Writer output shape (architecture freedom, per issue #1)
This module always emits nested tables and table-arrays as inline
TOML ({ k = v } / [ v, v ]), never [section]/[[section]] headers.
This is a deliberate divergence from tomli_w’s (and most hand-written
TOML’s) header-based style, chosen because it is unambiguous, needs no
header-nesting state machine, and is fully spec-valid TOML – the “one
constraint” from issue #1 is observable behavior (what a round trip
produces), not byte-for-byte resemblance to tomli_w’s pretty-printing
choices, and inline tables/arrays parse back to an identical Doc
either way.
Functions§
- check_
toml - Report what writing TOML would adjust, without producing output.
- read_
toml - Parse TOML text into a
Doc. - read_
toml_ with read_tomlunder explicitLimits.- write_
toml - Project a
Docto TOML text. See this module’s doc comment for the null-adjustment, integer-cap, temporal, and output-shape decisions.