Skip to main content

Module infer

Module infer 

Source
Expand description

Schema inference: draft a record Schema that accepts a set of sample Documents.

Ported from ~/dev/omnist/omnist/infer.py (issue #14). Given one or more sample Documents, infer drafts a record schema that accepts them:

  • a label present in every sample with count 1 becomes a required field ([1,1]); absent in some samples -> [0,1]; seen more than once -> an array ([min, None], permissive on length);
  • scalar children become one crate::schema::Scalar (nullable if any sample was null). Samples disagreeing on scalar shape raise, except integer/number mixing, which collapses to number (the one subset relation between scalars – see docs/design/model.md);
  • object children become a nested, named record (recursively).

Since the model has no inline records, nested records are given generated names derived from their label.

infer deliberately does not auto-normalize: the raw result keeps a 1:1 correspondence between sample labels and generated record names, which may therefore contain structurally-identical duplicate records. Call crate::ops::normalize on the result where a canonical minimal schema is wanted (issue #12, itself resolving issues #143/#151 in the Python reference).

§allow_any and AnyFallback

infer keeps its original two-argument signature and always infers with allow_any: false (matching its long-standing behavior). Two scenarios can’t be resolved to one precise type from the samples alone:

  1. a label whose samples mix objects and scalars;
  2. a label whose scalar samples disagree on kind in a way that isn’t the integer/number subset relation (e.g. string and boolean seen under the same label).

With allow_any: false (via infer, or infer_with_report called that way), both scenarios are a SchemaError. infer_with_report additionally accepts allow_any: true, matching Python’s infer_with_report/AnyFallback: instead of erroring, the field is opened as crate::schema::FieldType::Any and one AnyFallback is recorded (location is RecordName.label; reason says why).

§No native temporal input

crate::document::Scalar has no date/time/datetime variant (see document.rs’s module doc), so unlike the Python reference (which can receive real datetime.date sample values), every string sample here infers as string – never date/time/datetime – regardless of its shape. A schema wanting a temporal field has to be authored (or edited in after inference), not inferred from string-shaped samples. This is a deliberate architecture consequence of issue #4’s Value model, not a bug.

Structs§

AnyFallback
A single field infer_with_report opened as any under allow_any: true. location reads RecordName.label; reason says why the field could not be given a single precise type. Mirrors Python’s AnyFallback dataclass.

Functions§

infer
Infers a record Schema (rooted at root_name) that accepts every sample in samples. Every sample’s root must be an object (a record shape) – an empty samples list, or any sample whose root is a bare scalar, is a SchemaError. Always infers with allow_any: false – see infer_with_report for the allow_any: true variant.
infer_with_report
Like infer, but also takes allow_any and returns every AnyFallback recorded along the way (empty when allow_any is false, since every ambiguous field is a hard error in that mode instead). Mirrors Python’s infer_with_report.