Expand description
Schema inference: draft a record Schema that accepts a set of
sample Documents.
Ported from ~/dev/omnist/omnist/infer.py (issue #14). Given one or
more sample Documents, infer drafts a record schema that accepts
them:
- a label present in every sample with count 1 becomes a required field
(
[1,1]); absent in some samples ->[0,1]; seen more than once -> an array ([min, None], permissive on length); - scalar children become one
crate::schema::Scalar(nullable if any sample wasnull). Samples disagreeing on scalar shape raise, exceptinteger/numbermixing, which collapses tonumber(the one subset relation between scalars – seedocs/design/model.md); - object children become a nested, named
record(recursively).
Since the model has no inline records, nested records are given generated names derived from their label.
infer deliberately does not auto-normalize: the raw result keeps
a 1:1 correspondence between sample labels and generated record names,
which may therefore contain structurally-identical duplicate records.
Call crate::ops::normalize on the result where a canonical minimal
schema is wanted (issue #12, itself resolving issues #143/#151 in the
Python reference).
§allow_any and AnyFallback
infer keeps its original two-argument signature and always infers
with allow_any: false (matching its long-standing behavior). Two
scenarios can’t be resolved to one precise type from the samples alone:
- a label whose samples mix objects and scalars;
- a label whose scalar samples disagree on kind in a way that isn’t the
integer/number subset relation (e.g.
stringandbooleanseen under the same label).
With allow_any: false (via infer, or infer_with_report called
that way), both scenarios are a SchemaError. infer_with_report
additionally accepts allow_any: true, matching Python’s
infer_with_report/AnyFallback: instead of erroring, the field is
opened as crate::schema::FieldType::Any and one AnyFallback is
recorded (location is RecordName.label; reason says why).
§No native temporal input
crate::document::Scalar has no date/time/datetime variant (see
document.rs’s module doc), so unlike the Python reference (which can
receive real datetime.date sample values), every string sample here
infers as string – never date/time/datetime – regardless of
its shape. A schema wanting a temporal field has to be authored (or
edited in after inference), not inferred from string-shaped samples.
This is a deliberate architecture consequence of issue #4’s Value model,
not a bug.
Structs§
- AnyFallback
- A single field
infer_with_reportopened asanyunderallow_any: true.locationreadsRecordName.label;reasonsays why the field could not be given a single precise type. Mirrors Python’sAnyFallbackdataclass.
Functions§
- infer
- Infers a
recordSchema(rooted atroot_name) that accepts every sample insamples. Every sample’s root must be an object (a record shape) – an emptysampleslist, or any sample whose root is a bare scalar, is aSchemaError. Always infers withallow_any: false– seeinfer_with_reportfor theallow_any: truevariant. - infer_
with_ report - Like
infer, but also takesallow_anyand returns everyAnyFallbackrecorded along the way (empty whenallow_anyisfalse, since every ambiguous field is a hard error in that mode instead). Mirrors Python’sinfer_with_report.