mostlyright.core.schema

Declarative schema framework for mostlyright.

Defines the Schema base class, ColumnSpec description records, and SchemaRegistration audit-trail container. These are the shape contracts that the Validator, KnowledgeView, and adapter layers consume.

See docs/design.md §A (canonical schemas), §BB.3 (settlement schema), §BB.4 (audit log API), and §J (source_drift_reason reason-string semantics).

TimePoint will eventually wrap timestamp handling; this module accepts raw datetime objects for now and validates that they are timezone-aware.

Module Attributes

`SCHEMA_ID_GRAMMAR`	`schema.<domain>[.<variant>].vN`, DOT-separated components only.

Functions

`validate_schema_id`(schema_id)	Validate `schema_id` against the canonical grammar; return it unchanged.

Classes

`ColumnSpec`(name, dtype, units, nullable[, …])	Description of a single column in a `Schema`.
`Schema`()	Declarative schema base class.
`SchemaRegistration`(schema, source, …[, _audit])	Audit-trail container produced by `Schema.register(...)`.

class mostlyright.core.schema.ColumnSpec(name, dtype, units, nullable, enum_values=None, notes=”)

Bases: object

Description of a single column in a Schema.

Parameters:
- name (str)
- dtype (str)
- units (str | None)
- nullable (bool)
- enum_values (tuple [str , … ] | None)
- notes (str)

name

Column name as it appears in the canonical (metric) projection.

dtype

One of the canonical dtype tags. "enum" requires enum_values to be populated.

units

Free-form unit label ("celsius", "meters", "kt") or None for dimensionless columns.

nullable

Whether the column may contain NaN/NULL values.

enum_values

Allowed values when dtype == "enum". Must be None for non-enum columns.

notes

Free-form documentation surfaced in catalog metadata.

dtype: `str`

enum_values: `tuple`[`str`, `...`] | `None`

name: `str`

notes: `str`

nullable: `bool`

units: `str` | `None`

mostlyright.core.schema.SCHEMA_ID_GRAMMAR = re.compile(‘^schema\\.[a-z]+(\\.[a-z_][a-z])?\\.v\\d+$’)*

schema.<domain>[.<variant>].vN, DOT-separated components only. The domain component is pure lowercase letters; an underscore may appear WITHIN the optional single variant component but NEVER at a component boundary — so an underscore-hierarchy id like schema.observation_qc.v1 (domain carries the boundary as an underscore) fails LOUDLY and is never silently normalized. The .vN integer version is required. One regex gates source-schema registration AND the exporter; the TS codegen guard (packages-ts/codegen/src/schema-id.ts) shares this grammar so both runtimes reject the same fixtures.

Type: The canonical schema-id grammar (DX-H7 / A12)

class mostlyright.core.schema.Schema

Bases: object

Declarative schema base class.

Subclasses declare:

schema_id — stable identifier ("schema.observation.v1").
columns — ordered list of ColumnSpec.
imperial_renames (optional) — metric → imperial column-name map; columns not mentioned keep their metric name.

The base class is intentionally light: no I/O, no DataFrame coupling. Validator and adapter layers consume columns to perform their work; Schema itself only holds the contract.

classmethod column(name)

Look up a ColumnSpec by its (metric) name.

Raises: KeyError – if name does not name a declared column.
Return type: ColumnSpec
Parameters: name (str)

classmethod column_names(mode=‘metric’)

Return the column names in declaration order for the given mode.

mode="metric" returns the canonical names declared in columns. mode="imperial" applies imperial_renames (columns absent from the rename map keep their metric name).

Raises: ValueError – if mode is not "metric" or "imperial".
Return type: list[str]
Parameters: mode (str)

columns: `ClassVar`[`list`[`ColumnSpec`]]

classmethod from_dataframe(df, source, retrieved_at)

Infer a schema from a DataFrame (BYO data path). Deferred to v0.1.1.

Per docs/design.md Premise 5 (and §”Layer responsibilities”): schemas can be constructed declaratively (subclasses of Schema) OR inferred from a sample DataFrame. The declarative path ships in v0.1.0; the inference path (this method) ships in v0.1.1.

Track at:

Return type: Schema
Parameters:
- df (DataFrame)
- source (str)
- retrieved_at (datetime)

imperial_renames: `ClassVar`[`dict`[`str`, `str`]]

classmethod register(source, retrieved_at, rows)

Records the source and retrieved_at provenance and produces a fresh SchemaRegistration carrying the audit log. The registered event is appended automatically.

NOTE

v0.1.0 is a per-call factory. Every call returns a brand-new SchemaRegistration with an independent audit log; there is no cross-call registry that deduplicates by (schema_id, source) or merges audit events across pulls.

Design §BB.4 describes an audit log that persists with the schema across calls; that persistence model lands in v0.1.1 alongside the catalog adapter, which owns the (schema, source) keying and on-disk storage decisions (parquet metadata, JSON sidecar). Callers in v0.1.0 that need a long-lived registration must hold onto the returned object themselves.

Track at:

Parameters:
- source (str) – Source identity in "<source>.<endpoint>" form ("iem.archive", "awc.live").
- retrieved_at (datetime) – Wall-clock timestamp of the pull that produced the rows. MUST be timezone-aware.
- rows (int) – Row count after schema validation.
Raises:
- TypeError – if source is not str, retrieved_at is not a datetime, or rows is not int.
- ValueError – if source is empty, retrieved_at is naive, or rows is negative.
Return type: SchemaRegistration

schema_id: `ClassVar`[`str`]

class mostlyright.core.schema.SchemaRegistration(schema, source, retrieved_at_min, retrieved_at_max, rows, _audit=)

Bases: object

Audit-trail container produced by Schema.register(...).

Records the (source, retrieved_at range, row count) provenance of a training-time pull and accumulates an append-only audit log. The audit log captures source-drift opt-outs and reproducibility-audit outcomes per docs/design.md §BB.4.

Validator wires audit events in via _append_audit(); this module intentionally does not import Validator (it is implemented elsewhere).

Audit-log seam (contract — read this). The single-underscore prefix on _audit and _append_audit() denotes module-private with one cross-module writer: the Validator (a sibling module) calls _append_audit to record source_drift_allowed and temporal_drift_audit events. Any caller OTHER than Schema.register() or the Validator that invokes _append_audit — or any direct mutation of _audit — is a contract violation: the audit log is meant to be append-only and stamped by trusted writers only. The list is intentionally not frozen so the Validator can append without indirection; this is the explicit seam, documented here so code review can catch unauthorised callers.

Deferred to v0.1.1: volatility_warning (per design §B/§P, surfaced by catalog_search and flagged on schemas whose registered rows fall within the 30-day volatile window of iem.archive) will land as an attribute on this dataclass populated by the catalog adapter at registration time. v0.1.0 carries no such attribute.

Parameters:
- schema (type [Schema ])
- source (str)
- retrieved_at_min (datetime)
- retrieved_at_max (datetime)
- rows (int)
- _audit (list [dict [str , Any ] ])

audit_log()

Return a defensive copy of the chronologically-ordered audit log.

Both the outer list and each inner entry are shallow-copied so callers cannot mutate stored entries (e.g. reg.audit_log()[0]["event"] = "tampered" is a no-op against the underlying log). The list contains dict entries with at minimum event and ts (ISO-8601 UTC) keys. See _append_audit() for the supported event vocabulary.

Return type: list[dict[str, Any]]

retrieved_at_max: `datetime`

retrieved_at_min: `datetime`

rows: `int`

schema: `type`[`Schema`]

source: `str`

mostlyright.core.schema.validate_schema_id(schema_id)

Validate schema_id against the canonical grammar; return it unchanged.

The single enforcement point for DX-H7: an id that does not match SCHEMA_ID_GRAMMAR (e.g. an underscore-hierarchy id like schema.observation_qc.v1, a missing schema. prefix, uppercase, or a missing .vN version) raises loudly — it is NEVER silently normalized.

Raises: ValueError – if schema_id is not a str or does not match the grammar.
Return type: str
Parameters: schema_id (str)

mostlyright.core.schema

Module Attributes

Functions

Classes

class mostlyright.core.schema.ColumnSpec(name, dtype, units, nullable, enum_values=None, notes=”)

name

dtype

units

nullable

enum_values

notes

dtype: str

enum_values: tuple[str, ...] | None

name: str

notes: str

nullable: bool

units: str | None

mostlyright.core.schema.SCHEMA_ID_GRAMMAR = re.compile(‘^schema\\.[a-z]+(\\.[a-z_]*[a-z])?\\.v\\d+$’)

class mostlyright.core.schema.Schema

classmethod column(name)

classmethod column_names(mode=‘metric’)

columns: ClassVar[list[ColumnSpec]]

classmethod from_dataframe(df, source, retrieved_at)

imperial_renames: ClassVar[dict[str, str]]

classmethod register(source, retrieved_at, rows)

NOTE

schema_id: ClassVar[str]

class mostlyright.core.schema.SchemaRegistration(schema, source, retrieved_at_min, retrieved_at_max, rows, _audit=)

audit_log()

retrieved_at_max: datetime

retrieved_at_min: datetime

rows: int

schema: type[Schema]

source: str

mostlyright.core.schema.validate_schema_id(schema_id)

dtype: `str`

enum_values: `tuple`[`str`, `...`] | `None`

name: `str`

notes: `str`

nullable: `bool`

units: `str` | `None`

mostlyright.core.schema.SCHEMA_ID_GRAMMAR = re.compile(‘^schema\\.[a-z]+(\\.[a-z_][a-z])?\\.v\\d+$’)*

columns: `ClassVar`[`list`[`ColumnSpec`]]

imperial_renames: `ClassVar`[`dict`[`str`, `str`]]

schema_id: `ClassVar`[`str`]

retrieved_at_max: `datetime`

retrieved_at_min: `datetime`

rows: `int`

schema: `type`[`Schema`]

source: `str`