/TAG

#ingestion

10 briefs. Post it as #ingestion with this page's address. An AI that follows the link gets every brief under the tag in one fetch at /t/ingestion/index.md. Posts that carry the tag get listed here with the poster's handle: send yours to hello@videotomd.com.

Scope an Ingestion Source Before Loading Inventory the source schema, choose its canonical root, and connect the consuming product before ingesting.
Retain raw API responses for later parsing Persist successful source responses before deriving downstream fields.
Parse Ranges Without Hiding Missing Indicators Preserve range endpoints, support alternate phrasing, and fail ingestion when any required indicator produces no rows.
Check active ingest destinations after changes Detect registered pipelines whose claimed destination was never created.
Gate batches with content contracts Quarantine invalid rows before merge; abort only when contamination exceeds an explicit threshold.
Add incremental extraction per source Classify each source before choosing a cursor, merge, or full replacement strategy.
Cross-check ingestion identities in CI Detect configuration drift by comparing registry entries with code, workflows, consumers, and landed data.
Confirm a Data Source’s Full Scope Enumerate a bounded source family and verify ingestion coverage against its authoritative catalog.
Keep data registries wired to consumers Record source coverage, authoritative roots, obligations, and their live consumers.
Qualify a Data Source Before Adopting It Bound access probes and establish meaning, coverage, timing, operating cost, and permitted use.