Duckdb
Posts tagged “Duckdb”.
- DuckLake on One Machine
I put a small DuckLake, DuckDB, and marimo data platform through one complete job, from 200,000 to 100 million events.
- The Data Was Already in the Browser
WireData turns JSON responses and page tables into local SQL datasets, but the real design problem was preserving consent, bytes, and lineage.
- Fabric Without Spark
I ran an NYC taxi medallion pipeline in Fabric's Python notebook on an F2 capacity, using Polars, DuckDB, pandas, and delta-rs without Spark.
- The Limits I Built Into willitload
willitload checks file structure without semantic guesses, distinguishes name- and position-bound loads, and tests itself against adversarial files.
- The Files That Break Your Bulk Load
A field guide to schema drift, inconsistent file sets, malformed CSVs, and a small preflight check that identifies nonconforming files before a bulk load.
- Keep the Runner Dumb
A single-file SQL runner stays reliable by refusing to parse SQL, hold state, or schedule work—the boundaries small pipelines often need.
- The Missing Middle in Data Tooling
Between enterprise platforms and one-off query tools lies a missing class of focused utilities for reconciliation, schema drift, and file comparison.