5 min read

The Missing Middle in Data Tooling


Map the data tooling landscape by weight, and it splits into two clumps with a hole in the middle.

On one end are the platforms — Snowflake, Databricks, the warehouses, the orchestration suites, the “AI-powered enterprise data quality” products. They are real and they are powerful, but a lot is demanded before anything is returned: accounts, provisioning, connectors, billing, governance, onboarding. The data is expected to live inside them.

On the other end is a sprawl of tiny browser tools, most of them spun up on DuckDB-WASM in the last two years. SQL playgrounds. CSV-to-JSON converters. Query editors that open in a tab. They are free, private, local, and nearly interchangeable — there are dozens of them, because the technology made them trivial to build.

Between them is a less crowded category: tools for recurring file-level work that does not justify adopting a platform but is too error-prone to repeat by hand.

Two crowded clumps with an empty gap between them: tiny SQL boxes on the light end, enterprise platforms on the heavy end, and the mid-sized layer sitting empty in the middle.

The two ends are crowded; fewer tools address the recurring work between them.

Much of the recurring work is file-sized

The work that eats a week is rarely warehouse-scale and rarely a one-liner.

A vendor sends an extract that does not match last month’s. Finance flags a number and the rows that moved have to be found. Two files that should reconcile do not. A schema drifts and nothing warns anyone. A CSV carries a column that is mostly null, and that should be caught before it breaks a downstream job. None of this needs Spark. None of it needs a cluster, a workspace, or a six-month rollout. But none of it is comfortable to do by hand every single time either.

These jobs are too small for a platform and too repetitive to keep solving manually. They account for a substantial amount of routine engineering and analysis time.

Why the middle stays empty

DuckDB and WASM made local analytical tooling easier to build, as the number of browser SQL editors demonstrates. The remaining scarcity is mostly a product problem.

The wrong user gets designed for. Almost every data tool is built by a data engineer, and engineers build for people like themselves. So the default interface is a SQL box in the center of the screen. Fine for the SQL-literate. But it quietly shuts out the analyst, the finance person, the ops lead, the consultant — everyone who has the messy file and the real question but does not want to write a windowed aggregate to answer it. The tool that would fill the middle has to care about the non-technical user, and that is the product work most engineers skip, because it is not the fun part.

Generic wins on ambition and loses on usefulness. The instinct is always to build the general thing — the universal data quality platform, the everything-workbench. But a generic tool speaks nobody’s language. It does not know what a freight manifest looks like, which columns matter to a property manager, what the standard export from a given industry’s ERP actually contains. The useful middle is mostly vertical, and vertical is unfashionable. It means picking one narrow audience and learning their files cold instead of building the impressive abstract thing.

There is no clean business model, so nothing gets finished. The middle does not monetize like a platform. Seat-based SaaS pricing cannot be charged for “give me the file, I will tell you what changed, then you leave.” So the funded players ignore it — too small — and the indie builders abandon it half-done — no revenue. What is left is a graveyard of weekend projects that nailed the first 80% and never shipped the part that made them trustworthy.

Feature creep drags everything toward the platform end. Every tool that starts in the middle feels the pull. Inspect a file, and then saved projects are wanted. Then history. Then scheduled checks, then notifications, then users to receive them, then permissions for the users, then audit logs for the permissions — and the nimble little tool is now a junior platform fighting the giants on their own turf. Staying in the middle takes discipline, because staying small reads as a lack of ambition rather than a deliberate boundary.

Small workbenches fit this gap

The tools that belong in this gap share a shape. They are workbenches, not platforms. A platform wants to own the workflow; a workbench helps with one job and gets out of the way. The files stay where they are. The work does not get trapped inside somebody’s product.

What the middle is hungry for is sharp, single-purpose tools for the thankless work — file comparison and diffing, schema-drift detection, profiling a messy extract, inspecting Parquet, a validation report that can be handed to someone, a reconciliation check that answers what changed without a project plan. Local-first, so sensitive files never have to be uploaded anywhere. Deterministic at the core, so the output is evidence and not vibes — and where an AI layer exists, it should narrate and summarize, never be the source of truth. And ideally vertical: built for one audience’s real file formats instead of for an abstract “data user” who does not exist.

None of those tools requires a new analytical engine. They require a narrow audience, knowledge of its files, and discipline about where the tool stops.

Why I build in this category

A disclosure is fair here: I build small browser-based data tools at Anatini.dev, so this is not a neutral observation. Shipping them made the constraint concrete. The analytical engine was usually straightforward; the difficult part was resisting saved projects, accounts, notifications, and the other features that would turn a workbench into a small platform.

Platforms have budgets and established buying paths; free SQL editors have a large technical audience. A focused reconciliation or schema-drift tool has neither advantage. DuckDB nevertheless makes that class of local, disposable workbench practical, and I think more of the opportunity lies there than in building another general-purpose query editor.

More on Data engineering The Data Was Already in the Browser →