Data Warehouse, Data Lake, or Just a Better Database? The Architecture Decision Mid-Market Companies Get Wrong
Most growing companies buy a data warehouse before they need one, or avoid one until it is too late. Here is the decision framework we use in our data engineering consulting work before recommending either.
Somewhere around the fortieth data engineering engagement, we stopped being surprised by this: a company with twelve employees and a data warehouse, sitting next to a company with two hundred employees and none. Neither decision was made on evidence. Both were made on what a vendor, a conference talk, or a hire's previous employer happened to use.
The architecture question — warehouse, lake, lakehouse, or simply a well-modeled operational database with a reporting layer on top — is treated as a technology choice. It is actually a business maturity question, and getting it wrong in either direction is expensive. Buy too much infrastructure too early and you are paying six figures a year to query data that would fit in a spreadsheet. Wait too long and you rebuild your entire reporting stack under pressure, usually right when the business can least afford the disruption.
The Question We Ask Before Naming a Product
Before any conversation about Snowflake, BigQuery, Redshift, or a lakehouse pattern on top of object storage, we ask a much less exciting question: how many distinct sources does your business actually need to join together to answer a question a leader asks in a normal week? Not in theory. In the last ninety days.
For a large share of the companies we engage with, the honest answer is three or four sources — a CRM, an accounting system, a product or operations database, and maybe a marketing platform. That is not a data lake problem. That is a modeling and pipeline problem, and it can usually be solved with a properly built warehouse layer at a fraction of the cost and complexity that a "modern data stack" conversation tends to imply.
Three Tiers, Not One Answer
- 1Operational reporting layer: A well-indexed, well-modeled replica of your core systems with scheduled transformations. Right for companies under roughly 50 employees or under 10 recurring cross-system reports. Cheapest to run, fastest to build, and honestly the right answer for more companies than admit it.
- 2Data warehouse: A dedicated analytical store (Snowflake, BigQuery, Redshift, or Postgres-based alternatives) with a real transformation layer (dbt or equivalent) sitting between raw sources and the tables the business queries. Right once you have five or more source systems, need historical snapshots for trend analysis, or have more than two teams building reports independently and stepping on each other's numbers.
- 3Data lake or lakehouse: Right when a meaningful share of your data is unstructured or semi-structured — logs, sensor data, documents, images, event streams at volume — and you need to retain raw data cheaply before deciding how to model it. In our experience, fewer than one in five mid-market companies actually meet this bar, even though it is the pattern most often pitched to them.
The single most common mistake we see is companies adopting lakehouse architecture to solve a problem that was actually a modeling problem. The data was never the bottleneck. The absence of a clean, agreed-upon transformation layer was.
The Cost Nobody Puts In The Slide Deck
Warehouse and lake platforms are usually sold on their query performance and elasticity. What rarely gets mentioned is the ongoing cost of pipeline engineering — the people and process required to keep data flowing correctly, monitored, and documented once it is in place. A platform is not a solution. It is a foundation that still requires disciplined data pipeline engineering on top of it: source connectors that survive schema changes upstream, transformation logic that is tested rather than trusted, and alerting when a pipeline silently stops updating for three days and nobody notices until the Monday report looks wrong.
We have walked into organizations paying for enterprise-tier warehouse compute while running the same eight scheduled queries every morning, and organizations running critical financial consolidation off a single unmonitored script on someone's laptop. Both are architecture failures. Neither is a tooling failure.
Where to Start
Before choosing a platform, we run a short audit with any client considering this decision: map every system that holds business-critical data, count how many of those systems need to be joined to answer your ten most common leadership questions, and check whether your current pain is query speed, data trust, or simply the absence of a single place anyone looks first.
- If the pain is trust ("which number is right?"), the fix is modeling and governance, not a new platform.
- If the pain is speed on genuinely large or unstructured data, a warehouse or lakehouse is justified — but scope it to the specific workloads that need it.
- If the pain is simply that nobody owns the pipeline, no platform migration will fix that until ownership is assigned.
We have never once recommended a platform migration as the first step. We have recommended it as the third or fourth step, after the modeling, ownership, and query patterns were already clear.— Quantivo Inc. SARL, Data Engineering Practice