Contents
In brief
An LLM can confidently supply a total or an average even when it has not actually processed all the underlying rows. A Habr case study argues for moving calculation out of the model: let DuckDB and a controlled execution environment handle the data, while the model operates the tools.
The approach combines schema discovery before querying, read-only access, and a sandbox for more complex analysis. It cannot make data work error-free, but it substantially narrows the situations where a model can fill a gap with a plausible guess.
What happened
The author starts with a familiar failure mode: handing a CSV file directly to a model is not a reliable analysis workflow. To an LLM, a table is still text. When a dataset is large, column names are unexpected, or a question is slightly off target, the model may complete a likely-looking answer rather than acknowledge that it lacks the evidence.
Claude Desktop appears to handle large datasets more effectively, although the same question can still lead to different analysis paths. Instead of claiming to recreate the product’s internals, the author extracts a portable idea for an MCP server: the model writes an instruction, while a deterministic environment retrieves and calculates the data.
DuckDB runs arithmetic, aggregations, grouping, and joins from model-authored SQL. When the task needs statistics, forecasting, or transformations that do not fit SQL well, a separate Python sandbox executes the code.
Why it matters
An incorrect number is more dangerous than a vague sentence. It can sound authoritative, make its way into a report, and influence a budget or customer decision. Even a correct database engine cannot help if the model selects the wrong field or aggregates only a truncated result set.
That makes the order of operations important. The model should first inspect a catalog and schema: available datasets, field names, types, missing values, units, and currencies. It can then write a query with facts rather than guessed table or column names.
The tools should also be explicit about their output. A zero-row result can include the actual values that exist in a filter column. A truncated response should say so plainly. In both cases, the model receives evidence to revise the query instead of an invitation to improvise.
In practice
Teams can adopt the core controls without first building a large analytics platform:
- Run arithmetic and aggregations in a database or analytical engine, not in an LLM response.
- Expose a schema-inspection tool before query execution so the model does not invent tables and fields.
- Allow read-only queries only, and validate SQL before execution. Block data mutation, arbitrary file access, and extension loading.
- Execute model-written Python in an isolated environment with a known package set and limited network access.
- Return row counts, truncation status, and useful errors. When a filter finds nothing, help the model compare it with real values.
The boundaries still need review. A simple SQL filter can miss a join that duplicates rows, and a cached schema can become stale. A warning about mixed currencies is helpful, but it is weaker than an enforced rule.
Takeaway
The goal is not to promise that an AI will never be wrong. Reliability improves when code controls calculations, file access, and data state, while the model cannot silently turn missing evidence into a convincing answer.
For teams connecting LLMs to tables, the practical target is traceability: every reported number should lead back to an executed query and a concrete dataset. Boundaries implemented in tools make an answer easier to verify and reproduce.