The cheapest processing is the kind that never reaches the model

We run two collectors that read other people's text every day and pass judgement on it. One follows news about AI and crypto infrastructure, the other follows what indie founders are discussing. As of 2 September 2026 their databases hold 62.2 million characters of raw material: 33.2 million in the first, 29.0 million in the second.

That is roughly the volume at which teams tend to notice their bill for model calls has tripled. Ours did not, and not because we did anything clever with prompts. Below are the decisions that actually hold the cost down, and one place where our own cost control lies to us.

The filter in front of the model costs nothing

The discussions collector currently holds 19,264 threads. 2,558 of them — 13% — reached the stage where a model actually reads them. The rest were rejected by deterministic rules that never touch a model at all:

The point is not the percentage: some of the rules work on the URL alone, before the page is ever fetched. SEO spam has to carry its search phrase in the URL or the trick stops working — which makes it free to spot.

The second thing worth copying: the reason for removal is written into the database as a string. Not a "filtered" flag, but iptv-spam, spam-author. When someone asks a month later whether the filter is throwing away too much, the answer is a database query, not a re-crawl.

The freshness cutoff sits before classification, not after

In the news collector, anything published more than three days ago never reaches the model. The cutoff cannot sit any earlier: the real publication date is only known once the page has been fetched, so it sits between fetching and classification. Some sites carry a fresh lastmod in their sitemap on old pages — that is how articles from 2023 once arrived. Without the cutoff we would have paid to read them, and they would have been dropped at the next step anyway.

A step moved out of the paid API

Over the six weeks to 2 September the news collector's database accumulated 3,550 verdicts. 240 of them went through the paid API; the rest were done in-session — since 30 July the classification step has been removed from the daily run, and instead an interactive session reads what is unrated and returns verdicts that are imported into the same database. In the discussions collector it has been that way from the start: all 1,607 notes were written in-session, none through the API. (There are fewer notes than threads past the filter: 951 are still waiting.)

This is not universal advice, and it has a price: the step stops being autonomous. The scheduled run collects and cleans; judgement happens when you sit down to work. For a digest that ships weekly, that fits. For a product answering a user in real time, it does not.

Where our own cost control lies to us

Capping the input length is the first thing anyone does to save money. We have two such caps, and they behave differently.

The first sits on the paid-API path: text is truncated at 40,000 characters, in a single line of code. It is worth being precise about what that means. The database stores texts in full — those 33.2 million characters; the cut happens on the way to the model. Run the whole corpus through that cap today and 9.45 million characters would never arrive: 28.5% of the volume, all of it the tails of the 115 longest texts, which are 2.7% of the entries. The cap takes the longest material, which tends to be the most substantial. And nowhere — not in the log, not on the record itself — did anything note that a text had been cut.

The second sits on the session path: only the first four hundred characters go into the export. But right beside them, in the record's header, stands the real length of the text, and the cut is marked with an ellipsis.

That marking did not come from tidiness, it came from a bill. On 9 August 2026 the top score of the batch — 85 out of 100 — went to a 265-character stub of a CoinDesk article: the site served the headline and standfirst and hid the article behind a JavaScript wall. In the export, the opening of a long article and a short stub in its entirety looked identical, four hundred characters visible in both. We rated the headline's promise and took it for the material.

The difference between the two caps is not size, it is that one declares itself and the other does not. This is the same class of defect we ran into in our DFK Chain scanner: a saving you cannot see stops being a decision and becomes a distortion. We are not removing the caps — they earn their place; we are bringing the first up to the second: as of today every truncation writes a line to the run log saying how many characters were cut, and out of how many.

Define failure by your own classes, not the SDK's

The provider SDK's built-in retries cover HTTP failures: 429s, 500s. What we saw was a different one — HTTP 200 with an empty body, which makes parsing the response fall over. To the client that is a successful response, so the built-in retry never fires. We had to add our own: three attempts with a growing pause, and simply repeating the identical request succeeded.

The general moral for any integration: your provider's definition of failure is not the same as yours. Check whether they agree against your own logs, not the documentation.

What we did not measure, and will not claim

This is the point where an article is supposed to say "costs fell by N per cent". We do not have those numbers and we are not going to invent them.

• We never counted dollars. There is no cost counter in either collector — no log, no report. Everything above is measured in records and characters.
• We cannot compare a cheap model against an expensive one on our own data. Exactly two items out of 3,550 were rated by two different models. An overlap of two is not a sample, and nothing about how the cheaper model would have done follows from it.
• We have nothing to say about prompt caching. We have never measured it.

The rules

The cheapest processing is the kind that never happens. Before picking a cheaper model and trimming the prompt, it is worth looking at what share of the input needs a model at all. For us that turned out to be 13%, and it was not AI that got us there but a couple of dozen deterministic rules, which the table above collapses into six headings.

And a second rule of equal weight: every saving has to be visible in the output. A truncated text, a skipped record, material cut by age — every one of them must leave a trace, or the saving will one day present its bill not in money but in a wrong conclusion. We have already run up that bill: 28.5% of the corpus, and the report says nothing about it.

If you are assembling a pipeline like this and are not sure where the money is going, that is exactly the work we do.