You pay an auditor to rebuild context. Here is what to hand over up front

An auditor charges for time, and the first few days are spent rebuilding context. Four things we built for our own scanner that reads other people's contracts.

Updated:

A caveat first: we have never had a contract of our own audited. What we have is a tool that reads 166 of other people's contracts — a registry for the now-retired DFK Chain and a scanner that shows what is left at any address. That is the other side of that table: for a month we did exactly what an auditor bills for in the first few days, which is rebuilding context for someone else's code that came with none.

Hence this list of what is worth handing over on day one. We built every item for ourselves, and every item cost us something before we had it.

1. Provenance, not a list of addresses

Our registry is split in two, and that split turned out to be the important part.

The larger part is provenance: a 196 kB file in which every address carries either a link to source documentation plus a note recording that it was verified against the chain, or an explicit "not established". The other is a slim projection of 166 contracts that the code actually reads; it is generated from the first and never edited by hand.

We wrote the rule down early on: every claim resolves to either a source link or a recorded "not established". The second outcome is first-class, and it does appear: the fallback RPC is marked not established because it has no documentation.

What an auditor needs is that first part, the provenance. They can pull an address list from your deployment in ten minutes; what they cannot pull is what justifies each entry, and which entries you could not justify yourself.

2. Audit your own arithmetic before someone audits the code

Two weeks before the chain shut down we sat down to check the formula that computes a user's share of a liquidity pool. We had three suspicions:

  1. the share is computed with eighteen decimals hard-coded;
  2. when the price is known for only one side of the pool, the code doubles that side's value to get the total;
  3. a token's price comes from whichever USDC pool turns up last, with no regard for pool depth.

The data killed two of the three. The eighteen decimals appear in both the numerator and the denominator — the factor cancels, and the share is correct whatever the LP token's decimals are. The price doubling never fired in a month of running: all fourteen documented pairs had a price on both sides, and the skew between sides never exceeded 1%. The caveat has to be said in the same breath: that evidence covers documented pairs only. Undocumented pairs were never tested for it, and that is exactly where the real defect turned out to live.

The real defect was not in the formula but in where its parts came from. For undocumented pairs the reserves were read live from the chain while totalSupply came from a registry cache built weeks earlier. Fresh numerator, stale denominator. Measured against the chain, the average supply drift was 13.2% and the maximum 61%. The arithmetic: live supply had fallen to 39% of the cached figure, so dividing by the stale denominator understates the share by 2.6×. And it understates in the dangerous direction, the one where people never come to claim their funds because they concluded there was little there.

The useful part is not the bug but the order of work: three hypotheses written down before testing, two of which the data disproved. That is the second document to hand over — the list of what you have already checked, and what disproved each one.

3. A report that declares its own gaps

Our report carries a mandatory notCovered field: a list of what the check does not cover — assets held in auction contracts, asset classes with no official guidance, contracts that failed to respond.

The README puts it plainly: a scanner that silently skips a class of contracts and prints a clean report is worse than no scanner. Exactly the same holds for audit preparation. "Here is the test coverage, here is what is outside it and why" saves the auditor the first two days; "here is the test coverage" on its own costs you those two days — they will go looking for the boundary anyway, only now at your expense.

We broke this principle ourselves, and we know what it cost: a defect of our own turned a failed read into "nothing here", and the report came out clean where the check had not run at all.

4. Module isolation and a fixed block

Two technical points, both of which save someone else's time.

Isolation. The check is split into eight modules, and a module that fails adds an error line to the report instead of aborting the whole report. A pipeline where one step in eight fails loses the entire run; one where a single module fails loses only that module.

A fixed block. Every measurement we take is pinned to a specific block number and is therefore reproducible by anyone. That same property catches mistakes: a discrepancy in historical data cannot be blamed on the market moving, because data at a pinned block does not move.

What to assemble, in order

Highest return on effort first; item one is an afternoon, the other three fit into a fortnight:

  1. A list of claims with sources, and an honest "not established" line wherever you could not establish it. This is a document rather than code, and it is the fastest thing here.
  2. A walk through your own assumptions: what you suspected, what you checked, what disproved it. Even if none of the suspicions held — all the more so then.
  3. Explicit coverage boundaries next to the tests.
  4. A reproducible run: fixed block, isolated steps, errors in the output rather than in a log.

None of the four requires touching your contracts. All of it is about letting the person you are paying to find defects spend their time finding defects.

Sergei Palii

Founder, Sepia Software

About me

Read next

All articles