Sealion Projects

Peeked

Drag a verifier along an AI agent's backtest and watch the trades that used tomorrow's price deflate the equity curve.

research ยท desktop-firstPublished 17 Nov 2026financeai-agentsbacktestingverification

Interactive prototype

Product pitch

The problem

A person asks an AI assistant whether a simple trading rule would have worked. The assistant returns a confident report and a smooth, rising equity curve. Nothing on screen says whether each trade only used information that existed on the day. A single day of look-ahead is invisible in a chart, yet it can turn an average rule into an impressive one. The person cannot audit code they did not write, so they trust the curve.

The research

FINCOSMOS evaluated four coding agents across 150 sessions, recomputing every reported number by program. No session passed: every session the simulated user approved still failed a point-in-time or recomputation check, usually because the backtest used future data or tested a strategy the user never asked for. This build runs a faithful, narrow reduction of one check: it computes the honest and the one-day-leaky backtest in the browser on a seeded synthetic price series and flags each trade whose fill precedes its signal.

What it is

A mocked-up assistant chat holding one large equity curve. The visitor types a moving-average rule, and a verifier line they drag along the curve sweeps the point-in-time check. Left of the line, the claimed curve deflates into the recomputed one, the gap turns red, and each trade that peeked is marked. Rules can be saved and compared. The assistant's prose is scripted and labelled simulated; the backtests and the check are real computation. It does not use real market data.

Who it's for

Retail investors who already lean on a chatbot for money questions, and the friends who forward them warnings. It is not for professional quants, who have their own pipelines and need real data, transaction costs and walk-forward tests that this demo does not model. It is also not a way to judge a strategy's merit: it shows one specific failure, not whether a rule is good.

How it makes money

Not yet, and that is correct for now: a free checker earns trust faster than a priced one, and the check only matters if people believe it has no angle. The plausible path is a paid verification layer that brokerages or assistant makers embed beside agent reports, charged per verified report. The consumer page stays free so the evidence stays public.

Go to market

Start with retail-investing communities and newsletters that already warn about backtest overfitting, where a draggable demonstration of a one-day leak is something people forward as proof. The first partners are assistant and brokerage teams who need a visible safeguard after the paper's result; the pitch to them is the same page run on their own agent's output.

A focused prototype from Sealion Projects. Anything it saves stays in this browser, on this device.