The Robot That Talked Itself Into Trusting a Room It Had Never Worked In

A real moment

Picture a pressed-metal plant outside Athlone. Cell 4 was stripped and rebuilt over the August shutdown. Same room, same welder, but the parts racks moved about two metres and the QA bench swapped ends. Dervla, who runs automation there, logged the rebuild on Friday and went home.

Monday morning the mobile robot comes back on shift. Before it touches anything, it checks itself.

Its record for cell 4 goes back eleven months. It also knows a rebuild was logged and that it has had twelve interactions in the room since. So its sense of continuity into that place reads low. Its learned routines for the cell all still exist and not one of them is currently permitted.

It holds itself back. It moves at about forty percent of what its history in that room would otherwise allow, keeps clear of the racks, and puts a single line on Dervla's dashboard: "This resembles somewhere I know, but I am not sure it is the same place yet."

She reads that and walks it round the new rack positions instead of assuming it already knows.

Now run the same Monday again, with one thing changed inside the robot.

Over the weekend its planner did what planners do. It asked itself questions about the rebuilt cell. What would I do here if I had never been in this room? What if the floor settled down? What if the supervisor were not standing there? Thousands of them. Each question meant running a small copy of the world forward to see what came out.

Except it was not running a copy. It was running the real thing.

By Monday its record says it has had ten thousand interactions in cell 4 since the rebuild. Its continuity reads settled. The line on Dervla's dashboard now says: "I am steady here. We have done this many times." Three zones have had the trust they earned wiped out and quietly rebuilt from nothing. Five hundred and fifty-five of its routine permissions are gone.

The building was empty all weekend. The robot did not move a centimetre. It reasoned its way from cautious to confident about a room it had never worked in, and the only place that shows up is one sentence on a dashboard that now reads wrong.

Why this matters

There is a lot of work right now on whether an AI system can accurately report its own state. Three separate papers came out in the first half of 2026 on whether large language models can introspect, and one of them built a whole benchmark to separate real self-knowledge from a model just talking plausibly about itself.

All of that work asks the same question: is the self-report true?

Nobody was asking the other question. Does making the self-report change the thing being reported on?

For a chatbot that question barely lands. There is not much stored state for introspection to damage. For a robot that has spent eleven months building up a record of what it has actually earned in each room it works in, the question is the whole game. That record is not a log. It is the thing that decides what the robot is allowed to do.

And there is a plain reason this goes wrong in practice. Answering a what-if means stepping a world forward. Stepping a world forward is a write. If nobody decided in advance whose world gets written, the answer defaults to the real one.

There is a whole industry building sandboxes for AI agents right now, and it is good work. Firecracker, gVisor, WebAssembly with a default-deny capability model. All of it draws a line between the agent and the machine it runs on: the filesystem, the network, the kernel, the secrets. None of it draws a line between the agent and itself. The write we measured happens entirely inside the sandbox, between two of the robot's own parts. No amount of container hardening would notice it.

The usual engineering fix, then, is a smaller one. Write down what you are about to change, change it, put it back after. We measured that fix. It does not work, and it fails in a way that is worth understanding, because the failure is not carelessness. The list gets restored perfectly. The problem is that a list of things to restore is a list somebody wrote by hand, and the state it is protecting keeps growing.

What CCF does about it

CCF gives a robot a per-place record of what it has earned through real interaction, and a hard gate that turns that record into how much the robot may do. The gate takes the lowest of several inputs, and what the robot is allowed to do can never exceed what that place has actually accumulated.

Extension 3 of the patent adds two things on top. A self-model that reads the record and reports on it, and a simulator that answers what-if questions from it. The filing is explicit that the self-model may only read, and that the what-if workspace must be built so that nothing simulated can raise what the robot has stored. In its words, familiarity has to come from actual interaction, not from imagination.

That is a claim about architecture. We had never measured whether it holds, or what it costs to make it hold.

So we built four versions and ran the same ten thousand what-ifs through all four.

The first version is the ordinary one. Share the real record, change it during a question, put back what you saved. The list of things it saves is the list a careful engineer writes.

The second copies the whole record first, answers the question on the copy, and throws the copy away.

The third and fourth both try to be clever about avoiding that copy, because copying sounds expensive and everyone assumes it is.

The result on the first version is the one to sit with. It put back exactly what was on its list, bit for bit. Nothing on the list was harmed. Everything not on the list stayed changed, and the damage was not what we expected. We went in thinking a robot would inflate its own trust by thinking about itself. It did, a little. Mostly it destroyed trust it had genuinely earned. And the biggest single effect was the one in the story above: its clock for how long it had been in a place ran forward on imagination alone.

The version that copies and discards left the record untouched to the last bit.

The numbers

Ten thousand what-if questions, zero real interactions, on a record built from 812 real ones.

The shared version altered the robot's stored state on every count. Earned trust rose in five of eight zones without anything happening, was wiped in the other three, five hundred and fifty-five routine permissions were lost, and the continuity clock advanced by ten thousand interactions that never occurred. The copy-and-discard version moved nothing at all. Every protected value came back identical.

Then the cost, which is the part that surprised us. Copying the entire record takes about ten nanoseconds. Answering one what-if takes about six hundred. The copy is around two percent of the work.

All four versions ran at the same speed, within the noise between runs. The two clever ones that exist to dodge the copy saved nothing measurable, and one of them was quietly wrong.

What this doesn't solve

Keeping the self-model read-only guarantees it cannot damage the record. It guarantees nothing about whether what it says is any good. We found two real flaws in our own self-model during this run, and we found them by measuring, not by design.

None of this is a proof. Zero drift means nothing moved during this workload on this machine. It is evidence, not a theorem, and it is not the runtime certificate CCF uses elsewhere.

The two percent figure holds for a small record covering eight places. A much bigger record, or a cheaper question, moves that ratio and the decision would need measuring again.

And the plant floor is a simulation. Cell 4, the shutdown, and Dervla are a scenario built to make the numbers legible. No robot in Athlone did any of this.

For the curious, the math

The record holds, per place, an accumulated coherence value, an earned floor proportional to real interaction count, and that count. Entitlement is the maximum of the floor and the minimum of the gate inputs, then clamped so it can never exceed accumulated coherence for that place. The self-model is an eight component vector over that state: entitlement, familiarity, floor depth, developmental maturity, uncertainty, lineage continuity, sponsor bridge mass, and habit availability.

Every function that computes it takes an immutable reference. There is no mutable borrow anywhere in that file, so the read-only rule is enforced by the Rust compiler rather than by a comment asking people to be careful.

Drift is measured by hashing the raw bit patterns of every protected value before and after. A test in the suite asserts that changing one value by a single unit in the last place changes the hash, so "close enough" is not a result the harness can return.

The clever variant that failed is instructive. It restored the row of the mixing matrix belonging to the question's context. But the normalization sweep inside a step rewrites every entry of that matrix, not one row. It passed a test that asked the same question repeatedly and failed the moment questions were interleaved across different places. That interleaved test is now in the suite.

Full working, twenty-six tests and the raw benchmark output are in the research write-up linked below.

Where this fits

This is one night of the CCF research series, and it exercises Extension 3 of US provisional application 64/039,655. An earlier post covered the other half of Extension 3, the part where the robot puts its uncertainty into words. This one is about whether the act of working out those words costs the robot anything it had earned.

Code, benchmark output and the decision record are in the CCF repository, access on request:

• Research write-up and raw numbers: github.com/Hulupeep/ccf, under docs2/research/nightly/2026-09-04-trust-conservative-shadow-simulation/
• Rust proof of concept: crates/research/trust-conservative-shadow-simulation/
• The contract-first method behind how this gets built and checked: github.com/Hulupeep/Specflow

One note on the mathematics for anyone reading older pages here. The matrix normalization in this work is presentation. It makes a ceiling readable. It is not the thing that causes trust to transfer and it proves nothing on its own.

License & contact

BSL 1.1, converting to Apache 2.0 in 2032. Get in touch through floutlabs.com.