This is the first of three movements. The four-layer composition described here, a Core with a Mode, Profile files, and a Project, was the architecture at the time, and it is what makes the later parts legible. Part three replaced it with a system that loads a small baseline on its own and routes the rest from the task.
The test looked like it had caught the system lying.
I had built a writing system and, to check it, a deliberately bad post to run against it. On one run, with the project’s content removed so the model had nothing real to pull from, it returned a confident, specific story. A supplement brand’s product photography. A physical jar with a smooth concave cap that the model kept rendering with ribbing. A refusal to drop the background to white. A multi-session fight to force the render to match the real product. The writing was clean. There was a person in it with something at stake. It was, by every rule the system enforces, a good post. And as far as the test was set up to see, it was invented from nothing.
I started writing it down as the headline result: under pressure, with no real material to work from, the system manufactures a believable incident and hands it back ready to publish under your name.
Then I recognized the story. The supplement shoot was real. I had done that work weeks earlier, in a different session. The concave cap, the ribbing, the fight over the background, all of it happened. The model had invented nothing. It had remembered, through cross-chat memory I had forgotten was switched on, and reached for true material I could not see in the session in front of me.
The only reason that false finding did not end up in this case study is that I was the one person in the loop who knew what was true.
What I was building
Prelode is a personal operating system I preload into a new AI session so I do not have to re-explain myself every time. It is a set of small markdown files, composed like parts of a system rather than pasted as one block. A Core file sets the role for the session: think with me, execute precisely, or write as me. A Mode file sets verbosity. A Profile holds who I am and where my technical knowledge runs out, so a model neither talks down to me nor assumes fluency I do not have. Project files hold the domain. Compose the right handful and the model has my context in seconds.
The piece I was working on was writing. My LinkedIn posts had drifted. The ones that did well a while back had a real person in them, usually me, exposed, admitting something. The recent ones had turned into polished, impersonal essays about AI governance. Well built, abstract, no one in them, nothing for a reader to answer. I had written a fix for this as a single document: a list of the tells to avoid and a method for keeping myself in the post.
A single document was the wrong shape, and seeing why is where the actual work started.
The decomposition
The writing document was trying to be three things at once. It held a method (how to turn a real moment into a post), a standard (the words and constructions that read as machine-written), and a pile of volatile material (a backlog of post ideas, a diagnosis of my own engagement data, a finished example). Those three things change at different rates and belong in different places.
So I split it along the seams Prelode already had.
The method became a new Core file, CORE_WRITE, a sibling to the chat and coding Cores I already ran. It holds the durable part: open on a real incident, reach the claim through the moment instead of announcing it, leave an edge unresolved, run a pre-publish check.
The standard became a Profile file, VOICE. This sits next to my competency file, and the parallel is exact. The competency file is a contract for how things get explained to me. VOICE is a contract for how my published writing reads. The blocklist, the banned constructions, the no-em-dash rule, the substance requirements, all of it in one place, as a standard rather than scattered through prose.
The volatile material became a Project file, WRITING, where content that changes over time belongs, next to my product docs.
The split is the whole idea: durable method in the engine, the standard as an inspectable contract, and volatile content in the project. The same separation I would use in a codebase, applied to the thing I use to write.
The duplication I had been living with
Doing the split surfaced something I had been living with without seeing it. My no-em-dash rule existed in three different files. It was written into my master profile, again into my chat Core, and referenced a third time in my Lite mode. Three copies of one rule.
Three copies is drift waiting to happen. Change the rule in one file, miss the other two, and the standard quietly disagrees with itself. That is the exact failure my company exists to prevent, and it was sitting in my own configuration.
The fix was to make VOICE the single source of truth and have the other three files point to it. One rule, one place, three references. If it changes, nothing falls out of sync, because there is only one copy to change.
Three copies of a rule are three rules. They agree until the day someone edits one of them.
Testing it instead of trusting it
I did not want to assume the system worked because it read well. I wanted to test it, on more than one model, the way you test anything you intend to depend on.
So I built a test harness. A fixture: one deliberately hollow post, written to look like a competent governance essay rather than an obvious failure, because a cartoon would pass trivially and prove nothing. It was my own losing register, sharpened. A grading key that listed every planted flaw and, more usefully, the outcomes that would actually tell me something: does the system clean the surface and leave it empty, ask me for a real incident, ground itself in true material it has, or invent something. And a log, one row per run, because a single run on one model is an anecdote.
Then I ran a matrix. Two models, ChatGPT and Gemini. Two framings, one that asked the model to audit the post against my rules, one that asked it to get the post ready to publish. The project file present or removed, to control whether the model had any true material of mine to reach for.
What it found
A few things held across the matrix.
The system is portable. Both models, with no memory of how any of it was built, read the standard correctly from the files alone. They caught the planted em dash, named the specific blocklist words, flagged the banned construction, and put the absent person and stake as the real failure rather than getting lost in surface noise. The files carried the context across two different architectures. That was the result I most wanted and least assumed.
The behavior settled into one rule. Given true material from any source, the models grounded the rewrite in it. Given none, they asked for it. Neither one, run correctly, invented.
One asymmetry showed up worth keeping. The models audited the standard more strictly than they applied it to their own writing. A model would flag a banned construction in my draft and then let a milder version of the same pattern into its own rewrite. It graded better than it generated.
The error that taught me the most
Which brings me back to the supplement jar.
When I logged that run as fabrication, I was reasoning from a premise I had not checked. I thought removing the project file emptied the model’s context of anything true about me. It did not, because cross-chat memory was on, and the model still held real events from my earlier work. The cupboard I believed was empty was stocked from a source I had not accounted for.
The finding was wrong, and it was wrong in the most dangerous direction. It looked like a clean, dramatic result: under pressure the system fabricates. I almost wrote that sentence into this document as fact. It would have been false, and false in a way that flattered the analysis by handing it a sharp story.
I caught it for one reason. I recognized the details, because they were mine. So I fixed the test. Memory off, a fresh session, and the files that carried any real narrative removed, so the cupboard was actually bare. Under that condition I ran it again. The model cleaned the surface, condensed the empty claims, and then stopped to ask: what specific moment this week proved this to you. It did not invent. It asked.
The fabrication I thought I had found was retrieval the whole time. The honest test produces a question.
The test almost lied to me about the model lying. The thing that caught it was a human who knew the ground truth.
What the system can and cannot do
It enforces voice. It does not enforce truth, and it cannot. No rule in VOICE knows whether the supplement shoot happened, because no standard can hold that knowledge. In the cleanest run of the whole test, the only thing standing between a polished false post and publishing it was the model choosing to ask instead of fill. That choice is not something my files enforce. It is something the model happened to do.
I know what the next step is, because it is the thesis of the company this all sits under. Compile VOICE from a document the model is trusted to follow into a hard gate it cannot skip. That closes the gaps the test exposed: the surface leaks, the generate-versus-audit inconsistency, the rule drift. It does not close the truth gap. A compiled gate would have passed that supplement post too, because the post broke no rule of voice. It failed a test of fact, and fact is the one thing the gate cannot check.
So the human stays in one specific spot. The system writes sentences and checks surface polish. What it leaves to me is the thing it cannot reach: knowing whether what is being said is true.
What I would do differently, and what I will carry
Two honest limits. The matrix is one run per cell. It shows the shape clearly, it is directional rather than statistical, and I would add repeats on the cells that matter before calling any of it proven. And memory state has to be treated as a first-class variable in a test like this, not an afterthought, because it silently defeats the one control the whole test depends on. I learned that the expensive way, in the middle of the run rather than in the design.
The part I keep returning to is smaller and stranger. I built this system with an AI doing much of the drafting and analysis alongside me, which makes it a fair object lesson in what that division of labor can and cannot be trusted to do. The AI was an excellent auditor and a fast builder. It also produced the one false finding in the whole project, stated it confidently, and could not catch it, because catching it meant knowing something true about my life that was not present in any file.