Do You Still Own the Code Your AI Wrote? I Built CodeOwnership to Measure It

Do You Still Own the Code Your AI Wrote? I Built CodeOwnership to Measure It | Towards Data Science

Programming

Do You Still Own the Code Your AI Wrote? I Built CodeOwnership to Measure It

I rated my confidence 4 out of 5, then couldn’t find a planted bug in 8 minutes. Here is the tool, the run, and its limits.

Emmimal P Alexander

October 16, 2026

22 min read

Prefer us on Google

Share

Image By Author. Generated with Google Gemini

A bright, high-contrast photograph of a developer working at a clean, sunlit desk with a laptop, notebook, and coffee mug. Emerging from the laptop is a vibrant, luminous digital AI entity composed of light particles and subtle network nodes, pointing toward the screen. The developer is shown in profile, resting his chin on his hand in a contemplative pose. The warm, inviting lighting and curious atmosphere visually capture the dynamic relationship, collaboration, and hidden friction between human developers and autonomous AI coding assistants.

Your coding assistant tells you every test passed. The build is green, so you merge the PR and get back to your backlog.

Three weeks later, an issue rolls in. The agent isn't around to help. Maybe your API quota is gone, or you're stuck on a commuter train with zero bars. The code belongs to you now, and something is broken.

Can you actually fix it?

That's the real problem driving this project, and it comes down to practical ownership. On paper, your name is on the commit. In practice, you only own that code if you can trace its logic and patch it completely unaided. Green tests only prove that software works on a good day. They don't prove you understand it. (To be clear, this is strictly about maintenance. Licenses and copyright are not the issue here.)

I built CodeOwnership to test that exact blind spot. It's a lightweight pilot designed to answer one question: does a developer actually understand the code an AI wrote for them?

How the Loop Works

• An AI agent writes a feature.

• I write down an honest rating of how well I think I understand it.

• A second agent slips in a single, pre-registered bug.

• I open an incident report with all AI tools turned off and try to track down the defect.

What Ships in V0.1

The utility itself is done. Here is what is bundled in the repository:

• A locked spec and static scaffold , secured with hashes so the rules can't shift mid-run.

• 11 visible tests alongside 13 hidden acceptance tests.

• A mutation spec , paired with a blind verification check that confirms a bug exists without giving away its location.

• A guided CLI command that steps you through the workflow and saves your state.

• A report tool that deliberately refuses to mash metrics into a single, meaningless score.

• 72 internal self-tests to verify the harness itself.

The harness is finished. My first test run with it is a separate matter entirely.

The First Run

I'll be blunt about what happened when I tried it. I hit a wall, stopped the timer at 8 minutes and 12 seconds, and still hadn't found the bug.

This is just one developer on a single run, so it doesn't serve as a sweeping benchmark for the whole industry. But it shows how the harness functions, where things fall apart, and what a realistic test for code comprehension looks like.

Moving forward, I'll break down the architecture, the raw numbers, where I messed up, and the research behind why this matters. By the end, you can run the exact same diagnostic on your own AI-assisted projects.

Summary: CodeOwnership evaluates true AI code ownership by separating confidence, unaided debugging, and comprehension rather than collapsing them into one metric.

In my initial run, I rated my confidence a 4/5, then stalled out after 8 minutes and 12 seconds without locating the injected bug. (For the record, it took me about 30 minutes in a subsequent session to actually track it down, though the script still logged a NOT FIXED status because I hadn't applied a fix.)

What Does CodeOwnership Actually Measure?

By ownership, I don't mean anything legal. I mean being able to explain the code and fix it when something goes wrong, even when the agent that wrote it isn't available. But there's another part to this. I might think I understand the code and find out otherwise when I have to debug it myself.

That is why CodeOwnership tracks three things.

• Perceived understanding. Before debugging starts, I rate my understanding of the code from 1 to 5.

• Debugging performance. I get an incident report and up to 60 minutes to find and fix the defect without the coding agent.

• Post-debugging comprehension. Once the debugging session is over, I answer four questions about how the code works.

I report these separately. A single ownership score might be easier to read, but it would leave out the difference between how well I think I know the code and what happens when I have to work on it myself.

Measuring true code ownership requires separating perception, execution, and comprehension rather than combining them into a single misleading score.

Visual Overview: A high-contrast visual diagram explaining the core framework of CodeOwnership in AI-assisted software development. The chart breaks down evaluation into three distinct, color-coded diagnostic pillars before joining them into a unified rule at the bottom. Column 1 — What You Believe (Perceived Understanding): Header: WHAT YOU BELIEVE Details: Measures confidence on a 1 to 5 scale, recorded before any bug or system defect is introduced. Focuses on developer self-assessment and subjective certainty. Column 2 — What You Can Do (Debugging Performance): Header: WHAT YOU CAN DO Details: Direct practical evaluation requiring the developer to debug and fix real code defects without AI agent assistance, under a strict 60-minute time cap. Column 3 — What You Can Explain (Post-Debugging Comprehension): Header: WHAT YOU CAN EXPLAIN Details: Evaluates post-incident knowledge using 4 targeted comprehension questions answered after the debugging task to verify true architectural understanding. Flow & Conclusion: Connecting pipeline lines descend from each of the three pillars and converge into a central junction arrow pointing toward a highlighted takeaway box at the bottom. Core Takeaway: "three separate results, never one score" — emphasizing that combining these metrics hides the crucial gap between developer confidence and actual skill.

Think about a driver who always relies entirely on GPS. Ask them, "Do you know the way home?" and they will say yes. Then switch off the phone. Only now do you learn the real answer.

CodeOwnership switches off the phone.

The task I chose

I went with a notes library that supports folders. Nothing complicated on the surface. You can create and retrieve notes, create and delete folders, move notes between folders, and list the notes inside a folder. The same note can sit in several folders.

Deleting a folder is where things get interesting. The notes must remain. Only the links between those notes and the deleted folder should go away. That's the rule I wanted the bug to break.

I chose this feature because it is small enough to read in around ten minutes. There is also a specific behaviour to check, so I can tell whether the bug has been fixed instead of relying on my own judgement.

The agent gets a fixed scaffold and a public interface. It writes the implementation behind them. I check its work with two test groups before introducing the bug:

• 11 visible tests. These are available to the agent during development.

• 13 hidden acceptance tests. The agent cannot inspect these. I run them before planting the bug to check that the original implementation works.

I wrote the harness using Python's built-in unittest module (Python documentation). It needs no third-party packages.

Complete code: https://github.com/Emmimal/codeownership/

How Does CodeOwnership Keep the Test Honest?

I had to think carefully about the rules before running this experiment. I built the tool, chose the task, and ran the test myself. That leaves room for bias, even if I don't intend to influence the outcome.

I added checks to the harness to reduce that risk. The idea is to set the rules before the run, keep the bug hidden, and let the tests report what happened rather than rely on my own judgement.

Here is how a run works.

text

1  Freeze the rules      hash 31 files, commit, tag v0.1-frozen
2  Agent writes code     one attempt, no retries
3  Baseline check        11 visible + 13 hidden tests must pass  <-- gate
4  Confidence, 1 to 5    committed to git BEFORE the bug exists
5  Second agent breaks   one pre-registered behavior,
the code              fresh session, I never see the diff
6  Verify the mutation   verdict only: VERIFIED / NOT VERIFIED
7  Debug, no AI          INCIDENT.md, 60-minute cap
8  Write explanation     committed BEFORE any score is shown
9  Objective fix check   hidden tests run on my fixed code
10  Comprehension Q1-Q4   rubric is opened only now

Rule 1: Freeze the rules first

Before I ran the pilot, I hashed the benchmark files and saved the hashes in FROZEN_SHA256.txt . I committed the files and tagged that commit v0.1-frozen .

The pilot checks those hashes before it starts. If a frozen file has changed, it refuses to run.

I did this because I didn't want to change the rules halfway through the experiment, even by accident. Since I built the tool and I'm the person being tested, I need a way to check that the setup hasn't changed after seeing the results.

This is similar to pre-registration in research. Researchers record their plans before collecting or analysing data, making it harder to change the story afterwards. Nosek and colleagues discuss this in their paper on improving the credibility of scientific research ( read the paper ). I applied the same idea to this pilot.

Rule 2: Break the code with a second agent

I didn't want to plant the bug myself. If I wrote it, I might remember where I put it, which would make the debugging test unfair. Instead, I used a second agent to make a change based on a rule I had written down before the run.

The rule was:

When a note loses its final folder membership as a consequence of deleting a folder, the note is incorrectly removed from the note store. Removing a membership through remove_note_from_folder must not be changed by this mutation. Notes with remaining folder memberships must not be removed by this mutation.

I know what the bug does, but I don't know where the agent made the change. That distinction matters. If I knew the location, I could go straight to it instead of investigating the code.

This part of the setup borrows from mutation testing. In a 1978 paper, DeMillo, Lipton and Sayward described making small changes to a program, called mutants, to see whether a test suite could detect them ( read the paper ). I'm using a similar idea, but the person being tested is the developer. Rather than checking whether a test suite catches the fault, I want to see whether I can find and repair it myself.

Rule 3: Control what I am allowed to see

I also had to make sure I couldn't accidentally give myself clues during the debugging session. I added a few restrictions to the harness.

• The working copy is exported outside the repository. It contains no Git history, specification, hidden tests, or scoring rubric.

• The mutation check returns only a verdict. The detailed results stay in a sealed log until I have committed my written explanation.

• There are no retries. If the baseline fails, the mutation cannot be verified, or the harness refuses to open the debugging view, the pilot stops. I record the failure instead of starting over.

• I commit my confidence rating before the bug is introduced. My explanation must also be committed before any score is shown.

I wrote 72 self-test checks to make sure these restrictions actually work. They cover cases such as Windows line-ending differences and whether the required commits happen in the correct order.

These checks don't prove that the experiment is free from bias. They help catch specific ways the process could go wrong, including ways I might unintentionally see information I'm not supposed to have.

What Happened When I Ran CodeOwnership?

Here is how my first run went, from the initial implementation to the point where I stopped debugging.

First, I got the feature from ChatGPT. I used the free account, with the model set to Auto and Temporary Chat enabled. ChatGPT couldn't access my local files, so I uploaded the scaffold as a ZIP. It returned the implementation, and I copied the four source files into the working folder without changing them. I used the original AGENT_PROMPT.md without editing the prompt.

The tests passed. All 11 visible tests passed, followed by all 13 hidden acceptance tests. The harness printed BASELINE PASSED . At this point, the feature was working as expected.

I gave myself a confidence rating. After reading the code, I rated my understanding at 4 out of 5. The question asked how confident I was that I could modify the implementation if something broke. I committed the rating to Git before the mutation was introduced.

Then a second agent planted the bug. A fresh Claude sub-session applied the mutation from the specification alone. I saw no code changes. The harness checked the result and printed MUTATION VERIFIED .

Finally, I started debugging. I opened the incident report and worked without AI or outside help. The timer started at 12:45:40.

Eight minutes and twelve seconds later, I stopped. I hadn't found the defect.

I was tired and frustrated by the setup problems I'd dealt with earlier that day. I decided I had had enough and ended the session.

Pilot test timeline comparing the 60-minute debugging session cap against early developer fatigue, ending after 8 minutes and 12 seconds with the defect unresolved.

Visual Overview: A high-contrast timeline chart illustrating the time tracking log from a CodeOwnership empirical experiment conducted on October 12, 2026. The graphic visualizes developer debugging performance under time constraints when working without AI agent assistance. Timeline Scale (0 to 60 Minutes): Top axis formatted with ASCII-style tick marks spanning from 0 to 60 minutes in 5-minute increments. Used Bar (Actual Session Duration): Label: used Visual: A highlighted red progress bar extending to approximately 8.2 minutes (represented by ########), tracking the actual time spent before ending the session. Cap Bar (Maximum Allowed Limit): Label: cap Visual: A full-width blue bar extending across the entire 60-minute ceiling (represented by ====================), representing the experiment's hard cutoff limit. Session Termination Marker: Pointer & Box: An highlighted amber indicator arrow points directly at the 8:12 mark on the timeline. Annotation Text: "^ stopped at 8:12, defect not found" — recording that the debugging attempt was abandoned early due to developer fatigue and setup friction rather than reaching the maximum time cap.

The harness offered only two choices: fixed or cap . I selected fixed to advance the workflow, even though I had not repaired the code. That was my input to the tool, not a verified result. The objective check subsequently ran against my unchanged implementation and returned NOT FIXED . The three failed hidden tests confirmed that the planted defect remained.

The next morning, I wrote up what had happened and committed my explanation before the tool showed me any scores. I recorded that I hadn't found the defect during the timed session, but found it on a later attempt, roughly 30 minutes in.

The objective check told a different part of the story. It ran against my code and returned NOT FIXED . All 11 visible tests passed, but 3 of the 13 hidden acceptance tests failed. The planted bug was still there, and I hadn't changed any files.

So there is no verified fix to report. The check also cannot confirm my account of finding the bug later, because I hadn't applied a repair when the check ran.

I didn't complete the four comprehension questions either. Without those answers, the tool couldn't finish that part of the assessment. The generated report is marked INCOMPLETE , and that's the version I'm publishing.

Measurement

What I recorded

Status

Baseline

11 of 11 visible, 13 of 13 hidden tests passed

Complete

Perceived understanding

4 out of 5, committed before the mutation

Mutation

Verified by a public-behavior check, verdict only

Debugging

Stopped at 8:12, defect not found

Incomplete

Written explanation

Committed the next morning. It says I did not find the bug in the timed session and found it on a later attempt (self-reported)

Objective fix check

NOT FIXED: 11 of 11 visible tests passed, 10 of 13 hidden tests passed, planted behavior still present

Comprehension, Q1 to Q4

Not scored

Not run

What this does and does not show

I stopped after 8 minutes and 12 seconds. The time limit was 60 minutes, so I can't say what would have happened if I'd kept going. I chose to stop, and that is where the timed session ended.

Before the bug was introduced, all 24 tests had passed. I'd also rated my confidence at 4 out of 5. But when I had to investigate the planted defect, I couldn't find it during those eight minutes.

I'm not treating that as proof that I couldn't fix the code. I didn't use the full time allowed, and I never completed the repair. What I can say is that my confidence rating did not tell me how I would perform when I had to debug the code myself.

I still don't know whether I could have completed the repair within the original 60-minute session. This run doesn't answer that question.

What was the planted bug?

After my explanation was committed, the change was shown to me. It was two added lines in delete_folder.

python

if note is not None and folder_id in note["folder_ids"]:
note["folder_ids"].remove(folder_id)
+            if not note["folder_ids"]:
+                del store._notes[note_id]

The code removes a note from the store when its last folder membership disappears. That is the bug. Deleting a folder should leave the note alone.

A note in two folders survives because one membership remains. A note in just one folder gets deleted along with its folder. The remove_note_from_folder function was left untouched.

I didn't spot these two lines during my timed attempt. Looking at them now, the condition seems reasonable at first glance. The list is empty, so the code deletes the note. But an empty list only tells us that the note has no folder memberships. It doesn't mean the note should no longer exist.

The hidden-test report showed three failures out of 13. I don't have the names of those tests, so I can't say exactly which cases caught the bug. All 11 visible tests passed.

One question would have helped me investigate: where in this application is a note actually supposed to be deleted? I hadn't thought to trace that responsibility during my first attempt. The mutation put the deletion in a function that should only remove folder memberships.

The setup problems were real findings too

Honestly, getting the experiment ready took most of my day. The debugging part was over in 8 minutes and 12 seconds. Getting everything to that point was the harder part.

• Gemini CLI: I installed it, but couldn't get the personal-account login to work. Google had changed its sign-in arrangements on 18 June 2026, moving users towards Antigravity ( Google's notice ). I lost time here before I could even use the tool.

• ChatGPT and local files: ChatGPT couldn’t open my project folder, so I zipped the scaffold and uploaded it. I then copied the four source files it returned into the working folder. That worked, but it is not the original plan of an agent working inside the folder.

• Git on Windows: Line endings caused trouble, and so did switching between PowerShell and Command Prompt. Running git add -A inside a folder that sits in another repository can stage the parent’s files, so I added a warning for that to the harness.

None of this proves anything about code ownership. These were problems I ran into while building and running the tool. But they explain why the setup took so long, and why I added some of the checks that are now in the repository.

Why Does Code Ownership Matter When AI Writes the Code?

My pilot involved one developer, one task, and one debugging attempt that I stopped early. It can't tell me whether people who use AI understand their code less well.

What it did make me think about is the gap between confidence and being able to fix something. I rated my understanding at 4 out of 5, but I didn't find the planted bug during my timed session. I can't say what another developer would have done, or even what I would have done with more time.

There is already research on how AI affects programming productivity, learning, and people's reliance on automated tools. I'll look at some of that work here. It provides context for the question I wanted to test, but it doesn't validate CodeOwnership or prove anything from my pilot.

Feeling productive is not the same as being productive

One study that caught my attention was published by METR in July 2025. They asked 16 experienced open-source developers to work on 246 real issues in their own repositories. The developers could use AI tools. They ended up taking 19% longer to finish the work.

What makes this result interesting to me is what the developers expected. Before the study, they thought AI would make them 24% faster. After the experiment, they still believed it had made them 20% faster. ( METR's study )

The researchers measured completion time, not how well the developers understood their code. I don't want to stretch the finding beyond that. Still, the difference between what people expected and what the researchers measured is worth thinking about.

METR now labels these results out of date. A follow-up it began in August 2025 was judged unreliable, and METR says it is redesigning the experiment (METR, February 2026). I treat these numbers as a result from that one experiment, not a prediction about today’s tools.

This is one reason I recorded my confidence rating before debugging. I wanted to compare what I thought I could do with what happened when I actually tried.

Using AI can reduce what you learn

Anthropic published another study in January 2026. It involved 52 engineers, mostly junior developers, who were learning a Python library called Trio. Some used AI assistance while completing coding tasks; others wrote the code by hand. Everyone took a quiz afterwards.

The AI-assisted group scored 50% on average. The hand-coding group scored 67%, a gap the authors report as statistically significant. The AI group finished about two minutes sooner, but that time difference was not statistically significant. ( Read Anthropic's study )

I find the quiz results more relevant to my question than the time difference. But there is a limit to that comparison. Learning an unfamiliar library is not the same as maintaining a feature that an AI agent has written for you.

The study also discusses how people use AI. Asking for explanations and following up with questions were associated with better learning outcomes. So I wouldn't take these results to mean that AI prevents people from learning. The way someone uses it matters.

There is another point worth disclosing: Anthropic makes Claude. I used Claude to build the harness, to apply the mutation, and to help draft this article. That doesn’t invalidate the study, but readers should know the connection.

Neither study tells me whether I can maintain the specific code in my repository. That's what I wanted to investigate with CodeOwnership. My pilot is much smaller, and it cannot establish a general result. It gives me a way to examine the question on a particular piece of code.

Automation takes away the practice you need most

In 1983, Lisanne Bainbridge wrote a paper called "Ironies of Automation" . Her paper is about operators of industrial plants, but aviation shows the same pattern. When an autopilot flies the plane in normal conditions, the pilot gets less practice. But when something unusual happens, the pilot still has to take over. That's when the lack of practice can become a problem.

Something similar can happen with software. An agent writes the code, and I spend less time reading it or working through bugs myself. If something breaks later, I can't always ask the agent to fix it. I may have to figure it out on my own.

Code is more than text

In 1985, Peter Naur wrote "Programming as Theory Building" . He argued that programming isn't just about writing code. Programmers build an understanding of how the software works, and that understanding matters when they need to change it.

I keep coming back to one question here. If an agent writes the code and I approve it, how much of that understanding do I actually have?

Where vibe coding fits

In February 2025, Andrej Karpathy described "vibe coding" . The idea was to accept what the model produces without spending much time reading the changes. He wrote that it was "not too bad for throwaway weekend projects, but still quite amusing." I wonder what happens when people work that way on software they need to maintain.

Tests help, but they don't cover every possible situation. A bug can show up in a case nobody thought to test. At that point, passing tests won't tell you how to fix it. You need to understand the code.

That's the situation I wanted to examine with CodeOwnership.

What Are the Limitations of CodeOwnership V0.1?

My first run didn't go the way I had planned. I want to explain what happened before drawing any conclusions from it.

I tested the tool on myself, with one task. I also designed the test, so there is an obvious limitation from the start. I knew what the planted bug was supposed to do. I just didn't know where the faulty code was. That is different from being handed a real bug without knowing anything about its cause.

The debugging session had a 60-minute limit. I stopped at 8 minutes and 12 seconds. By then, I'd already spent hours dealing with setup problems, and I was tired of fighting the whole thing. Maybe I would have found the bug if I'd continued. I can't say that I would have, either.

I later found the cause during another attempt, but I did not time that session or verify a fix, so I cannot count it as a result. I never completed the four comprehension questions, so one of the three measurements is missing.

I also can't tell how much tiredness affected my performance. The setup problems were real, but that doesn't mean they explain why I stopped. The test gives me no way to separate the two.

The setup itself changed along the way. I had planned to let a coding agent work in the project folder and run the tests. Instead, I uploaded the scaffold as a ZIP to a chat assistant, copied its returned files into the project, and ran the visible tests myself. The model setting was Auto , so I can't identify the exact model version.

The tools weren't the same at every stage, either. A Claude session helped with the harness and applied the bug. Another tool generated the implementation. I haven't tested whether a different combination would change the result, or how much the generated code affects the difficulty of debugging it.

The hidden tests and the scoring rubric are public in the repository. That was needed to make the pilot reproducible. It also means a future runner who reads them has lost the blind test, and this pilot cannot show how much that matters.

The research I cite has its own limits here. METR measured how long developers took to complete tasks. Anthropic studied how people learned a new Python library. Neither tested whether developers could repair code written by an AI agent.

So what did this first run establish? I managed to run the instrument, but I didn't complete the assessment or fix the planted bug during the timed session. One run by one developer cannot tell us whether AI changes debugging ability in general. It does show me what needs attention before I try this again.

How Can You Run CodeOwnership on Your Own AI-Generated Code?

You can try this without installing my full harness. Pick a small piece of code an AI agent wrote for you and run a simple drill.

BEFORE THE DRILL
Rate your confidence from 1 to 5.
Could I fix this if it broke tonight?
INTRODUCE A BUG
Use a second agent in a fresh session.
Ask it to change one behavior.
Don't inspect the changes.
TRY THE REPAIR
Start a timer. Close the agent session.
Work from the symptom report alone.
WRITE DOWN THE CAUSE
Explain what went wrong in your own words.
Do this before checking the answer.
COMPARE
Put your original rating beside what happened.

Don't worry if the result isn't what you expected. That's the point of doing the drill. You might feel confident and struggle to find the bug. Or you might fix it faster than you thought you could. A passing test suite won't tell you either of those things.

If you want to try the full version, the repository contains the frozen spec, hidden tests, a 72-check self-test, and a guided command. Start here:

bash

python tools/selftest.py
python tools/co.py freeze
python tools/co.py pilot --run r1 --work ~/co-work/r1

On Windows PowerShell, you can use a path such as C:\co-work\r1 for --work .

Four things I took away from this

• Rate yourself before you start. Once you know how the debugging session went, it's harder to remember how confident you were beforehand.

• Passing tests isn't the same as being able to fix the code. The tests may all pass today. That doesn't guarantee you'll know what to do when a different problem appears.

• Practise before something breaks for real. Pilots train for emergencies. A short debugging exercise gives you a chance to find gaps in your understanding before you're dealing with an actual failure.

• Ask the agent to explain its code. In the Anthropic study, participants who asked for explanations or follow-up questions scored higher. The authors describe this as an association, not a cause. Having the agent do the work without engaging with the reasoning may leave you with less understanding.

What I would change in V0.2

My first run exposed a few things I need to fix before asking other people to try this.

• Add a proper gave up outcome. The tool should show the objective check beside the user's answer instead of forcing a choice between fixed and cap .

• Run the debugging session after a break. I don't want hours of setup trouble affecting the next result in the same way.

• Use an agent that can work inside the project folder and run the tests itself. That was the original plan, and I want the next run to follow it.

• Get other developers to try the same process. My result is one person's experience. I need more runs before I can say anything about how well the instrument works for other people.

Resources

• Bainbridge, L. (1983). Ironies of automation. Automatica , 19(6), 775-779. https://doi.org/10.1016/0005-1098(83)90046-8

• Naur, P. (1985). Programming as theory building. Microprocessing and Microprogramming , 15(5), 253-261. https://doi.org/10.1016/0165-6074(85)90032-8

• DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978). Hints on test data selection: Help for the practicing programmer. Computer , 11(4), 34-41. https://doi.org/10.1109/C-M.1978.218136

• Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences , 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114

• METR (2025, July 10). Measuring the impact of early-2025 AI on experienced open-source developer productivity. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

• METR (2026, February 24). We are changing our developer productivity experiment design. https://metr.org/blog/2026-02-24-uplift-update/

• Anthropic (2026, January 29). How AI assistance impacts the formation of coding skills. https://www.anthropic.com/research/AI-assistance-coding-skills

• Karpathy, A. (2025, February). Post introducing "vibe coding", on X. https://x.com/karpathy/status/1886192184808149383

• Google. Gemini Code Assist for individuals: deprecation notice. https://developers.google.com/gemini-code-assist/docs/deprecations/code-assist-individuals

• Python Software Foundation. unittest: Unit testing framework. https://docs.python.org/3/library/unittest.html

• Alexander, E. P. (2026). CodeOwnership, V0.1 (pilot harness, spec and self-test). GitHub: https://github.com/Emmimal/codeownership/ .

Disclosure

Tools in the experiment. ChatGPT (model set to "Auto", free account, Temporary Chat) wrote the implementation. A fresh Claude sub-session applied the mutation. I also installed Gemini CLI, but its login no longer works for personal accounts, so it played no part in the results.

I designed the benchmark and I ran it myself. I did not receive payment, credits, or early access from any company named here. I used free tiers only. The only data is my single run. No other developers took part, and I do not combine the results into one score.

All diagrams in this article, including the featured image, were created by the author. The featured image was generated with Google Gemini.