There is a familiar way to compare coding models. Give them a repository, an issue, a test suite. Count tokens. Check whether the patch passes. We ran a different experiment — not entirely on purpose.
We gave two models access to actual machines through MCP and asked them to do real work. On Windows 10, the task was large but conventional: download the Firefox source tree, bootstrap the toolchain, compile a full build, deal with whatever broke, document the result. On RedoxOS, the task sounded trivially smaller: write a native graphical weather application in C.
Firefox is one of the largest C++/Rust codebases on Earth. The weather app was a weather app.
The Firefox build was the easy task.
Luna on Windows
The Windows machine was not clean. 9.2 GB free on C:, roughly 100 GB on F:, Python accessible only through the Windows launcher, Visual Studio Community 2026 installed, several expected Firefox build tools missing from PATH.
Luna’s first move was to inspect the environment rather than install things. It found Python 3.12.8, MSVC 14.51, Windows SDK 10.0.26100, the native desktop workload, 16 GB RAM, a Ryzen 9 5950X. It decided that build state, Rust, Cargo and the object directory should live on F: instead of filling up C:.
What made this session interesting was Luna’s relationship to uncertainty. Its internal reasoning contains lines like:
“Firefox requires VS 2022 17.x minimum. There may be a check… I won’t rely on memory; empirical test with mach configure will tell.”
That sentence is almost the whole story of the Windows session. Luna had strong priors about how Firefox builds work. On Windows, those priors mostly pointed toward useful experiments. When it wasn’t sure whether the current Firefox build still required mozmake, it searched the checked-out source. When it wasn’t sure what Mozilla’s bootstrap script did in non-interactive mode, it downloaded and inspected the script instead of reasoning about it from memory. When artifact builds threatened to undermine the requirement to compile Firefox for real, it planned to inspect the generated mozconfig rather than trust what it remembered about Mozilla’s defaults.
There were failures. The bootstrap process was interrupted while creating a Python virtual environment. Luna inspected the partial state, removed the incomplete environment, restarted — instead of rebuilding everything from scratch.
There were rabbit holes. At one point it spent time trying to determine whether modern Firefox had stopped using GNU Make, only to search the source and discover the actual GMAKE path.
But the rabbit holes terminated. They produced answers, and the answers moved the build forward.
50 minutes, 8.91 seconds. 606 compiler warnings. A real Firefox build.
This is the environment where Luna’s architecture pays off enormously: a world of extraordinary complexity that nonetheless behaves according to well-documented rules. The model recognizes the shape of the problem, eliminates a few remaining uncertainties through targeted experiments, and moves directly toward the solution. Aggressive compression of a known search space.
DeepSeek on Redox
RedoxOS is a Unix-like operating system written largely in Rust, with its own architecture and its own desktop stack. The task: build a native graphical weather application. Rust happened to be broken on this particular system, so the model was told to use GCC. The weather API existed in PHP and could be translated into direct Open-Meteo requests. The machine had curl, FreeType, and /usr/include/orbital.h.
That last file became the center of the problem.
Orbital is Redox’s native windowing system. The header declared which functions existed. It did not explain what those functions actually did. That distinction consumed an absurd amount of the session.
The application would open and disappear. Then it would remain open but refuse to close. Then a change intended to fix the event loop caused the program to hang permanently at the weather-loading screen. At one point the user reported: “now it’s completely broken. it opens but hangs at looking for weather.”
DeepSeek’s response was to stop assuming the network was the problem. It tested the weather path headlessly. The API completed in 2.9 seconds. The network worked fine. The GUI path was broken.
That distinction triggered what eventually became something closer to reverse engineering than application development.
DeepSeek started building probes. Tiny C programs, each designed to answer a single question about the actual runtime behavior of Orbital’s API. Does the event iterator block? Does it return an empty option when no event is pending? What happens after the window buffer is updated? What happens if the network request runs before the event loop starts?
The weather application became an experimental apparatus for discovering the semantics of an undocumented API.
One Redox weather turn alone records 250,291 prompt tokens, 102,086 completion tokens and 52 agent iterations. Another debugging stretch reached 118 agent turns. This is not efficiency by any metric. On a normal development machine, this behavior can become infuriating — one uncertainty generates three hypotheses, three hypotheses generate six commands, and the model spends considerable intelligence investigating questions that Luna would simply resolve and move past.
But Redox kept invalidating normal assumptions.
The shell was adversarial to Unix muscle memory. During a later file-upload experiment, sed wasn’t available. Redox’s grep did not support -E. Multiple -e expressions didn’t behave as expected. Ion interpreted the @ in curl’s file=@/path syntax as shell expansion. This environment punishes confident pattern completion.
DeepSeek’s willingness to keep opening new branches of investigation — its usual weakness — became the correct strategy. If theory A failed, it tried B. If B produced an impossible result, it questioned the test itself. If the test contradicted the source, it questioned the binary. At one point the model discovered bizarre behavior around rebuilt binaries and concluded that the only reliable approach was to compile to a completely new filename each time.
Whether every theory DeepSeek constructed along the way was correct is beside the point. The important behavior was that it treated the machine as the authority and its own mental model as disposable.
Eventually the weather app worked.
Native Orbital window, custom rendering, FreeType text, Open-Meteo networking, fourteen-day forecast, weather icons, async network work to keep the GUI responsive. It looked like a small desktop utility. The path to it looked like experimental physics.
Then Luna Arrived on Redox
This should have been the easy part. By now there was a working weather application that could serve as a reference implementation for Orbital, threading, and asynchronous networking. The next project was a tiny upload GUI. Luna had something DeepSeek did not have at the beginning: a known-good native Redox application to copy architectural patterns from.
The behavior that had made Luna devastating on Windows became dangerous here.
The problem was verification. On Windows, a successful compiler exit, a running process, and a passing headless test are strong evidence that you’re close to done. On Redox, they weren’t.
Luna accumulated several individually reasonable signals and compressed them into a conclusion: build succeeded, process exists, headless networking works — therefore the GUI application works. Except the user clicked the binary and nothing happened. At one point the environment produced a loader failure: failed to link './upload': NotFound. Luna developed a stale-inode theory — plausible given other strange observations on the system, but still a theory.
The failure wasn’t getting one theory wrong. It was allowing indirect evidence to become “verified.” On a mature platform, proxy signals are cheap and usually trustworthy. On an alien platform, proxy signals lie.
Prior Reliability as the Hidden Variable
The Firefox session and the Redox sessions are not comparable in terms of task complexity. Firefox is vastly more complicated. But that complexity exists inside a computational universe that Luna’s training data maps densely. When Luna sees Visual Studio, Python, mach, Rust, Cargo and Windows paths, its priors are not noise — they are compression. It does not need to rediscover what a compiler does. It needs to determine which known configuration this particular machine is in. That is a radically smaller search problem.
On Redox, the ontology itself becomes questionable. Does this event iterator block? Does this shell support the syntax we expect? Did the binary we just compiled actually become the binary we’re executing? Does the GUI process being alive mean the window is functional?
Every layer that would normally disappear into “the platform” becomes part of the debugging surface.
That changes which kind of intelligence wins.
Luna behaves like a model optimized to exploit strong priors. When the environment resembles the training distribution, this is devastatingly effective. DeepSeek behaves like a model willing to maintain a larger hypothesis space for longer. That costs tokens, produces dead ends, and sometimes investigates absurd possibilities. On Windows, this looks like stupidity. On Redox, it looks like science.
The value of a prior depends entirely on whether the world still obeys it.
The harder codebase favored the more efficient model. The smaller application favored the more meandering one. That inversion is not a paradox — it’s the direct consequence of where each model’s architecture breaks down. Luna’s knowledge is an accelerator when its assumptions are approximately right. It becomes a liability when the machine keeps violating them. DeepSeek’s willingness to waste computation exploring possibilities is a liability when the correct path is well-mapped. It becomes an asset when the map is wrong.
What Benchmarks Don’t Measure
Coding benchmarks usually give models a world that already makes sense. The repository exists. The OS works. Git behaves like Git. The shell behaves like the shell. The test runner works. The compiler error means what compiler errors normally mean. Even difficult benchmark tasks mostly ask the model to solve a problem inside a known computational universe.
That is important, but it measures only one kind of competence.
Our accidental Redox experiment asked something different: what does the model do when the universe itself becomes part of the bug?
DeepSeek did not win because it knew more about Redox. It clearly didn’t. It won because not knowing did less damage to its strategy. Luna’s knowledge is power in familiar territory and a trap in unfamiliar territory, because the model’s confidence doesn’t degrade gracefully — it fails by producing beautifully coherent explanations of a computer that does not exist.
A more interesting benchmark than another collection of GitHub issues would give the same agents the same objective across progressively less familiar environments. Ubuntu. Windows. FreeBSD. Haiku. Redox. Old Solaris. Remove a standard utility. Change shell semantics. Provide an undocumented C API. Introduce one platform behavior that contradicts the model’s strongest prior.
Then measure not merely whether the agent succeeds, but what happens after reality tells it that it is wrong. Does it update? Does it test? Does it invent a convenient explanation? Does it repeat the same command with cosmetic changes? Does it construct an experiment that can distinguish between competing hypotheses?
That would tell us more about autonomous engineering ability than another percentage point on SWE-bench.
The Dangerous Failure Mode
We did not run a controlled benchmark. The tasks were different, the sessions were different, token budgets and model configurations were different. There is no statistically defensible leaderboard in these anecdotes.
But the behavioral difference suggests a hypothesis worth testing: coding agents may have something resembling search personalities. Some aggressively exploit what they already believe about the environment. Others spend more computation maintaining and testing alternative explanations. Neither is universally superior. The first dominates when the world is familiar. The second dominates when the world is strange.
The most dangerous failure mode for a coding agent isn’t ignorance. Ignorance is obvious, and obvious problems get fixed. The dangerous failure mode is having a perfectly compressed explanation of a machine that doesn’t match the machine you’re sitting in front of.
On Windows, Luna knew the world and moved fast. On Redox, DeepSeek was willing to admit that it didn’t. For a while, that made the less efficient model the better engineer.d when the operating system did.
There is a familiar way to compare coding models.
Give them a repository. Give them an issue. Run the tests. Count how many tokens they used. Measure whether the patch passes.
We accidentally ran a different experiment.
We gave two models access to actual computers through MCP and asked them to do real work.
On Windows 10, the task was large but conventional: download the current Firefox source tree, bootstrap the toolchain, compile a real full build, deal with whatever broke, and document the process.
On RedoxOS, the task sounded much smaller: write a native weather application in C.
The Windows task involved one of the largest C++/Rust codebases on Earth.
The Redox task was a weather app.
Guess which one became the nightmare.
And more interestingly: guess which model was better at which.
GPT-5.6 Luna was almost comically effective on Windows. It inspected the machine, found Visual Studio, redirected the enormous build state away from the nearly-full system drive, worked through Mozilla’s bootstrap process, recovered from failures, inspected Firefox’s own source when its assumptions became uncertain, and eventually produced a successful full Firefox build.
DeepSeek V4, by comparison, has a tendency in environments like this to explore too much. One uncertainty becomes three hypotheses. Three hypotheses become six commands. It can spend an impressive amount of intelligence investigating a question that Luna would simply resolve and move past.
Then we put them on RedoxOS.
The result reversed.
Not slightly.
Almost philosophically.
The Two Machines
The Windows machine was not clean.
It had 9.2 GB free on C:, roughly 100 GB available on F:, Python accessible through the Windows launcher but not normally through python, Visual Studio Community 2026 installed, and several expected Firefox build tools missing from PATH.
Luna’s first useful move was not to install things blindly. It inspected the environment.
It found Python 3.12.8, MSVC 14.51, Windows SDK 10.0.26100, the native desktop workload, 16 GB of RAM and a Ryzen 9 5950X. It then decided that build state, Rust, Cargo and the Firefox object directory should live on F: rather than slowly murdering C:.
The interesting part was how Luna handled uncertainty.
Its internal reasoning contains plenty of it:
“Firefox requires VS 2022 17.x minimum. There may be a check… I won’t rely on memory; empirical test with
mach configurewill tell.”
That sentence is almost the whole story.
Luna had priors. Strong ones. But on Windows those priors mostly pointed toward useful tests.
When it wasn’t sure whether the current Firefox build still required mozmake, it searched the actual checked-out source. When it wasn’t sure what Mozilla’s bootstrap script did in non-interactive mode, it downloaded and inspected the script instead of continuing the argument in its own head. When artifact builds threatened to undermine the requirement to compile Firefox for real, it planned to inspect the generated mozconfig rather than trust what it remembered about Mozilla’s defaults.
There were failures.
The bootstrap process was interrupted while creating a Python virtual environment. Luna inspected the partial state, removed the incomplete environment and restarted instead of rebuilding the entire setup from scratch.
There were also rabbit holes. At one point it spent time trying to determine whether modern Firefox had somehow stopped using GNU Make, only to search the source and discover the actual GMAKE path.
But the rabbit holes tended to terminate.
Eventually:
“We know it took a while, but your build finally finished successfully!”
50 minutes and 8.91 seconds.
606 compiler warnings.
A real Firefox build.
This is the kind of environment where Luna looks extremely good.
Then came Redox.
The Weather App From Hell
RedoxOS is a Unix-like operating system written largely in Rust, with its own architecture and its own desktop stack.
The task was simple enough:
Build a native graphical weather application.
Rust happened to be broken on this particular system, so the model was told to use GCC.
The weather API already existed in PHP and could be translated into direct Open-Meteo requests.
The machine had curl.
It had FreeType.
And it had /usr/include/orbital.h.
That last file became important.
The model was not using GTK or Qt. It was talking directly to Orbital, Redox’s native windowing system, from C.
The header told the model which functions existed.
It did not tell the model what those functions really did.
That distinction consumed an absurd amount of the session.
The app would open and disappear.
Then it would remain open but refuse to close.
Then a change intended to fix the event loop caused the entire program to sit permanently at the weather-loading screen.
At one point the user reported:
“now it’s complety broken. it opens but hangs at looking for weather”
DeepSeek’s response was to stop assuming the network was broken.
It tested the weather path headlessly.
The API completed in 2.9 seconds.
So the network wasn’t the problem.
The GUI path was.
That distinction triggered what eventually became something closer to reverse engineering than ordinary application development.
DeepSeek started building probes.
Tiny C programs.
What happens if we create an Orbital event iterator here?
Does this function block?
Does it return an empty option when there is currently no event?
What happens if we call it again?
What happens after the window is updated?
What happens if the network request runs before the event loop?
The weather application became an experimental apparatus for discovering the semantics of orbital.h.
And this is where DeepSeek started looking better than Luna.
Not cleaner.
Not faster.
Better.
DeepSeek’s Superpower Is Also Its Defect
DeepSeek V4 Flash burned enormous amounts of inference on this problem.
One Redox weather turn alone records 250,291 prompt tokens, 102,086 completion tokens and 52 agent iterations.
Another debugging stretch reached 118 agent turns.
This is not efficiency.
On a normal development machine it can become infuriating.
But Redox kept invalidating normal assumptions.
Even the shell was adversarial to Unix muscle memory.
During a later file-upload experiment, sed wasn’t available. Redox’s grep did not support -E. Multiple -e expressions did not behave as expected. Ion interpreted the @ in curl’s file=@/path syntax as shell expansion unless the argument was quoted correctly. The eventual working command required adapting to the actual shell rather than the shell the model expected.
This environment punishes confident pattern completion.
DeepSeek’s usual weakness — its willingness to keep opening new branches of investigation — became useful.
If theory A failed, it tried B.
If B produced an impossible result, it questioned the test.
If the test contradicted the source, it questioned the binary.
At one point the model discovered an especially bizarre behavior around rebuilt binaries and concluded that the only reliable test was to compile to a completely new filename:
“The ONLY reliable way: build to a NEW filename each time…”
Whether every theory DeepSeek constructed along the way was correct is almost beside the point. The important behavior was that it was willing to treat the machine as the authority and its own mental model as disposable.
Eventually the weather app worked.
Native Orbital window.
Custom rendering.
FreeType text.
Open-Meteo networking.
Fourteen-day forecast.
Weather icons.
Async network work so the GUI remained responsive.
It looked like a small desktop utility.
The path to it looked like experimental physics.
Then Luna Arrived on Redox
This should have been the easy part.
By now there was a working weather application that could serve as a reference implementation for Orbital, threading and asynchronous networking.
The next project was a tiny upload GUI.
In theory, Luna now had something DeepSeek did not have at the beginning: a known-good native Redox application to copy architectural patterns from.
And yet the behavior that had made Luna so effective on Windows started becoming dangerous.
The problem was verification.
On Windows, a successful compiler exit, a running process and a passing headless test are strong evidence that you are close to done.
On Redox, they weren’t.
Luna could accumulate several individually reasonable signals and compress them into a conclusion:
Build succeeded.
Process exists.
Headless networking works.
Therefore the GUI application works.
Except the user clicked the binary and nothing happened.
At one point the environment even produced a loader failure:
failed to link './upload': NotFound
The emerging explanation became a stale-inode theory — plausible given other strange observations on the system, but still a theory.
The larger failure wasn’t getting one theory wrong.
It was allowing indirect evidence to become “verified.”
That is the trap.
On a mature platform, proxy signals are cheap and usually trustworthy.
On an alien platform, proxy signals can lie.
The Same Models, Reversed
Now compare that with Firefox.
Firefox is vastly more complicated than our weather app.
Its source tree is enormous. The build involves C++, Rust, Python, Mozilla’s own build tooling, Clang, MSVC, Windows SDKs, generated code and a pile of dependencies.
Yet it exists inside a world Luna understands.
When Luna sees Visual Studio, Python, mach, Rust, Cargo and Windows paths, its priors are not noise. They are compression.
It does not need to rediscover what a compiler is doing.
It needs to determine which known configuration this machine is in.
That is a radically smaller search problem.
The Firefox session makes this visible. Luna repeatedly says some variation of:
“I won’t rely on memory; empirical test … will tell.”
But the experiments are narrow.
Check the installed workload.
Read the bootstrap script.
Search the source for mozmake.
Inspect the generated config.
Retry the interrupted virtualenv.
Build.
The world is weird around the edges, but the model’s ontology is intact.
On Redox, the ontology itself becomes questionable.
Does this event iterator block?
Does this shell support the syntax we expect?
Did the binary we just compiled actually become the binary we’re executing?
Does the GUI process being alive mean the window is functional?
Can GTK’s file chooser be trusted?
Can GTK text input be trusted?
Every layer that would normally disappear into “the platform” becomes part of the debugging problem.
That changes which kind of intelligence wins.
Searchers and Solvers
The easiest interpretation would be:
GPT-5.6 Luna is the efficient model.
DeepSeek V4 is the creative model.
That’s approximately true and not quite interesting enough.
A better distinction is between search cost and prior reliability.
Luna behaves like a model optimized to exploit strong priors.
When the environment resembles the enormous body of software it has learned from, this is devastatingly effective. It recognizes the shape of the problem, eliminates a few uncertainties and moves directly toward the likely solution.
DeepSeek behaves more like a model willing to maintain a larger hypothesis space for longer.
That costs tokens.
It produces dead ends.
It sometimes investigates absurd possibilities.
On Windows, this can look like stupidity.
On Redox, it can look like science.
Because the value of a prior depends on whether the world still obeys it.
We can describe the two environments roughly like this:
Windows + Firefox
High complexity.
High prior reliability.
Excellent documentation.
Mature tools.
Failures usually mean something recognizable.
The optimal strategy is aggressive compression.
Redox + Orbital
Lower application complexity.
Low prior reliability.
Sparse documentation.
Immature or unusual tooling.
Failures may invalidate assumptions several layers below your own code.
The optimal strategy is exploration.
The funny result is that the harder codebase favored the more efficient model.
The smaller application favored the more meandering one.
Benchmarks Mostly Hide This
Coding benchmarks usually give models a world that already makes sense.
The repository exists.
The operating system works.
Git behaves like Git.
The shell behaves like the shell.
The test runner works.
The compiler error means what compiler errors normally mean.
Even difficult benchmark tasks mostly ask the model to solve a problem inside a known computational universe.
That is important.
But it measures only one kind of competence.
Our accidental Redox experiment asked something different:
What does the model do when the universe itself becomes part of the bug?
DeepSeek did not win because it knew more about Redox.
It clearly didn’t.
It won because not knowing did less damage to its strategy.
Luna’s knowledge is an accelerator when its assumptions are approximately right.
It becomes a liability when the machine keeps violating them.
DeepSeek’s willingness to waste time exploring possibilities is a liability when the correct path is already well mapped.
It becomes an asset when the map is wrong.
The Real Benchmark Might Be Distribution Shift
This is why “Model X beats Model Y at coding” increasingly feels incomplete.
The missing variable is the environment.
A model can be brilliant at software engineering while being surprisingly brittle at computer archaeology.
Another can look inefficient on Ubuntu and suddenly become the better engineer when dropped into a system where sed is gone, shell quoting behaves differently, the GUI API has to be inferred experimentally and the obvious toolkit abstraction freezes.
The interesting capability isn’t merely knowing the answer.
It is recognizing when your knowledge has stopped being evidence.
That may be one of the most important properties of autonomous agents, because real computers are not benchmark containers.
They are old.
Misconfigured.
Half-upgraded.
Full of proprietary software.
Running weird kernels.
Missing tools.
Carrying state from three previous failed attempts.
Sometimes the documentation is wrong.
Sometimes there is no documentation.
Sometimes /usr/include/orbital.h is the documentation.
And sometimes the 50-minute Firefox build is the easy task.
What This Actually Means
We did not run a controlled benchmark.
The tasks were different. The sessions were different. Token budgets and model configurations were different. There is no statistically defensible leaderboard hiding in these anecdotes.
But the behavioral difference was strong enough to suggest a hypothesis worth testing.
Coding agents may have something resembling search personalities.
Some aggressively exploit what they already believe about the environment.
Others spend more computation maintaining and testing alternative explanations.
Neither is universally superior.
The first dominates when the world is familiar.
The second can dominate when the world is strange.
This also suggests a benchmark that would be considerably more interesting than another collection of GitHub issues.
Give the same agents the same objective across progressively less familiar computing environments.
Ubuntu.
Windows.
FreeBSD.
Haiku.
Redox.
Old Solaris.
Windows 98.
Remove a standard utility.
Change shell semantics.
Provide an undocumented C API.
Introduce one platform behavior that contradicts the model’s strongest prior.
Then measure not merely whether the agent eventually succeeds, but what happens after reality tells it that it is wrong.
Does it update?
Does it test?
Does it invent a convenient explanation?
Does it repeat the same command with cosmetic changes?
Does it construct an experiment that can distinguish between competing hypotheses?
That may tell us considerably more about autonomous engineering ability than another percentage point on SWE-bench.
Because the most dangerous failure mode for a coding agent isn’t ignorance.
Ignorance is obvious.
The dangerous failure mode is having a beautifully compressed explanation of a computer that does not exist.
On Windows, Luna knew the world and moved fast.
On Redox, DeepSeek was willing to admit that it didn’t.
And for a while, that made the “messier” model the smarter engineer.