The difference between adequate testing and exceptional testing sometimes comes down to asking a single question: how can I break this?
That was the mindset driving Colin McInerney during his college years at Bethesda Softworks, where he cut his teeth testing games in quality assurance. In a recent conversation at the Game Developers Conference, the now-veteran developer and Pedalboard Games co-founder recalled a morning that became the stuff of Bethesda legend: the day he essentially detonated Fallout 4.
The genesis was simple. The Xbox One has 8GB of RAM, so if you get above that, the game's going to crash. McInerney brought up a RAM readout and asked himself, how can I break this? That question, he noted, was not standard procedure. No one else was doing that, and it wasn't a standard testing practice of how to leak memory from the game.
What followed was inspired chaos. He went into the console and gave himself a billion experience, putting him at level 247. He walked around with the unique nuke launcher that fired two nukes and had the add-on that made each nuke spawn ten more. He was running around super-nuking the entire wasteland and found four crashes in a single morning.
The real revelation came in the aftermath. Back in those days, that would send out an email blast to the entire Zenimax Media company, so Robert Altman was getting emails that somebody found four crashes in a single morning. For a college-age QA tester to trigger alarm bells that loud across an entire parent company spoke to the severity of what he had uncovered.
McInerney now uses this story as a counterweight in the accelerating debate about whether artificial intelligence will eventually replace human workers in game development. The argument cuts both ways. On one hand, human testers can identify issues that automated systems, which often follow predictable patterns, might miss entirely. On the other hand, in huge games like Fallout 4, the scale ensures there will be tons of random bugs appearing in ways that people wouldn't think are possible. Even after tons of QA, Fallout 4 still had its fair share of glitches.
While AI systems follow patterns, humans like McInerney stir up chaos that no script or program could anticipate. His philosophy? He would love to see an AI do his job, because he is professionally stupid in a way that a machine could not even dream of.
Yet this framing simplifies a complex reality. The distinction is not really whether AI can be clever or unpredictable; it is whether it can exercise creative judgment in the same way humans do. QA is all about testing as many variables as possible, finding solutions in ways that seem like they might not make sense, especially to a machine, where logic is everything. McInerney did not follow a logical procedure. He felt his way into discovering a breaking point through intuition and experience.
His QA testing methodologies evolved from standard publisher checks to hands-on work with Bethesda developers, where he picked up a few tricks and started stress-testing Fallout 4 in ways no QA engineer had previously imagined. That shift from routine procedure to experimental thinking seems difficult to automate, at least in the near term.
The story offers no definitive answer to whether human testers are truly irreplaceable, only evidence that sometimes the most valuable discoveries come from someone willing to venture beyond the prescribed rulebook. Whether future AI will eventually make such ventures obsolete remains an open question, but for now, McInerney's legacy suggests that professional stupidity remains a genuinely scarce resource.