ROBOTS.TXT - WILL IT RUN DOOM?

THE QUESTION

Can your robots.txt run DOOM?

Not as a redirect. Not as an easter egg link. The actual game, rendered directly when you visit the file in a browser.

Previously, I turned my robots.txt into a Mystery Science Theater 3000 experience using XML and XSLT. This led to a discussion where John Mueller mentioned:

“I think this paves the way to running Doom on robots.txt … ;-)”

A challenge like that can’t be ignored.

The answer is yes. And getting there required compressing a 9MB game into 800KB and some creative abuse of web standards.

THE SETUP

Every website has a robots.txt. It’s a plain text file that tells web crawlers what they can and can’t visit. Typically it looks like this:

User-agent: *
Allow: /

User-agent: GPTBot
Disallow: /

Boring. Functional. But here’s the thing: browsers don’t actually care what’s in a text file. They care about the Content-Type header.

This header, sent alongside the document, tells the browser how to treat the file. Send the signal that it’s an audio file, and the browser will try to play it—which is exactly what John Mueller did with his robots.txt, turning it into a playable WAV file. Tell the browser it’s XML, and it will parse it as XML.

What’s interesting about XML is that it’s a human-readable format. The content remains visible and understandable, even as the browser processes it differently.

THE DREAM: PURE XML

The elegant solution would be pure XML with an embedded XSLT stylesheet:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="#doom-style"?>
<doom-robots xmlns="https://playground.wunsch.dk/doom-robots">
  User-agent: *
  Allow: /

  <xsl:stylesheet id="doom-style" version="1.0" ...>
    <!-- XSLT that outputs HTML with a DOOM canvas -->
  </xsl:stylesheet>
</doom-robots>

Firefox handles this beautifully—it supports inline XSLT via href="#id" fragment references. One file, completely self-contained, transforms itself into a DOOM game. Try it in Firefox.

Chrome? Not so much.

THE BROWSER PROBLEM

Chrome, Safari, and Edge don’t support inline XSLT stylesheets. They require an external .xsl file. This breaks the “single self-contained file” dream for the XML approach (for now).

We could use a shim—an external XSLT that fetches the original XML, extracts the embedded stylesheet, and applies it via JavaScript. But that defeats the elegance. It’s no longer one file.

So for cross-browser compatibility, we went with HTML.

THE HTML APPROACH

Serve the robots.txt with Content-Type: text/html and browsers render it as a webpage. The robots.txt directives sit at the top in a <pre> block—the first visible content:

<!DOCTYPE html>
<html>
<head><!-- styles --></head>
<body>
<pre>
User-agent: *
Allow: /

User-agent: GPTBot
Disallow: /

Sitemap: https://playground.wunsch.dk/sitemap.xml

# This is both a valid robots.txt AND a playable DOOM game!
</pre>
<!-- DOOM game here -->
</body>
</html>

(The examples here are formatted for readability. The actual file concatenates the HTML onto as few lines as possible, so crawlers hit the robots directives sooner.)

Why does this still work for crawlers? The robots.txt specification is line-based and forgiving. Crawlers look for lines they understand—User-agent:, Allow:, Disallow:—and treat everything else as garbage to skip. DOCTYPE declarations, HTML tags, style blocks, JavaScript? All garbage. Skipped.

One crawler’s garbage is a browser’s gold. The same lines that crawlers discard are exactly what browsers need to render a page. When a crawler hits our file, it ignores the HTML and finds valid directives. When a browser hits it, the directives are just text to display—and then it renders DOOM.

THE SIZE PROBLEM

But DOOM is big. A typical WebAssembly port is 6.8MB—the entire DOOM engine plus the shareware WAD embedded in the binary. Base64-encode that and embed it in HTML, and you’re looking at 8.8MB.

A 9MB robots.txt felt… excessive.

COMPRESSION ROUND 1: GZIP

Modern browsers have native DecompressionStream support. Compress the WASM with gzip before base64 encoding:

async function decompressGzip(compressedData) {
  const stream = new Response(compressedData).body
    .pipeThrough(new DecompressionStream('gzip'));
  return new Uint8Array(await new Response(stream).arrayBuffer());
}

Result: 2.9MB. A 67% reduction with zero additional code.

COMPRESSION ROUND 2: LZMA

Gzip is convenient, but LZMA compresses better. The trade-off is needing a JavaScript decoder. After testing several options, lzma-js won—it’s only 6.8KB minified and achieves significantly better compression.

Result: 2.2MB. Another 24% smaller than gzip.

THE WAD PROBLEM

At this point, we’d squeezed the compression as far as it could go. The WASM was 1.7MB compressed—but most of that was the embedded doom1.wad file (4MB uncompressed). The actual DOOM engine code is relatively small.

Could we use a smaller WAD?

Enter squashware, a project that creates minimal DOOM WADs:

WAD Variant	Size	Contents
Original doom1.wad	4.2MB	Full shareware (9 levels, all assets)
newdoom1.wad	1.7MB	Full shareware, compressed
newdoom1_1lev.wad	657KB	E1M1 only + sounds
newdoom1_1lev_silent.wad	513KB	E1M1 only, no sounds

The E1M1-only WAD is 84% smaller than the original. One level is enough to prove the concept.

RECOMPILING CHOCOLATE DOOM

The solution was straightforward: recompile Chocolate Doom with the smaller WAD embedded. The WAD is included at build time, so swapping in the E1M1-only version meant:

Replace the WAD file
Update the size constant
Rebuild to WebAssembly

The Chocolate Doom project compiles cleanly to WASM using Emscripten. With the smaller WAD baked in, the resulting binary dropped dramatically in size—and compresses even better since there’s less redundant game data.

THE FINAL ARCHITECTURE

The robots.txt file structure:

<!DOCTYPE html>
<html>
<head><!-- styles, LZMA decoder --></head>
<body>
<pre>
User-agent: *
Allow: /

User-agent: GPTBot
Disallow: /

# This is both a valid robots.txt AND a playable DOOM game!
</pre>
<div class="game">
  <h1>DOOM</h1>
  <canvas id="screen" width="640" height="400"></canvas>
</div>
<script id="wasm-data" type="application/wasm-lzma-base64">
XQAAAAT//////////wA... <!-- ~800KB of LZMA-compressed DOOM -->
</script>
</body>
</html>

The HTML page contains:

Robots directives in a <pre> block at the top
A <canvas> element for rendering
The WASM binary as LZMA-compressed base64 in a script tag

FINAL RESULTS

Version	Size	Compression
Inline base64 (uncompressed)	8.8MB	—
Inline gzip	2.9MB	67% smaller
Inline LZMA (full shareware)	2.2MB	75% smaller
Inline LZMA E1M1 (recompiled)	~800KB	91% smaller

The final file is a single HTML document containing:

Valid robots.txt directives (in a <pre> block)
The Chocolate Doom engine compiled to WebAssembly
E1M1 game data
LZMA decoder (6.8KB)
All necessary JavaScript

You can save it, email it, host it anywhere. One file, no dependencies, plays DOOM.

LESSONS LEARNED

Compression choice matters. LZMA beats gzip by 24% for this data, and the 6.8KB decoder cost pays for itself immediately.

Start with less data. Compressing a smaller WAD beats compressing a larger one. The E1M1 build shrank the source material by 84% before compression even started.

Browser compatibility is pain. The pure XML/XSLT solution works perfectly in Firefox. Chrome’s limitations forced us to HTML. Sometimes elegance loses to pragmatism.

The web is weird and wonderful. File formats are just conventions. Sometimes you look at a robots.txt and think “this could be DOOM.”

THE FUTURE

Ironically, XSLT support is being phased out of most browsers. Chrome and Safari are deprecating native XSLT processing. The dream of self-contained XML files seems to be fading.

But excellent work is being done on XSLT polyfills. These JavaScript implementations can take over where native support ends—and crucially, a polyfill could be made to support inline stylesheets. The very feature Chrome never implemented.

By late 2026, when native XSLT is gone, we’re probably closer to being able to use a single-file XML than ever before. The elegant solution becomes possible precisely when the “native” path disappears.

Sometimes the web giveth by taking away.

WHY?

Because crawlers don’t care. They parse their directives and move on. Browsers render a first-person shooter. Everyone gets what they came for.

Robots.txt - will it run doom?