Impersonating LLMs as an Adblocking Strategy
If you're an SEO (and if you're not why are you here), you probably have seen all the conversation about how brands should now be serving markdown to LLMs - the thought being that LLMs prefer Markdown because they don't have to parse an entire DOM, yadda yadda yadda.
There's a problem with this, though:
One, this only really works for sites or pages that are purely content led; (Markdown doesn't officially support inputs and the like, so most ecom sites are going to struggle with them), and Two, if you're creating LLM friendly pages that strip away all the guff you put readers through and just provide the content, it is trivially easy to write a browser plugin that prioritises accepting markdown and spoofs a user agent... and now human users get access to that markdown too.
That means there's no ads, no upsell banners, no mammoth JS libraries that do eye tracking, no analytics, none of the stuff that the web industry relies on to make money. Just users getting the actual content they want, stripped of all the things that annoy them - it's like adblocking, but in this scenario you're pretending to be a bot because a Bot has been given access to the content with fewer gates and hoops to jump through. Essentially, as soon as you give a bot markdown and force a user to deal with the overhydrated mess that is most Next.js sites, you're treating users as second class citizens and I think they're within their right to try and sneak into first class.
So I wrote Doubledealer, after the Transformer who was both an Autobot and a Decepticon.
It's a teeny plugin that tries pretending to be an LLM when you load a webpage, and if it doesn't work it tries again using your standard browser settings. It also remembers sites that don't work and bypasses them for next time.
You can see it working on Fastly's documentation site. - Thanks to the wonders of prewritten Markdown parsers, Doubledealer will even parse and style it for you.
Use it at your own risk, stuff might break , YMMV.
To install it:
- Click the link under Doubledealer below
- Unzip the file somewhere
- Go to about:debugging in Firefox
- Click "Load Temporary Plugin"
- Find the manifest.json file and select it
"Ah but users won't use that will they?" They will if their adblockers start including it as an option.
