SEOs, your competitors may be covertly switching their content from HTML to markdown for AI agents. You can analyze by auditing their Accept Headers. Here's the process:

Covertly? lol Just use and tick the “content” checkbox. That accept header test is covered of course

We tested AI bots to see what request headers they actually send on real traffic. Surprisingly, nobody except one accepts Markdown. I personally don't understand this obsession with Markdown. Converting HTML to Markdown during parsing isn't a big deal.

Serving the markdown from the same URL with Vary: Accept keeps it off the index and normal crawlers still get the HTML.

Tested and approved

Gives me “amp version” vibes all over again.

I think that to the extent that we want agents to be able to take action on our sites, we shouldn’t rely on crutches but work on aria labels etc.

Interesting discussion here. In my opinion, I think JSON is the preferred language of ai agents, particularly for datapoints on a page. Markdown is more like a bridge for humans to copy over info to their AI for a chat discussion.

Should have this working (better) in Rocketship SEO for WordPress here soon. We've been doing it for about a year or so. ChatGPT and other LLMs sometimes try and work around it.