The Questions I'll Never See

I've been noticing some traffic on my blog from LLM based search. Small little drips of visitors from ChatGPT, Perplexity, and similar search tools. Not huge amounts, but enough to make me curious about what's actually happening.

Someone out there is asking these LLMs questions, and my lowly little blog is ending up in the answers. The problem is I have absolutely no idea what questions led them here or what context my writing appeared in. Did the AI misrepresent what I said? What were people even searching for that made my blog seem relevant?

At least with Google, you could sometimes reverse engineer what people were looking for. Search Console would tell you the queries that brought people to your site. You'd see "iPhone review" or "best camera for street photography" and think "right, that makes sense." With ChatGPT and Perplexity, there's nothing. Just a referral from their domain and complete darkness about why.

I wrote about how AI summaries stop people clicking through to sources back in July. The data from Pew Research showed only 8% of people click links when they see an AI overview, compared to 15% when they don't. So the fact that anyone's reaching my site at all from these tools feels almost accidental, and also means that I am showing up much more than these small numbers that I can see.

Part of me is glad someone's reading. I don't obsess over stats, mainly because I'd drive myself mad if I started worrying about page views. But I'd genuinely love to know what searches are bringing people here. Are they asking about Perplexity's advertising plans? My thoughts on Siri versus Gemini? Something I wrote years ago and forgot about?

I'm not precious about my writing. If I was, I wouldn't put it on the internet where anyone can read it or feed it to their large language model. I'd at least like to understand how it's being used and whether the AI is representing my actual position or just cherry-picking sentences that fit whatever point it's trying to make.

The reality is this is probably just what publishing on the web looks like now. Your work gets ground up, fed through an LLM, and spat out as part of someone else's answer. You get a referral if you're lucky. Most of the time you probably don't even get that.