Post

I am sure this will be hugely unpopular with a lot of people; but I think AI prompt visibility tracking is generally a big and expensive waste of time. 🤷‍♂️

The SEO industry has lazily tried to apply the 'old' paradigm of 'rank tracking' to LLMs, and frankly, it doesn't work well because it doesn't reflect how LLMs operate.

Traditional search engines rank documents in a linear order based on their perceived relevance to a query, LLMs contextually and non-deterministically generate an answer in language, not rankings.

Some search professionals herald the dawn of conversational AI because humans can interact with 'real' language instead of queries; yet still want to try and measure results with the former, rather than the latter.

My thoughts have been shaped by discussions with @myriam.dopamine.builders on this, but I believe @dixonjones.bsky.social is heading in the right direction with Waikay; it's more important to get an understanding of how the model perceives your brand, your product, your service.

1980s: Brand is what you tell people with billboards, TV ads, and writing things in the sky with planes 2000s: Brand becomes what people collectively say about you, as they themselves become hyper-connected on the web

It makes no sense to try and explore the latter by bombarding it with prompts; using a handful as a 'dipstick' to see where you are might be interesting, but this should not be the focus.