I’ve spent the last few weeks attending various conferences and a hot topic was Agentic Search, namely the idea that search has a new user – AI agents. There’s a lot of excitement about this topic – how it will affect e-commerce search, how APIs and backends will need to change – but I think the picture is a little more complicated.
The view from the conference circuit on agentic search
At Commerce AI Summit in London (thanks Vespa for the invite), which was a networking event with us all moved around between different tables, I spoke to a lot of people on the retail side who were indeed worried about agentic search – although they were generally a little unclear what this actually means. Anecdotes about ‘my kids use ChatGPT more than Google’ were common, but actual numbers about the traffic from AI agents were more vague. AI consultants were more enthusiastic (but then again, they were at the event to market their services). As ever, the confusion between SEO and site search was ever present, and the fear of Amazon and other big players was high – mainly because by giving their agents access to your catalogue could make it possible to carry out transactions entirely outside your own site, with users never visiting (and perhaps discovering other things to buy).
Berlin Buzzwords had a number of talks about agentic search and it was a common topic in the chats in between the talks (my favourite part of the event!). Jo Kristian Bergum, who left Vespa to become CEO of startup Hornet, was clear on the differences – benchmarks assume a single query whereas agentic search is more of an automated loop of several queries, evaluations and refinements, so we’re going to need new benchmarks. Hornet is very much focused on being the search platform most suited to agentic workloads. His colleague Lester Solbakken had some interesting points on diminishing attention (a key part of how transformer models work) as the whole set of results will be scanned by the agent, rather than by a human who rarely gets past the first page and is more interested in the top few results. He also talked about how irrelevant results can be poison to AI reasoning and should perhaps be judged as -1, not just zero (in binary judging we use 1 to mark relevance, 0 for irrelevant). I liked his concept of ‘defensive retrieval’. Hajer Bouafif from AWS had a great take on personalisation with agentic memory with OpenSearch – I was very pleased to see the mention of User Behaviour Insights, an idea I’ve championed, as a way to capture user behaviour for future use. There were some other relevant talks I didn’t attend, but everything is now available on Youtube.
At MICES, the e-commerce event that follows Buzzwords, there weren’t any talks specifically focused on agentic retrieval but it was a common topic of conversation. The approaches from retailers vary between ‘we know it’s coming but there’s not enough traffic for us to worry about it yet’ to ‘we need to get ahead of the game so let’s spin up a MCP server right away’. However, the talks that did happen were often focused on today’s problems, not tomorrow – why Otto’s hybrid search raised conversion but led to more complaints, whether adding chatbots just provides another confusing user interface, to my own presentation on how much searchandizing is too much (slides are here, the videos for my talk and all the others should appear soon).
Back to reality
Back in the real world, my consultancy practice is also focused on today’s (and sometimes yesterday’s) problems: bad relevance, a lack of ways to measure search quality, poor explainability and imperfect tuning processes. There are still thousands of companies – some very large – out there running old versions of search engines rather than the latest AI platform (I joked with Torsten Bøgh Köster at MICES about how many times we’d seen Solr 4 – released in 2012 – still in use in the wild). AI initiatives themselves are great ways to spent money, but often don’t get much of a return. Agentic search itself is a thing many teams would love to investigate but it’s in the ‘when we have time’ pile. Today’s problem is dealing with human, not agentic users.
Should companies be considering agentic search? Well, yes, but the jury is still out on how much of an impact this will have on search platform workloads. That’s not to say we shouldn’t be cheering on innovative companies like Hornet, or platforms like Vespa and OpenSearch who are adding features to handle this new agentic user. We’ll need this technology to be available and their work pushes on the state of the art.
A topic to return to
The autumn conference season will give us a chance to return to this topic – at Vespa AI Live for example which I’m pleased to be co-hosting in London (tickets are available, check out the slides from my MICES talk above for a 20% discount code), Haystack Europe the week after or London Search Week including Search Solutions in November. See you then!
If you need help with traditional, semantic, hybrid or even agentic search – get in touch.