Let’s pan out from technologies, models, the firehose of news for a few minutes and consider the wider problem of connecting users to information using search & AI.
AI systems are often presented as a panacea, able to solve all business problems if only we throw enough computing power at them. But businesses serve people – external and internal users – and it’s the people we need to help.
Solving user problems
Search and information retrieval in general are ways to solve user problems. I want to buy a washing machine for my family, with a limited budget and an eco-conscious mindset. I’m a lawyer looking for a past case, but I can’t quite remember the details. I work in a factory and I need to buy some specific parts for a machine – I miss the old paper catalogue from our supplier and don’t really want to use a computer, I’d rather talk to an expert. I need a holiday, but I’m not sure where to go, I need inspiration!
Over the years we’ve constructed complex information systems to gather, categorise and structure information in ways that (we hope) users can access effectively. We hope, because there’s a huge issue – our users don’t know what information we hold, or how we’ve organised and described it. To them, it’s often a black box. In the past they’d engage the services of an information expert deeply familiar with this box and what’s in it – a librarian or shop assistant for example – and through the medium of a conversation, they’d work together to find the information they need to carry out their task. As information volume grew, this became an unsustainable approach and we now rely on information retrieval systems to find what we need.
A question of translation
Many of these retrieval systems force the user to describe their information needs in very short sequences of words. These words probably don’t fully describe their needs and the words themselves may not match the contents of the black box. This is the central problem of search, a problem of mis-matched language: my user is not an expert in my field, writes their question in a different way to what we expect, or is using text to attempt to find something that isn’t textual, like an image.
For more complex or sector-specific information, some retrieval systems introduced complex query syntaxes that the user would have to learn to describe their information need, or a plethora of interface elements (filters, checkboxes etc.) they would have to choose from. Again, this complex query may not match the content, but at least with some training and practice the user could learn how to formulate an effective question. This type of search is common in areas such as law, patent examination, medical and recruitment, often described as ‘professional search’.
As search evolved, we developed many other ways to help our users: enhancing the findability of our content with metadata, structure and entity identification; replacing filters we might have to choose (without knowing exactly how they would restrict results) with facets allowing the user to cut down a long list of results; adding auto-suggest and auto-complete features to the search box to shortcut the query process; suggesting ways to better structure the query; using curated synonyms as a translation aid; adding spelling suggestions based on the words we knew from our content; developing complex ranking algorithms to better order the results.
All of these can help of course, but the central problem still remains: our users and our content are speaking different languages.
Search can be regarded as a translation problem (watch this great talk from Doug Turnbull & Tommaso Teofili at Berlin Buzzwords) – turning the language of the user into the language of our content for better matching. We may need to reformulate the user’s query into something more useful, without necessarily knowing the context of this query – if we have only been given two or three words to work with, we lack context and the user’s intent is unclear.
Modelling language for search
The rise of AI and specifically the rise of language models over the last few years promises us new tools for solving this issue of mis-matched language. Vector embeddings, derived from language models, give us a way to encode meaning, not just index specific words or terms. Language models can also help us reformulate and expand user queries into something more likely to match our content. Fine-tuned models can learn about our own content with all its specific terms, acronyms and shortforms.
Embeddings perform better than traditional keyword search in many cases where there is a language mismatch, but keyword search is still useful when the query is exactly specified, for example when searching for a part number, or for items which must exactly match a particular quality. A hybrid of keyword and vector-based retrieval is often best, though how to mix the results from these two very different worlds remains a hard problem. Fine-tuning is expensive and time consuming, so we often have to rely on open source or commercial language models that don’t know about our business or area of interest.
Multi-modal models give us an opportunity to help those users searching for an image that isn’t labelled with the words we type. Models trained on images of documents help reduce errors due to difficult and potentially faulty extraction of text from certain file formats such as PDFs.
Changing the way we search
We can also consider how AI can help us to move away from traditional models of information retrieval – a typed query and ranked list of results. Conversational search (via a chatbot) lets us replicate that interaction with a librarian or subject matter expert, drawing out more context from the user during a chat session as we zero in on what they actually need. LLMs can also be used to summarise a set of results into a single answer – but due to the well-known hallucination problem, they cannot always be relied on: if they don’t contain the information we need, they may make up a plausible but incorrect answer. This is hugely risky for businesses that depend on reliable information, for example in the legal or medical sectors.
Retrieval Augmented Generation (RAG) is a technological solution to the need for a single answer to our user’s question, grounding a LLM in truth, backed by some kind of search engine. This engine – the R part of RAG – doesn’t necessarily need to be AI-driven, it can be a traditional search. The LLM is prompted to summarize an answer based on what the search engine retrieves – and to admit when it doesn’t have enough information to do so. Not everyone is convinced RAG is the best way forward and there are some other approaches using automation to combine search and LLMs.
Interestingly, a lot of the activity in this area comes from outside the world of information retrieval – companies want to “do AI”, build RAG or a chatbot, but they don’t always consider the vital search component, or that traditional search techniques may be a simpler way to solve the user problem.
And so we come full circle – search needs AI to succeed, but often AI also needs great search. We’re not going to replace every search engine with a chatbot, or use only vector embeddings for every search. Our toolbox has gained some more tools, we’ve not thrown it out and bought a robot that will make everything for us.
Choosing the best tool for the job with search & AI
I’ll finish with a very basic example of how to improve user experience (thanks Eric Pugh for suggesting this). In the Firefox web browser I use, if you type into the address bar something that looks like a URL (e.g. bbc.co.uk) and hit Enter, it will take you directly to that site. If you type something more like a multi-word query (e.g. bbc website) it will trigger a web search. This deceptively simple behaviour shortens the user journey.
Let’s expand on this concept a little using our enhanced toolbox, remembering that we’re here to solve a user problem. Our information systems should take the path of least resistance whenever possible, and adapt to the user’s language. If I type in 'big washing machine' then let’s do a hybrid search, using embeddings to cope with the multiple meanings and synonyms of 'big‘; if I type in 'hotpoint nswr 846' then let’s be more specific, searching using keywords and a brand filter for that model as I’m probably not interested in Indesit or Samsung machines. If I type in just 'washing machine' let’s trigger a chatbot to ask for more detail, rather than giving the user a confusing list of thousands of items. If I type 'what is the best washing machine with large capacity under £300 for a family of 4 that doesn't want to use much electricity' then we can use RAG to provide a single answer. Vision models can support our user if they take a picture with their phone of their neighbour’s washing machine they hear good things about!
Forcing our users to learn our language causes frustration – remember they don’t know what we have or how we’ve described it. We have an arsenal of tools at our disposal that can help solve their problems, speaking their language with search & AI – so let’s be creative!
Are you struggling to combine AI and traditional search techniques? Need to figure our your search & AI strategy? Let me help.