I’m currently working as a fractional Search Lead at Moonpig, the first company to sell greeting cards and related gifts online in the UK. The job title is somewhat vague, so I thought I’d give you a general insight into how I approach consulting around search & AI – and show you it’s not just about technology. What I write below isn’t specific to Moonpig and is based on my wider experience in search engine consulting.
Learning the World
The first thing any consultant has to do when starting a new engagement is to learn the world of the organisation they’ve been brought in to help. Every organisation is different, even those operating in the same sector and of relatively the same size. You need to understand the organisation’s purpose (selling greeting cards, or providing medical information, or assisting lawyers), rough structure, the internal language and terms, business processes and how people around search interact. I’m not a lawyer, doctor or greeting card expert, but I need to know enough about these sectors to figure out how great search can help.
There are several ways to learn: from structured approaches where you systematically ask a series of questions of the search team, to reading background documents and/or shadowing key individuals. The principle that ‘no question is a stupid question’ applies here – and you may need to ask many stupid questions! I make copious notes during this process, often record online calls and I’m beginning to use AI transcription tools to assist me. I’m relaxed about the fact that I’ll probably repeat myself occasionally and I try to stay humble: after all, my new colleagues know much, much more about their organisation than I do, especially about the special cases, workarounds, shortcuts and hacks that always exist. Being judgemental (‘why on earth have you done it that way?’) isn’t at all helpful – there are usually good reasons why things are set up a particular way, and there is always technical debt, missed opportunities and an endless backlog of things to fix one day, when there is time.
If appropriate, one might provide a list of written questions to be answered ahead of the start date – but remember the answers may not be definitive, or can be just plain wrong, as you might discover when you actually arrive.
Plugging in
Fitting into an existing team is always difficult: there may be suspicion of your consultant status, existing social norms you don’t understand, processes and rituals that seem to make no sense – my approach is always to try to disturb things as little as possible. If there’s a regular standup or other meetings then I start attending them, but mainly to listen. No-one wants to hear your grand opinions when there’s a Kanban board to be got through.
If you’re lucky (and I have been at Moonpig) the onboarding process will be smooth, and you’ll have the various logins, equipment and other things you’ll need to participate on day one. You’ll also be grateful for friendly greetings, occasional check-ins to make sure you’re settling well and invitations to meet the team in more informal settings. This ‘social glue’ helps immensely, but some organisations struggle with it – often because there are many more urgent things to do. Politeness, patience – and an understanding that yours are not always the most urgent questions to answer – are vital.
Wait and see
It’s quite tempting to come up with a Grand Plan within days of arriving at a new engagement. In glorious technicolor detail, this will show how your recommendations will change everything, please all the customers, make more money and generally Do Search Right. However the reality is that you simply don’t know enough yet to go into any detail. I expect it to take a good few weeks before I can recommend anything more than general initiatives – for example, I’ll certainly recommend offline relevance testing (if it’s not being done already) but I have no idea what queries we’ll test, what tools we’ll be able to use and how often we’ll be able to do it. That said, a general outline of the areas you hope to work it will be useful, if for nothing else to check against your client’s expectations.
Get your hands dirty
At some point you’ll want to figure out what’s really going on with search. With some experience, you can try out various types of queries and identify whether or how certain features have been implemented – for example, how does search cope with typos, are there some dead ends where you hit zero results, do the facets make sense. You can do most of this with the main search interface, but at some point you’re going to have to go deeper.
I have a good general understanding of how search works at a low level – terms, frequencies, indexes etc. – and can find my way around most search engine specific query options (usually Domain Specific Languages or DSLs, or collections of named parameters). With a suitable test bed (for example Quepid or the OpenSearch DevTools console) I can try out actual search queries using a provided example, fiddle with parameters, chop bits out and see what happens. This can be a great way to find odd bugs, part-implemented features and ways in to improve search quality: but remember there may also be very good reasons why something is how it is. This will lead you to more questions for the team: why is that boost level 15.5? Where do the vector embeddings come from? Why aren’t we searching this text field?
You’ll also need to see the code that actually calls the search engine – I’m lucky that over a long career in IT I’ve worked with everything from assembler to Java, C++ to scripting languages, so I can usually figure out what code is doing (just don’t ask me to read LISP). Once I’ve pulled things apart (and asked more stupid questions) I can make some more concrete recommendations for improvement. I’m unlikely to write any code – the team should do this, as they’ll have to maintain it – but I can show examples and general patterns.
It’s also worth looking into how the index is generated from content, as content quality is the Achilles heel of so many search applications. Wear your protective goggles for this task!
Education, education, education
One of the main tasks of a consultant is to teach – and this is necessary at many different levels. You may have to explain high-level search concepts to an executive team, how spelling correction differs from typo tolerance to a product owner or how the Snowball stemmers work to an engineer. If you don’t know the answers they seek, do some research rather than glossing over the subject. No-one will expect you to be the all-seeing oracle but you are a search expert so you should at least know where to look!
More formal training is also an option – if you have experience and can provide this (and the team have time available) it can be a great way to level up search knowledge, especially for those relatively new to search and information retrieval.
I like to provide a rolling list of useful education resources – conferences to go to (I maintain a calendar of these), books to read, relevant links to follow, people to be inspired by – and remind people about this regularly, mentioning any exciting new additions. I’m very willing to recommend resources created by other search consultants, many of which I hugely admire – here’s a great example from Peter Fries, the ‘penny farthing’ of offline and online testing from his Haystack 2018 talk:
Be enthusiastic
One of the most vital things in my view is to be enthusiastic about what can be done to improve search – perhaps with better testing, or an upgraded infrastructure, or (of course) with AI. No-one wants to hear that everything is broken, they’re years behind the competition and nothing can be done. Be realistic about the challenges and don’t over-promise, but with your experience guiding them any search team should be re-invigorated by your suggestions and ideas. There’s always something more that can be done to improve search – it’s a never-ending journey!
The search consulting endgame
The aim of any search consultant should be to leave things in a better state than before they arrived – but remember not all your suggestions will be implemented, often for very good reasons of time and budget. If you have made a real difference to search quality, then hopefully both you and the client will be able to announce this publically, which can be hugely satisfying.
However one should also remember that it’s people that really matter – so if you’ve helped others gain knowledge and experience that will serve them in the future as they move towards search Nirvana, this can be a great reward.
If you need help building better search, then get in touch to discuss a future engagement.