This publication is a series of posts devoted to query understanding. Each post is self-contained, but there is a natural progression. If you’d like to read them all, start with the first one.
Regular readers know that search is about much more than ranking. If you’re new here, you might want to read this post about ranking vs. relevance.
But a topic I’ve neglected is search result diversity. I’ll remedy that in this post.
Search result diversity is not for ambiguous queries.
Search can be computationally intensive. Indeed, search has been the driving force behind many advances in computational efficiency, from MapReduce for distributed indexing to approximate nearest-neighbor methods.
But not all computational investments yields equal return. A search engine has a limited computational budget, so it should allocate that budget wisely…
As a high-class consultant, I don’t just talk to companies about the technical aspects of search. I also help them them address organizational challenges.
For example, how should a search team be structured? Should search be combined with recommendations? …
The first rule of the Free Advice Club is: you talk about the Free Advice Club! Not like that other club.
But I’m getting ahead of myself. I’ve had an extremely fortunate career trajectory. When I reflect on what — besides a massive amount of luck — has made it…
Several weeks ago, I wrote a post advocating the use of AI for query understanding. …
In 1668, John Wilkins published An Essay Towards a Real Character, and a Philosophical Language. In it, he proposed a universal language that would represent every concept with its own symbol.
Here is an excerpt from his treatise:
Needless, to say, Wilkins did not succeed in this monumental and aspirational…
Most folks who work on search worry about relevance. But it’s surprisingly difficult to find a useful definition of relevance.
Defining Relevance
Merriam-Webster defines relevance as “the ability (as of an information retrieval system) to retrieve material that satisfies the needs of the user.”
William Goffman defines it as “a measure of…
For feeds and recommendations, ranking is critical. All the inputs are implicit, so machine-learned ranking is the only practical way to optimize engagement.
Search is different.
A search engine elicits the searcher’s explicit intent, expressed as keywords, and this explicit intent is, by far, its most valuable input. Searchers, quite understandably, expect results…
In improv, there’s a fun game called “new choice”. At any point during the improvised scene, the host can call for a “new choice”. The player who last spoke has to backtrack and substitute a new line. …
