Regular readers know that search is about much more than ranking. If you’re new here, you might want to read this post about ranking vs. relevance.
But a topic I’ve neglected is search result diversity. I’ll remedy that in this post.
Search can be computationally intensive. Indeed, search has been the driving force behind many advances in computational efficiency, from MapReduce for distributed indexing to approximate nearest-neighbor methods.
But not all computational investments yields equal return. A search engine has a limited computational budget, so it should allocate that budget wisely…
In 1668, John Wilkins published An Essay Towards a Real Character, and a Philosophical Language. In it, he proposed a universal language that would represent every concept with its own symbol.
Here is an excerpt from his treatise:
Most folks who work on search worry about relevance. But it’s surprisingly difficult to find a useful definition of relevance.
Merriam-Webster defines relevance as “the ability (as of an information retrieval system) to retrieve material that satisfies the needs of the user.”
William Goffman defines it as “a measure of…
For feeds and recommendations, ranking is critical. All the inputs are implicit, so machine-learned ranking is the only practical way to optimize engagement.
A search engine elicits the searcher’s explicit intent, expressed as keywords, and this explicit intent is, by far, its most valuable input. Searchers, quite understandably, expect results…