Ranking vs. Relevance

Defining Relevance

Merriam-Webster defines relevance as “the ability (as of an information retrieval system) to retrieve material that satisfies the needs of the user.”

Ranking

Ranking is simply an ordering of results determined by the search engine. Typically, the search engine computes a score for each retrieved result and sorts the retrieved results by that score. We can think of retrieval itself as the most significant bit of the score used for ranking: for the purposes of ranking, unretrieved results have a score of zero.

Ranking vs. Relevance

There is clearly a relationship between ranking and relevance: searchers expect the top-ranked results to be relevant to their information needs. But beyond that, things can get a bit messy.

Summary

Ranking and relevance are related but distinct concepts. Relevance is essentially a binary measure of whether a result addresses the searcher’s need, while ranking sorts relevant results based on searcher and business objectives. Relevance provides the most significant bit to the ranker, while the ranker takes into account query-independent signals, such as quality or popularity, as well as factors like prototypicality. Ranking is also a place to ensure result diversity, through reranking the top-scoring results. Finally, ranking is an opportunity to make tradeoffs between searcher and business objectives, e.g., through promoted search results.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store