In grade school, we were taught the three Rs: reading, writing, and ‘rithmetic. In search, we can be thankful that the three Rs actually start with the letter R: relevance, recall, and ranking.

Relevance

Relevance is the prime directive of search: the guiding principle for a search engine is to return results that satisfy the searcher’s information need. That means understanding what the searcher wants and retrieving relevant results.

Achieving relevance is a trade-off between precision and recall. We’ll discuss recall in a moment, but precision is the measure that people associate most with relevance: the fraction of results that satisfy…

Search developers tend to focus most of their efforts on the first page of results. As a result, they prioritize investment in ranking models, with the goal of improving quality and business metrics, such as relevance and conversion.

Precision and Recall

In information retrieval terms, this focus on the first page corresponds to an emphasis on precision, the fraction of results that are relevant. To be more precise — no pun intended — it corresponds to an emphasis on position-biased precision measures like discounted cumulative gain (DCG).

But precision isn’t the only measure of search quality. There’s also recall, which measures the fraction…

Francis Fukuyama, Barak Richman, and Ashish Goel recently published a piece in Foreign Affairs, which they ambitiously titled “How to Save Democracy From Technology: Ending Big Tech’s Information Monopoly”.

The gist of their proposal is to take away the role of giant platforms (i.e., Google, Facebook, Twitter) as gatekeepers of content by allowing users to choose from among middleware companies to manage information access. They see this approach as addressing the threat that the concentration of information platforms poses to democracy — and to society generally.

Ads? Hold That Thought…

I believe that this threat is compounded by an ad-supported…

Faceted search is a fascinating topic. It’s a standard feature of site search, and one could write an entire book on the subject. In this post, I’ll focus on some nuances of faceted search that I feel have been neglected in the literature.

Broad Queries vs. Ambiguous Queries

Both search engine developers and users treat facets as useful for refining broad search queries. But there’s a tendency to conflate broad queries with ambiguous queries. There’s an important distinction between the two.

Broad queries are unambiguous but underspecified. For example, the query “shirts” expresses a clear but underspecified intent: it includes…

A few years ago, I proposed a non-adversarial ad-supported model: an explore-exploit model, applicable to search engines and feeds, that aligns the interests of consumers and advertisers.

This model would treat ads as new, potentially relevant content to be thrown into an exploration funnel. Participating in exploration would be the price that users collectively pay in order to benefit from the exploitation of signals derived from that exploration.

Since advertisers benefit from users seeing their relevant ads, they would be willing to pay for that exploration work with cash. Some ads would prove worthless, in which case advertisers would have…

In 1984, Robin Williams starred in Moscow on the Hudson as a Russian saxophonist who decides to defect from the USSR during a shopping trip to a New York department store. One scene stands out, particularly to those of us who focus on search experience.

This scene, shown below, depicts the protagonist going to the supermarket to buy coffee. Expecting to find a “coffee line” where he will have to queue up to get a single kind of coffee, he is literally overwhelmed by the abundance of coffee options available to him in the “coffee aisle”.

Some people interpret…

When we design search applications, we aspire to make the user experience frictionless. A search engine should “just work”, enabling searchers to easily express their intent and responding with exactly the results searchers need.

Unfortunately, our search applications tend to fall short of our aspirations.

Most search analytics focuses on measuring friction in the search experience. For example, the mean reciprocal rank (MRR) of clicks tells us how far users have to scan down the results to find what they want. Search abandonment rate reflects how often users don’t find what they want at all. …

Evaluating and improving search experience starts with analyzing the queries searchers are making. But search queries are not the same as search intents.

I’m not talking about ambiguous queries like “java” or “jaguar” — examples that information retrieval researchers often use to illustrate how a single search query can map to multiple search intents. Ambiguous queries are fascinating in theory, but in practice they tend to be rare edge cases.

I’m talking about the opposite: when multiple queries mapping to the same intent. For example, queries like “mens shoes” and “shoes for men”.

Recognizing when two or more search queries…