Mapping Search Queries To Search Intents

  • Surface query similarity. Queries that only differ in stemming or lemmatization often express the same intent — especially if they represent the singular and plural form of a noun phrases. So do queries that only differ in word order or the inclusion of stop words. In synthetic languages like German, there may also be differences in compounding.
  • Similar post-search behavior. Queries that express the same intent will be followed by the same behavior, i.e., engagement with the same kinds of results. Post-search behavior can be represented in a vector space, e.g., by using the embeddings of result titles. Of course, this behavioral similarity is only possible if both queries allow users to find the same kinds of results.
  • Group queries into equivalence classes based on surface similarity, canonicalizing each query by stemming each token and sorting the tokens alphabetically, and removing stop words, e.g., “mens dress shirts” and “dress shirts for men” both map to the canonical form “dress men shirt”.
  • Split these equivalence classes into clusters of similar post-search behavior based on the cosines of vectors associated with queries, thus ensuring that queries like “dress shirt” and “shirt dress” are not considered equivalent despite their surface similarity.
  • Query rewriting. Given a pair or set of equivalent queries, we can rewrite all of them to a common query representation that ensure the optimum retrieval, ranking, and other search features.
  • Analytics. Rather than analyzing the performance of each query separately, we can group queries into equivalence classes, thus aggregating behavioral signals that would otherwise be fragmented.
  • Machine Learning. The same aggregation that improves analytics also allows us to capture more robust signals to train machine-learned models.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store