Thanks. And sorry that slide was cryptically brief. What I mean is that spelling correction should not restrict itself to correcting each token separately. Many spelling mistakes violate token boundaries because the user either omitted a space or included one that wasn’t necessary. In addition, the tokenization of queries with punctuation and special characters may not match how the text was tokenized in the index. But, regardless of how tokenization works, it should be possible — and practical — to correct a misspelled head query by thinking of the query as a single string; rather than first tokenizing the query and performing spelling correction one token at a time.

High-Class Consultant.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store