Interesting. I don't question their results, but I'd say that indicates something fundamentally wrong with how folks are doing ranking. Retrieval should essentially be a coarse-grained way to separate good from bad results, while ranking makes finer-grained distinctions between better results and worse ones. A lack of alignment between retrieval and ranking strikes me as a fundamental problem. Your thoughts?