It has been a while since I posted about my continuing research, being a librarian can get rather busy! Especially during the summer it would seem, when we have time to dust off old projects and get the ball rolling on new ideas, my life was full of meetings, planning, and implementing.
I mentioned in my last post that I would be researching Bing and Google in-depth, with well-selected search terms to determine the relative recall and precision of the two search engines. It was a long process, but I got some insightful results.
First, a few words about relative recall. If you haven’t heard of this before, relative recall is the number of results retrieved by a search engine divided by the number of total results. So for Bing, relative recall is:
# of results retrieved by Bing / # of resuls retreived by Bing + Google
My research showed that Google surpassed Bing by a huge margin when it comes to single-word seraches, but the multi-word and complex searches had more interesting results:
You can see that for simple multi-word searches, the two are more evenly matched, and for complex searches, Bing surpasses Google for some of the search terms.
So, what does this even mean? Is this really a good way to judge how well a search engine works? The number of results was in the hundreds of millions, and sometimes billions, for each search engine. Research shows that most searchers do not even look past the first page of results, so do you really need 100 million results vs. 10 million? Since the first few pages of searches were very similar for both, does recall really matter? In my opinion, it doesn’t. However, I wanted to duplicate the research from the Usmani article. At least I learned not to worry about recall so much!
Up next: precision. It’s a much better judge of search engine character.