More Research: Relative Recall

It has been a while since I posted about my continuing research, being a librarian can get rather busy!  Especially during the summer it would seem, when we have time to dust off old projects and get the ball rolling on new ideas, my life was full of meetings, planning, and implementing.

I mentioned in my last post that I would be researching Bing and Google in-depth, with well-selected search terms to determine the relative recall and precision of the two search engines.  It was a long process, but I got some insightful results.

First, a few words about relative recall.  If you haven’t heard of this before, relative recall is the number of results retrieved by a search engine divided by the number of total results.  So for Bing, relative recall is:

# of results retrieved by Bing / # of resuls retreived by Bing + Google

My research showed that Google surpassed Bing by a huge margin when it comes to single-word seraches, but the multi-word and complex searches had more interesting results:

Graph of Relative Recall

Graph of Relative Recall, values are %

You can see that for simple multi-word searches, the two are more evenly matched, and for complex searches, Bing surpasses Google for some of the search terms.

So, what does this even mean?  Is this really a good way to judge how well a search engine works?  The number of results was in the hundreds of millions, and sometimes billions, for each search engine.  Research shows that most searchers do not even look past the first page of results, so do you really need 100 million results vs. 10 million?  Since the first few pages of searches were very similar for both, does recall really matter?  In my opinion, it doesn’t.  However, I wanted to duplicate the research from the Usmani article.  At least I learned not to worry about recall so much!

Up next: precision.  It’s a much better judge of search engine character.


Categories: Bing vs. Google, Project Details, Research

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: