Jump to content
  • 0

How To Compare Two Ranking Algorithms?


mk65
 Share

Question

Hello all,
 
I want to compare two ranking algorithms. In these algorithms, client specifies some conditions in his/her search. According to the client`s requirements, these algorithm should assign a score for each items in data base and retrieve items with highest scores.
 
I have searched the net. According to my searches, the most relevant article which explains about some metrics for comparing ranking algorithms, was this:
http://www.icml2010.org/papers/504.pdf
I think prec@k, MAP, MRR, and NDCG, are good metrics to use, but I have a problem:
 
My algorithm sort results, so the first item in my result list is the best one with highest score, the second result have the second top score, and so on. I limit my search algorithm to for example find 5 best results.The results are the most top 5 items. So,precision will be 1. When I limit my search to find best result,It finds the best one. Again, precision will be 1.But the problem is that,it is unacceptable for people who see this result.
 
What can I do? How I can I compare these algorithms and show one is better than the other?
Please help me. Thanks a million.

Link to comment
Share on other sites

3 answers to this question

Recommended Posts

  • 0

Hello mk65,

 

If I understand correctly, it is better to look for "the best algorithm" for you, because algorithms were created to solve specific tasks. So, some algorithm are better for something, and other are better for others.

 

Do you want to use algorithms on your Search DataPages? :)

Link to comment
Share on other sites

  • 0

Hello mk65,

 

If I understand correctly, it is better to look for "the best algorithm" for you, because algorithms were created to solve specific tasks. So, some algorithm are better for something, and other are better for others.

 

Do you want to use algorithms on your Search DataPages? :)

 

Hello.

Thanks for your attention.

No. I don`t want to use a search algorithm.

I have written an algorithm for a specific search in a topic, which have used ranking algorithms. In addition, there is another algorithm in this topic. I have implemented both of these algorithms.  I should prove my algorithm is better than the other. I think prec@k, MAP, MRR, and NDCG, are good metrics to use, but I have the  problem mentioned in my question. I don`t know how to solve this? How to show my algorithm is better than the other in this topic?

Please help me.

Thanks a lot.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...