Comparison of citation-based journal ranking schemes.

<p>We present results for 13 journals that the ISI classifies primarily in experimental psychology, and 36 journals that the ISI classifies primarily in ecology (see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0001683#pone.0001683.s003" target="_blank">Appendix S3</a> for other fields). For every pair of journals, <i>J<sub>i</sub></i> and <i>J<sub>j</sub></i>, belonging to the same field, we obtain the probability <i>p<sub>ij</sub></i> that a randomly selected paper published in <i>J<sub>i</sub></i> has received more citations than a randomly selected paper published in <i>J<sub>j</sub></i>. We rank the journals in each field according to three schemes: (A) optimal ranking R<sup>AUC</sup>, that is, the ranking that maximizes <i>p<sub>ij</sub></i> for R(<i>i</i>)j); (B) ranking according to decreasing <i>q̅</i>(<i>J</i>); (C) ranking according to decreasing JIF. We plot {<i>p<sub>ij</sub></i>} matrices for each of the fields and ranking schemes using the color scheme on the right. Green indicates adequate ranking, whereas red indicates inadequate ranking. It is visually apparent that the ranking according to decreasing <i>q̅</i>(<i>J</i>) provides nearly optimal ranking, whereas ranking according to decreasing JIF does not. As an example, consider the journals <i>Brain and Cognition</i> and <i>Journal of Experimental Psychology: Learning, Memory, and Cognition</i>. The JIF ranks <i>Brain Cogn.</i> third and <i>J. Exp. Psy.</i> fourth. However, the median number of cumulative citations to the papers published in the latter is 34, and only 3 for papers published in the former. Not surprisingly, the probability of a randomly selected paper published in <i>J. Exp. Psy.</i> to have received more cumulative citations than a randomly selected paper published in <i>Brain Cogn.</i> is 0.88.</p>