Comparison of citation-based journal ranking schemes.

We present results for 13 journals that the ISI classifies primarily in experimental psychology, and 36 journals that the ISI classifies primarily in ecology (see Appendix S3 for other fields). For every pair of journals, Ji and Jj, belonging to the same field, we obtain the probability pij that a randomly selected paper published in Ji has received more citations than a randomly selected paper published in Jj. We rank the journals in each field according to three schemes: (A) optimal ranking RAUC, that is, the ranking that maximizes pij for R(i)j); (B) ranking according to decreasing (J); (C) ranking according to decreasing JIF. We plot {pij} matrices for each of the fields and ranking schemes using the color scheme on the right. Green indicates adequate ranking, whereas red indicates inadequate ranking. It is visually apparent that the ranking according to decreasing (J) provides nearly optimal ranking, whereas ranking according to decreasing JIF does not. As an example, consider the journals Brain and Cognition and Journal of Experimental Psychology: Learning, Memory, and Cognition. The JIF ranks Brain Cogn. third and J. Exp. Psy. fourth. However, the median number of cumulative citations to the papers published in the latter is 34, and only 3 for papers published in the former. Not surprisingly, the probability of a randomly selected paper published in J. Exp. Psy. to have received more cumulative citations than a randomly selected paper published in Brain Cogn. is 0.88.