Comparison of citation-based journal ranking schemes.

We present results for 13 journals that the ISI classifies primarily in experimental psychology, and 36 journals that the ISI classifies primarily in ecology (see Appendix S3 for other fields). For every pair of journals, J_i and J_j, belonging to the same field, we obtain the probability p_ij that a randomly selected paper published in J_i has received more citations than a randomly selected paper published in J_j. We rank the journals in each field according to three schemes: (A) optimal ranking R^AUC, that is, the ranking that maximizes p_ij for R(i)j); (B) ranking according to decreasing q̅(J); (C) ranking according to decreasing JIF. We plot {p_ij} matrices for each of the fields and ranking schemes using the color scheme on the right. Green indicates adequate ranking, whereas red indicates inadequate ranking. It is visually apparent that the ranking according to decreasing q̅(J) provides nearly optimal ranking, whereas ranking according to decreasing JIF does not. As an example, consider the journals Brain and Cognition and Journal of Experimental Psychology: Learning, Memory, and Cognition. The JIF ranks Brain Cogn. third and J. Exp. Psy. fourth. However, the median number of cumulative citations to the papers published in the latter is 34, and only 3 for papers published in the former. Not surprisingly, the probability of a randomly selected paper published in J. Exp. Psy. to have received more cumulative citations than a randomly selected paper published in Brain Cogn. is 0.88.