The ranking terminology and methodology used by ScholarGPS™ was developed by a panel of distinguished scholars from academia and other non-academic institutions over a period spanning several years. After its initial development, the methodology was submitted to various academic leaders for their comments and critical review for further refinement. Previous ranking methodologies have been plagued with problems such as use of subjective qualitative components, a lack of Field /Discipline /Specialty sensitivity, confusion over the inclusion or exclusion of self-citations, whether to account for the number of authors on individual publications, and a perceived bias in favor of senior scholars. ScholarGPS™ Rankings were developed on several foundational principles: that a ranking terminology and methodology should exhibit simplicity, transparency and fairness, and that the population of scholars considered within the methodology should encompass every individual (active or retired) across the entire globe who meets our definition of a scholar. In consideration of these foundational principles, ScholarGPS™ Rankings are determined purely quantitatively, and are based on metrics which capture the productivity, the impact, and the quality of individual scholars as well as institutions. The ranking methodology is presented in graphical form in the Ranking Flowchart.
Rankings for both scholars and institutions are evaluated in four ranking categories:
Four ranking metrics are calculated across each of the preceding four categories for scholar rankings:
Additionally, three ranking options are available for each ranking metric for scholar rankings:
Ranking metrics are presented both in terms of standard competition rank[1] (e.g., "1,2,3,3,5,6...") and top percentage ( based on the complement of the percentile rank[2], e.g., "Top 3%").
Scholar rankings form the basis of the institutional rankings -- See Institutional Rankings below.
Scholars are ranked in each category and by each ranking metric for various criteria. The ranking proceeds as follows.
First, the top percentage rank[2] of a scholar within a category for any ranking metric is calculated as follows:
`N` = Total scholars within a category
`R` = Standard competition rank of the scholar within a category
`F` = Frequency of scholars with rank `R`
Top Percentage Rank (`TPR`) = `100 - [((N - R + 1) - (0.5 xx F))/N xx 100]`
where
Productivity `TPR` = `TPR_p` = Top percentage rank by publication
Impact `TPR` = `TPR_c` = Top percentage rank by citation count
Quality `TPR` = `TPR_h` = Top percentage rank by h-index
Note that `TPR` is based on the standard Percentile Rank[2], but is not rounded to the nearest integer i.e., is given as a decimal number.
Next, the ScholarGPS™ Ranksof each individual scholar, which is reported in terms of a pair of rankings `[SR, STPR]` is determined by calculating the composite score (defined as the geometric mean[4])
`S = \root{3}{TPR_p xx TPR_c xx TPR_h}`
From the distribution of these composite scores, we determine the pair
`text(ScholarGPS™ Ranks) = [text(ScholarGPS Rank),text(ScholarGPS Top Percentage Rank)]`
where
`text(ScholarGPS Rank (SR)) = text(Standard Competition Rank(S))`
and
`text(ScholarGPS Top Percentage Rank (STPR)) = text(Top Percentage Rank(SR))`
As evident, a scholar's ScholarGPS™ rank `SR` and ScholarGPS™ top percentage rank `STPR` are determined by that scholar's `S` score relative to the distribution of `S` scores of scholars in the population to which the scholar is being compared. A scholar's ScholarGPS™ Rankspair can be determined relative to the universe of scholars (Overall), relative to scholars in the scholar's Field or Discipline, or relative to scholars in any Specialty associated with the scholar.
Recall that each ranking metric can be determined relative to a Field, a Discipline, or a Specialty; can be computed with or without author and citation weighting; and can be over the scholar's lifetime or the prior five-year period. In fact, each ranking metric `M` is a multi-indexed entity `M_{F,T}^{\alpha \beta \gamma}` where `\alpha` is the duration over which the metric is determined (lifetime, prior five years), `\beta` indicates the weighting type (weighted/unweighted authors/citations), `\gamma` indicates whether self-citations are or are not included, `F` indicates the category over which the metric has been determined (Overall, Field, Discipline, or Specialty) and `T` indicates the metric type (productivity, impact, quality, or composite score). To avoid this cumbersome notation, and because it will be clear from the context what the complete specification of the metric is, we drop all but the most necessary indices. For example, `M_p` would be some metric related to productivity, and the context will identify the other circumstances under which it was determined.
Highly Ranked Scholars™ are those with ScholarGPS™ Ranksof 0.05% or better. The data used to identify the ScholarGPS™ Highly Ranked Scholars™ are based on lifetime activity, weighting each publication and citation by the number of authors, and excluding self-citations.
Top experts are also determined based on their ScholarGPS™ Rankswith publications and citations weighted by the number of authors and excluding self-citations. Top experts rank in the top 0.1% of ranked scholars in various Fields, Disciplines, and Specialties. Top experts can be determined on either a lifetime or prior five-year basis.
Institutional Rankings are based on the quality and quantity of active scholars in each institution who are highly productive (number of publications) and have generated outstanding work of meaningful impact (citations) and excellent quality (h-index). Data used in the calculations of institutional rankings are based on the lifetime contributions of the included scholars, weighting each publication and citation by the number of authors, and excluding self-citations. Uniquely, ScholarGPS™ rankings are available separately for academic institutions (universities and colleges), and non-academic institutions (private or public industries including health care systems, and research institutions).
In general, the ranking categories for an institution are representative of the following areas within an institution:
The critical mass of quality scholars necessary for institutional rankings was determined based on an extensive optimization analysis to (1) make institutional rankings nearly independent of institution size and (2) avoid penalizing institutions for scholars who are inactive, or active in only a small number of focused research areas.
Since Specialties are characterized by focused scholarly activities that may span across multiple Disciplines, and because publication and citation traditions can vary substantially from Discipline-to-Discipline and Specialty-to-Specialty, care must be taken in the ranking of institutions relative to this ranking category. However, regardless of the Specialty, the Institution's ranking in any Specialty must reflect both the quality and quantity of outstanding scholars associated with the Specialty. Based on extensive internal research and the recommendations of our panel of distinguished advisors, we calculate the institutional rankings in Specialties based on the active scholars in each Specialty.
The institutional Specialty Rank Score is defined as:
`l_s` = `sum_{a \in A}(100-STPR_a)`
where
`l_s` is the Institutional Rank Score in Specialty `s`
`A` is the set of admissible scholars in the institution relative to Specialty `s`
`STPR_a` is the ScholarGPS™ Top Percentage Rank (see Scholar Ranking above) of Scholar `a`
As for Specialties, Disciplines are characterized by focused scholarly activities, and because publication and citation traditions can vary substantially from Discipline-to-Discipline, care must be taken in the ranking of institutions relative to Disciplines. Based on extensive internal research and the recommendations of our distinguished advisors, we calculate the institutional rankings in Disciplines based on the active scholars in each Discipline.
The institutional Discipline Rank Score is defined as:
`l_d` = `sum_{a \in A}(100-STPR_a)`
where
`l_d` is the Institutional Rank Score in Discipline `d`
`A` is the set of admissible scholars in the institution relative to Discipline `d`
`STPR_a` is the ScholarGPS™ Top Percentage Rank (see Scholar Ranking above) of Scholar `a`
ScholarGPS associates each Discipline with one and only one Field. Therefore, institutional excellence in the Disciplines that comprise a Field will yield a high institutional ranking in the Field. Based on our research and the recommendations of our advisors, we calculate the institutional rankings in Fields based on the active scholars in each Discipline for those Disciplines belonging to the Field of interest.
The institutional Field Rank Score is defined as:
`l_f` = `sum_{a \in A}(100-STPR_a)`
where
`l_f` is the Institutional Rank Score in Field `f`
`A` is the set of admissible scholars associated with the institution relative to the Disciplines belonging to Field `f`
`STPR_a` is the ScholarGPS™ Top Percentage Rank (see Scholar Ranking above) of Scholar `a`
An institution's overall ranking correlates to the institution rankings in the Disciplines that comprise the institution. Based on our research and the recommendations of our advisors, we calculate the institutional rankings based on the active scholars in each Discipline.
The institutional Overall (All Fields) Rank Score is defined as:
`l_o` = `sum_{a \in A}(100-STPR_a)`
where
`l_o` is the Overall (All Fields) Institutional Rank Score
`A` is the set of admissible scholars associated with the institution across all Disciplines
`STPR_a` is the ScholarGPS™ Top Percentage Rank (see Scholar Ranking above) of Scholar `a`
Publications with over 30 authors are not included in scholarly profiles or in ScholarGPS™ Ranking schemes because in such cases it is often too difficult to ascertain individual contributions to the publication.
A scholar's ranking in a category is presented only if it is in the top 50%.
In order for a Specialty to be included in Institutional Rankings, it must have a critical mass of at least 6,000 scholar profiles associated with it.
A maximum of up to 100 institutions may be ranked for a Specialty.
It is likely impossible to find any encompassing set of metrics that would create the perfect scholarly ranking model — one that would be embraced by all scholars and which would rank scholars with absolute and complete fairness and accuracy. ScholarGPS™ recognizes that great care should be taken in using any scores (whether those from ScholarGPS™ or any other ranking system) as the final statement of any scholar’s true productivity or value. Users should therefore not construe a lower score or ranking as necessarily representative of lesser influence or prestige. While ScholarGPS™ metrics are derived largely from the traditional metrics used in the sciences and social sciences, other parameters including quality of teaching, outreach activities, as well as other modes of scholarly or artistic dissemination such as exhibitions, performances, and musical compositions should be considered by others when warranted.
[1] Ranking. (2022, March 24). In Wikipedia.
https://en.wikipedia.org/w/index.php?title=Ranking&oldid=1078999715#Standard_competition_ranking_(%221224%22_ranking)
[2] Percentage Rank. (2022, January 10). In Wikipedia.
https://en.wikipedia.org/w/index.php?title=Percentile_rank&oldid=1064785741
[3] Koltun, V., & Hafner, D. (2021). The h-index is no longer an effective correlate of scientific reputation.
PLOS ONE, 16(6). doi: 10.1371/journal.pone.0253397
[4] Geometric mean. (2023, February 16). In Wikipedia.
https://en.wikipedia.org/w/index.php?title=Geometric_mean&oldid=1139765330
[5] Standard score. (2023, March 29). In Wikipedia.
https://en.wikipedia.org/w/index.php?title=Standard_score&oldid=1147224272
