Using test data to evaluate rankings of entities in large scholarly citation networks

[摘要] ENGLISH ABSTRACT : A core aspect in the field of bibliometrics is the formulation, refinement, and verificationof metrics that rate entities in the science domain based on the information containedwithin the scientific literature corpus. Since these metrics play an increasingly importantrole in research evaluation, continued scrutiny of current methods is crucial. For example,metrics that are intended to rate the quality of papers should be assessed by correlatingthem with peer assessments. I approach the problem of assessing metrics with test databased on other objective ratings provided by domain experts which we use as proxies forpeer-based quality assessments.This dissertation is an attempt to fill some of the gaps in the literature concerningthe evaluation of metrics through test data. Specifically, I investigate two main researchquestions: (1) what are the best practices when evaluating rankings of academic entitiesbased on test data, and (2), what can we learn about ranking algorithms and impactmetrics when they are evaluated using test data? Besides the use of test data to evaluatemetrics, the second continual theme of this dissertation is the application and evaluationof indirect ranking algorithms as an alternative to metrics based on direct citations.Through five published journal articles, I present the results of this investigation.

[发布机构] Stellenbosch University