Annex A: IREG Berlin Principles Mapping

A IREG

ICO Std 2002 — Annex A (Informative)


Purpose

This annex provides a detailed mapping between the IREG Berlin Principles on Ranking of Higher Education Institutions (adopted 20 May 2006, Berlin) and the corresponding provisions of ICO Std 2002. It demonstrates the degree of alignment between the two frameworks and identifies areas where ICO Std 2002 extends beyond the requirements of the Berlin Principles.

IREG《》(2006520,)ICO Std 2002。,ICO Std 2002。


A.1 Overview of the Berlin Principles

A.1.1 History and Context

The Berlin Principles on Ranking of Higher Education Institutions were adopted on 20 May 2006 at the second meeting of the International Ranking Expert Group (IREG), founded in 2004 by the UNESCO European Centre for Higher Education (UNESCO-CEPES) in Bucharest and the Institute for Higher Education Policy (IHEP) in Washington, DC. The principles consist of 16 items organised into four sections:

IREG《》2006520(IREG)。IREG2004(UNESCO-CEPES)(IHEP)。16,:

Section Title Principles
A Purposes and Goals of Rankings R1–R5
B Design and Weighting of Indicators R6–R9
C Collection and Processing of Data R10–R14
D Presentation of Ranking Results R15–R16

Note: The Berlin Principles were originally designed for higher education rankings. ICO Std 2002 applies to a broader range of ranking domains (cultural heritage, consumer products, brands, cities, etc.). The mapping below accounts for this domain expansion by identifying where ICO Std 2002 generalises or extends the Berlin Principles beyond their original higher education context. :。ICO Std 2002(、、、)。ICO Std 2002,。

A.1.2 Scope of Application

The Berlin Principles apply to rankings of higher education institutions. ICO Std 2002 applies to any ranking system that evaluates entities using a structured methodology, regardless of domain. The mapping in this annex covers all 16 Berlin Principles and identifies, for each principle:

。ICO Std 2002,。16,:

a) the specific ICO Std 2002 clause(s) that address the same requirement; ICO Std 2002;

b) the degree of coverage ( Full / Partial / Exceeds);

c) any significant extensions or differences.


A.2 Principle-by-Principle Mapping

A.2.1 R1: Ranking as One of Multiple Approaches

Berlin Principle R1 R1

“Be one of a number of diverse approaches to the assessment of higher education inputs, processes, and outputs. Rankings can provide comparative information and improved understanding of higher education, but should not be the main method for assessing what higher education is and does.”

Aspect Berlin Principle ICO Std 2002 Coverage
Ranking as complementary assessment R1 §4.1 (Independence), §01 (Scope) Full
Limitations of ranking scope R1 §9.1.4 (Limitations Disclosure) Full
ICO Std 2002 Mapping ICO Std 2002

ICO Std 2002 §01 (Scope) establishes that the standard applies to ranking systems as one form of comparative assessment, not as a substitute for comprehensive evaluation. §9.1.4 requires explicit disclosure of the conceptual limitations of the indicator system, including “any aspects of the ranking purpose that are not fully captured by the indicators.” This fully satisfies R1’s requirement that rankings be positioned as complementary rather than definitive.

ICO Std 2002 §01(),。§9.1.4,”“。R1。


A.2.2 R2: Clear Purpose and Target Groups

Berlin Principle R2 R2

“Be clear about their purpose and their target groups. Rankings have to be designed with due regard to their purpose.”

Aspect Berlin Principle ICO Std 2002 Coverage
Purpose statement R2 §4.1.1 (Organisational Independence), §9.1.2(a) Full
Target group identification R2 §9.1.2(a) Full
Design aligned with purpose R2 §05.1 (Indicator Selection Criteria) Full
ICO Std 2002 Mapping ICO Std 2002

§9.1.2(a) requires the methodology statement to include “the ranking purpose and target audience.” §05.1 (Indicator Selection Criteria) requires that indicators be selected according to their relevance and validity relative to the stated ranking purpose. This ensures that the indicator system is designed with due regard to the ranking’s purpose, as required by R2.


A.2.3 R3: Recognise Diversity of Institutions

Berlin Principle R3 R3

“Recognize the diversity of institutions and take the different missions and goals of institutions into account.”

Aspect Berlin Principle ICO Std 2002 Coverage
Diversity recognition R3 §4.4.3 (Uniform Application of Criteria), §05.2 (Indicator Hierarchical Structure) Full
Mission differentiation R3 §06.1 (Weight Assignment Methods), SE modules Full
ICO Std 2002 Mapping ICO Std 2002

ICO Std 2002 generalises R3 beyond higher education institutions. §4.4.3 requires that criteria be applied uniformly across all ranked entities, while §05.2 allows for indicator systems that accommodate different entity types through hierarchical structures. The extension modules (SE-01 through SE-09) define domain-specific indicators that reflect the diversity of different ranking domains. §06.1 provides for differential weighting that can account for the varying missions and goals of ranked entities.

ICO Std 2002R3。§4.4.3,§05.2。(SE-01SE-09)。§06.1,。


A.2.4 R4: Information Sources and Messages

Berlin Principle R4 R4

“Provide clarity about the range of information sources for rankings and the messages each source generates.”

Aspect Berlin Principle ICO Std 2002 Coverage
Information source identification R4 §7.1 (Data Source Classification), §4.2.2 (Data Source Traceability) Full
Source-specific messages R4 §9.1.3 (Data Summary) Full
ICO Std 2002 Mapping ICO Std 2002

§7.1 establishes a three-tier data source classification system (Tier 1: Official/Audited; Tier 2: Survey/Expert; Tier 3: Unverified/Public), which goes beyond the Berlin Principles’ general requirement by providing a structured framework for assessing the reliability and limitations of each data source. §4.2.2 requires traceability of all data to its source. §9.1.3 requires publication of a data summary including the proportion of data from each tier.


A.2.5 R5: Specify Linguistic, Cultural, Economic, and Historical Contexts

Berlin Principle R5 R5

“Specify the linguistic, cultural, economic, and historical contexts of the educational systems being ranked.”

Aspect Berlin Principle ICO Std 2002 Coverage
Context specification R5 §4.2.2 (Data Source Traceability), §9.1.4 (Limitations Disclosure) Full
Cultural sensitivity R5 §4.2.2 (Cultural Sensitivity extension), SE-05 (Cultural Heritage) Exceeds
ICO Std 2002 Mapping ICO Std 2002

ICO Std 2002 satisfies R5 through §9.1.4, which requires disclosure of conceptual and methodological limitations including contextual factors. Furthermore, ICO Std 2002 extends beyond R5 by incorporating explicit cultural sensitivity requirements in SE-05 (Cultural Heritage) and by requiring domain-specific contextual considerations in all extension modules. The DPP-CQ integration framework (§10.4) provides a mechanism for addressing cultural and contextual differences in cross-domain rankings.

ICO Std 2002§9.1.4R5,。,ICO Std 2002SE-05(),,R5。DPP-CQ(§10.4)。


A.2.6 R6: Methodology Transparency

Berlin Principle R6 R6

“Be transparent regarding the methodology used for creating the rankings. The choice of methods used to prepare rankings should be clear and unambiguous. This transparency should include the calculation of indicators as well as the origin of data.”

Aspect Berlin Principle ICO Std 2002 Coverage
Methodology disclosure R6 §4.2.1 (Methodology Disclosure), §9.1.2 (Methodology Statement) Full
Indicator calculation transparency R6 §8.2 (Normalisation), §8.3 (Aggregation), §4.2.3 (Scoring Rule Verifiability) Full
Data origin transparency R6 §4.2.2 (Data Source Traceability), §7.1 (Data Source Classification) Full
Four-level transparency system §9.2 (Transparency Levels) Exceeds
ICO Std 2002 Mapping ICO Std 2002

ICO Std 2002 fully satisfies R6 through its comprehensive transparency framework. §4.2.1 requires publication of a complete methodology description. §4.2.3 requires that scoring rules be specified with sufficient precision for third-party verification. §9.2 establishes a four-level transparency classification system (Level 1: Summary Transparency; Level 2: Methodology Transparency; Level 3: Data Transparency; Level 4: Full Reproducibility) that goes significantly beyond the Berlin Principles’ general transparency requirement by providing a graduated framework for assessing and improving transparency.

ICO Std 2002R6。§4.2.1。§4.2.3。§9.2(:;:;:;:),,。


A.2.7 R7: Indicator Relevance and Validity

Berlin Principle R7 R7

“Choose indicators according to their relevance and validity. The choice of data should be grounded in recognition of the ability of each measure to represent quality and academic and institutional strengths, and not availability of data.”

Aspect Berlin Principle ICO Std 2002 Coverage
Relevance criterion R7 §05.1 (Indicator Selection Criteria) Full
Validity criterion R7 §05.1 (Indicator Selection Criteria), §8.5 (Ranking Result Verification) Full
Data availability warning R7 §05.1 (prohibition of data-driven selection) Full
ICO Std 2002 Mapping ICO Std 2002

§05.1 establishes indicator selection criteria that require indicators to be selected based on their relevance and validity to the ranking purpose, not merely on data availability. §8.5 (Ranking Result Verification) provides for internal consistency checks (Cronbach’s alpha ≥ 0.70), external validity checks, and robustness checks, which together validate the indicator system’s effectiveness. This fully satisfies R7.


A.2.8 R8: Measure Outcomes in Preference to Inputs

Berlin Principle R8 R8

“Measure outcomes in preference to inputs whenever possible. Measures of outcomes provide a more accurate assessment of the standing and/or quality of a given institution or program.”

Aspect Berlin Principle ICO Std 2002 Coverage
Outcome preference R8 §05.1 (Indicator Selection Criteria), §05.2 (Indicator Hierarchical Structure) Full
Input-output balance R8 §05.2 (Indicator classification by type) Full
ICO Std 2002 Mapping ICO Std 2002

§05.2 requires that the indicator system include a classification of indicators by type (input, process, output, outcome), and §05.1 requires that outcome indicators be given priority in the selection process. The weighting provisions in Chapter 6 allow for differential weighting that can reflect the higher importance of outcome indicators. This satisfies R8.


A.2.9 R9: Weight Transparency and Stability

Berlin Principle R9 R9

“Make the weights assigned to different indicators (if used) prominent and limit changes to them. Changes in weights make it difficult for consumers to discern whether an institution’s or program’s status changed in the rankings due to an inherent difference or due to a methodological change.”

Aspect Berlin Principle ICO Std 2002 Coverage
Weight prominence R9 §9.1.2(d) (Weighting scheme disclosure) Full
Weight stability R9 §06.3 (Weight Change Control) Full
Weight change justification R9 §06.3.2 (Methodological Change Documentation) Full
ICO Std 2002 Mapping ICO Std 2002

§9.1.2(d) requires disclosure of the weighting scheme and the method by which weights were determined. §06.3 (Weight Change Control) establishes requirements for limiting and documenting changes to weights, including the requirement that changes be based on methodological or conceptual considerations and not be used to produce different results. This fully satisfies R9.


A.2.10 R10: Ethical Standards and Good Practice

Berlin Principle R10 R10

“Pay due attention to ethical standards and the good practice recommendations articulated in these Principles. In order to assure the credibility of each ranking, those responsible for collecting and using data and undertaking on-site visits should be as objective and impartial as possible.”

Aspect Berlin Principle ICO Std 2002 Coverage
Ethical standards R10 §4.1 (Independence), §4.4 (Impartiality) Full
Objectivity and impartiality R10 §4.1.2 (Conflict of Interest Identification), §4.4.2 (Prohibition of Stakeholder Interference) Full
Paid listing prohibition §4.4.1 (Prohibition of Paid Listing) Exceeds
ICO Std 2002 Mapping ICO Std 2002

ICO Std 2002 satisfies R10 through its comprehensive independence and impartiality framework. §4.1 establishes the independence principle with specific requirements for conflict of interest identification, disclosure, and recusal. §4.4 establishes the impartiality principle, including the prohibition of paid listing (§4.4.1) and stakeholder interference (§4.4.2), which go beyond the Berlin Principles by providing specific operational rules for ensuring objectivity.

ICO Std 2002R10。§4.1,、。§4.4,(§4.4.1)(§4.4.2),,。


A.2.11 R11: Audited and Verifiable Data

Berlin Principle R11 R11

“Use audited and verifiable data whenever possible. Such data have several advantages, including the fact that they have been accepted by institutions and that they are comparable and compatible across institutions.”

Aspect Berlin Principle ICO Std 2002 Coverage
Verifiable data requirement R11 §7.1 (Data Source Classification), §7.2 (Data Collection Procedures) Full
Data audit trail R11 §10 (Audit and Traceability) Full
Data quality assurance R11 §7.3 (Data Quality Metrics) Exceeds
ICO Std 2002 Mapping ICO Std 2002

§7.1 classifies data sources by reliability tier, with Tier 1 (Official/Audited) data preferred. §7.2 requires that data collection procedures be documented and verifiable. §10 (Audit and Traceability) establishes a complete audit trail for all data and methodological decisions. §7.3 (Data Quality Metrics) goes beyond R11 by establishing quantitative data quality requirements including accuracy, completeness, timeliness, and consistency metrics.


A.2.12 R12: Scientific Data Collection Procedures

Berlin Principle R12 R12

“Include data that are collected with proper procedures for scientific data collection. Data collected from an unrepresentative or skewed subset of students, faculty, or other parties may not accurately represent an institution or program and should be excluded.”

Aspect Berlin Principle ICO Std 2002 Coverage
Scientific data collection R12 §7.2 (Data Collection Procedures), §7.4 (Missing Data and Outliers) Full
Representativeness R12 §7.2 (Sampling requirements), §7.3 (Data Quality Metrics) Full
Exclusion of skewed data R12 §7.4 (Outlier handling) Full
ICO Std 2002 Mapping ICO Std 2002

§7.2 establishes data collection procedures that must conform to scientific data collection standards, including requirements for sampling design, response rate thresholds, and data validation. §7.4 provides rules for handling missing data and outliers, including the requirement to exclude data from unrepresentative or skewed subsets. §7.3 establishes data quality metrics that can be used to identify and exclude low-quality data. This fully satisfies R12.


A.2.13 R13: Quality Assurance of Ranking Processes

Berlin Principle R13 R13

“Apply measures of quality assurance to ranking processes themselves. These processes should take note of the expertise that is being applied to evaluate institutions and use this knowledge to evaluate the ranking itself. Rankings should be learning systems continuously utilizing this expertise to develop methodology.”

Aspect Berlin Principle ICO Std 2002 Coverage
QA of ranking process R13 §10 (Audit and Traceability), §11 (Compliance) Full
Continuous improvement R13 §10.3 (Continuous Improvement), §06.3 (Weight Change Control) Full
Internal audit R13 §10.2 (Internal Audit) Full
ICO Std 2002 Mapping ICO Std 2002

§10 establishes a comprehensive audit and traceability framework, including internal audit requirements (§10.2) and continuous improvement provisions (§10.3). §11 (Compliance) specifies the requirements for demonstrating conformance with the standard. The DPP-CQ integration framework (§10.4) provides a mechanism for continuous quality improvement. This fully satisfies R13.

§10,(§10.2)(§10.3)。§11()。DPP-CQ(§10.4)。R13。


A.2.14 R14: Organisational Measures for Credibility

Berlin Principle R14 R14

“Apply organizational measures that enhance the credibility of rankings. These measures could include advisory or even supervisory bodies, preferably with some international participation.”

Aspect Berlin Principle ICO Std 2002 Coverage
Advisory/supervisory bodies R14 §10.1 (Governance), §10.2 (Internal Audit) Full
International participation R14 §10.1 (Governance requirements) Full
External audit R14 §10.2 (External audit provisions) Full
ICO Std 2002 Mapping ICO Std 2002

§10.1 establishes governance requirements for ranking entities, including the establishment of advisory and supervisory bodies. §10.2 provides for both internal and external audit mechanisms. The standard recommends international participation in advisory bodies, particularly for international rankings. This satisfies R14.


A.2.15 R15: Consumer Understanding and Choice

Berlin Principle R15 R15

“Provide consumers with a clear understanding of all of the factors used to develop a ranking, and offer them a choice in how rankings are displayed. In addition, they should have some opportunity to make their own decisions about how these indicators should be weighted.”

Aspect Berlin Principle ICO Std 2002 Coverage
Factor disclosure R15 §9.1.1 (Minimum Disclosure Set), §9.1.2 (Methodology Statement) Full
Display choice R15 §9.2 (Transparency Levels), §8.6 (Ranking Publication Format) Full
User-weighted ranking R15 §06.4 (User-Adjustable Weights) Exceeds
ICO Std 2002 Mapping ICO Std 2002

§9.1.1 and §9.1.2 require comprehensive disclosure of all ranking factors. §8.6 provides for flexible ranking publication formats. §06.4 (User-Adjustable Weights) goes beyond R15 by establishing a framework for interactive ranking tools that allow users to adjust indicator weights and recompute rankings in real time, subject to the constraint that the default (ranking entity) weights are prominently displayed alongside any user-adjusted results.


A.2.16 R16: Error Correction and Data Integrity

Berlin Principle R16 R16

“Be compiled in a way that eliminates or reduces errors in original data, and be organized and published in a way that errors and faults can be corrected. Institutions and the public should be informed about errors that have occurred.”

Aspect Berlin Principle ICO Std 2002 Coverage
Error prevention R16 §7.2 (Data Collection Procedures), §7.3 (Data Quality Metrics) Full
Error correction R16 §9.3 (Error Correction and Retraction) Full
Public notification R16 §9.3.1 (Error Notification) Full
Retraction procedures §9.3.2 (Retraction Procedures) Exceeds
ICO Std 2002 Mapping ICO Std 2002

§7.2 and §7.3 establish data quality measures that prevent and detect errors. §9.3 (Error Correction and Retraction) goes beyond R16 by establishing a formal error correction and retraction procedure, including requirements for error notification (§9.3.1), correction timelines, and retraction procedures for significant errors (§9.3.2). This exceeds R16’s requirements.


A.3 Summary Mapping Table

Principle Berlin R# ICO Std 2002 Reference Coverage
Ranking as complementary approach R1 §01, §4.1, §9.1.4 Full
Clear purpose and target groups R2 §4.1.1, §9.1.2(a), §05.1 Full
Recognise diversity R3 §4.4.3, §05.2, §06.1, SE modules Full
Information sources clarity R4 §7.1, §4.2.2, §9.1.3 Full
Specify contexts R5 §4.2.2, §9.1.4, SE-05 Exceeds
Methodology transparency R6 §4.2.1, §4.2.3, §9.2 Exceeds
Indicator relevance and validity R7 §05.1, §8.5 Full
Outcome preference R8 §05.1, §05.2 Full
Weight transparency and stability R9 §06.3, §9.1.2(d) Full
Ethical standards R10 §4.1, §4.4 Exceeds
Audited and verifiable data R11 §7.1, §7.2, §7.3, §10 Exceeds
Scientific data collection R12 §7.2, §7.3, §7.4 Full
QA of ranking processes R13 §10, §11 Full
Organisational credibility measures R14 §10.1, §10.2 Full
Consumer understanding and choice R15 §9.1, §8.6, §06.4 Exceeds
Error correction R16 §7.2, §7.3, §9.3 Exceeds
Coverage Summary
Coverage Level Count Percentage
Full 10 62.5%
Exceeds 6 37.5%
Partial 0 0%
Not Covered 0 0%

A.4 ICO Extensions Beyond Berlin Principles

The following features of ICO Std 2002 have no direct counterpart in the Berlin Principles and represent extensions beyond the scope of the original IREG framework:

ICO Std 2002,IREG:

A.4.1 Four-Level Transparency System

The Berlin Principles require transparency in general terms (R6). ICO Std 2002 establishes a graduated four-level transparency classification (§9.2) that provides:

(R6)。ICO Std 2002(§9.2),:

This graduated system enables stakeholders to assess the transparency of a ranking publication against a defined standard and provides a pathway for ranking entities to progressively improve their transparency.

A.4.2 DPP-CQ Integration Framework

The Data Processing Pipeline — Continuous Quality (DPP-CQ) integration framework (§10.4) provides a systematic mechanism for continuous quality improvement of ranking processes. This has no counterpart in the Berlin Principles, which address quality assurance only in general terms (R13).

—(DPP-CQ)(§10.4)。,(R13)。

The DPP-CQ framework includes: DPP-CQ:

a) automated data quality monitoring throughout the ranking lifecycle;

b) trigger-based quality alerts for data anomalies;

c) feedback loops from ranking results to methodological improvement;

d) version control and change tracking for all methodological decisions.

A.4.3 Domain Neutrality and Multi-Domain Architecture

The Berlin Principles were designed specifically for higher education rankings. ICO Std 2002 generalises the ranking methodology framework to apply across all domains through a core-layer + extension-module architecture. This domain-neutral design (§4.2.1) ensures that the standard’s requirements are applicable to any ranking context while allowing domain-specific adaptations through extension modules (SE-01 through SE-09).

。ICO Std 2002+,。(§4.2.1),(SE-01SE-09)。

A.4.4 Cultural Sensitivity Requirements

While the Berlin Principles acknowledge the importance of cultural context (R5), ICO Std 2002 extends this by incorporating explicit cultural sensitivity requirements in the SE-05 (Cultural Heritage) extension module and by requiring that all ranking systems consider the cultural implications of their indicator design and data collection methods. This is particularly relevant for the Tianji Ranking’s intangible cultural heritage track.

(R5),ICO Std 2002SE-05(),,。。

A.4.5 Anti-Fraud and Anti-Manipulation Mechanisms

The Berlin Principles do not address the risk of deliberate manipulation of ranking data by ranked entities. ICO Std 2002 includes explicit anti-fraud provisions (§7.5) that address:

。ICO Std 2002(§7.5),:

a) detection and prevention of data manipulation (e.g., fabricated survey responses, inflated metrics);

b) anti-brush-clicking () mechanisms for consumer product rankings;

c) sanctions for entities found to have manipulated data.

A.4.6 UN DCP Alignment

ICO Std 2002 aligns with the United Nations Declaration on the Rights of Indigenous Peoples (UNDRIP) and the UNESCO Convention for the Safeguarding of the Intangible Cultural Heritage (2003), providing a framework for ranking systems that respect the rights and dignity of communities and cultural practitioners. This alignment has no counterpart in the Berlin Principles.

ICO Std 2002《》(UNDRIP)《》(2003),。。


A.5 Comparative Analysis

A.5.1 Overall Assessment

ICO Std 2002 achieves full coverage of all 16 Berlin Principles, with 6 of 16 principles exceeded by ICO Std 2002’s more detailed or broader provisions. No Berlin Principle is partially covered or uncovered. The areas of extension are primarily in:

ICO Std 200216,6ICO Std 2002。。:

a) Transparency granularity — the four-level transparency system provides a graduated framework not present in the Berlin Principles;

b) Data quality rigour — quantitative data quality metrics and the DPP-CQ framework provide systematic quality assurance beyond the Berlin Principles’ general requirements; ——DPP-CQ;

c) Anti-fraud provisions — explicit mechanisms for preventing and detecting data manipulation;

d) Domain scope — generalisation from higher education to all ranking domains;

e) Cultural sensitivity — explicit requirements for cultural sensitivity in heritage and community contexts;

f) User empowerment — the user-adjustable weights framework provides a mechanism for consumer choice beyond the Berlin Principles’ general recommendation.

A.5.2 Implications for IREG Ranking Audit

An ICO Std 2002-compliant ranking system that covers the higher education domain would satisfy all IREG Ranking Audit criteria derived from the Berlin Principles. The additional provisions of ICO Std 2002 (transparency levels, data quality metrics, anti-fraud mechanisms) would provide additional evidence of quality that could strengthen the ranking’s audit assessment.

ICO Std 2002IREG。ICO Std 2002(、、),。

A.5.3 Recommendations for Future Alignment

a) ICO Std 2002 should be submitted to IREG for formal review and potential recognition as a complementary framework to the Berlin Principles. ICO Std 2002IREG,。

b) Future revisions of the Berlin Principles may consider incorporating the four-level transparency system and anti-fraud provisions pioneered by ICO Std 2002. ICO Std 2002。

c) Cross-certification pathways between IREG Ranking Audit and ICO Std 2002 compliance assessment should be explored. IREGICO Std 2002。


Note: This annex is informative and part of ICO Std 2002 (Tianji Ranking Methodology Standard). The mapping is based on the Berlin Principles as adopted on 20 May 2006 and does not reflect any subsequent revisions by IREG. :,ICO Std 2002()。2006520,IREG。