SE-04 : Technology Innovation

ICO Std 2002 — Extension SE-04

This extension module specifies the requirements and methodologies for ranking and evaluating technology innovation capability. Technology innovation ranking addresses the distinctive challenge of assessing entities on their capacity to create, develop, and deploy new technologies — a process that spans from fundamental research through commercialisation, and whose outcomes are inherently uncertain and long-cycle. This module extends the core-layer methodology (Chapters 4 through 10) by defining domain-specific indicators for innovation capability assessment, incorporating Technology Readiness Level (TRL) mapping, establishing methods for evaluating technology frontier positioning, and aligning with the conceptual frameworks of ISO 56002 and the Oslo Manual. It is applicable to rankings of enterprises, research institutions, technology clusters, and national innovation systems.

SE-04.1 Scope and Applicable Objects

SE-04.1.1 Scope

This module applies to the design, execution, and publication of rankings that evaluate technology innovation capability. It covers:

a) enterprise-level innovation capability assessment;

b) research institution innovation output assessment;

c) technology cluster and regional innovation ecosystem assessment;

d) national innovation capability assessment.

SE-04.1.2 Applicable Objects

This module shall apply to rankings of:

a) technology enterprises across sectors;

b) universities and public research organisations;

c) technology parks, incubators, and innovation clusters;

d) national or regional innovation systems as defined by established frameworks (e.g., the European Innovation Scoreboard methodology).

SE-04.1.3 Exclusions

This module does not apply to:

a) the evaluation of individual research projects or grant applications;

b) patent litigation or intellectual property valuation for legal purposes;

c) the ranking of individual researchers or scientists (which falls under SE-03 Individual Influence in the core framework).

SE-04.2 Normative References

a) ISO 56002:2019 — Innovation management — Innovation management system — Guidance;

b) ISO 56000:2020 — Innovation management — Fundamentals and vocabulary;

c) Oslo Manual 2018 — Guidelines for Collecting, Reporting and Using Data on Innovation (4th edition, OECD/Eurostat);

d) ICO Std 2002 Core Layer (Chapters 4 through 10);

e) NASA Technology Readiness Assessment (TRA) Guide (2017).

SE-04.3 Indicator System

SE-04.3.1 Indicator Hierarchy

The technology innovation indicator system shall be organised into the following three-level hierarchy:

a) Level 1 — Dimensions: R&D Investment, Patent and IP Output, Technology Commercialisation, Innovation Ecosystem, Market Impact;

b) Level 2 — Sub-dimensions: Decompositions of each dimension as specified in SE-04.3.2 through SE-04.3.6;

c) Level 3 — Specific measures: Operationalised variables for which data is collected.

SE-04.3.2 R&D Investment

R&D investment assesses the commitment of resources to research and development activities as a leading indicator of innovation capability.

a) R&D Expenditure Intensity: The ratio of R&D expenditure to revenue (for enterprises) or to total budget (for research institutions). The ranking entity shall:

1) use the Frascati Manual definition of R&D expenditure;

2) distinguish between basic research, applied research, and experimental development expenditure;

3) report both the aggregate R&D intensity and the breakdown by research type.

b) R&D Human Capital: The quantity and quality of R&D personnel, including:

1) the number of R&D full-time equivalents (FTEs) as a proportion of total workforce;

2) the educational composition of R&D staff (proportion with doctoral, masters, and bachelors degrees);

3) the retention rate of key R&D personnel.

c) R&D Infrastructure: The quality and scale of research infrastructure, including:

1) the number and type of research laboratories and testing facilities;

2) the investment in research equipment and instrumentation;

3) access to shared research infrastructure (e.g., national laboratories, supercomputing centres).

d) R&D Portfolio Diversity: The breadth and balance of the R&D portfolio, including:

1) the number of distinct technology areas in which R&D is conducted;

2) the balance between short-term applied projects and long-term exploratory research;

3) the TRL distribution of R&D projects (see SE-04.4).

SE-04.3.3 Patent and IP Output

Patent and IP output assesses the quantity, quality, and impact of intellectual property generated by the entity.

a) Patent Volume: The number of patents filed and granted. The ranking entity shall distinguish:

1) invention patents (utility patents) versus design and utility model patents;

2) domestic versus international patent filings (PCT, EPO, USPTO, JPO, CNIPA);

3) first filings versus subsequent filings in the same patent family.

b) Patent Quality: The quality and significance of the patent portfolio, assessed through:

1) citation impact — the average forward citation count relative to the technology field average;

2) patent family size — the number of jurisdictions in which patent protection is sought;

3) grant rate — the proportion of filed patents that are ultimately granted.

c) Patent Relevance and Concentration: The degree to which patents are concentrated in the entity’s core technology domains and contribute to its competitive advantage, including:

1) the proportion of patents in the entity’s top-3 technology domains;

2) the strategic alignment of patents with the entity’s stated technology roadmap;

3) the patent clustering coefficient — the degree to which patents form interconnected technology clusters.

d) Non-Patent IP Output: Other forms of intellectual property, including:

1) published peer-reviewed research papers and their citation impact;

2) software copyrights and open-source contributions;

3) trade secrets and proprietary know-how (assessed through expert evaluation where quantifiable metrics are unavailable).

SE-04.3.4 Technology Commercialisation

Technology commercialisation assesses the entity’s ability to transform research outputs into market-viable products, services, or processes.

a) Technology Transfer Rate: The proportion of R&D outputs that are successfully transferred to commercial application, including:

1) the number of technology licensing agreements per unit of R&D expenditure;

2) the revenue from technology licensing and transfer;

3) the number of spin-off companies created from research outputs.

b) Time-to-Market: The average elapsed time from initial R&D project initiation to commercial product or service launch. The ranking entity shall:

1) define the start and end milestones for time-to-market measurement;

2) report the median and interquartile range of time-to-market;

3) compare time-to-market against industry benchmarks.

c) Product Innovation Rate: The proportion of revenue derived from products or services introduced within a defined recent period (typically three to five years), consistent with the Oslo Manual’s definition of product innovators.

d) Process Innovation Rate: The proportion of production or service delivery processes that have been significantly improved or newly introduced within a defined recent period, consistent with the Oslo Manual’s definition of process innovators.

SE-04.3.5 Innovation Ecosystem

Innovation ecosystem assesses the breadth, depth, and vitality of the entity’s innovation network and collaborative relationships.

a) Collaboration Breadth: The diversity and extent of the entity’s innovation collaboration network, including:

1) the number and type of collaboration partners (universities, research institutions, enterprises, government agencies);

2) the geographic diversity of collaboration partners;

3) the number of joint research projects and co-authored publications.

b) Open Innovation Engagement: The degree to which the entity participates in open innovation practices, including:

1) contribution to and adoption of open-source technologies and standards;

2) participation in industry consortia and standard-setting bodies;

3) the number and impact of hackathons, innovation challenges, or crowdsourcing initiatives.

c) Innovation Talent Attraction: The entity’s ability to attract and retain innovation talent, including:

1) the net inflow rate of technology talent;

2) the proportion of new hires with advanced degrees in relevant fields;

3) the number of industry-leading researchers or “highly cited” scientists affiliated with the entity.

d) Innovation Culture Indicators: Proxy measures for the entity’s innovation culture, including:

1) the proportion of employees participating in innovation or improvement programmes;

2) the number of internally generated innovation proposals per employee;

3) the tolerance for failure as measured by the proportion of R&D projects that are discontinued without career penalty for project leaders.

SE-04.3.6 Market Impact

Market impact assesses the economic and competitive significance of the entity’s innovation outputs in the marketplace.

a) Market Share in Technology Segments: The entity’s market share in its primary technology segments, including:

1) revenue market share in the entity’s top-3 technology segments;

2) market share growth rate in technology segments;

3) the number of technology segments in which the entity holds a top-3 market position.

b) Technology Standard Influence: The entity’s influence on industry technology standards, including:

1) the number of standard-essential patents (SEPs) held;

2) leadership positions in standard-setting organisations;

3) the adoption rate of the entity’s proposed technical standards.

c) Revenue from New Products: The revenue generated from products or services based on technologies introduced within a defined recent period, as a proportion of total revenue.

d) Competitive Displacement: The degree to which the entity’s innovations have disrupted or displaced existing market solutions, including:

1) market share gains attributable to innovation-driven product advantages;

2) the entry of the entity into new market segments through innovation;

3) the adoption of the entity’s technology by competitors (indicating industry leadership).

SE-04.4 Technology Readiness Level Mapping

SE-04.4.1 TRL Framework

a) The ranking entity shall use the Technology Readiness Level (TRL) framework to classify the maturity of technologies under evaluation. The TRL scale shall follow the nine-level framework originally developed by NASA and widely adopted internationally:

TRL Description
1 Basic principles observed
2 Technology concept formulated
3 Experimental proof of concept
4 Technology validated in laboratory
5 Technology validated in relevant environment
6 Technology demonstrated in relevant environment
7 System prototype demonstration in operational environment
8 Actual system completed and qualified
9 Actual system proven in operational environment

b) The ranking entity shall document the TRL classification methodology, including the criteria used to assign each TRL level and the evidence required for each classification.

SE-04.4.2 TRL Distribution Analysis

a) The ranking entity shall calculate the TRL Distribution Profile for each ranked entity, showing the number and proportion of technology projects at each TRL level.

b) The ranking entity shall calculate the Technology Maturity Index (TMI) as:

\[TMI = \frac{\sum_{i=1}^{9} i \cdot n_i}{9 \cdot N}\]

where $n_i$ is the number of projects at TRL level $i$, and $N = \sum_{i=1}^{9} n_i$ is the total number of technology projects. TMI ranges from 0 (no technology activity) to 1 (all technologies at TRL 9).

c) The ranking entity shall report both the TMI and the TRL Distribution Profile, as they convey complementary information. A high TMI with a skewed distribution toward TRL 9 may indicate a mature but potentially declining innovation pipeline, whereas a moderate TMI with a broad distribution may indicate a healthy innovation pipeline.

SE-04.4.3 TRL Progression Rate

a) The ranking entity should assess the rate at which technologies progress through TRL levels, as a measure of innovation execution efficiency. The TRL Progression Rate shall be calculated as:

\[TPR = \frac{\Delta TRL_{\text{avg}}}{T_{\text{elapsed}}}\]

where $\Delta TRL_{\text{avg}}$ is the average TRL advancement of tracked technology projects over the measurement period, and $T_{\text{elapsed}}$ is the measurement period in years.

b) The TRL Progression Rate provides insight into the entity’s ability to move technologies from research through development to market — a critical capability that pure R&D investment metrics do not capture.

SE-04.5 Technology Frontier Assessment

SE-04.5.1 Definition

Technology frontier assessment evaluates the degree to which an entity’s technologies represent the leading edge of their respective fields. This dimension addresses the qualitative distinction between incremental innovation and frontier or breakthrough innovation.

SE-04.5.2 Frontier Positioning Method

a) The ranking entity shall assess each entity’s technology frontier positioning using the following multi-method approach:

1) Patent Landscape Analysis: Mapping the entity’s patent portfolio against the global patent landscape to identify the entity’s position relative to technology frontiers. The ranking entity shall use:

2) Expert Panel Assessment: A structured expert assessment of the entity’s technology position, using a Delphi or modified Delphi process as specified in Chapter 5 of the core layer. The expert panel shall include:

3) Publication Impact Analysis: For research-intensive entities, analysis of the citation impact and journal prestige of published research outputs, using standard bibliometric indicators:

b) The Frontier Positioning Score shall be calculated as:

\[FPS = 0.40 \cdot P_{\text{frontier}} + 0.35 \cdot E_{\text{frontier}} + 0.25 \cdot B_{\text{frontier}}\]

where $P_{\text{frontier}}$ is the patent landscape frontier score, $E_{\text{frontier}}$ is the expert panel frontier score, and $B_{\text{frontier}}$ is the bibliometric frontier score, each normalised to 0-1.

SE-04.5.3 Frontier Breadth vs. Depth

a) The ranking entity shall assess both the breadth and depth of frontier technology positioning:

1) Frontier Breadth: The number of distinct technology domains in which the entity has frontier-level technology. An entity with frontier presence in multiple domains shall score higher on breadth.

2) Frontier Depth: The degree of advancement beyond the current state of the art in the entity’s primary technology domains. An entity whose technologies significantly surpass the current frontier shall score higher on depth.

b) The ranking entity shall report both frontier breadth and depth scores alongside the overall Frontier Positioning Score.

SE-04.6 Data Collection

SE-04.6.1 Data Source Categories

The ranking entity shall collect data from the following categories of sources:

a) Patent Databases: Global patent databases (e.g., WIPO Patentscope, EPO Espacenet, national patent office databases) for patent filing, grant, and citation data.

b) Financial and Corporate Reports: Annual reports, R&D expenditure disclosures, and investor communications from the ranked entities.

c) Bibliometric Databases: Scientific publication and citation databases (e.g., Scopus, Web of Science) for research output analysis.

d) Government Statistics: National and international innovation statistics (e.g., OECD Main Science and Technology Indicators, national R&D surveys).

e) Industry and Market Data: Market research reports, industry surveys, and technology market analysis from established providers.

SE-04.6.2 Patent Data Quality Requirements

a) Patent data shall be collected from authoritative patent office databases and shall include complete bibliographic data, citation data, and legal status information.

b) The ranking entity shall apply patent family deduplication to avoid double-counting inventions filed in multiple jurisdictions.

c) Citation data shall be adjusted for citation lag (the time delay between patent publication and citation accumulation) using established normalisation methods.

SE-04.6.3 R&D Data Reporting Standards

a) R&D expenditure data shall comply with the definitions and classifications of the Frascati Manual (OECD, 2015) to ensure international comparability.

b) Where entities report R&D expenditure using different accounting standards, the ranking entity shall apply reconciliation adjustments and shall document the adjustments made.

SE-04.7 Scoring Methodology

SE-04.7.1 Scoring Scale

a) Each Level 3 indicator shall be scored on a normalised scale from 0 to 100.

b) Quantitative indicators shall be normalised using the min-max method or z-score method as specified in Chapter 8 of the core layer.

c) The Frontier Positioning Score (SE-04.5.2) shall be integrated into the overall score through the Market Impact dimension.

SE-04.7.2 Weighting

a) The default weight scheme for the five Level 1 dimensions shall be as follows:

Dimension Default Weight
R&D Investment 0.20
Patent and IP Output 0.20
Technology Commercialisation 0.25
Innovation Ecosystem 0.15
Market Impact 0.20

b) The default weights reflect the principle that technology innovation ranking shall emphasise the conversion of R&D investment into commercial outcomes, while recognising the importance of sustained investment and ecosystem participation.

c) For rankings of early-stage technology companies or research institutions where commercialisation outcomes are not yet available, the ranking entity may increase the weights of R&D Investment and Patent Output to a combined total of 0.55, with corresponding reductions in Technology Commercialisation and Market Impact.

SE-04.7.3 Aggregation

a) Scores shall be aggregated using the weighted sum method specified in Chapter 8 of the core layer.

b) The overall innovation score, the Technology Maturity Index (SE-04.4.2), and the Frontier Positioning Score (SE-04.5.2) shall all be reported in the ranking publication.

c) The ranking entity shall report the TRL Distribution Profile for each ranked entity.

SE-04.8 Alignment with International Standards

SE-04.8.1 ISO 56002 Alignment

a) The indicator system shall be consistent with the innovation management system guidance of ISO 56002:2019, particularly:

1) the emphasis on leadership and commitment to innovation;

2) the focus on innovation processes from ideation through implementation;

3) the importance of evaluation and improvement of innovation performance.

b) The ranking entity shall map each indicator in this module to the corresponding ISO 56002 process area and shall document the mapping.

SE-04.8.2 Oslo Manual Alignment

a) The definitions of product innovation, process innovation, and innovation activities used in this module shall be consistent with the Oslo Manual 2018 (4th edition).

b) The ranking entity shall use the Oslo Manual’s definitions when classifying innovation types and shall document any extensions or adaptations for ranking purposes.

c) Data collection on innovation activities should follow the Oslo Manual’s survey guidelines where applicable, to ensure comparability with official innovation statistics.

SE-04.9 Publication and Transparency

SE-04.9.1 Publication Requirements

In addition to the requirements specified in Chapter 9 of the core layer, technology innovation ranking publications shall include:

a) the Technology Maturity Index and TRL Distribution Profile for each ranked entity;

b) the Frontier Positioning Score, including frontier breadth and depth scores;

c) the patent data sources and deduplication methodology;

d) the R&D data reporting standard used and any reconciliation adjustments;

e) for entity types where commercialisation data is limited, a disclosure of the modified weighting scheme and its rationale.

SE-04.9.2 Entity Identification

a) Each ranked entity shall be uniquely identified using:

1) the legal entity name and registration details;

2) the scope of innovation activities included (e.g., consolidated group, specific division);

3) the reporting period for financial and R&D data.

Note: This extension module is part of ICO Std 2002 (Tianji Ranking Methodology Standard). Technology innovation evaluation faces a fundamental tension between measuring inputs (R&D investment), outputs (patents, products), and outcomes (market impact, competitive advantage). No single dimension fully captures innovation capability. The TRL framework and Frontier Positioning Score are distinctive contributions of this module that address the need to assess not just how much innovation activity occurs, but how advanced and how leading-edge that activity is. Ranking entities should be cautious about over-reliance on patent counts as proxies for innovation quality, and should complement quantitative metrics with expert assessment and frontier analysis.