Metascience Emerges as New Approach to Fix Structural Flaws in Research Systems

Metascience is gaining attention as a new research field within the scientific research system. This discipline seeks to address long-standing structural problems inherent in scientific research by analyzing research systems through scientific approaches including data, methods, and tools. The scientific research system operates under the premise that the determination, execution, and dissemination of research projects should be rigorous, transparent, and fair, with applied methods effective enough to generate better outcomes.

However, due to the high degree of specialization and uncertainty inherent in scientific research, actual operations show considerable limitations. For research requiring high levels of expertise, it is assumed reasonable that evaluation be conducted by peer experts possessing relevant knowledge and information in the field. Consequently, most research project selections are made through peer review.

Yet researchers in the field complain that peer review is conservative, preventing transformative and challenging research projects from being selected. Others argue that evaluation consistency among experts is lacking, with selections determined by the luck of evaluation panel composition. The peer review system that has sustained scientific research is losing credibility.

Most scientific research fields are seeing an increase in large-scale projects with substantial participant numbers due to specialization of scientific activities and growing problem complexity. We must examine whether the knowledge and technology produced by large projects differ in nature from small projects, and particularly whether large projects generate more innovative and disruptive outcomes than smaller ones. Demand for such analysis of research funding allocation is amplifying as growing research funding demands combine with strengthened efficiency requirements following R&D budget increases.

Scientific data analysis and verification are necessary to identify fundamental problems inherent in scientific research systems, as validating analyses through researcher complaint surveys or simple statistics alone proves difficult.

The U.S. National Science Foundation (NSF) and National Institutes of Health (NIH) began pursuing scientific analytical approaches from the mid-2000s. NSF established the Science of Science and Innovation Policy (SciSIP) program to advance related work. Results from long-term panel studies (Wu, Wang & Evans, 2019) reported that large-scale research tends to advance existing technologies rather than produce disruptive innovation. Over more than a decade, multiple NIH studies analyzing research reproducibility issues and grant allocation methods revealed deficiencies in peer review validity and consistency.

While U.S. government agencies have pursued scientific data analysis approaches and attempted some regulatory revisions, they have taken a rather conservative approach to improving existing systems. Recently, however, the UK government is boldly attempting to leap over these barriers. It has established a metascience unit within government and is conducting bold policy experiments based on scientific analysis of research systems. For example, to improve peer review methods, it is experimenting with randomly selecting and funding projects through a lottery system among proposals scoring above a certain threshold.

The UK's challenge carries significant meaning in applying bold changes and innovation through scientific approaches to solve fundamental problems in scientific research systems that have been addressed only passively until now. It is also implementing approaches to realize evidence-based science and technology policy on weak foundations. While environmental changes such as strengthened efficiency demands on R&D systems and increasing research system complexity have influenced these developments, the capability for big data analysis through AI is also playing a major role.

Korea's science policy approaches issues element by element, including evaluation system improvements, research ethics strengthening, and open science application. It relies on simple statistics or accepting field opinions about difficulties rather than scientific analytical approaches. The emerging field of metascience not only boldly addresses fundamental problems in scientific research systems but also considers overall system efficiency. It pursues active innovation that analyzes, infers, and experiments with causal relationships regarding structural problems rather than simple statistical information.

The global scientific research field is moving beyond the vague stereotype that science is inherently good, toward scientifically analyzing and innovating structural problems in research systems to generate better research outcomes and innovation. Korea must also build infrastructure and frameworks for new scientific research policy approaches. Particularly required is an open attitude from the science and technology community accepting that scientific research systems themselves are subjects for change and innovation.