2 Clinical Trial Portfolio Analysis
Ethical justification for the burdens placed on clinical research participants extends well beyond rigorous trial design. Modern scholarship stresses that trials should form part of an aligned research portfolio, where each study fills knowledge gaps and builds on prior evidence, contributing to a coherent, cumulative understanding of therapeutic questions. This portfolio approach is central for generating social value and ethically justifying participant burdens, as research efforts that are strategically linked offer a far greater return on participant contribution than isolated studies (London and Kimmelman 2019; Kimmelman, Carlisle, and Gönen 2017; Kimmelman and London 2015).
Evaluating clinical research from a bird’s eye perspective, particularly after drug approval, produces two key benefits:
Improving Future Trials: Analyses of existing research portfolios can highlight recurring challenges or risks; these insights directly inform the design, oversight, and safety of subsequent studies. For example, patterns identified at the portfolio level can help investigators anticipate methodological pitfalls or ethical concerns (Bittlinger, Peppercorn, and Kimmelman 2022).
Revealing Research Gaps: Only by viewing groups of related trials together do unmet needs and new research questions rise to the surface—questions invisible to single-study analysis, but crucial for continuous improvement and targeted innovation (Kimmelman, Carlisle, and Gönen 2017; London and Kimmelman 2019).
1. Contextualizing Research Within the Scientific Continuum
- Interdependence of Questions: Each study should not exist in isolation. Instead, it should build upon previous findings and lay the groundwork for subsequent inquiry. This continuity ensures that research is not a series of disconnected efforts, but a tapestry of knowledge where each thread informs and supports the others.
- Cumulative Value of Evidence: The justification for involving participants must consider how the current research contributes to a growing body of evidence. Studies should be positioned to fill genuine knowledge gaps, refine existing understanding, or resolve uncertainties left by prior trials.
2. Strategic Question Selection and Research Prioritization
- Meaningful Sequencing: What we choose to ask next matters. The process of formulating research questions must be deliberate, ensuring they are timely, necessary, and logically connected to what has been already established. This sequencing maximizes the efficiency and impact of participant commitments.
- Avoiding Redundancy and Waste: By coordinating research agendas and sharing data openly, the field can prevent unnecessary replication and avoid subjecting participants to studies of limited incremental value.
3. Ethical Stewardship: Participant Burden in Perspective
- Value Amplification: When research questions are interconnected and aligned towards elucidating a coherent “bigger picture,” the societal value gained for each participant’s sacrifice increases. Their contributions are part of a collective effort, not a solitary, potentially obscure, experiment.
- Transparency and Trust: Clearly articulating how each study fits within the broader research mission fosters public trust, honors participant altruism, and reinforces the legitimacy of the research burden.
4. Cultivating a Healthy Research Culture
- Interdisciplinary Collaboration: Encouraging collaboration across disciplines and institutions ensures that research priorities are shaped by a diversity of perspectives and pressing clinical realities.
- Vision and Foresight: Healthy research ecosystems invest in horizon scanning—anticipating future questions and challenges—to guide present study designs and ethical considerations.
2.1 Trial Portfolios and Research Trajectories
Let us examine some examples using EMA data from the EMA website. On the x-axis we plot time (years) and on the y-axis we plot the “volume” (counts) of EMA approvals per category (therapeutic areas). We do so using a stream graph (Byron and Wattenberg 2008), where the volume of individual streams (i.e. number of approvals per year and therapeutic area) is proportional to the values in each category.
Key features of stream graphs
- Trends Over Time: Easily see rises and falls in activity for different categories (e.g., trial phases, therapeutic areas).
- Relative Volumes: The thickness of each stream at a given time denotes the volume (number of trials or approvals) for that category.
- Research Shifts: Highlight when attention moves from one research area to another, or when new categories emerge.
It’s particularly powerful for showing the temporal development of research portfolios, such as how the number of clinical trials in different indications expand, contract, or shift across years.
This plot visually presents “trends” in approval rate per therapeutic area over time.
The width of each category’s stream fluctuates, letting us spot certain observations (e.g., surges in lung cancer trials after pembrolizumab’s first lung approval).
Events like regulatory approvals or new indication launches appear as “bursts.”
Declining or “sunset” indications shrink and eventually disappear, providing a living map of the research focus over time as described in the reference example
This streamgraph visualizes trends in EMA regulatory approval categories from the early 2000s to 2024. Each colored stream represents one category, with the stream’s thickness indicating the number of medicines approved under that category each year.
Interpretation by Category:
Additional monitoring (largest stream): This category has grown substantially since 2012, peaking in 2022. It reflects increased post-market surveillance requirements for newer medicines, possibly due to evolving safety monitoring standards.
Generic or hybrid: Steady presence over the years, with moderate peaks around 2009–2012 and a strong resurgence from 2021–2024. This suggests growing availability and approval of off-patent medicines or new combinations/formulations.
Orphan medicine: Consistent upward trend, especially post-2018, reflecting EMA’s ongoing support for therapies targeting rare diseases.
Biosimilar: Emerging around 2010 and slowly gaining momentum. The rise in recent years aligns with the expiration of patents on major biologics.
Accelerated assessment, Advanced therapy, Conditional approval, PRIME: priority medicine: These smaller streams show limited but increasing use, indicating selective but strategic application of fast-track or specialized pathways for innovative or high-need treatments.
In sum, the graph shows that regulatory pathways tailored for innovation, safety, and patient access have become more prominent. Particularly, Additional monitoring and Orphan medicine categories have seen major growth, while generics and biosimilars reflect market maturity and cost-containment policies.
2.2 Limitations of hitorical data analysis
Data Relevance and Timeliness: Historical data may not reflect recent changes in the environment, such as technological advancements, market disruptions, or shifts in consumer behavior. This can make insights obsolete or misleading.
Quality and Completeness Issues: Historical datasets can suffer from inaccuracies, incompleteness, or inconsistent formats, reducing the reliability of any conclusions drawn.
Survivorship Bias: The data might be affected by biases, such as focusing on entities that have survived or succeeded while ignoring those that have failed, leading to skewed interpretations (also known as survivorship bias).
Limited Predictive Power: Historical patterns do not always guarantee future outcomes, especially in volatile environments or during unprecedented events. Unexpected disruptions (like the COVID-19 pandemic) often break with historical trends.
Overfitting: There is a risk of creating models that fit historical data too closely but fail to generalize accurately to new situations or data—a problem especially in complex statistical modeling.
Stifling Innovation: Exclusive reliance on past data can limit creativity, making organizations less likely to explore new approaches or adapt to novel circumstances.
Cultural and Contextual Shifts: Societal, regulatory, or industry-specific changes can render historical data less useful for contemporary analysis, potentially leading to misaligned strategies
References
This book is in Open Review. I want your feedback to make the book better for you and other readers. To add your annotation, select some text and then click the on the pop-up menu. To see the annotations of others, click the in the upper right hand corner of the page