Completing an EU JCA in One Hour?
We used the public Tarlatamab assessment to test how quickly and accurately otto-SR could reproduce the evidence behind seven unique PICOs.
TL;DR: Using the public tarlatamab JCA, otto-SR reproduced the evidence behind seven PICOs in under one hour, recovering the same core study families used by the HTD while preserving a traceable evidence trail.
The EU Joint Clinical Assessment (JCA) creates a difficult evidence-operations problem. The scientific question is only one part of it; teams must also keep a complex, changing evidence package coherent under a compressed timeline.
Health technology developers (HTDs) need to anticipate and address multiple potential PICO questions, identify the relevant direct and indirect evidence for each, perform the necessary analyses, and make the entire package transparent and defensible.
Assessment teams face the same problem from the other side. They need to determine whether the evidence base is complete, whether every analysis can be traced back to the correct studies and populations, and whether new questions can be answered quickly enough to keep the assessment moving.
We wanted to understand: how much of the JCA evidence workflow could we reproduce in otto-SR?
We chose the tarlatamab JCA as a test case because it represents a particularly useful stress test. The assessment contained seven PICOs, spanning multiple populations, comparators, and evidence-generation approaches.
Across the seven questions, the submitted evidence relied on a combination of direct comparisons and indirect analyses, including NMA, MAIC, and other indirect treatment comparisons, highlighting the complexity of the evidence package.
Workflow
We built one inspectable evidence workflow. From defining the protocol to mapping the evidence, each step stayed inside the same system and preserved the context needed for the next.
Plan
Translate seven PICOs into one evidence search
We first mapped the seven PICO questions and the evidence required to address each one. Across the assessment, the relevant comparisons included topotecan, CAV, platinum-based retreatment, and platinum-etoposide regimens.
| PICO | Population | Comparator | HTD approach & sources |
|---|---|---|---|
| PICO 1 | TFI < 3 months (platinum resistant) | Topotecan | DirectDeLLphi-304 |
| PICO 2 | TFI < 3 months (platinum resistant) | CAV | Unanchored MAICDeLLphi-304GPFC 0501 |
| PICO 3 | TFI ≥ 3 months (platinum sensitive) | Platinum-based retreatment | NMADeLLphi-304Baize 2020 |
| PICO 4 | TFI ≥ 3 months (platinum sensitive) | Topotecan | DirectDeLLphi-304 |
| PICO 5 | TFI ≥ 3 months (platinum sensitive) | CAV | NMADeLLphi-304von Pawel 1999 |
| PICO 6 | RFI ≤ 6 months; not suitable for retreatment | Topotecan or CAV | DirectDeLLphi-304 |
| PICO 7 | RFI > 6 months; suitable for retreatment | Cisplatin or carboplatin with etoposide | ITCDeLLphi-304Baize 2020 |
- Population
- TFI < 3 months (platinum resistant)
- Comparator
- Topotecan
- Sources
- DeLLphi-304
- Population
- TFI < 3 months (platinum resistant)
- Comparator
- CAV
- Sources
- DeLLphi-304GPFC 0501
- Population
- TFI ≥ 3 months (platinum sensitive)
- Comparator
- Platinum-based retreatment
- Sources
- DeLLphi-304Baize 2020
- Population
- TFI ≥ 3 months (platinum sensitive)
- Comparator
- Topotecan
- Sources
- DeLLphi-304
- Population
- TFI ≥ 3 months (platinum sensitive)
- Comparator
- CAV
- Sources
- DeLLphi-304von Pawel 1999
- Population
- RFI ≤ 6 months; not suitable for retreatment
- Comparator
- Topotecan or CAV
- Sources
- DeLLphi-304
- Population
- RFI > 6 months; suitable for retreatment
- Comparator
- Cisplatin or carboplatin with etoposide
- Sources
- DeLLphi-304Baize 2020
Intervention across all questions: tarlatamab monotherapy
Rather than running seven disconnected literature reviews, we consolidated the questions into one broad evidence protocol, similar to the approach used by the original HTD.
Determine the efficacy and safety of tarlatamab relative to other common small-cell lung cancer treatments through direct and indirect evidence.
Include
- Adults (≥18 years) with relapsed or refractory SCLC who previously received one platinum-based regimen
- Eligible interventions: tarlatamab; CAV; platinum-based chemotherapy; topotecan; or cisplatin/carboplatin with etoposide
- Randomized controlled trials and single-arm trials
Exclude
- Case reports, small case series (≤10 patients), reviews, observational studies, preclinical studies, and in-vitro studies
- Studies that did not evaluate a pre-specified intervention of interest
- Studies that did not report efficacy or safety outcomes
- Other indications, including non-small-cell lung cancer and haematologic malignancies
- Populations with more than one prior line of systemic therapy
- Populations that had not received first-line platinum-based therapy
Search + screen
Find all available evidence
We entered the protocol into otto-SR and used its new agentic search feature to run the search. Reviewers can retrieve the exact PubMed search strategy, and every screening decision remains inspectable. For each record, researchers can review why it was included or excluded, open the source, and correct the decision when needed.
54 studies included across 57 reports.
Open full PRISMA diagramExtract
Quickly extract all relevant fields
We configured a data-extraction template around patient populations and subpopulations, interventions, comparators, and extractable outcomes. Source-level citations stayed attached throughout the process.
Analyze and surface insights
Identify potential direct/indirect comparisons
Finally, we used our "Ask Otto" analysis agent to identify the direct and indirect evidence relevant to each PICO. The resulting map recovered the same core study families used by the HTD across all seven questions.
Results
Developers normally have 100 days to prepare the initial JCA dossier. Assessors can request further specifications, clarifications, data analyses, or other evidence with a response window of 7 to 30 days. Traditional evidence synthesis workflows that might otherwise be manageable are quickly overwhelmed.
In our test, otto-SR reproduced the evidence map across all seven PICOs, recovering the same core study families used by the HTD. The exercise took under one hour.
| PICO | HTD approach & sources | otto-SR recommendations | Match |
|---|---|---|---|
| PICO 1TFI < 3 months · topotecan | DirectDeLLphi-304 | Direct DeLLphi-304 evidence; a <90-day intended-topotecan estimate would require supplementary analysis or sponsor IPD. | Correct |
| PICO 2TFI < 3 months · CAV | Unanchored MAICDeLLphi-304GPFC 0501 | Unanchored MAIC using DeLLphi-304 and GPFC 0501, with residual confounding kept explicit. | Correct |
| PICO 3TFI ≥ 3 months · platinum retreatment | Bayesian NMA; an anchored network can connect DeLLphi-304 to Baize 2020 through topotecan. | Correct | |
| PICO 4TFI ≥ 3 months · topotecan | DirectDeLLphi-304 | Direct DeLLphi-304 evidence in the ≥90-day intended-topotecan subgroup, with NMA as supporting indirect evidence. | Correct |
| PICO 5TFI ≥ 3 months · CAV | NMA using DeLLphi-304 and von Pawel 1999, with a ≥60-day versus ≥90-day population-alignment caveat. | Correct | |
| PICO 6RFI ≤ 6 months · topotecan or CAV | DirectDeLLphi-304 | Partial direct match in DeLLphi-304; an exact estimate requires sponsor IPD and a defensible definition of “not suitable for retreatment.” | Correct |
| PICO 7RFI > 6 months · platinum–etoposide | Anchored ITC using DeLLphi-304 and Baize 2020; feasibility depends on the corresponding DeLLphi subgroup effect. | Correct |
- Correct
PICO 1
TFI < 3 months · topotecan
- HTD approach & sources
- DirectDeLLphi-304
- otto-SR recommendations
- Direct DeLLphi-304 evidence; a <90-day intended-topotecan estimate would require supplementary analysis or sponsor IPD.
- Correct
PICO 2
TFI < 3 months · CAV
- HTD approach & sources
- Unanchored MAICDeLLphi-304GPFC 0501
- otto-SR recommendations
- Unanchored MAIC using DeLLphi-304 and GPFC 0501, with residual confounding kept explicit.
- Correct
PICO 3
TFI ≥ 3 months · platinum retreatment
- HTD approach & sources
- otto-SR recommendations
- Bayesian NMA; an anchored network can connect DeLLphi-304 to Baize 2020 through topotecan.
- Correct
PICO 4
TFI ≥ 3 months · topotecan
- HTD approach & sources
- DirectDeLLphi-304
- otto-SR recommendations
- Direct DeLLphi-304 evidence in the ≥90-day intended-topotecan subgroup, with NMA as supporting indirect evidence.
- Correct
PICO 5
TFI ≥ 3 months · CAV
- HTD approach & sources
- otto-SR recommendations
- NMA using DeLLphi-304 and von Pawel 1999, with a ≥60-day versus ≥90-day population-alignment caveat.
- Correct
PICO 6
RFI ≤ 6 months · topotecan or CAV
- HTD approach & sources
- DirectDeLLphi-304
- otto-SR recommendations
- Partial direct match in DeLLphi-304; an exact estimate requires sponsor IPD and a defensible definition of “not suitable for retreatment.”
- Correct
PICO 7
RFI > 6 months · platinum–etoposide
- HTD approach & sources
- otto-SR recommendations
- Anchored ITC using DeLLphi-304 and Baize 2020; feasibility depends on the corresponding DeLLphi subgroup effect.
“Correct” means otto-SR recovered the HTD's reported approach and core study family, or correctly identified where only a partial match was available. It does not validate an outcome estimate or change the JCA conclusion.
Takeaways
The exercise surfaced three practical priorities for JCA teams.
Finding evidence for every PICO
A single JCA can require an HTD to address several populations and comparators at once. Teams may need to screen thousands of citations to identify the comparator trials required for indirect treatment comparisons, network meta-analyses (NMAs), matching-adjusted indirect comparisons (MAICs), and other analyses.
otto-SR brings search, screening, and extraction into one traceable workflow, helping teams identify relevant trials quickly and extract the data points needed for downstream analyses.
Traceable and reproducible results
The catequentinib and Tacquell JCAs were discontinued after their dossiers failed the Article 9 completeness and transparency requirements. The Commission identified incomplete or missing search and study-selection documentation in both.
For catequentinib, the notice also cited the lack of justification for not searching for or selecting non-randomised studies for potential unanchored indirect comparisons.
With otto-SR, search strategies, screening decisions, and protocol revisions remain connected in one inspectable record.
Responding rapidly when assessors ask a new question
Not every assessor request requires a new analysis. Some can be answered by clarifying the dossier or retrieving information that is already available. But the questions are often highly specific: What was the result for this population, comparator, and outcome? Why was this study excluded? What supports this analysis?
otto-SR preserves citation-level provenance, helping teams trace extracted values to the relevant study, report, population, data cutoff, outcome definition, and analysis.
This exercise was not designed to test whether otto-SR would change the JCA or EMA outcome. Instead, it tested whether the platform could reproduce the underlying evidence workflow. In this case, otto-SR recovered all relevant study reports, mapped the key direct and indirect comparisons, extracted the required fields, and maintained a visible record of exclusions, corrections, and unresolved gaps. Together, these capabilities can make evidence packages easier to inspect, update, and defend as assessor questions arise.