# Can public LNP data rank the next library?

This directory is an executed, real-data screening audit of the public
[Lipid Nanoparticle Database](https://github.com/evancollins1/LNPDB).

The practical question is deliberately narrow:

> If an entire LNP publication is unseen, can formulation and assay metadata
> recover that publication's most active candidates, and can an unlabeled
> support diagnostic identify panels on which the model should abstain?

## Bottom line

The moderate-budget point estimates favor the pooled model, but the
seven-budget family-wise intervals include zero. The study also finds no
dependable small-campaign or abstention rule.

- A candidate-random split reaches **0.656 nDCG@K** and **0.295 top-K hit
  rate**.
- Leaving out a complete publication reduces them to **0.570** and **0.141**.
- The publication-cold advantages over random expectation are small and their
  95% publication-cluster bootstrap intervals include zero.
- Removing every ionizable lipid found in the test publication changes
  nDCG@K by only -0.002.
- The frozen support gate makes selection worse: it accepts 27 of 37 panels,
  whose mean nDCG@K is 0.544 versus 0.570 across all panels.
- A 90% rank interval covers 0.894 of candidates only by spanning 0.869 of
  the full percentile scale.
- At a 20% budget, top-decile recall is 0.284 versus 0.203 under exact random
  selection. Paired uplift is +0.081, with pointwise 95% interval
  [+0.016, +0.148] and Bonferroni-7 simultaneous interval [-0.005, +0.170].
- At 50%, recall uplift is +0.096, with pointwise interval
  [+0.008, +0.186] and simultaneous interval [-0.026, +0.212].
- Any-hit uplift is -0.078 [-0.246, +0.054] at 20% and
  -0.042 [-0.140, +0.010] at 50%, where brackets are simultaneous intervals.
- Target-free nearest-publication distance has Spearman +0.216
  [-0.196, +0.589], so it does not identify reliable transfers.

For recall uplift and any-hit uplift separately, the 95% family-wise intervals
apply Bonferroni across the seven declared budgets to 2,000
publication-cluster bootstrap draws. Each interval uses tail quantiles
0.05/(2 x 7) and 1 - 0.05/(2 x 7). The two interval sets do not cover all
14 estimates as one joint family.

The useful claim is narrow: moderate-budget point estimates remain worth
testing prospectively, while this retrospective curve does not establish a
budget-specific uplift, small-campaign assurance, or support-based abstention.

## Evidence boundary

The frozen input is LNPDB commit
fc7c38933b445eb54985014f2b8917606462eec1. The vendored CSV SHA-256 is:

    3493f27306419facd0958589030ed37f272f05c81ad47dd7bc1b8ce0284a57b3

The upstream repository distributes this CSV under the MIT License. Its full
copyright and permission notice is preserved beside the data at
`data/LNPDB-LICENSE.txt`. The analysis and manuscript cite the associated
Nature Communications paper; vendoring the source table does not transfer
authorship of the database.

Eligibility is restricted to numeric **luminescence_normalized** panels with a
PubMed identifier and at least 30 unique formulations. The executed set has
13,982 candidates, 37 assay-specific panels, and 28 publications.

The outer loop withholds a full PubMed publication. Exact test formulations
are removed from training. Model family and ridge penalty are chosen only
through inner whole-panel-held-out validation. Identifiers, source links, and
targets are forbidden as features. A second analysis also removes every test
ionizable lipid from training.

LNP Atlas is not pooled into the endpoint. Its archive is machine-readable,
but its biological activity field is heterogeneous free text rather than a
compatible row-wise normalized luminescence target.

This is retrospective ranking evidence. It is not prospective discovery,
laboratory validation, manufacturing evidence, safety or efficacy evidence,
or a clinical claim.

## Reproduce

No installation step is part of this study. It uses the repository's existing
Python scientific stack and existing TinyTeX:

    python run_analysis.py
    python second_cycle.py
    python -m unittest -v test_analysis.py
    python -m unittest -v test_second_cycle.py
    python run_analysis.py --check
    python second_cycle.py --check
    latexmk -pdf -interaction=nonstopmode -halt-on-error paper.tex
    python build_research_manifest.py
    python build_release_manifest.py
    python verify_release.py

The base and second-cycle check commands verify their recorded inputs and
outputs without recomputing the analyses.

`verify_release.py` is the wider publication check. It verifies
**results/release_manifest.json**, which binds the source license, CSV, code,
tests, manuscript source, compiled PDF, figures, and result files. Temporary
LaTeX files, `tmp/`, bytecode, and the manifest itself are explicitly excluded.

## Map

- **study_config.json** — rules frozen before outcome generation
- **data/LNPDB.csv** — pinned public source snapshot
- **data/LNPDB-LICENSE.txt** — upstream MIT copyright and permission notice
- **analysis_core.py** — eligibility, features, splits, metrics, bootstraps
- **study_run.py** — nested outer evaluation and negative-result summaries
- **study_plot.py** — publication figures
- **run_analysis.py** — deterministic runner and hash verifier
- **second_cycle.py** — budget replay, exact random baseline, target-free publication map
- **test_analysis.py / test_second_cycle.py** — base and second-cycle regression tests
- **build_research_manifest.py** — claim report, source registry, hashes, and pending approvals
- **research.json / inputs/question.md / outputs/report.md** — claim-traceable research package
- **test_analysis.py** — regression and leakage-control tests
- **results/** — panel inventory, predictions, fold metrics, manifests
- **figures/** — vector PDF and high-resolution PNG figures
- **paper.tex** — concise manuscript grounded only in executed evidence
- **results/release_manifest.json** — package-wide publication checksum inventory
- **build_release_manifest.py / verify_release.py** — rebuild and verify that inventory
