Can Public LNP Data Rank the Next Library?

The question

If a whole publication is unseen, can a model recover its most active lipid nanoparticle formulations and identify panels where its ranking should not be used?

My contribution

I evaluated formulation and assay metadata from the Lipid Nanoparticle Database (LNPDB), created by Evan Collins and collaborators. The eligible cohort contains 13,982 candidates in 37 assay-specific panels from 28 publications. Each panel has at least 30 unique formulations with numeric normalized-luminescence measurements. Rankings are evaluated within panels, rather than treating measurements from different assays as interchangeable.

Each outer fold holds out one complete PubMed publication and removes its exact formulations from training. Model family and regularization are chosen using whole-panel validation inside the remaining data. Publication identifiers, source links, and target values are excluded from the features. A separate analysis also removes every ionizable lipid found in the test publication.

The study compares transparent ridge models and then replays their frozen predictions at seven screening budgets. It also tests whether a distance measure built without target labels can identify unreliable transfers. The released package preserves the data snapshot, split checks, predictions, uncertainty estimates, and reproduction commands.

What the evidence shows

A candidate-random split gave a ranking score, nDCG@K, of 0.656. Holding out an entire publication reduced it to 0.570; higher is better. The stricter split's advantage over random selection was small, and its 95% publication-cluster interval included zero.

At a nominal 20% screening budget, the model recovered 28.4% of each panel's top-decile candidates on average, versus 20.3% under exact random selection. The paired improvement was 8.1 percentage points. Its pointwise 95% interval was +1.6 to +14.8 points, but the simultaneous interval across seven budgets was −0.5 to +17.0 points. The budget scan therefore did not establish a budget-specific improvement. The probability of finding at least one top-decile candidate was also not reliably improved.

The rule intended to reject poorly supported panels accepted 27 of 37 panels, but their mean nDCG@K fell to 0.544. Nearest-publication distance also failed to identify reliable transfers. These diagnostics did not provide a dependable rule for deciding when to abstain.

Screening-budget recall and any-hit improvements over random selection, a target-free publication distance map, and distance versus held-out ranking quality — Decision budget and publication support. Moderate-budget recall point estimates favor the model, but none remain resolved across the seven-budget scan. The bands are 95% Bonferroni-adjusted intervals, calculated separately for recall and any-hit improvement. Publication distance does not identify reliable transfers.

Where the evidence stops

This analysis ranks already-published candidates. It does not synthesize or test a new lipid nanoparticle, and it provides no manufacturing, safety, efficacy, or clinical evidence. The findings apply to these baseline models and this cohort; they do not establish that machine learning cannot improve LNP design.

Holding out publications reduces leakage but cannot remove publication bias, extraction errors, or unrecorded protocol differences. The models use supplied descriptors and string fragments. Uncertainty estimates contain only 28 publication clusters, and the seven budgets reuse the same predictions. The simultaneous intervals cover recall and any-hit outcomes as separate families, rather than all fourteen estimates together.

The next test should use a prospective campaign with its candidate pool, comparison rules, and abstention threshold fixed before outcomes are observed. Richer molecular models and support diagnostics can then face the same unseen-publication boundary.

Read and reproduce

Can Public LNP Data Rank the Next Library? A Publication-Cold Screening Audit of LNPDB

The package contains analysis code, tests, the pinned LNPDB CSV and its MIT license, predictions, result tables, figures, manuscript source, and checksum manifests. The source snapshot is LNPDB commit fc7c38933b445eb54985014f2b8917606462eec1. LNPDB remains the work of Collins and collaborators; this paper is an independent screening audit of their public data.

Code & data (ZIP, 3.8 MB)

View citation and BibTeX

Weale, George. Can Public LNP Data Rank the Next Library? A Publication-Cold Screening Audit of LNPDB. Author-hosted research paper. https://georgeweale.com/pdfs/research/2025-lipid-nanoparticle-surrogates.pdf

@misc{wealeLnpPublicationScreening,
  author = {Weale, George},
  title = {Can Public {LNP} Data Rank the Next Library? A Publication-Cold Screening Audit of {LNPDB}},
  note = {Author-hosted research paper},
  url = {https://georgeweale.com/pdfs/research/2025-lipid-nanoparticle-surrogates.pdf}
}