OAEI Bio-ML is a ML-friendly biomedical track for equivalence and subsumption matching over three whole biomedical ontologies — NCIT–DOID, SNOMED–FMA and SNOMED–NCIT. Systems are scored on two tracks across three CodaBench competitions.
The tasks
Track 1 measures whole-ontology equivalence matching two ways — globally (a full alignment) and locally (ranked candidates). Track 2 adds subsumption: a typed ranking that rewards the preferred relation and hierarchy-aware ordering.
Submit one full alignment per pair (full OWL IRIs). Public
refs_equiv/train.tsv per pair; the test reference is hidden and scored
organiser-side. Headline is the repaired, coherence-aware P/R/F1;
standard P/R/F1 and reasoner-checked Global Coherence are reported alongside.
For each source entity, rank a fixed candidate pool best-first. The public
*.test.cands.tsv are gold-stripped. Scored with
MRR and Hits@{1,5,10}, macro-averaged over the
three pairs.
A typed ranking over equivalence and subsumption candidates together,
serialised as CURIEs (e.g. NCIT:C101044). Scored with
Preferred Typed-MRR and Hierarchy-aware Typed-nDCG@10.
Data
The 2026 task data is distributed on the Hugging Face Hub as
OAEI-ML/bio-ml (edition tag
2026). It is publicly available
and downloads without gating (entity IRIs/CURIEs only); the self-contained
scoring_kit/ ships separately with the track repository, and the
licence-restricted source vocabularies (SNOMED CT, UMLS) are obtained separately under your
own licence.
git clone https://github.com/liseda-lab/OAEI-Bio-MLhf download OAEI-ML/bio-ml --repo-type dataset --revision 2026 --local-dir ./bio-ml
The Hugging Face dataset holds the per-pair task files only; the self-contained
scoring_kit/ (validators and self-scorers) comes from the track
repository, cloned above. The source ontologies are not re-hosted here —
see the ontologies page for where to obtain each,
and the quickstart for the full walk-through. Track
repository: liseda-lab/OAEI-Bio-ML.
Submit
Each scored track has its own CodaBench competition. Validate and (where possible) self-score locally with the scoring kit, then submit. Portals open on 12 July 2026.
| Competition | Submission portal |
|---|---|
| Track 1 · Subtrack 1 — Global equivalence alignment | Open on CodaBench |
| Track 1 · Subtrack 2 — Local equivalence ranking | Open on CodaBench |
| Track 2 — Mixed equivalence + subsumption typed ranking | Open on CodaBench |
Organiser-run baselines are published on the baselines page; participant standings appear on results once the window opens.
Timeline
The vocabularies
NCIT, DOID, SNOMED CT and FMA are matched across the three pairs; MONDO and the UMLS Metathesaurus underlie the reference construction. All are obtained from their original publishers — see ontologies for sources and licences.
Part of the OAEI
OAEI Bio-ML runs under the Ontology Alignment Evaluation Initiative — the long-running campaign that benchmarks ontology-matching systems. It continues the machine-learning-friendly Bio-ML datasets, which superseded the LargeBio track in 2022.
Bio-ML is one of the OAEI's biomedical tracks; its results feed the annual OAEI campaign. The earlier LargeBio track was retired when Bio-ML — a “larger LargeBio” over whole ontologies — launched in 2022. The mark above is a faithful, theme-adaptive re-creation of the OAEI logo.
Citation
A citation for the 2026 edition is pending and will be published here once minted. The benchmark design follows the original machine-learning-friendly Bio-ML datasets (He et al.). Until the edition DOI is available, cite the dataset release:
OAEI Bio-ML 2026 [Data set]. Hugging Face. huggingface.co/datasets/OAEI-ML/bio-ml (edition tag 2026).
People
OAEI Bio-ML 2026 is organised by the OAEI-ML team (LISEDA Lab, Universidade de Lisboa, and collaborators). The full organiser list will be confirmed with the resource paper at launch.
Questions or corrections: open an issue on the track repository or email contact@oaei-ml.org.
Part of the Ontology Alignment Evaluation Initiative. Full acknowledgements appear in the resource paper (available at track launch).