About the Track · OAEI Bio-ML

OAEI Bio-ML is an OAEI ontology-matching track over whole biomedical ontologies — a larger LargeBio. It evaluates equivalence and subsumption matching across three pairs — NCIT–DOID, SNOMED–FMA, and SNOMED–NCIT — with references grounded in curated biomedical knowledge (UMLS/Mondo) and repaired for logical coherence.

Tracks

Track 1 — Equivalence
- Subtrack 1 — Global equivalence alignment. Submit one full alignment per pair (full OWL IRIs). Semi-supervised: a public refs_equiv/train.tsv is provided per pair; the test reference is hidden and scored organiser-side. Headline metric: repaired, coherence-aware P/R/F1 (standard P/R/F1 and reasoner-checked Global Coherence reported alongside).
- Subtrack 2 — Local equivalence ranking. Rank a fixed, gold-stripped candidate pool per source entity. Metrics: MRR and Hits@{1,5,10}.
Track 2 — Mixed equivalence + subsumption typed ranking. A typed ranking over equivalence and subsumption candidates (CURIEs, e.g. NCIT:C101044). Metrics: Preferred Typed-MRR and Hierarchy-aware Typed-nDCG@10.

Full definitions are on the evaluation metrics page. All headline metrics are macro-averaged over the three pairs.

How the references were built

For each pair, the reference alignment is grounded in the UMLS Metathesaurus and Mondo. Because a reference assembled from these sources can be logically incoherent, it is repaired before use: the set of correspondences to remove (or weaken) is computed as a union over three repair tools — ALCOMO, LogMap, and AML — following the LargeBio repair tradition. Under the track’s annotation scheme, surviving correspondences keep their (possibly weakened) equivalence/subsumption relation; only fully-removed correspondences are marked uncertain (?) and ignored at scoring time.

The track therefore ships two references per pair: the standard (complete, possibly-incoherent) reference and the repaired (coherence-aware) reference. The repaired reference is the headline; the standard reference is reported alongside for comparison. The two are not directly comparable — see evaluation metrics.

Serialisation

By design, Track 1 uses full OWL IRIs and Track 2 uses CURIEs (e.g. NCIT:C101044). Public local- and typed-ranking candidate files (*.test.cands.tsv) are gold-stripped: they contain the source entity and its candidate list only.

Data

The 2026 datasets are publicly available on the Hugging Face Hub as OAEI-ML/bio-ml (edition tag 2026) — the task data is entity IRIs/CURIEs and downloads without gating. The Hugging Face dataset is data only; the self-contained scoring_kit/ (validators + self-scorers) ships separately with the track repository. The licence-restricted source ontologies (SNOMED CT, UMLS) are not re-hosted — see ontologies for where to obtain each and under which licence. The quickstart walks through cloning the scoring kit, downloading the data, and validating and self-scoring a submission.

Baselines & results

Organiser-run baseline systems (a naive lexical baseline, SapBERT, and the BERTMap family) are published before the competition on the baselines page, rendered directly from the machine-readable leaderboard.json. Participant standings appear on the CodaBench leaderboards, surfaced on the results page once the evaluation window opens.

Participate

Three CodaBench competitions — Track 1 Global Alignment, Track 1 Local Ranking, and Track 2 Typed Ranking — open on 12 July 2026. See the quickstart.

Organisers & contact

OAEI Bio-ML 2026 is organised by the OAEI-ML team (LISEDA Lab, Universidade de Lisboa, and collaborators); the full organiser list is confirmed with the resource paper at launch. The benchmark design follows the original machine-learning-friendly Bio-ML datasets (He et al.); full references appear in the resource paper.

Questions or corrections: open an issue on the track repository or email contact@oaei-ml.org.

OAEI Bio-ML 2026 (first edition). Track repository: https://github.com/liseda-lab/OAEI-Bio-ML.