MassSpecGym: A benchmark for the discovery and identification of molecules

The discovery and identification of molecules in biological and environmental samples is crucial for advancing biomedical and chemical sciences. Tandem mass spectrometry (MS/MS) is the leading technique for high-throughput elucidation of molecular structures. However, decoding a molecular structure from its mass spectrum is exceptionally challenging, even when performed by human experts. As a result, the vast majority of acquired MS/MS spectra remain uninterpreted, thereby limiting our understanding of the underlying (bio)chemical processes. Despite decades of progress in machine learning applications for predicting molecular structures from MS/MS spectra, the development of new methods is severely hindered by the lack of standard datasets and evaluation protocols.

To address this problem, we propose MassSpecGym - the first comprehensive benchmark for the discovery and identification of molecules from MS/MS data. Our benchmark comprises the largest publicly available collection of high-quality labeled MS/MS spectra and defines three MS/MS annotation challenges: de novo molecular structure generation, molecule retrieval, and spectrum simulation. It includes new evaluation metrics and a generalization-demanding data split, therefore standardizing the MS/MS annotation tasks and rendering the problem accessible to the broad machine learning community.

🧪 MassSpecGym dataset

MassSpecGym comprises the largest publicly available collection of 231 thousand high-quality MS/MS spectra labeled with molecular structures. The dataset includes spectra exhaustively collected from well-established public repositories (i.e., MoNA, MassBank, and GNPS), as well as new spectra generated from our in-house mass spectrometry measurements.

The curation of the dataset involves a pipeline of cleaning and standardization steps, ensuring high-quality data while preserving a broad coverage of molecular structures and mass spectrometry settings. The table below introduces all the variables present in MassSpecGym. The main ones are mzs and intensities, representing MS/MS spectra, and smiles, representing the corresponding molecular structures.

🏆 MassSpecGym benchmark

MassSpecGym provides three challenges for benchmarking the discovery and identification of new molecules from MS/MS spectra:

• 💥 De novo molecule generation (MS/MS spectrum → molecular structure)
  • ✨ Bonus chemical formulae challenge (MS/MS spectrum + chemical formula → molecular structure)
• 💥 Molecule retrieval (MS/MS spectrum → ranked list of candidate molecular structures)
  • ✨ Bonus chemical formulae challenge (MS/MS spectrum → ranked list of candidate molecular structures with ground-truth chemical formulae)
• 💥 Spectrum simulation (molecular structure → MS/MS spectrum)
  • ✨ Bonus chemical formulae challenge (molecular structure → MS/MS spectrum; evaluated on the retrieval of molecular structures with ground-truth chemical formulae)

The provided challenges abstract the process of scientific discovery from biological and environmental samples into well-defined machine learning problems with pre-defined datasets, data splits, and evaluation metrics.

📩 Contact

For any questions or suggestions, please contact the authors via email: roman.bushuiev@uochb.cas.cz.

🔗 References

📃 NeurIPS 2024 Splotlight paper: https://arxiv.org/abs/2410.23326.

💻 GitHub repository: https://github.com/pluskal-lab/MassSpecGym.

🤗 Hugging Face page: https://huggingface.co/datasets/roman-bushuiev/MassSpecGym.

If you use MassSpecGym in your work, please cite the following paper:

@article{bushuiev2024massspecgym,
      title={MassSpecGym: A benchmark for the discovery and identification of molecules}, 
      author={Roman Bushuiev and Anton Bushuiev and Niek F. de Jonge and Adamo Young and Fleming Kretschmer and Raman Samusevich and Janne Heirman and Fei Wang and Luke Zhang and Kai Dührkop and Marcus Ludwig and Nils A. Haupt and Apurva Kalia and Corinna Brungs and Robin Schmid and Russell Greiner and Bo Wang and David S. Wishart and Li-Ping Liu and Juho Rousu and Wout Bittremieux and Hannes Rost and Tytus D. Mak and Soha Hassoun and Florian Huber and Justin J. J. van der Hooft and Michael A. Stravs and Sebastian Böcker and Josef Sivic and Tomáš Pluskal},
      year={2024},
      eprint={2410.23326},
      url={https://arxiv.org/abs/2410.23326},
      doi={10.48550/arXiv.2410.23326}
}