Sequence

• Protein language models such as ESM and what their embeddings capture
• Genomic models and the tokenisation problem specific to DNA
• Structure prediction as the most convincing demonstration to date

Other Modalities

• Pathology and microscopy foundation models and their annotation dependence
• Biomedical literature models and single-cell models such as Geneformer
• Multi-omics and the difficulty of aligning modalities from different assays

Evaluation

• Benchmarks, leakage between pretraining data and test sets
• Zero-shot and few-shot claims and how often a simple baseline matches them
• Reproducibility, licensing and weights that are announced but not released

Use

• Embeddings as features against fine-tuning for a specific task
• Compute requirements and what is achievable on modest hardware
• Where these models fail biologically and the experimental check that catches it