Below is a story of how the beauty of biology meets the beast of frontier proteomic and genomic research, told through the tale of advanced machine learning methods.
Some time ago, we partnered with Jonathan Schmid-Burgk's lab at the University of Bonn to aid the newly developed Nuclear In-Situ Sequencing (NIS-Seq) with deep-learning guided analysis of fluorescence microscopy images of protein distributions in individual cells. This resulted in a shared publication in Nature Biotechnology (see Fandrey et al., 2024, and an introduction on our blog).

Since then we have collaborated on a new, wider and improved incarnation of some of the ideas used in NIS-Seq, called PATTERNS (Proteome Analysis Through Tagging Endogenous pRoteiNs and Sequencing). In the new project, both the biology and the machine learning analysis are pushed much further. From the biological perspective, CRISPR-guided fluorescent tagging of individual genes was combined with NIS-Seq barcode readout and scaled up to a library of more than 11,000 tagged genes, of which about 2,400 were barcoded and imaged, allowing us to uncover a myriad of protein patterns within live human cells. These patterns were confirmed by co-staining of individual organelles, and utilised for protein displacement studies during vaccinia virus infection, while changes in protein abundance were measured after dosing of several protein-altering drugs.
The magnificent datasets of half a million cells, imaged with high-resolution fluorescence microscopy, can be explored in an interactive Observable-based viewer presenting both the images and the outline of the analysis: https://jsb-lab.bio/patterns-viewer/pretty-proteins-viewer/index.html

A scientific article presenting the details of the method, analysis and results is currently in press. Here we share several lessons learned during this project.
Firstly, as always, the key to automating the analysis of fluorescence microscopy data is normalisation. We have played quite a bit with different approaches to normalising the images to optimise for various aspects: different normalisations are needed for the best human-facing display, for automated analysis with deep learning, and for quantification of fluorescent signal similarity across channels. This aspect is also related to image augmentation when training deep learning models, where we saw noticeable improvements in performance once careful consideration was given to which precise protocols should be used in our case. A lesson we learned many times over is to always try to visualise the methods used for image processing.
Secondly, protein localisation within cells is an active field of study, and universal atlases such as the Human Protein Atlas, OpenCell or others often contain contradictory information. It is also important to note that there are many biological caveats related to the interpretation of protein localisation and its shifts. Imaging fixed cells (e.g., in Cell Painting-based protocols) captures only a single snapshot, so the timing of the stress application and imaging is vital for capturing the information, while inherent cell variability and cell-type variation muddy the picture with complicated inter-relationships. One of the strengths and promises of a method like PATTERNS is to unravel new biological insights by retaining the live-imaging aspect.
Thirdly, we observed an interesting phenomenon in our analysis. With the goal of finding the embedding spaces of protein patterns most useful for our analysis (quantified in various ways: separation of genes, agreement with external localisation labels, or internal grouping metrics), we compared two pre-trained models. A recent foundation model pre-trained on cellular data, namely SubCell, provides better embeddings out of the box than a universal image processing network (Swin Transformer V2) pre-trained on a general image dataset. However, unexpectedly, the difference is erased, or on some compartment-level metrics even slightly reversed, when the two models are weakly supervised on the data in our study. This hints that, with fine-tuning on in-domain data, a general-use pre-trained model can catch up with or even overtake a domain-specific one.
Stay tuned for the full reveal in the published article!

