Constantin Pape

🔬 CochleaNet has been published in Molecular Therapy 🎉 CochleaNet provides deep learning-based analysis for cochlear connectomics and gene therapy validation using high-resolution isotropic light-sheet microscopy. Our goal: quantitative therapy validation for research on hearing restoration.

This project was only possible through an amazing collaboration between Auditory Neuroscience #MoserLab, Computer Science #PapeLab, and Light Sheet Microscopy #HuiskenLab. A big thank you to Lennart and Aleyna who lead this effort, Martin who drove the software design, and all other co-authors!

🔬 CochleaNet has been published in Molecular Therapy 🎉 CochleaNet provides deep learning-based analysis for cochlear connectomics and gene therapy validation using high-resolution isotropic light-sheet microscopy. Our goal: quantitative therapy validation for research on hearing restoration.

Gene therapies for deafness are moving towards the clinic, we provide a new tool for their pre-clinical analysis, by comprehsive quantification of SGNs, IHCs, and ribbon synapses, to analyze tonotopy, cell types, and therapy outcomes. It is also applicable to other species and other microscopy data.

I feel like giving up on reviewing for most biomedical imaging focused CS journals. Over the last year almost all papers I got where basically AI generated (probably the underlying experiments are real, but the descriptions seemed to be filtered through several AI rewrites)

Leading to well sounding, but extremely dense and jargon-heavy prose. Combined in many cases with very bad experiment design (a lot of baselines that don't fit --- asked coding agent to just compare to everything that seems relevant?).

Segmenting individual cells in microscopy is much easier these days thanks to foundation models. Can we use these models for other tasks, such as cell classification? We investigate in our latest work, finding big improvements for object and pixel classification compared to classical approaches.

We compare different foundation models (SAM and DINO variants) with classical features in ilastik and fully supervised learning. The FM features outperform classical approaches. Attentive probing yields better performance than RF, but is too slow for interactive use. See example results in the fig.

Segmenting individual cells in microscopy is much easier these days thanks to foundation models. Can we use these models for other tasks, such as cell classification? We investigate in our latest work, finding big improvements for object and pixel classification compared to classical approaches.

Initial thoughts on 5.6-Sol: actually makes codex useful (was not helpful for C++ with 5.5 for me at all). Still unhinged compared to Claude (large performance critical refactors without careful benchmarking when not being explicitly told -> lead to major regressions, Opus would not do this)

probably explaining some of the unhinged behavior. Interesting side-effect: generating reports on optimization strategies with Codex / 5.6 enables getting past Fable safeguards (which is still a superior model); it accepts validating and improving upon it while it would reject the work from scratch.

Initial thoughts on 5.6-Sol: actually makes codex useful (was not helpful for C++ with 5.5 for me at all). Still unhinged compared to Claude (large performance critical refactors without careful benchmarking when not being explicitly told -> lead to major regressions, Opus would not do this)

Sharing a big update: I started a group at the MPINAT in Göttingen! We will develop AI for analyzing how proteins interact in the cellular environment based on cutting edge imaging. This appointment is in parallel to the university, where I will retain my current group.

How can we use foundation models such as (micro)SAM to improve electron microscopy segmentation? Check out our new preprint where we found substantial improvements for nucleus, mito, and neurite-segmentation based on initialization and semi-supervised learning with foundation models.

Claude code is so funny. Ask it to generate optimized watershed code: Yes sir! Here's the validated optimized C++ code 10x faster than skimage. Ask it to generate meaningful seeds for the example image: undersegments everything and confidently declares that this are now great parameters.

After reviewing a couple of agentic tools for microscopy segmentation: I think this is a bad idea. Setting up these systems is so complicated that it negates any potential productivity gains.