Automated synaptic vesicles detection with deep learning
Synaptic vesicles (blue) are microscopic structures localized at synaptic terminals (green). They are located in all neurons of our brain and their function is to transport neurotransmitters, the molecules that allow neurons to communicate with each other.
Studying the density and distribution of these small vesicles is important to better understand many processes of our brain, such as learning and memory.
image by courtesy of Dr. Marta Orlando
Synaptic vesicles can be visualized with high resolution electron microscopy (EM) (right figure). However, their identification has been so far a tedious manual
task costing a lot of precious time to PhD students and PostDoc.
Taking advantage of modern deep learning algorithms, I developed an interactive analytic tool to automate the detection and localization of these small
structures.
Below on this page, you will find:
o Some relevant links;
o A simplified explanation of how the algorithm works (for a detailed description look at the link to our paper*);
Relevant links:
* If you are interested of a detailed explanation of our tool, please visit our paper at: https://www.eneuro.org/content/9/1/ENEURO.0400-20.2021
The source codes to use this tool can be found at: https://github.com/Imbrosci/synaptic-vesicles-detection
How the algorithm works and how to use it
To build this tool the first step was to teach an deep learning model how a synaptic vesicle looks like. To do so, we trained a convolutional neural network with
thousands of image patches, labeling each of them as either containing or not containing a vesicle.
Once the model learned the task, we let it screen a whole EM image and produce a probability map. The video demonstrates the model at work producing the probability map in pink. This map represents the prediction of our model and tells us the probability for each pixel to be part of a vesicle.
The pixels whose predictions were below 0.5 were set to 0.
The probability map was not our final product. What we want is the count of the vesicles, their position and maybe even their area. To do so, after some processing,
thresholding and binarization steps, we performed a connected-component analysis. We basically let an algorithm separate the "objects" (hopefully single vesicles). An example is shown
below.
Finally, we introduced some steps to minimize false positives and some merge errors and obtained our final result! As you can see below, the detected vesicles are
marked as white dots!
Finally, our tool offers the possibility to measure the area of the detected vesicles. This is achieved by an algorithm looking for the membrane of each vesicle. A membrane indeed delimites each vesicle and in EM images is darker than the vesicles' lumen.
The algorithm draws an ellipse of different radius on top of each vesicles and screen for darker regions (see video). The major (a) and minor radius (b) of the ellipse matching the most with the darker membrane are then used to calculate the area with this simple formula: a * b * π * pixel size2.
Note that, for illustration purposes, the video represents a simplification of what the algorithm does.
To guaranty the usage of the algorithm to people with no or little programming experience, the tool was embedded in a Graphical User Interface (GUI).
The results can be found in an automatically generated excel file and possible residual errors can be corrected via the GUI interactively.
This project has been conducted in collaboration with my friend and colleague Dr. Marta Orlando who took care of the biological aspects and produced the EM images.