Image Analysis
In this section, I will go through the image processing and analysis techniques that I used in my studies, including some innovative ones that I developed.
Total Variation (TV) Image Denoising
Image "denoising" (i.e. noise removal) is an important issue and is usually refered to as an image "pre-processing" step. This means that image denoising is usually required prior to (and strongly impacts the result of) later image processing steps such as image deconvolution, segmentation, or pattern recognition. This is especially true for low-light imaging, which occurs very often in biomedical and live-cell imaging. In Fig. 1, the "camera man" image was used as an example to demonstrate how an image could be denoised. The original image (left) was actually not a low-light image, so its inherent noise could be ignored. Then, we intentionally added noise (in this case, an uniform-level Gaussian noise was added) to simulate what it may look like in low-light imaging (middle). Finally, we processed the noise-corrupted image, using the "Total Variation" (TV) image denoising method with 2000 iterations (right). It can be clearly seen that a lot of noise has been removed, while the key features such as the camera, the face of the man, as well as the buildings behind are still preserved. We can also see that the processed image does look very similar to the original image.
TV denoising is one of the "local" denoising methods, and it removes noise from a local area basically by smoothing the intensities of the pixels in that area. In practice, it minimizes the "total variation" (imagine that noise gives variations in the image) under cerntain constraint such that the image is not over-smoothed. There are also "global" denoising techniques, such as Fourier domain filtering, that use the information from the entire image to do denoising. Global and local denoising methods have their own advantages, and one of the key advantage of local TV denoising (and why it is useful for biomedical and cell imaging) is that it is "edge-preserving" because it does directional smoothing - smoothing that occurs only along a certain direction, in this case, along the edge (low-gradient direction) but not across the edge (high-gradient direction). This effect is well-demonstrated in Fig. 1 (features preserved) as well as in Fig. 2 below.
TV denoising is one of the "local" denoising methods, and it removes noise from a local area basically by smoothing the intensities of the pixels in that area. In practice, it minimizes the "total variation" (imagine that noise gives variations in the image) under cerntain constraint such that the image is not over-smoothed. There are also "global" denoising techniques, such as Fourier domain filtering, that use the information from the entire image to do denoising. Global and local denoising methods have their own advantages, and one of the key advantage of local TV denoising (and why it is useful for biomedical and cell imaging) is that it is "edge-preserving" because it does directional smoothing - smoothing that occurs only along a certain direction, in this case, along the edge (low-gradient direction) but not across the edge (high-gradient direction). This effect is well-demonstrated in Fig. 1 (features preserved) as well as in Fig. 2 below.
Novel TV Denoising on FLIM
TV denoising methods can be applied to FLIM (you can find a detailed description on FLIM in the FLIM section), and I have developed novel TV methods (such as the "FWTV") that can deal with Poisson-distributed noise, which is commonly seen for photon counting devices. In addition, FWTV has been further modified to be capable of removing variant-level noise for low-light imaging. This is needed because in these imaging conditions, there is usually a combination of distinct forms of noise (shot noise, readout noise, Poisson noise, etc.) and the total noise level neither is constant nor has a simple relationship with the local intensity (like Poisson noise).
Fig. 2 demonstrates the results of applying the novel denoising methods (FWTV and modified FWTV) and the Poisson Wavelet method (Willett and Nowak, 2004), which is another method designed for Poisson noise removal, to time-gated FLIM images. It can be seen that only the modified FWTV can remove the variations of the fluorescence lifetime image while preserving the features - this can greatly enhance further processing such as pattern recognition, segmentation, or deconvolution. Note that in this case, denoising was implemented on the intensity images [the gray areas in Fig. 2(a) in the FLIM section] before the reconstruction of lifetime maps. In fact, denoising can even be implemented directly on lifetime maps. Check out my 2010 Journal of Biomedical Optics and 2010 Optics Express papers for more details. (Reference: Willett, R. M., Nowak, R. D., 2nd IEEE Int. Symp. Biomed. Imaging: Macro to Nano, 2004, Vol. 1–2; Chang, C. W., Mycek, M. A., Journal of Biophotonics, 2012, 5(5-6), 449-57; Chang, C. W., Mycek, M. A., Journal of Biomedical Optics, 2010, 15(5), 056013; Chang, C. W., Mycek, M. A., Optics Express, 2010, 18(8), 8688-8696)
Fig. 2 demonstrates the results of applying the novel denoising methods (FWTV and modified FWTV) and the Poisson Wavelet method (Willett and Nowak, 2004), which is another method designed for Poisson noise removal, to time-gated FLIM images. It can be seen that only the modified FWTV can remove the variations of the fluorescence lifetime image while preserving the features - this can greatly enhance further processing such as pattern recognition, segmentation, or deconvolution. Note that in this case, denoising was implemented on the intensity images [the gray areas in Fig. 2(a) in the FLIM section] before the reconstruction of lifetime maps. In fact, denoising can even be implemented directly on lifetime maps. Check out my 2010 Journal of Biomedical Optics and 2010 Optics Express papers for more details. (Reference: Willett, R. M., Nowak, R. D., 2nd IEEE Int. Symp. Biomed. Imaging: Macro to Nano, 2004, Vol. 1–2; Chang, C. W., Mycek, M. A., Journal of Biophotonics, 2012, 5(5-6), 449-57; Chang, C. W., Mycek, M. A., Journal of Biomedical Optics, 2010, 15(5), 056013; Chang, C. W., Mycek, M. A., Optics Express, 2010, 18(8), 8688-8696)
Novel TV Denoising on PET
I also applied my novel FWTV methods to positron emission tomography (PET) images. Fig. 3 demonstrates an example of quantitative parametric PET imaging of a human brain using 11C-Raclopride (Tohka and Reilhac, 2008). The first three images (Hann, ramp, and Van Cittert) were reconstructed by Tohka and Reilhac, and the last one (ramp with FWTV) was produced with my FWTV method. The first two images were reconstructed with the filtered backprojection (FBP) with the Hanning filter (denoted as ‘Hann’) and with the ramp filter (denoted as ‘ramp’). These reconstruction filters represent the two extremes in terms of the resolution/noise tradeoff. Namely, the Hann filter produces images with a low noise level but with a poor resolution. The ramp filter produces noisy images with a good resolution. The reblurred Van Cittert iteration (denoted as ‘Van Cittert’) included spatial deconvolution (the deconvolution of the signals and point spread functions) and assumed the noise process is Gaussian. The ‘Van Cittert’ method was considered by Tohka and Reilhac to be the best method in their study. Indeed, ‘Van Cittert’ produces an image that has better resolution than ‘Hann’ and less noise than ‘ramp’.
After I applied the novel FWTV method to denoise the ‘ramp’ image, I found that we could actually produce an image with significantly reduced noise compared to ‘ramp’, and, surprisingly, also with enhanced resolution compared to ‘Van Cittert‘. The sharper image produced by FWTV should be attributed to the ability to preserve edges. Another thing worth noting in Fig. 3 is that the Van Cittert method has a very obvious ‘ringing’ artifact (the edge with brighter signals), which occurs very commonly during deconvolution possibly due to inappropriate denoising. This kind of artifact does not exist in our FWTV-denoised image. (Reference: Tohka, J., Reilhac, A., Neuroimage, 2008. 39(4): p. 1570-1584; Chang, C. W., Ph.D. dissertation)
After I applied the novel FWTV method to denoise the ‘ramp’ image, I found that we could actually produce an image with significantly reduced noise compared to ‘ramp’, and, surprisingly, also with enhanced resolution compared to ‘Van Cittert‘. The sharper image produced by FWTV should be attributed to the ability to preserve edges. Another thing worth noting in Fig. 3 is that the Van Cittert method has a very obvious ‘ringing’ artifact (the edge with brighter signals), which occurs very commonly during deconvolution possibly due to inappropriate denoising. This kind of artifact does not exist in our FWTV-denoised image. (Reference: Tohka, J., Reilhac, A., Neuroimage, 2008. 39(4): p. 1570-1584; Chang, C. W., Ph.D. dissertation)
Image Segmentation, Object Tracking, and Morphometric Analysis
I also applied various image analysis techniques in my cell biology studies. In the example shown in Fig. 4, the sub-cellular structure "focal adhesions" (FAs; the locally brighter spots in the images; see the Cell Mechanics section for more details) are individually identified and each given a number for further analysis (upper left, Fig. 4). In this study, a time-course experiment was conducted, and the FAs were identified in all the images at different time points, such that these FAs could be tracked, and the trajectories of FA movement could be plotted (upper right, Fig. 4). Although most FAs did not move for a long distance, FAs in the part close to the upper left "tail" of the cell showed greater movement. A morphometric analysis was also performed to quantify the orientation of each FA (lower left, Fig. 4). Finally, all FAs were individually segmented and the FRET ratio in each FA was averaged to study the correlation between the tension and the orientation of FAs (lower right, Fig. 4; see both the Cell Mechanics section and FRET section for more details). (Reference: Chang, C. W., Kumar, S., Journal of Cell Science, 2013, 126(14), 3021-3030)