Implementation of hybrid image fusion technique for feature enhancement in medical diagnosis
- Jyoti Agarwal^{1}Email author and
- Sarabjeet Singh Bedi^{2}
DOI: 10.1186/s13673-014-0020-z
© Agarwal and Bedi; licensee Springer. 2015
Received: 20 September 2014
Accepted: 21 December 2014
Published: 4 February 2015
Abstract
Image fusion is used to enhance the quality of images by combining two images of same scene obtained from different techniques. In medical diagnosis by combining the images obtained by Computed Tomography (CT) scan and Magnetic Resonance Imaging (MRI) we get more information and additional data from fused image. This paper presents a hybrid technique using curvelet and wavelet transform used in medical diagnosis. In this technique the image is segmented into bands using wavelet transform, the segmented image is then fused into sub bands using curvelet transform which breaks the bands into overlapping tiles and efficiently converting the curves in images using straight lines. These tiles are integrated together using inverse wavelet transform to produce a highly informative fused image. Wavelet based fusion extracts spatial details from high resolution bands but its limitation lies in the fusion of curved shapes. Therefore for better information and higher resolution on curved shapes we are blending wavelet transform with curvelet transform as we know that curvelet transform deals effectively with curves areas, corners and profiles. These two fusion techniques are extracted and then fused implementing hybrid image fusion algorithm, findings shows that fused image has minimum errors and present better quality results. The peak signal to noise ratio value for the hybrid method was higher in comparison to that of wavelet and curvelet transform fused images. Also we get improved statistics results in terms of Entropy, Peak signal to noise ratio, correlation coefficient, mutual information and edge association. This shows that the quality of fused image was better in case of hybrid method.
Keywords
Image fusion Wavelet transform Curvelet transform Hybrid image fusionIntroduction
Fusion of two or more images of the same scene to form a single image is known as image fusion. Image fusion process combines the relevant information from two or more images into single image therefore the resultant fused image will be more informative and having important features from each image. Image fusion is important in many different image processing fields such as satellite imaging, remote sensing and medical imaging. Several fusion algorithms have been evolved such as pyramid based, wavelet based, curvelet based, HSI (Hue Saturation Intensity), color model, PCA (Principal Component Analysis) method. All of them lacks in one criteria or the other [1]. Fusion of medical images should be taken carefully as the whole diagnosis process depends on it. Medical images should be of high resolution with maximum possible details [2]. The medical images should represent all important characteristics of the organ to be imaged so the integrated image should present maximum possible details. Therefore our aim is to adopt the best method of image fusion so that the diagnosis should be accurate and perfect [3].
Wavelet method was supposed to be one of the most promising methods of image fusion due to its simplicity and ability to preserve the time and frequency details of the image to be fused. Wavelet Fusion transforms the images from spatial domain to wavelet domain. The wavelet domain represents the wavelet coefficient of the images [4,5]. The wavelet decomposition is performed by passing the image into series of low pass and high pass filters. In this method the input signal goes through two one digital filters. One of them performs high pass filtering and the other performs low pass filtering. The various filter bands are produced and each band producing images of different resolution levels and orientations. These sub bands are then combined using inverse wavelet transform [6-8].
The curvelet transform is used to represent the curved shapes. This transform represents edges better than wavelets. The fused image obtained yields higher details than the original image due to edge representation thereby preventing image denoising. It is based on the segmentation of the whole image into small overlapping tiles and then the ridgelet transform is applied to each tile [9]. The studies on curvelet fusion of MR and CT images shows that the application of curvelet transform in the fusion of MR and CT images is superior to the application of traditional wavelet transform. Segmentation approximates the curved lines by small straight lines [10]. Geetha et al. [11] suggested that the performance of algorithms can be improved by introducing the directional oriented multiresolution transforms such as steerable pyramids, contourlets etc., overlapping of tiles prevents edge effects. Bindu and Kumar [12] further evaluated the performance analysis of multi source (CT, PET and MRI images) fused medical images using multiresolution (combination of DWT and contourlet) transforms. While denoising of computer tomography images using curvelet transforms it has been found that the curvelet transform outperforms the wavelet transform in terms of signal to noise ratio [13,14].
Material and methods
CT and MRI images
Simulation tool/software used (MATLAB 7.1)
Proposed research work has been developed by using MATLAB ® for simulation of image fusion algorithms. This high performance language for technical computer, integrates computation, visualization, and programming in an easy-to-use environment. One of the reasons of selecting MATLAB in this research is because it fits perfectly in the necessities of an image processing research due to its inherent characteristics. MATLAB basic data element is an array that does not require dimensioning. This is especially helpful to solve problems with matrix and vector formulations. And an image is nothing but a matrix or set of matrices which define the pixels value of the image, such a grey scale value in black and white images, and red, green and blue or Hue, Saturation and Intensity values in color images.
Image Processing Toolbox of MATLAB provides a comprehensive set of reference-standard algorithms and graphical tools for image processing such as analysis, image enhancement, feature detection, noise reduction and image registration etc.
Image processing toolbox supports a diverse set of image types, including high dynamic range, high resolution. Graphical tools explore an image, examine a region of pixels, adjust the contrast, create contours or histograms, and manipulate regions of interest (ROIs) Figure 2.
Image fusion using wavelet transform
Curvelet transform
- A)
The image P is split up into three sub bands Δ1, Δ2 and P3 using the additive wavelet transforms [15].
- B)
Tiling is performed on the sub bands Δ1 and Δ2.
- C)
The discrete Ridgelet transform is performed on each tile of the sub bands Δ1 and Δ2.
- 1.
Sub band filtering
- 2.
Tiling
- 3.
Ridgelet Transform
Tiling divides the image into overlapping tiles. These tiles are then used for transformation of curved lines into straight lines in the sub bands formed during sub band filtering. Tiling thus helps in handling curved edges.
Is the ridgelet basic function for each a > 0 and θ є [0,2π]. This function is constant along with lines X _{1} cosθ + X _{2}sinθ = constant.
Hence, the ridgelet transform is the application of the 1-D wavelet transforms to the slices of the Radon transform where the angular variable θ is constant and it is varying. To make the ridgelet transform discrete, both the radon transform and the wavelet transform have to be discrete. Thus curvelet transform can be used to analyze dense structures in human body by visualising its shape, morphology of soft tissues and few other.
Hybrid image fusion technique
A single method of fusion may not be as efficient as it always lacks in one point or the other. Therefore their exists the need of developing a method which takes into consideration the advantages of various different fusion rules. Thus the hybrid image fusion is used. It performs processing of the image based upon the different fusion rules and then integrates these results together to obtain a single image. The results of various fusion techniques are extracted and then they are again fused by implementing a hybrid method presenting better quality results. A single method may not effectively result in removing the ringing artifacts and the noise in the source images. These inadequacies result in development of fusion rules which follow a hybrid algorithm and improve to great extent the visual quality of the image [17-19].
Therefore Hybrid Image fusion leads to minimum Mean Square Error Value and maximum Signal to Noise (S/N) Ratio value. The proposed work in this paper will describe the hybrid of two methods that is the wavelet based image fusion and the curvelet based image fusion.
Proposed work (hybrid of wavelet and curvelet fusion rules)
A hybrid of wavelet and curvelet integrates various pixel level rules in a single fused image. Pixel based rules operates on individual pixels in the image but ignores some important details such as edges, boundaries of the image. Wavelet based rule alone may reduce the contrast in some images and cannot effectively remove the ringing effects and noise appearing in the source images. Curvelet method can work well with edges and boundaries and curve portions of the images using ridgelet transforms. In the hybrid method first the decomposition of the input images is done up to level N by passing the image through series of low and high pass filters. The low and high pass bands are then subjected to curvelet transform by decomposing it further into small tiles and then fused using wavelet transform and inverse wavelet transform to get full size images. This will take into account the drawbacks of wavelet and effectively remove it using curvelet transform and visual quality of the image is improved. Wavelet transform of an image up to level N till different resolution is obtained. This gives various frequency bands. Figure 3 shows the procedure of combining image 1 and image 2 into single fused wavelet coefficients.
These bands obtained are then passed through curvelet transform which segments it into various additive components each of which is subband of the image. These bands are then passed through tiling operation which divides the band to overlapping tiles. The tiles are small in dimension to transform curved lines into small straight line and overlapping is done so as to avoid the edge effects. Tiling operation is performed after subband filtering of the filter bands. Finally these overlapped tiles are passes through ridgelet transform which is a kind of 1-D wavelet transform helps in wavelet transform in Radon Domain which helps in shape and edge detection (Figure 4) band obtained by wavelet transform. Now these fused bands after curvelet transform are fused again using Inverse wavelet Transform. The Inverse Wavelet Transform fuses together all the bands and result in a full size integrated image.
Performance parameters for determining the quality of fused Image
- A.
Entropy (H)
The Entropy (H) is the measure of information content in an image. The maximum value of entropy can be produced when each gray level of the whole range has the same frequency. If entropy of fused image is higher than parent image then it indicates that the fused image contains more information.$$ H=-{\displaystyle {\sum}_{g=0}^{L-1}p(g) \log {}_2p(q)} $$(5) - B.
Correlation Coefficient (CC)
The correlation coefficient is the measure the closeness or similarity in small size structures between the original and the fused images. It can vary between −1 and + l .Values closer to +1 indicate that the reference and fused images are highly similar while the values closer to −1 indicate that the images are highly dissimilar.$$ CORR = \frac{2{\boldsymbol{c}}_{\boldsymbol{r}\boldsymbol{f}}}{{\boldsymbol{C}}_{\boldsymbol{r}+{\boldsymbol{C}}_{\boldsymbol{f}}}} $$(6)Where$$ Cr={\displaystyle {\sum}_{j=1}^M{\displaystyle {\sum}_{j=1}^N{I}_{r\ }{\left(i,\ j\right)}^2}} $$(7)$$ {C}_f={\displaystyle {\sum}_{j=1}^M{\displaystyle {\sum}_{j=1}^N{I}_f{\left(i,\ j\right)}^2}} $$(8)$$ {C}_{rf=}{\displaystyle {\sum}_{i=1}^M{\displaystyle {\sum}_{j=1}^N{I}_f\left(i,j\right){I}_r\left(i,\ j\right)}} $$(9) - C.
Root Mean Square Error (RMSE)
A commonly used reference based assessment metric is the RMSE. The RMSE will measure the difference between a reference image, R, and a fused image, F, RMSE is given by the following equation$$ RMSE=\sqrt{\frac{1}{\mathrm{MN}}{\displaystyle {\sum}_{\mathrm{n}=1}^M{\displaystyle {\sum}_{\mathrm{n}=1}^N\left(R\left(m,n\right)-Fm,n\right)\Big){}^2}}} $$(10)where R(m, n) and F(m, n) are the reference (CT or MR) and fused images, respectively, and M and N are image dimensions. Smaller the value of the RMSE, better the performance of the fusion algorithm.
- D.
Peak Signal to Noise Ratio (PSNR)
PSNR is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. The PSNR of the fusion result is defined as follows$$ \mathrm{PSNR}=10\mathrm{X} \log \frac{\left({f}_{max}\right)2}{\left( RMSE\ \right)2} $$(11)where f_{max} is the maximum gray scale value of the pixels in the fused image. Higher the value of the PSNR, better the performance of the fusion algorithm.
- E.
Mutual Information (MI)
Mutual information is the basic concept of measuring the statistical dependence between two random variables and the amount of information that one variable contains about the others. Mutual information here describes the similarity of the image intensity distributions of the corresponding image pair. Let A and B be two random variables with marginal probability distributions p_{A}(a) and p_{B}(B) and joint probability distribution p_{AB}(a, b)$$ {I}_{AB}\left(a,b\right) = {\displaystyle {\sum}_{x,\ y}{P}_{AB}\left(a,b\right) \log \frac{P_{AB}\ \left(a,b\right)}{P_A(a){P}_B(b)}} $$(12)Considering two input images A, B and a fused image F we can calculate the amount of information that F contains about A and B according to above equation$$ {I}_{FA}\left(f,a\right) = {\displaystyle {\sum}_{x,\ y}{P}_{FA}\left(f,a\right) \log \frac{P_{FA}\ \left(f,a\right)}{P_F(f){P}_A(a)}} $$(13)$$ {I}_{FB}\left(f,b\right) = {\displaystyle {\sum}_{x,\ y}{P}_{FB}\left(f,b\right) \log \frac{P_{FB}\left(f,b\right)}{P_F(f){P}_B(b)}} $$(14)Thus the mutual information is given by$$ {M}_F^{AB} = {I}_{FA}\left(f,a\right) + {I}_{FB}\left(f,\ b\right) $$(15) - F.
Edge association
It is a measure of important visual information with the edge that is present in each pixel of an image. The visual to edge information association is well supported by HVS (Human Visual System) studies and is frequently used in compression systems and image analysis. The amount of edge information that is transferred from input images to the fused image can be obtained as$$ {Q}_F^{AB/F} = \frac{{\displaystyle {\sum}_{x=1}^M}{\displaystyle {\sum}_{y=1}^N}{Q}^{AF}\left(x,y\right){w}^A\left(x,y\right) + {Q}^{BF}\left(x,y\right){w}^B\left(x,y\right)}{{\displaystyle {\sum}_{x=1}^M}{\displaystyle {\sum}_{y=1}^N}\left({w}^A\left(x,y\right) + {w}^B\left(x,y\right)\right)} $$(16)Where Q^{AF}, Q^{BF} are edge preservation values and W^{A}, W^{B} are the corresponding weights.
Results
The hybrid image fusion has been performed using CT and MRI images. The quality of image obtained by hybrid technique has been verified using various criteria such as entropy, correlation coefficient, peak signal to noise ratio and root mean square error. The original input images and their corresponding fusion results using the proposed technique are depicted in detail. Wavelet and Curvelet transform are applied on the source images and then transform coefficients obtained are obtained for five different fusion methods.
Hybrid image fusion using GUI interface of MATLAB
Statistics results of various fusion methods
Fusion methods | Metrics | |||||
---|---|---|---|---|---|---|
Entropy | RMSE | PSNR | CC | MI | Q ^{ AB/F } | |
Select maximum | 6.63 | 4.248 | 29.56 | 0.61 | 5.23 | 0.55 |
Select minimum | 2.89 | 15.23 | 23.25 | 0.67 | 6.92 | 0.74 |
Simple average | 4.23 | 13.23 | 27.32 | 0.71 | 4.81 | 0.65 |
Principle component | 6.34 | 3.421 | 36.12 | 0.88 | 5.89 | 0.79 |
Laplace transform | 7.45 | 3.921 | 39.24 | 0.90 | 4.56 | 0.80 |
Wavelet transform | 7.77 | 3.442 | 29.33 | 0.92 | 6.23 | 0.89 |
Curvelet transform | 8.54 | 3.436 | 38.77 | 0.89 | 6.85 | 0.75 |
Hybrid transform | 8.81 | 3.316 | 41.91 | 0.85 | 7.44 | 0.91 |
Effect of fusion methods on entropy and root mean square error
Effect of fusion methods on peak signal to noise ratio and correlation coefficient
Effect of fusion methods on mutual index and edge association
From the above analysis it has been concluded that hybrid transform works well with all the fusion methods also the value of testing parameters are optimum for hybrid transform as visualization is clear and image is more intact by the combination of the two transform methods i.e. wavelet and curvelet transforms. Furthermore the proposed hybrid fusion scheme in this research work compensates all the short comings of wavelet and curvelet transform. It also removes the ringing effect and produced smooth corners and edges in the fused image. From the image quality assessment tables, it is clear that the proposed fusion technique outperforms other methods based on performance evaluation criteria’s i.e. Entropy, Correlation Coefficient, Peak signal to noise ratio, Root mean square error, Mutual index and Edge information. The fusion methods also focuses on the fact the finally obtained image is much clearer and contains more information in comparison to the other existing fusion methods.
Conclusion
In this research work, attention was drawn towards the current trend of the use of multiresolution image fusion techniques such as wavelet transform and curvelet transform. An efficient image fusion technique has been proposed here which is formed by combining the features of both wavelet and curvelet image fusion algorithms. In our proposed technique of image fusion we get more enhanced image and work well for edges, corners and helps in minimization of the localized errors. The high pass filter mask enhances the edges whereas averaging filter mask helps in removing noise by taking mean of grey values surrounding the centre pixel of the window. The response to image fusion is found to have higher values of Entropy, Peak signal to noise ratio, correlation coefficient, mutual index and edge association. The root mean square error also gets reduced. Finally the smoothness parameter should be taken relatively high value to decrease the slope of the filter function reducing the oscillations of the filter response function in the time domain.
Thus the two different modality images are fused using the various fusion rules based on the Wavelet, Curvelet and hybrid transforms. Moreover the difference in performance for these transforms is clearly exhibited using six performance measures. It is observed that, fusion methodology based on the Curvelet transform has given curved visual details better than those given by the Wavelet fusion algorithm. The fused image obtained using hybrid transform contains more useful information than the fused image using wavelet or curvelet transform. The proposed technique compensates all the shortcomings of either wavelet or curvelet transform method of fusion. Thus enabling the radiologists to locate the imperfections accurately, making the treatment easier and perfect.
From the various image quality assessment table and graphs, it has been clear that the proposed fusion technique outperforms other methods in terms of entropy, correlation coefficient, peak signal to noise ratio, root mean square error, mutal index information and edge association.
Declarations
Authors’ Affiliations
References
- K P Soman, K I Ramachandran (2005) Insight into Wavelets from Theory to Practice, 2nd edn. PHI Learning Pvt. Ltd, New Delhi -110001, IndiaGoogle Scholar
- Ping YL, Sheng LB, Hua ZD (2007) Novel image fusion algorithm with novel performance evaluation method. Syst Eng Electron 29:509–513Google Scholar
- Sahu DK, Parsai MP (2012) Different image fusion techniques – a critical review. Int J Modern Eng Res 2:4298–4301Google Scholar
- Hall D, Llinas J (1997) An introduction to multisensory data fusion. Proc IEEE 85:6–23View ArticleGoogle Scholar
- Wu H, Xing Y (2010) Pixel – based image fusion using wavelet transform for SPOT and ETM + Image. IEEE Trans 19:6744–6789Google Scholar
- Kirchgeorg M, Prokop M (1998) Increasing spiral CT benefits with post processing applications. Eur J Radiol 28:39–54View ArticleGoogle Scholar
- Tan Y, Shi Y, Tan KC (2010) A multi-modality medical image fusion algorithm based on wavelet transforms. Adv Swarm Intell 6146:627–633View ArticleGoogle Scholar
- Godse DA, Bormane DS (2011) Wavelet based image fusion using pixel based maximum selection rule. Int J Eng Sci Technol 3:5572–5577Google Scholar
- Chandana M, Amutha S, Kumar N (2011) A hybrid multi-focus medical image fusion based on wavelet transform. Int J Res Rev Comput Sci 2:1187–1192Google Scholar
- Ali FE, El-Dokany IM, Saad AA, Abd El-Samie FES (2008) Curvelet fusion of MR and CT images. Prog Electromagnetics Res C 3:215–224View ArticleGoogle Scholar
- Geetha G, Raja Mohammad S, Murthy YSSR (2012) Multifocus image fusion using multiresolution approach with bilateral gradient based sharpness criterion. J Comput Sci Inf Technol 10:103–115Google Scholar
- Bindu CH, Prasad KS (2012) Performance analysis of multi source fused medical images using multiresolution transforms. Int J Adv Comput Sci Appl 3(10):54–62Google Scholar
- Sivakumar R (2007) Denoising of computer tomography images using curvelet transform. ARPN J Eng Appl Sci 2(1):21–26Google Scholar
- Stark JL, Candes E, Donoho DL (2002) The curvelet transform for image denoising. IEEE Trans Image Process 11(6):670–684View ArticleMathSciNetGoogle Scholar
- Chao Z, Zhang K, Li YJ (2004) An image fusion algorithm using wavelet transform. Chin J Electron 32:750–753Google Scholar
- Choi M, Kim RY, Kim MG (2004) The curvelet transform for image fusion. Int Soc Photo Grammetry Remote Sensing 35:59–64Google Scholar
- Tsai IC, Huang YL, Kuo KH (2012) Left ventricular myocardium segmentation on arterial phase of multi-detector row computed tomography. Comput Med Imaging Graph 36:25–37View ArticleGoogle Scholar
- Zhan GQ, Guo BL (2006) Fusion of multi sensor images based on the curvelet transforms. J Optoelectronics Laser 17:1123–1127Google Scholar
- Bedi SS, Agarwal J, Agarwal P (2013) Image fusion techniques and quality assessment parameters for clinical diagnosis: a review. Int J Adv Res Comput Commun Eng 2:1153–1157Google Scholar
Copyright
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.