Veterinary Thermographic Image Analysis



[pic]

Veterinary Thermographic Image Analysis

Data and Temperature Normalization

Project Number 7- 64878

Report Number 4878-3

Scott E Umbaugh, BSE, MSEE, PhD

Patrick Solt, BSEE, MSCS, PhD Candidate

Computer Vision and Image Processing Laboratory

Southern Illinois University Edwardsville

January 23, 2008

Submitted to:

Dr. Catherine Loughin

Dr. Dominic Marino, Chief of Staff

Long Island Veterinary Specialists

TABLE OF CONTENTS

Executive Summary 2

Introduction and Overview 4

Materials and Methods 5

Images 5

Software 5

Data Normalization 7

Color Normalization 8

Experimental Results and Discussion 11

Preliminary Experiments 11

Data Normalization Results 11

Feature Results 12

Final Experiment Parameters 12

Results from Experiment Set #1. 13

Results from Experiment Set #1 – PCA Scatterplot 14

Results from Experiment Set #2. 15

Results from Experiment Set #2 – PCA Scatterplot 16

Results from Experiment Set #3. 17

Results from Experiment Set #3 – PCA Scatterplot 18

Results from Experiment Set #4. 19

Results from Experiment Set #4 – PCA Scatterplot 20

Conclusions 21

Future Work 22

References 23

Appendix – Excel Result File Names 24

Executive Summary

Research and development was undertaken to investigate the application of image analysis and pattern classification to thermographic images of 34 canines. Specifically, thermographic images of canines of the breed Cavalier King Charles Spaniel were examined to investigate the Chiari malformation, or COMS, pathology. Pattern classification algorithms were developed for: severe, moderate and mild classes of the pathology. Comparision between clinical and nonclinical images was explored. Additionally, the impact of sedation on the canines was investigated. The eventual goal for the research is to be able to differentiate normal and abnormal thermographic patterns in canines as a diagnostic and research tool.

The first task was to continue development of application-specific software for this research. Software from Phase II of the research was enhanced. For this phase, software was written to explore various methods of data and color normalization. The software used the CVIPtools image libraries. Continued development and enhancement of this software will be quite useful for this research area.

Four sets of experiments were performed using six data normalization methods and five color normalization methods for a total for of 1,804,200 experiments. Each experiment consisted of a specific body part, specific classes, a specific data normalization method, a specific color normalization method and specific features. Experiments were performed with all permutations of the ten selected features on each selected body part. After preliminary experimentation the MinMax, Softmax and Standard Normalization data normalization methods provided the best results and were selected for further experimentation. The texture distances of 6 and 7 provided the best results and were selected for further experimentation. Regarding color normalization, the Luminance and Norm-RGB-Lum methods provided the best results.

The first set of experiments used the clinical images that were classified as severe and moderate classes of the pathology. Norm-RGB-Lum provided the best color normalization results. Here, the front of head (A1), top of head (A1D), left side of head (A1LL) and right side of head (A1LR) images were used. With top and front of head images, 100% successful classification was achieved. With the left and right side of head images, success rates were 67% and 78%, respectively. This indicates that the top and front of head images are better for the classification than the side of head images, and Norm-RGB-Lum is the best color normalization method. These results indicate that there is a difference between the moderate and severe classes with the clinical images. This finding supports previous results.

The second set of experiments used the nonclinical images that were classified as severe, moderate and mild classes of the pathology. Here, the front, top and side of head images were used. With the top of head images, a classification success rate of 83% was achieved. The success rate for the front of head images, as well as for both the left and right side of head images was 74%. Here, results improved from previous results by about 5% to 10% due to variation of texture distance and data normalization methods. They indicate that the top of head images are best suited for classification. These results are inconclusive but indicate that there may be a difference between the mild, moderate and severe classes in the nonclinical images.

After the first two sets of experiments were performed, comparisons of success rates were made between results on the clinical and nonclinical canine images. Comparing the results we find that the separate classes in the clinical (moderate and severe) images are more homogeneous than the classes (mild, moderate and severe) in the nonclinical images.

The third set of experiments explored the pattern difference between unsedated (CS) and sedated (CSS). Here, the front, top and side of head images were used. Four sets of experiments were performed on the different body parts using two classes: sedated and unsedated. The highest classification rate of 94% was found with the top of head images and the luminance color normalization method. This represents an improvement of 4% over previous results. These results indicate that there is a pattern difference between the sedated and unsedated canines. However, we believe this finding is biased by the position of the canines when sedated versus unsedated.

The fourth set of experiments involved investigation into the similarity of the clinical and nonclinical canine images. Eight sets of experiments were performed on different body parts using two classes: clinical and nonclinical. The front, top and side of head images with moderate and severe diagnoses were used. With the moderate diagnosis, the highest classification rate was with the right side of head images at 100% differentiation between the clinical and nonclinical classes and a low classification rate of 82% (previously 65%) with the front of head images. With the severe diagnosis images, a classification rate of 100% was achieved with the front of head images, as well as for both the left and right side of head images. The success rate for the top of head images was 88% (previously 75%). The improved success rates are due to use of the luminance color normalization method. The best success rate of 100% for each diagnosis indicates that the classes clinical and nonclinical are different as groups.

Introduction and Overview

Thermographic images of canines were analyzed using feature extraction and pattern classification tools. The purpose of this research is to examine the thermographic images of canines of the breed Cavalier King Charles Spaniel to investigate the Chiari malformation, or COMS, pathology.

The primary question here was to explore various color normalization methods to determine the optimal method as applied to:

1. Can we successfully differentiate between the classes moderate and severe in the clinical images? Here four different experiments were performed:

➢ Use all front of head images (A1)

➢ Use all top of head images (A1D)

➢ Use left side of head images (A1LL)

➢ Use right side of head images (A1LR)

2. Can we successfully differentiate between the classes mild, moderate and severe in the nonclinical images?

➢ Use all front of head images

➢ Use all top of head images

➢ Use left side of head images

➢ Use right side of head images

3. Is there pattern difference between unsedated (CS) versus sedated (CSS)? Four sets of experiments were performed using two classes: sedated and unsedated.

➢ Use all front of head images (A1)

➢ Use all top of head images (A1D)

➢ Use left side of head images (A1LL)

➢ Use right side of head images (A1LR)

4. Is there a pattern difference between the clinical and nonclinical canine images? Sets of experiments with severe and moderate classes were performed on head images.

➢ Use all front of head images

➢ Use all top of head images

➢ Use left side of head images

➢ Use right side of head images

Materials and Methods

Images

The thermal images are from Long Island Veterinary Specialists taken with a Meditherm Med2000 IRIS. The images are in TIF file format as RGB images with 319 columns by 238 rows, 8-bits per pixel per color band. Note that a total of 18 colors are used in these images. The images are thermal images of 34 canines of the breed Cavalier King Charles Spaniel. Included in the data set are images of front of head, top of head, left and right side of head.

Software

The CVIPtools [1] (Computer Vision and Image Processing Tools) software was used to investigate and analyze the images. The primary tools used include Analysis->Features and Analysis->Pattern Classification. The features considered include (see Figure 1) spectral (Fourier), histogram, and texture features. The histogram features include mean, standard deviation, skew, energy and entropy; and the texture features include energy, inertia, correlation, inverse difference, and entropy. The pattern classification tools utilized include various normalization methods, distance and similarity measures and three classification methods (see Figure 2).

[pic]

Figure 1 – CVIPtools Feature Extraction window

[pic]

Figure 2 – CVIPtools Pattern Classification window

In the course of the image analysis it was determined that the previously developed special purpose software would be further enhanced for this phase of the research. The software is designed to ease the analysis process and enable us to perform a significantly greater number of experiments. We added the capability to run groups of experiments simultaneously, with the same parameters, which allowed for the analysis of comparative results. One of the primary factors under investigation was color normalization, and the software was employed to experiment with a number of normalization techniques and methods.

Data Normalization

Data normalization is used to compensate for potential bias due to the varying range on different components of the feature vectors. For example, one component may only range from 1 to 5 and another may range from 1 to 5,000, so a difference of 5 would be a maximum for the first feature, while this same difference might be insignificant for the second feature. Here we explored five data normalization methods:

• Range-normalization: The vector components are divided by the range on each vector component, where the range is simply the maximum value for that component minus the minimum.

• Unit vector normalization: Modifies the feature vectors so that they all have a magnitude of 1. This will retain only directional information about the vector, which preserves relationships between the features, but loses magnitudes.

• Standard normal density (SND): A statistical-based method to normalize feature vectors by taking each vector component and subtracting the mean and dividing by the standard deviation. The probability distributions are estimated by using the existing data.

• Min-max normalization: Maps the data to a specified range, but still retains the relationship between the values

• Softmax scaling: A nonlinear method for skewed data distributions, that is, not evenly distributed about the mean. It compresses the data into the range of 0 to 1. The process compresses the data exponentially as it gets farther away from the mean.

Color Normalization

For this phase of the research, several different methods for normalizing the colors in the images were compared. Each of the images uses the same color palette consisting of 18 colors. Within an image, each color represents a specific temperature. The color to temperature mapping for an image is based on camera settings that were in effect at the time the image was captured. Since the camera may be recalibrated between each image capture, to obtain best results for each image, within a set of images one color may map to several different temperatures. This shifting of temperature scale between images could introduce noise in the pattern classification process. To compensate for this, a remapping was applied to the color values of each image so that all of the images are mapped to a common temperature scale. Four different color normalization methods were used:

• Luminance: For each pixel in the original image, the color components are weighted as follows to generate a gray level value:

[pic]

• Norm-Gray: Temperatures from 15° C to 45° C are mapped to gray levels from 0 to 255. Each pixel in the normalized image is colored with the gray level that is mapped to the temperature represented by the color of the corresponding pixel in the original image. Figure 3 (below) shows the color remapping that occurs when this normalization method is applied to an image for which the temperature range is 21° C to 34° C. The final step of this normalization method is a histogram stretch that occurs when the image is saved as a TIFF file.

[pic]

Figure 3 – Norm-Gray remapping for image with temperature range of 21° C to 34° C

• Norm-RGB: Similar to Norm-Gray except that instead of temperatures being mapped to gray levels from 0 to 255, they are mapped to a continuous version of the original color palette. An example of the remapping that occurs when this method is applied to an image for which the temperature range is 21° C to 34° C is shown in Figure 4.

[pic]

Figure 4 – Norm-RGB remapping for image with temperature range of 21° C to 34° C

• Norm-RGB-Lum: This method produces a gray-scale image by applying the Norm-RGB normalization method to an image and then applying the luminance normalization method to the result.

Figures 5a through 5e show an original image, along with the images resulting from each of the normalization methods used:

[pic]

Figure 5a – Original Image

[pic]

Figure 5c – Norm-Gray Normalized Image

[pic]

Figure 5e – Norm-RGB-Lum Normalized Image

[pic]

Figure 5b – Luminance Normalized Image

[pic]

Figure 5d – Norm-RGB Normalized Image

Experimental Results and Discussion

Preliminary Experiments:

Preliminary experiments were performed to determine the best data normalization method for the final experiments. Under consideration were the following:

DATA NORMALIZATION METHODS

1. None

2. MinMax with range 0 to 1

3. Range

4. Softmax with r = 1

5. Standard Normal

6. Unit Vector

FEATURES

1. Histogram features: Mean, Standard deviation, Skew, Energy and Entropy

2. Texture features: Energy, Inertia, Correlation, Inverse difference, and Entropy. The pixel distance was varied from five to seven.

Data Normalization Results:

The table below shows the experiments with 100% success rates using top of head with the various data normalization methods and texture distances. Based on these results, the MinMax, Softmax and Standard Normalization methods were selected for further experimentation.

|Experiment #1 A1D - Top of Head |

|Data |Texture |No. of |Success |Color Normalization |TOTALS |

|Norm |Distance |Expments | | | |

| | | | |Orig |Lum |Norm- |Norm- |Norm- | |

| | | | | | |Grey |RGB |RGB-lum | |

|None |

|Data |Texture |No. of |Success |Color Normalization |TOTALS |

|Norm |Distance |Expments | | | |

| | | | |Orig |Lum |Norm- |Norm- |Norm- | |

| | | | | | |Grey |RGB |RGB-lum | |

|Min |

|Max |

|Body |Color |Data |Classification |Number of |

|Part |Normalization |Normalization |Success Rate |experiments at |

| | | | |that success rate |

|Top of head |Norm-RGB-Lum |MinMax |100% |135 |

|Top of head |Norm-RGB-Lum |Standard Normal |100% |122 |

|Top of head |Norm-RGB-Lum |Standard Normal |100% |121 |

|Top of head |Norm-RGB-Lum |Softmax |100% |120 |

|Top of head |Norm-RGB-Lum |Softmax |100% |116 |

➢ We believe that the 100% classification success indicates that the top of head is the most useful for differentiation of the two classes.

➢ These results indicate that there is a difference between the moderate and severe classes in the clinical images.

➢ With the left and right side of head, the best success rates were 67% and 78%, respectively.

➢ Combining results from top, front, left and right side of head images indicates that the classes are easier to differentiate with top and front images than with the left and right side head images.

➢ The best rate for front of head images was also 100%, but only 12 of the experiments from this group had a 100% classification rate.

➢ Norm-RGB-Lum provided the best color normalization results

➢ The results are limited by the small size of the data sets

Results from Experiment Set #1 – PCA Scatterplot

The principal components transform shows the clustering of the two classes. Here the original 10-dimensional feature space is linearly remapped to three dimensions in a manner that maximizes the retained information – in this case 80.8%.

This plot resulted from one of the 2,048 experiments in this set. This experiment used the top of head images and the Norm-RGB-Lum color normalization method. The classification success rate was 100%.

[pic]

Results from Experiment Set #2

Images: Nonclinical group, the front, top and side images of the head.

Classes: mild, moderate and severe.

Note: for complete results see the Excel spreadsheet file Exp2-stats-success.xls.

|Top five classification results for Experiment #2 |

|Body |Color |Data |Classification |Number of |

|Part |Normalization |Normalization |Success Rate |experiments at |

| | | | |that success rate |

|Top of head |Original Image |Standard Normal |83% |2 |

|Top of head |Original Image |Softmax |83% |2 |

|Top of head |Original Image |Softmax |78% |28 |

|Top of head |Original Image |Standard Normal |78% |22 |

|Top of head |Original Image |Softmax |74% |102 |

➢ These results indicate that the top of head images are best suited for classification.

➢ These results are inconclusive but indicate that there may be a difference between the mild, moderate and severe classes in the nonclinical images.

Results from Experiment Set #2 – PCA Scatterplot

The principal components transform shows the clustering of the three classes. Here the original 10-dimensional feature space is linearly remapped to three dimensions in a manner that maximizes the retained information – in this case 73.3%.

This plot resulted from one of the 2,048 experiments in this set. The top of head images were used, and the classification success rate was 83%.

[pic]

Results from Experiment Set #3

Images: The CS (COMS screening unsedated) and the CSS (COMS screening sedated).

The front, top and side images of the head and the left right side body images.

Classes: sedated and unsedated.

Note: for complete results see the Excel spreadsheet file Exp3-stats-success.xls.

|Top five classification results for Experiment #3 |

|Body |Color |Data |Classification |Number of |

|Part |Normalization |Normalization |Success Rate |experiments at |

| | | | |that success rate |

|Top of head |Luminance |MinMax |94% |1 |

|Top of head |Luminance |Standard Normal |94% |1 |

|Top of head |Luminance |Standard Normal |92% |1 |

|Top of head |Luminance |MinMax |92% |7 |

|Top of head |Luminance |MinMax |90% |29 |

➢ The highest classification rate for the front of head images was 89%

➢ The highest classification rate for the left side of head images was 87%

➢ The highest classification rate for the right side of head images was 76%

➢ These results indicate that there is a pattern difference between the sedated and unsedated canines

➢ The pattern difference may be biased by the fact that the sedated canines are lying down and the unsedated canines are standing. Even though we are not using any shape features, the position skews the thermographic patterns.

Results from Experiment Set #3 – PCA Scatterplot

The principal components transform shows the clustering of the two classes. Here the original 10-dimensional feature space is linearly remapped to three dimensions in a manner that maximizes the retained information – in this case 82.5%.

This plot resulted from one of the 2,048 experiments in this set. This experiment used the top of head images and the Luminance normalization method. The classification success rate was 94%.

[pic]

Results from Experiment Set #4

Images: Clinical group, the front, top and side images of the head.

Classes: clinical and nonclinical.

Note: for complete results see the Excel spreadsheet file Exp4-stats-success.xls.

|Top five classification results for Experiment #4 |

|Experiment |Body |Color |Data |Classification |Number of |

| |Part |Normalization |Normalization |Success Rate |experiments at |

| | | | | |that success rate |

|Exp4-severe |Front of head |Luminance |Softmax |100% |205 |

|Exp4-mod |Right side of head |Luminance |Softmax |100% |194 |

|Exp4-severe |Front of head |Luminance |Standard Normal |100% |177 |

|Exp4-severe |Front of head |Luminance |MinMax |100% |174 |

|Exp4-mod |Right side of head |Luminance |Standard Normal |100% |84 |

➢ These results indicate that there is a difference between the clinical and nonclinical canines as groups

➢ The highest classification rate for the front of head moderate images was 82%

➢ The highest classification rate for the top of head moderate images was 91%

➢ The highest classification rate for the left side of head moderate images was 91%

➢ The highest classification rate for the top of head severe images was 88%

➢ The highest classification rate for the left side of head severe images was 100%

➢ The highest classification rate for the right side of head severe images was 100%

➢ The results are limited by the small size of the data sets

Results from Experiment Set #4 – PCA Scatterplot

The principal components transform shows the clustering of the two classes. Here the original 10-dimensional feature space is linearly remapped to three dimensions in a manner that maximizes the retained information – in this case 85.8%.

This plot resulted from one of the 2,048 experiments in this set. This experiment used the right side of head images and the Luminance normalization method. The classification success rate was 100%.

[pic]

Conclusions

Four sets of experiments were performed for a total for of 1,804,200 experiments. Preliminary experimentation showed that the MinMax, Softmax and Standard Normalization data normalization methods provided the best results and were thus selected for further experimentation. The texture distances of 6 and 7 provided the best results and were selected for further experimentation. Regarding color normalization, the Luminance and Norm-RGB-Lum methods provided the best results.

The first set of experiements was to investigate classification of the moderate and severe classes with the clinical images. With top and front images, 100% successful classification was achieved. Norm-RGB-Lum provided the best color normalization results. With the left and right side of head success rates were 67% and 78%, respectively. This indicates that the top and front of head images are better for the classification than the side of head images. These results indicate that there is a difference between the moderate and severe classes with the clinical images.

The second set of experiments used the nonclinical images and classified that were classified as severe, moderate and mild classes of the pathology. Here, the front, top and side images of the head were used and the original images provided the best results. Success rates from 74% to 83% lead to inclusive results but indicate that there may be a difference between the mild, moderate and severe classes in the nonclinical images.

Comparing the results from the first two sets of experiments we find that the separate classes in the clinical (moderate and severe) images are more homogeneous than the classes (mild, moderate and severe) in the nonclinical images.

The third set of experiments explored the pattern difference between unsedated (CS) and sedated (CSS). The highest classification rate of 94% was found with the top of head images using the luminance color normalization method. This represents an improvement of 4% over previous results. These results indicate that there is a pattern difference between the sedated and unsedated canines. However, we believe this finding is biased by the position of the canines when sedated versus unsedated.

The fourth set of experiments involved investigation into the similarity of the clinical and nonclinical canine images. The best color normalization results were from the luminance transform. The results indicate that the severe diagnosis is more easily separable into the clinical and nonclinical classes than the moderate diagnosis. The best success rate of 100% for each diagnosis indicates that the classes clinical and nonclinical are different as groups.

Future Work

• Collect more images for differentiation of the mild, moderate and severe classes of both clinical and nonclinical images

• Determine a methodology to unbias the sedated versus unsedated images

• Further analyze the results from the1,804,200 experiments, including analysis of frequency of features relative to classification success and feature set size relative to success

• Analyze Experiment Sets #2 and #3 to determine why specific examples were misclassified

• Investigate use of support vector machines for analysis of veterinary thermographic images

• Research the use of neural networks for the analysis of veterinary thermographic images

• Investigate use of genetic algorithms for the analysis of veterinary thermographic images

• Research methods to determine the “best” feature set(s)

• Enhance the graphical user interface (GUI) for the automatic pattern classification and application-specific software

• Explore finding of feature vector “norms” for various pathologies/abnormalities

• Explore multimodal image analysis

• Investigate development of automatic preprocessing software for thermographic images

• Investigate development of automatic segmentation software for thermographic images

• Observe first hand the image acquisition process to explore standardization of this process and observation of the methods

• Consult the veterinarians to learn more regarding analysis and diagnostic uses of the thermographic images

• Confer with the veterinarians and/or technicians in the development and use of the application-specific software

References

1. Veterinary Thermographic Image Analysis, Project Number 7-64878, Report Number 4878-1, December 22, 2006

2. Veterinary Thermographic Image Analysis, Project Number 7-64878, Report Number 4878-2, August 31, 2007

3. Computer Imaging: Digital Image Analysis and Processing , Scott E Umbaugh, The CRC Press, Boca Raton, FL, January 2005

4. Pattern Recognition Engineering, Morton Adler and Eric Smith, John-Wiley & Sons, NY, 1993

5. Pattern Classification, 2nd Edition, Richard Duda, Peter Hart, David Stork, John-Wiley & Sons, NY, 2001

6. Pattern Recognition, Sergios Theodoris and Konstantinos Koutroumbas, Academic Press, NY, 2006

Appendix – Excel Result File Names

The resultant files are not included here, but are in Excel spreadsheet files with the following file names.

Result Data Files

|Experiment |File name |

|Experiment Set #1 |Exp1-Stats-Success.xls |

|Experiment Set #2 |Exp2-Stats-Success.xls |

|Experiment Set #3 |Exp3-Stats-Success.xls |

|Experiment Set #4 |Exp4-Stats-Success.xls |

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download