SEQUENTIAL ANALYSIS OF MEDICAL IMAGES USING NEURAL …



Sequential Analysis of Medical Images

Using Neural Networks

Marek Czeleń, Grzegorz Wojtas, Zbigniew Mikrut[1]

Institute of Automatics, University of Mining and Metallurgy

Mickiewicza 30, 30-059 Kraków, Poland

e-mail: zibi@biocyb.ia.agh.edu.pl

Abstract. A problem frequently encountered in the analysis of medical images is reaching a correct and reliable diagnosis of the examined organ, particularly by persons with little experience, e.g. first year medical students. That fact has become an inspiration for construction of an assisting program, based on application of three neural networks: 1) moving the attention point along the selected object, 2) elaborating the diagnosis for the area in the current viewport, 3) reporting the end of the examination.

In the present work a summary is done for almost a hundred of experiments, testing the efficiency of all those neural networks in the analysis of raw medical images. The final version of the algorithm uses backpropagation type networks: it operates on square areas which are 31x31 pixels in size for the first and third networks and 91x91 pixels in size for the second (diagnosing) network. The achieved level of correct recognition (about 80%) is a very good result, if one takes into account rather low quality (dynamic range) of the analysed images and the observed natural interferences.

1 Introduction

With the increasing presence of the equipment for digital recording and visualisation of medical images more and more needs are expressed, concerning not only the quality increase of the registered images but also their intelligent analysis. Such systems are narrowly specialised applications, the role of which is to suggest the answer, leaving the final decision to the human staff. Most of such applications performs an overall analysis of the selected object, applying various preprocessing methods, segmentation, detection and classification of preselected features, both by neural [3, 8] and non-neural [5] methods.

An interesting alternative is provided by a sequential object analysis, for which a distant and unequal led ideal is the human vision system. It seems the neural networks are one of the few possibilities for realisation of that technique [6, 1]. This concerns both implementation of moving the so called "attention point" and the analysis (diagnosing) of the currently "seen" image window.

In order to preserve a consistent approach to the problem for the present study it was decided that raw data or slightly preprocessed medical image data will be used as input. It reflects the authors' belief that an adequately complicated neural network will be able to extract the interesting image areas by itself. Other concepts of recognition based on raw images can be found in papers [7, 2, 4].

The material selected for the study consisted of ERCP[2] X-ray images, registered for several patients (see Fig. 3). The task for the created application was a preliminary examination of the pancreatic duct area preselected by the human operator. During the pancreas area tracking the application should be able to provide a current diagnosis, looking for the following pathological changes:

1. branches characteristic for the acute pancreas inflammation

2. swellings and constrictions characteristic for the pancreas cancer

3. presence of cysts.

2 Methods and Experiments

In order to select an appropriate form of the input data and the neural network structures, several experiments have been carried out, during which the networks were learned and their structure was being adjusted. Therefore it is not possible to strictly divide the description of the study into the classical stages, i.e. design of the input data form, choice of the network structure and the experiments themselves. The concepts presented below are results from positively completed tests.

2.1 Task Distribution

The artificial neural networks operating on raw images are able to perform well in only one learned task. That is a limitation for the sequential image analysis procedure, because for a complete analysis of the pancreatic duct image in each step it is necessary to determine the motion direction, elaborate the local diagnosis and possibly evaluate the condition for the algorithm termination. Therefore it has been decided that three independent networks will be used:

1) the first network determined the motion direction for the "focus of attention" point using a small (31x31 pixels) area of the image, what was equivalent to determination of a new actual position

2) the second network elaborated the local diagnosis at the actual position using a bigger image area than the previous network (91x91 pixels), which was reduced to the area 31x31 pixels in size

3) the third network, operating on the same image area as the diagnosing network, determined whether any part of the pancreatic duct can be seen. It was one of the conditions required for program termination, in the case when all the objects has been analysed or in a situation when the first network lost the tracked object.

2.2 Construction of the Learning Sequences

During the construction of the learning sequences the main objective was to ensure that they are properly representative. For the network setting the motion direction the images and the respective motion directions have been selected in such a way that the direction distribution was reasonably uniform or the forward motion was slightly preferred. Obviously the images should be consistent (non-contradicting) what has been finally ensured by proposing a concept of "virtual vehicle" (see sect. 2.4).

Another problem was the necessity of making a decision concerning the choice of the algorithm used for determination of the motion direction for the "narrow" and "wide" objects (see Fig. 3a and 3b respectively). When the pancreatic duct is narrow, then bright areas can be seen on both sides. On the other hand when it is wide, then the bright area can be detected on one side only. Frequently other objects are visible in the image window, which can interfere in the process of direction determination (e.g. in almost every image a duodenoscope tube - a device for dosage the contrast medium - can be seen). Such situations have also been taken into account during the construction of the learning sequence.

It was also ensured that the numbers of images representing particular diseases (pathological conditions) were approximately the same. The images have been registered at different angles, so that after the learning stage the network would be able to carry out a proper evaluation. In the construction of the learning set for the network responsible for analysis termination both the images containing parts of the pancreatic duct and images without such parts have been taken into account. The ratio of the respective images was kept close to the 1:1.

2.3 Preprocessing

The ERCP image express particularly low dynamic range (see Fig. 1 upper row). The preliminary experiments have shown that the neural network was not able to correct the dynamic range on its own, and - what is a consequence of the above - that execution of a simple preprocessing essentially affects the obtained results. From several examined methods of contrast enhancement based on the histogram analysis one was selected, consisting of histogram stretching to the full brightness range accompanied by aggregation of 10% brightest and 10% darkest pixels. Several examples of application of the above mentioned operation are shown in Fig.1.

Fig. 1. Image contrasting using the histogram analysis

2.4 Moving the Focus of Attention

The best results have been obtained by implementing the "virtual vehicle" concept, in which the window to be analysed behaves like a small vehicle, moving along the pancreatic duct as if it was a road. During consecutive steps the window is turned in such a way that its centre is kept in the same location, while the lateral edge (with respect to the motion direction) is kept parallel to the segment connecting the actual analysis window centre with its previous location (Fig.2)

Fig.2.The virtual vehicle

concept

For such a situation the network "sees" similarly oriented images and is able to determine the direction of motion for the attention point with its proper orientation ("forward" direction). The rotation angle is determined using the actual and previous location of the window centre by the program. Therefore the neural network should be able to recognise relative direction (with respect to the previous motion direction) within the [-90,90] degrees.

The tests have not confirmed the utility of methods based on the log-polar transform and similar concepts. It was a direct consequence of considerable in-formation averaging near the edges of the input image area, what is an advantage only when the tracked object is within the window. The attempts of application of "receptor fields", detecting lines of various inclinations have lead to better results than the transforms mentioned above, but still the results were worse then for the pure contrast enhancement. An attempt of applying a Kohonen network, as a feature detecting network, did not work at all. Also the combining of contrasted image with the information taken from the receptor field or with the Kohonen network did not improve the correctness of the network operation (see Tab. 1).

3 Results and Discussion

The structures of all the networks tested are listed in Table 1, while several examples of analysis are presented in Fig.3.

In the cases when the pancreatic duct was located too close to other objects entering the "region of interest" of the direction determining network or when it crossed other objects (Fig.3a) the network was not able to determine properly the motion direction because the information about context (surrounding) was not sufficient. Possible extension of the window on one hand would increase the amount of available information, but on the other hand it would complicate the analysis by introducing more visible organs.

Fig. 3. (a) Analysis of a regular pancreas; NN diagnosis: 84.6% of the pancreas length does not exhibit any pathological changes. (b) Pancreatic duct with cysts; NN diagnosis: 38.5% of the pancreas length exhibits cysts, 22.9% exhibits constrictions. (c) Regular pancreatic duct; NN diagnosis: 78.1% of the pancreas length does not exhibit pathological changes.

Table 1. Summary of data representations and network architectures tested (X)

|Data |Network structure | |NN | |

|representation |[type, inputs, structure] |1 |2 |3 |

|Log-Polar |Bp, (8x5), (100:100:2) |X | | |

| |Bp, (12x6), (100:100:8) |X | | |

|Retina |Bp, (68), (15:15:5) |X | | |

| |Bp, (130), (15:15:5) |X | | |

| |Bp, (68), (100:100:16) |X | | |

|Square window of |Bp, (31x31), (15:15:5) |X | | |

|analysis | | | | |

| |Bp, (31x31), (100:100:2) |X | | |

| |Bp, (31x31), (100:100:5) |X | | |

| |Bp, (31x31), (25:25:5) | |X | |

| |Bp, (31x31), (10:2) | | |X |

|Receptor fields |Bp, (var), (15:15:5) |X | | |

|Hybrid network |Kh, (31x31), (5:5) + Bp, (25), (15:15:5) |X | | |

| |Kh, (31x31), (5:5) + Bp, (25), (5:5:5) |X | | |

| |Kh, (31x31), (5:5) + Bp, (31x31+5), (15:15:5) |X | | |

Bp – backpropagation

Kh – Kohonen SOM

The pancreases of regular shapes are better suited for the analysis than the pathological ones (see Fig. 3). Particularly difficult cases are the ducts with cancer symptoms, because the constrictions in their sections are seen as discontinuities, what may lead to errors in the procedure of determination of motion direction.

For the diagnosing network the interference from another objects (e.g. the spine) has lead to erroneous detection of branching. However the network used for determination of the conditions of analysis termination correctly reported the fact of absence of the pancreatic duct on its ends or in the case of deviation from the proper route within the duct area. The errors were generated mainly on discontinuities, which the network was not able to distinguish from the situation when two various objects were present.

4 Conclusions

The obtained results prove that the analysis of raw images using neural networks is possible and effective. The diagnostic recognition level about 80% presents a very good result, if the low quality of the ERCP images and the presence of natural interference are taken into account. The result is comparable to the 85% recognition level reported in the paper [5], whereas it should be mentioned that in the cited work structural descriptions (grammars) have been applied to a preselected (binary) object.

The presented method can be also applied to the analysis of medical images containing objects of classes similar to the ERCP images, e.g. coronary vessels or eyeball vessels. Having this in mind refinement of the whole algorithm is planned, by introduction of feedback worked-out in the diagnosing network. The feedback's task would include a temporary reduction of the step length between consecutive attention shifts (at present the step length is constant) in order to provide a possibility of more detailed analysis by the recognising network in the cases which are difficult to diagnose. That would be a next step towards the functional similarity of the proposed algorithm to the movements of the human eyeball, executed during realisation of the diagnostic task, which have been registered in our Biocybernetic Laboratory.

References

1. van de Laar P., Heskes T., Gielen S. (1997) Task-Dependent Learning of Attention. Neural Networks vol.10 no 6, Pergamon

2. Lindblad T., Lindsey C., Padgett M., Vanyachine A., Zanevsky I. (1998) Digital X-Ray image processing using biologically inspired methods. Proc. IJCNN, Anchorage, Alaska.

3. Mikrut Z., Tadeusiewicz R. (2000) Neural Networks applied to digital image processing and recognition, in: Biocybernetics and Biomedical Engineering, Vol. 6, Ed. Nałęcz M. AOW EXIT, Warsaw (in Polish)

4. Neubauer C. (1998) Evaluation of Convolutional Neural Networks for Visual Recognition. IEEE Trans. Neural Networks, Vol. 9, No. 4

5. Ogiela M., Tadeusiewicz R. (2000) Syntactic Pattern Recognition for X-Ray Diagnosis of Pancreatic Duct. Eng. in Medicine and Biology, IEEE Magazine, 19, 6.

6. Postma E., Herik H., Hudson P. (1997) SCAN: A Scalable Model of Attentional Selection. Neural Networks vol.10 No. 6, Pergamon

7. Sjostrom P. J., Wahlberd L.U. (1999) Automated cell recognition and counting based on combination of artificial neural networks and standard image analysis methods. Proc. 3rd Int. Conf. Cognitive and Neural Systems, Boston

8. Żurada J. M. (1992) Introduction to artificial neural networks. West Publ., St. Paul New York.

-----------------------

[1] The work was partially supported by the Polish Committee for Scientific Research under grant No. T11E 007 17.

[2] Endoscopic Retrograde CholangioPancreatography

-----------------------

previous locations

[pic]

c)

b)

a)

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download