? (1) Classification of pathological changes in high myopia. In this part, firstly, the method of "multi-reconfiguration channel basic unit network" based on "reconfiguration channel network basic unit" is proposed, and the experiments of "reconfiguration channel network" and "reconfiguration channel network" are carried out, and their performance is analyzed. Secondly, referring to the idea of machine learning Bagging algorithm (similar to random forest, abbreviated as RF), the three basic units of "reorganization channel network" are combined into "multi-reorganization channel basic unit network", in which each basic unit is regarded as an independent network branch; Finally, according to the forecast results of each network branch, the category with more votes is the forecast category of "multi-reorganization channel basic unit network".
? (2) The development of fundus image optimization algorithm. In this part, an algorithm based on BGR information of digital image is proposed to optimize the relevant influencing factors of fundus image. In this algorithm, firstly, by analyzing the gray brightness histograms of several groups of pathological high myopia fundus images with good quality and poor quality, the differences of brightness, contrast, color balance and other parameters of different color channel images are analyzed. Then, the formula of brightness space transformation is put forward, and the parameters such as brightness, contrast and color balance of images with different color channels are changed by this formula to achieve the effect of image optimization.
? (3) Segmentation of pathological high myopia focus area. Considering that it is often not enough to predict the severity of high myopia lesions, it is more important to accurately segment the related lesion areas. Therefore, this part proposes a virtual network FPN network segmentation method. Firstly, the architecture of virtual network and FPN network is studied. Then, design the virtual network FPN network; Finally, experiments are carried out on the original data set and several data sets optimized by the image optimization algorithm to comprehensively analyze the segmentation performance of the network for fundus lesions.
? The research content of this paper can assist ophthalmologists in the diagnosis and treatment of high myopia, and is of great significance to the project of combining deep learning with medical technology.
At present, the main methods of fundus examination are ophthalmoscope, fundus fluorescein angiography, fundus camera imaging and optical coherence tomography. The color image of the fundus collected by the fundus camera will show the main tissue structure on the retina, as shown in figure 1- 1. As can be seen from the figure 1- 1, in the fundus color photographic image, blood vessels are most widely distributed in the retina, showing a dark red reticular structure, and they and visual nerve fibers enter the retina from the optic disc area. As can be seen from the figure, the optic disc is usually disc-shaped with clear boundaries and high brightness. In addition, the dark area in the middle of the fundus image is called macula, which is oval, and the center of the depression is called fovea. Fovea retinae is the most sensitive area of human vision, and once the disease occurs in this area, vision will be seriously affected.
In view of the lack of research on the classification of high myopia at present, this paper mainly draws lessons from the research methods of severity classification of diabetic retinopathy (DR). In the research of DR classification, there are two main classification standards, namely, secondary classification standard and tertiary classification standard.
On the issue of binary classification, researchers divide color fundus images into two categories: non-DR and DR. Gardner et al. [4] use the pixel intensity value of an image as a feature and train a neural network to classify. Roychowdhury[5] and others put forward a two-step hierarchical classification method, which comprehensively uses four machine learning algorithms: Gaussian mixture model (GMM), k- nearest neighbor classifier (kNN), support vector machine (SVM) and AdaBoost to train the classifier. Priya[6] and others first extract the features of blood vessels, fundus hemorrhage and exudate, and then input these features into three classification models for training, namely probabilistic neural network (PNN), Bayesian classifier and support vector machine (SVM). In the three-classification problem, researchers usually divide color fundus images into three categories: non-DR, non-proliferative DR and proliferative DR Nayak[7] and others extract the area of exudate and the area and texture features of blood vessels, and then use these features to train neural networks.
From the above research methods, it can be seen that the research methods of fundus color image disease classification at home and abroad are mostly to extract the underlying visual features of the image (such as pixel value, texture, contrast, etc.). ), or fundus hemorrhage, exudation and other local features such as biological damage or blood vessels related to diabetic retinopathy, and then use different machine learning algorithms to classify images. The performance of these methods depends largely on the effectiveness of hand-designed features. Compared with binary classification, the classification standard of multi-classification problem is more detailed, and the classification of some types of retinopathy often needs to consider very subtle features, so the multi-classification problem is more complicated.
(1) Leopard fundus in high myopia
? Generally speaking, the fundus images with high myopia have relatively obvious checkerboard fundus. Leopard-print fundus is mainly caused by high myopia, which makes the axial length longer, and the retinal blood vessels become thinner and straighter after leaving the optic disc. In addition, the changes of choroidal capillaries cause nutritional disorders of retinal pigment epithelium, leading to the decrease or disappearance of surface pigments. In this way, the brown pigments between the choroidal red blood vessels and the vascular network are intertwined, forming a so-called leopard-print fundus, which clearly presents a leopard-print fundus with high myopia. Shi Yining [20] and others studied 154 people with high myopia over 4 1 year old, and found that non-pathological changes accounted for 45.5%, of which leopard-print fundus accounted for 33.8%. However, when the axial length is obviously prolonged and the diopter is high, it can be increased to 90%- 100%. Especially in the posterior pole, with the gradual deepening of myopia diopter, the leopard-print fundus is more obvious.
? Because the leopard-print fundus has a certain relationship with age, that is, the older the age, the more obvious the leopard-print fundus is. Therefore, the definition of leopard-print fundus of high myopia in this chapter must meet the following two conditions at the same time:
? 1) obvious leopard print changes can be seen near the macula (leopard print fundus area in macula >: 1/2)
? 2) Obvious leopard pattern changes can be seen at the vascular arch (leopard pattern base area of vascular arch >; 1/2)
(2) Pathological features of pathological high myopia
High myopia is a common and high-incidence fundus disease variant, which can lead to many diseases, such as myopic macular degeneration, leopard-print fundus, diffuse retinal choroidal atrophy, patchy retinal choroidal atrophy, macular atrophy, lacquered striation, choroidal neovascularization, fuchs's spot, posterior scleral staphyloma and so on. This paper mainly studies the common patchy retinal choroidal atrophy and macular atrophy, and other diseases have not been studied. We mainly briefly introduce these two types of lesions, as follows:
[If! Support list ]( 1)[endif] patchy chorioretinal atrophy. Gray-white lesions with clear boundaries around the macula or optic disc, only myopic foxes without leopard lines at the fundus will not consider such lesions. As shown in Figure 2-3 below:
? (2) Macular atrophy. Macular atrophy is a well-defined round choroidal and retinal atrophy focus, gray or white, surrounded by degenerated fibrovascular membranes, and it gets worse with time. Macular atrophy must be distinguished from macular retinal atrophy, but patients generally have both lesions. Usually, macular atrophy is a circle centered on the fovea (not the geometric center of the atrophic area, in fact, as long as atrophy involves the fovea), while macular retinal atrophy is not centered on the fovea and has irregular edges. Figure 2-4 shows the fundus images of macular atrophy and patchy choroidal retinal atrophy.
We use the high myopia lesion category label data set provided by Baidu competitors to evaluate the classification performance of the multi-recombination channel basic unit network proposed in this chapter, and compare it with the classification performance of the recombination channel basic unit network and the recombination channel network. This data set * * * contains more than 1009 color fundus images. We divided the data set into 800 training sets, 100 verification sets and 109 test sets by stratified sampling.
In the next part of this section, we will discuss fundus image data preprocessing and data enhancement, network implementation and parameter setting, as well as experimental results and analysis in turn.
(1) data preprocessing and data expansion
1) data preprocessing
Based on the training characteristics of deep convolutional neural network, if some problems in the data set are not dealt with, the performance of the trained network will be seriously affected, so we preprocess the data set as shown in the figure.
(1) Intercept the region of interest in the fundus image. Because the data sets were collected by ophthalmologists in different hospitals in different environments and different equipment, there are black background areas with different image sizes, as shown in figure A), which are not helpful to the classification of images and even seriously reduce the classification performance of the network, so we removed the redundant background areas and reserved the fundus areas, and the result is shown in figure B). Because the image is not a rectangular pixel distribution, according to the input structure requirements of the classification network, in order to prevent the image from being deformed after the adjustment operation of changing the image size through the network input, we extend the black edge of the non-rectangular fundus image to solve the problem of image information loss.
? (2) Adjustment of fundus image size. The image size of the original data set is about 1444* 1444, which is too large for the network. Therefore, according to the experience of deep learning algorithm on the influence of image size on the performance of classification network, we finally adjust the image size to 256*256 in the training process of the network, mainly for the following two reasons: First, the image size is too small, which easily leads to the loss of feature information between different types of images. Secondly, considering the memory and processing power of GPU, if the image size is too large, it is not conducive to the training of deep convolution neural network.
2) data amplification
Compared with the classification of natural scene images, the data set used in this chapter for the classification of high myopia lesions is obviously smaller. In order to avoid over-fitting in the training network, we adopted the following online data expansion methods in the training process:
? (1) Flip. Randomly flip the fundus color image horizontally, vertically, horizontally and vertically.
? (2) Contrast adjustment. In this paper, the square of the pixel is used to change the contrast and brightness of the image, and the formula is as follows:
g(x)=αf(x)+β
? Where α adjusts contrast and β adjusts brightness. In this paper, α and β are set in the random range of 0.8- 1.0.
Selected random number.
(3) rotation. Randomly select an offset angle from 0 to 15 to rotate the image.
(4) cutting. Randomly select an offset, for example, the original image size is 1024* 1024, and the offset range is [0, 128], and then cut it into an image of 1444 *] 1364.
Practice classic classification networks such as Resnet, Densenet and ShuffleNetV2.
In the previous chapters, the automatic classification of the severity of high myopia lesions and the optimization algorithm of fundus color images are introduced in detail. In the classification of high myopia lesions, we adopt the multiple recombination network method based on Bagging thought; In the fundus image optimization algorithm, a fundus image optimization algorithm based on image space transformation method is proposed.
In the actual computer-aided diagnosis system for high myopia, only the severity of high myopia is predicted, and the auxiliary diagnosis information provided by ophthalmologists is obviously insufficient. Therefore, after obtaining the fundus image of pathological high myopia, it is necessary to segment the lesion area so that doctors can further confirm the patient's condition, which will greatly increase the efficiency and accuracy of high myopia diagnosis and provide feasible measures for large-scale screening of high myopia lesions. Therefore, this chapter has carried out the segmentation of high myopia lesions under the condition of multi-scale fusion. Specifically, knowing the severity of high myopia in the fundus image, the optimized algorithm is used to process the pathological high myopia fundus image, and then the pathological region is automatically segmented by the method in this chapter.
? As mentioned above, the accurate segmentation of high myopia lesion area is of great significance for the diagnosis and treatment of high myopia patients. According to the current mainstream fundus color image focus segmentation method, the steps are as follows: fundus image preprocessing, high myopia focus candidate set extraction, focus feature extraction and classification, fundus image post-processing. These methods combine pre-built features with several common machine learning algorithms, such as support vector machine (SVM) and K-means clustering algorithm (K-means), to segment the focus region. This chapter adopts the segmentation method of deep learning, which reduces the difficulty of data preprocessing. However, because the deep convolutional neural network needs more image data, we extend the data to a certain extent and enlarge the data set.
? After the classification of the severity of high myopia in the second chapter, we need to evaluate and screen the image quality of pathological high myopia, that is, manually screen and filter the images with poor quality, and finally get about 350 fundus color photos with normal quality as a data set. This step is mainly due to a series of problems such as underexposure caused by focusing failure and overexposure caused by environmental illumination changes when collecting fundus color images. These images are too large to meet the normal clinical requirements, as shown in the figure.
For the above Vnet network and Vnet-FPN network, they are trained from scratch on the basis of the same initialization method. The data set in this chapter contains 400 images, including 300 training sets and 300 test sets, 100 verification sets and 100 test sets. In this chapter, Dice, TPR and PPV, which are commonly used to evaluate the performance of medical image processing algorithms, are selected as the evaluation indexes of the network, which are actually the proportion of the overlapping parts between the predicted area and the real area on the ground.
Dice is the similarity index, that is, the overlapping area between the results of the segmentation algorithm and the results of the gold standard divided by the average of their two areas:
? The pathological high myopia lesion data set used in this paper is relatively small. Therefore, in network training, in order to avoid over-fitting, we mainly preprocess the data set, that is, practice the following data augmentation methods in real time in network training:
? (1) Flip. Experimental experience shows that horizontal inversion of fundus color images is more effective than vertical inversion, so this paper adopts horizontal inversion data amplification method.
? (2) Contrast adjustment of image brightness. In this paper, the brightness of the image is changed by the square of pixels, and it is superimposed on the original fundus image according to the randomly generated brightness value.
? (3) Rotation and proportional transformation. Randomly select an angle between 5 and15 to rotate the image and scale the coordinates of the fundus color image, which is equivalent to scaling the image.
? (4) Gaussian filter. Gaussian filter is a linear smoothing filter, which is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of image processing. Generally speaking, Gaussian filtering is a process of weighted average of the whole image, and the value of each pixel is obtained by weighted average of itself and other pixels in the neighborhood.
The left, middle and right are label visualization, Vnet segmentation result and Vnet-FPN segmentation result respectively.