(1) Grading the degree of high myopia lesions. This part proposes a "multi-reorganization channel basic unit network" method based on the "reorganization channel network basic unit". Firstly, we experiment and analyze the performance of "reorganization channel network" and "reorganization channel network" reorganization channel basic unit; secondly, we borrow the idea of machine learning Bagging algorithm (similar to the Random Forest, Random Forest, RF for short), three basic units of "reorganization channel network" are combined as a "multi-reorganization channel basic unit network", in which each basic unit is an independent branch of the network; finally, according to the prediction results of each branch of the network, a voting process is carried out, and the category with the most votes is voted on. Finally, based on the prediction results of each network branch, the category with the most votes is the prediction category of the "multi-reorganization channel basic unit network".
(2) Development of fundus image optimization algorithms. In this section, an algorithm is proposed to optimize the factors affecting the fundus image based on the BGR information of the digital image. In this algorithm, firstly, by analyzing the grayscale luminance histograms of several groups of pathological high myopia fundus images with better and worse quality, the differences in parameters such as luminance, contrast, and color balance of different color channel images are analyzed; then, the luminance space transformation formula is proposed, which is applied to change the parameters such as luminance, contrast, and color balance of different color channel images in order to achieve the effect of image optimization.
(3) Segmentation of pathological high myopia lesion region. Considering that it is often not enough to predict the category of severity of the obtained high myopia lesion, it is more important to be able to accurately segment the relevant lesion region, for this reason, the segmentation method of Vnet-FPN network is proposed in this part. First, the architecture of Vnet network and FPN network is studied. Then, the Vnet-FPN network is designed; finally, experiments are conducted on two data sets optimized by the original and image optimization algorithms to comprehensively analyze the network's segmentation performance on the fundus lesion region.
? The research content of this paper can assist ophthalmologists in the diagnosis and treatment of high myopia lesions, which is very important for the engineering landing of the combination of deep learning and medical technology.
? Currently, fundus examination is mainly performed manually by doctors using ophthalmoscopy, fundus fluorescence imaging, fundus camera imaging and optical coherence layer imaging. The color photo image of the fundus captured using the fundus camera will present the main tissue structures in the retina, as shown in Figure 1-1. As shown in Figure 1-1, blood vessels (Vessel) are the most widely distributed in the retina and appear as a dark red mesh structure, which together with the visual nerve fibers enter the retina from the Optic Disc region. As can be seen in the image, the Optic Disc usually appears as a disc-like structure with clearer borders and higher brightness. In addition, the darker area in the middle of the fundus image is called the macula lutea, which is an oval-shaped depression with a center called the fovea. The center of the depression is called the Fovea. The Fovea is the most sensitive area of the human eye's vision, and once the area is diseased, vision will be seriously affected.
? In view of the paucity of research work on this problem of classifying high myopia, for this purpose, we mainly draw on the research methodology of classifying the severity of diabetic retinopathy (DR). There are two main categorization criteria in the study of classification of DR, which are dichotomous and trichotomous.
? In the binary classification problem, researchers classified color fundus images into two categories: non-DR and DR.Gardner et al[4] used pixel intensity values of images as features and classified them by training neural networks.Roychowdhury[5] et al. proposed a hierarchical classification method containing two steps, which integrates the Gaussian Mixture Model (GMM), k-Nearest Neighbor (kNN), Support Vector Machine (SVM), and AdaBoost machine learning algorithms to train the classifier.Priya [6] et al. first extracted several biological damage features such as blood vessels, fundus hemorrhage, and exudates. Priya [6] et al. first extracted the features of blood vessels, fundus hemorrhage and exudates, and then input these features into three classification models for training, namely Probabilistic Neural Network (PNN), Bayesian Classifier and Support vector machine (SVM). The researchers have also used the PNN, Bayesian Classifier and Support vector machine (SVM). In the triple classification problem, researchers usually classify color fundus images into three categories: non-DR, non-proliferative DR, and proliferative DR.Nayak [7] et al. have developed a method by extracting the area of exudates as well as the area of blood vessels and texture features, and then using these features to train the neural network.
? From the above research methods, it can be seen that most of the current domestic and international research methods on disease classification of fundus color photography images are based on extracting visual features (such as the pixel value, texture, and contrast of the image) at the lower layers of the image, or features of localized biological damage or blood vessels related to diabetic retinopathy such as hemorrhages and exudates in the fundus, and then classifying the images using different machine learning algorithms. The performance of these methods is largely determined by the effectiveness of the hand-designed features. Compared to binary classification, the multiclassification problem is more complex as the classification criteria are more refined, and the classification of certain classes of retinal lesions often requires the consideration of very subtle features.
(1) Leopard's foot in high myopia
? In general, fundus images with high myopia have a relatively distinct tessellated fundus. The tessellated fundus is mainly due to the prolongation of the eye axis caused by high myopia, the retinal blood vessels leave the optic disk, that is, thinning and straightening, coupled with changes in the choroidal capillaries, resulting in nutritional deficiencies in the retinal pigment epithelium, resulting in the reduction or disappearance of the superficial pigmentation. The brown pigment between the exposed choroidal red blood vessels and the vascular network is thus interspersed to form the so-called leopard-shaped fundus, and the figure clearly shows the leopard-shaped fundus in high myopia. Shi Yining [20] et al. studied 154 people over 41 years of age with high myopia and found that non-pathologic changes accounted for 45.5% of the cases, of which leopard-shaped fundus accounted for 33.8%. And it can increase to 90%-100% when the eye axis is significantly prolonged and the refractive error is higher. This is especially significant in the posterior pole, and the leopard-shaped fundus becomes more pronounced as the refractive error of myopia progresses.
? Because the leopard-shaped fundus has a certain relationship with age, that is, the older the leopard-shaped fundus is more significant. For this reason, this chapter defines a highly myopic panuveticular fundus as one in which the following two conditions are met:
? 1) Significant leopard-like changes are seen near the macula (area of leopard-like fundus in the macula>1/2)
? (2) Obvious leopard-like changes can be seen at the vascular arch (vascular arch leopard-like fundus area>1/2)
(2) Characteristics of lesions in pathologic high myopia
? High myopia is a common and highly prevalent type of fundus lesion, which leads to many lesions, such as myopic maculopathy, leopard-shaped fundus, diffuse retinal choroidal atrophy, plaque retinal choroidal atrophy, macular atrophy, lacunar crackles, choroidal neovascularization, Fuch's spots, posterior scleral staphyloma, and other pathologic lesions. In this paper, we focus on the more common plaque retinal choroidal atrophy and macular atrophy; other lesions were not studied. We focus on these two types of lesions briefly as follows:
[if !supportLists](1)? [endif]Patchy choroidal retinal atrophy. A well-defined grayish-white lesion around the macula or optic disc; myopic foxing alone without a leopard-like fundus is not considered to do this type of lesion. As shown in Figure 2-3 below:
(2) Macular atrophy. Macular atrophy is a well-defined, rounded, choroidal retinal atrophic lesion that is gray or white in color, appears surrounded by a degenerated fibrovascular membrane, and increases in size over time. Macular atrophy must be distinguished from patchy retinal atrophy, but patients typically have both. Typically, macular atrophy is centered in the central sulcus (not the geometric center of the atrophic area; in fact, any atrophy involving the central sulcus is considered to be atrophic) and is circular, whereas patchy retinal atrophy is not centered in the central sulcus and has irregular edges. Figure 2-4 shows a fundus image of macular atrophy with patchy choroidal retinal atrophy.
? We evaluated the classification performance of the multi-reorganization channel basic unit network proposed in this chapter using a dataset labeled with high myopia lesions provided by Baidu Competitor, and compared the classification performance with the "reorganization channel network" and the "reorganization channel network" of the reorganization channel basic unit. The classification performance of the two networks is compared with the "Recombination Channel Network" and the "Recombination Channel Network" of recombination channel basic units. The dataset*** contains more than 1009 color fundus images. We used stratified sampling by category to divide the dataset into the 800-image training set, 100-image validation set, and 109-image test set that we used in our experiments.
? In the next part of this section, we will discuss the four aspects of fundus image data preprocessing and data augmentation, network implementation and parameter settings, and experimental results and analysis in turn.
(1) Data preprocessing and data augmentation
1) Data preprocessing
? Based on the training characteristics of deep convolutional neural networks, if we do not deal with some of the problems in the dataset, it will seriously affect the performance of the trained network, for this reason, we have preprocessed the dataset as shown in Fig.
(1) Intercept the region of interest of fundus image. Since the dataset was collected by ophthalmologists in different hospitals in different environments and with different equipment, there are inconsistent image sizes and a large black border background region as shown in Figure a), and these regions do not help the classification of the images, and even seriously reduce the classification performance of the network, so we remove the excess background region and retain the fundus region, and the result is shown in Fig. b). Since the image is not distributed in rectangular pixels, according to the requirements of the classification network input structure, to prevent the image from being deformed after the Resize operation that changes the image size by the network input, we expand the black edge of the non-rectangular fundus image to solve the problem of the loss of image information.
(2) Fundus image resizing. The image size of the original dataset is around 1444*1444, which is too large for the network, and thus according to the experience of deep learning algorithms at this stage on the impact of image size on the performance of classification networks, we finally adjusted the size of the image size to 256*256 during the training of the network, mainly for the following two reasons: firstly, the image size is too small and easy to lead to the loss of feature information between different categories of the image, which is not favorable to the image information loss. First, too small an image size can easily lead to the loss of feature information between different categories of the image, which is not conducive to the network's feature learning for regions with classification features; second, considering the GPU memory and processing power used in this paper, it is not conducive to the training of deep convolutional neural networks if the image size is too large.
2) Data augmentation
Compared with the classification problem of natural scene images, the data set used in this chapter to classify high myopia lesions is obviously smaller in size, and in order to avoid the phenomenon of over-fitting in the training network, we use the following aspects of the training process of the online data augmentation (Online Data Augmentation methods:
? (1) Flipping. Randomly flip the fundus color photo images horizontally, vertically, and horizontally and vertically.
? (2) Contrast and brightness adjustment (Contrast Adjustment). In this paper, the main use of per-pixel party to change the image contrast and brightness, the formula is as follows:
? g(x)=αf(x)+β
? where α regulates the contrast and β regulates the brightness. In this paper, the classification sets α and β as random numbers selected in the random 0.8-1.0 interval
.
(3) Rotation. Rotation is performed by randomly selecting an offset angle between 0° and 15°.
(4) Cropping. Randomly select an offset (Offset), for example, the original image size 1024 * 1024, the offset value range of [0, 128], then the 1444 *]1444 image cut to 1316 * 1316.
? Practice classical classification networks Resnet, Densenet, ShuffleNetV2 and other classification networks.
? In the previous chapters, automatic classification of high myopia lesion severity and optimization algorithms for fundus color images were described in detail. In the classification of high myopia lesions, we used a method based on Bagging idea of multiple recombination network; in the fundus image optimization algorithm, we proposed a fundus image optimization algorithm based on image space transformation method.
? In the actual computer-aided diagnosis (Computer-Aided Diagnosis) system of high myopia disease, only predicting the severity of high myopia lesions, the auxiliary diagnostic information provided to ophthalmologists is obviously not enough. For this reason, after obtaining fundus images of pathologic high myopia, the lesion area needs to be segmented again for doctors to further confirm the patient's condition, which will largely increase the efficiency and accuracy of high myopia diagnosis and provide a feasible measure for large-scale screening of high myopia lesions. Therefore, in this chapter, the segmentation of high myopia lesion regions under multi-scale fusion conditions is carried out. Specifically, with the knowledge of the severity of high myopia lesions in the fundus images, the optimization algorithms are used to process the fundus images of pathologic high myopia, and then the methods in this chapter are used to automatically segment the lesion regions therein.
As mentioned earlier, the accurate segmentation of the lesion area of high myopia is of great significance for the diagnosis and treatment of patients with high myopia. According to the current mainstream methods for lesion segmentation in fundus color photography images, the several steps are as follows: fundus image preprocessing, extraction of candidate sets of high myopia lesion regions, lesion feature extraction and classification, and fundus image post-processing. These methods use manually pre-constructed features in combination with several more common Machine-Learning Algorithms, such as Support Vector Machine (SVM), k-means clustering algorithm (K -Means) and other methods for segmentation of the lesion region. In this chapter, the segmentation method of deep learning is used to reduce the difficulty of data preprocessing, but since deep convolutional neural networks require more image data, we have performed a certain degree of data augmentation as a way of augmenting the dataset.
? After the classification of the severity of high myopia lesions in Chapter 2, we need to evaluate and screen the image quality for pathologic high myopia, i.e., manually screen and filter the poor quality images, and ultimately a **** to get about 350 normal quality color images of the fundus of pathologic high myopia as a dataset. This step is mainly due to a series of problems such as underexposure due to focusing failure and overexposure due to changes in ambient lighting that may occur during the acquisition of funduscopic color photographs, most of which do not meet normal clinical requirements, as shown in Fig.
For the above Vnet network and Vnet-FPN network are trained from scratch based on the same initialization method. The dataset of this chapter contains 400 images, of which 300 are for the training set, 300 for the test set, 100 for the validation set and 100 for the test set. In this chapter, the review metrics Dice, TPR, and PPV, which are often used when evaluating the performance of medical image processing algorithms, were chosen as the review metrics for the network, which are actually counting the proportion of the portion of the region predicted by its segmentation method that overlaps with the region of the gold standard (ground truth).
? Dice is a similarity index, which is the Overlap Area between the results of the segmentation algorithm and the results of the gold standard divided by an average of their two areas:
? The dataset of pathologic high myopia lesions used in this paper has a relatively small data size. Therefore, in order to avoid Over-fitting during network training, we mainly preprocess the dataset, i.e., we practice some of the following data augmentation methods in real time during network training:
? (1) Flipping. Experimental experience shows that the fundus color photo image horizontal flipping is more effective than vertical flipping, so this paper adopts the horizontal flipping data augmentation method.
? (2) Image brightness adjustment (Contrast Adjustment). In this paper, the main use of pixel-by-pixel party to change the brightness of the image, according to the randomly generated brightness value superimposed on the original fundus image.
? (3) Rotation and Scale. Randomly select an angle between 5° and 15° to perform a rotation operation on the image and scale the coordinates of the fundus color image, which is equivalent to scaling the image.
? (4) Gaussian filter. Gaussian filter is a linear smoothing filter, suitable for eliminating Gaussian noise, widely used in image processing noise reduction process. In common parlance, Gaussian filtering is the process of weighted averaging the entire image, and the value of each pixel is obtained by itself and other pixel values in the neighborhood after weighted averaging.
The label visualization, Vnet segmentation results and Vnet-FPN segmentation results are shown in the middle left and right, respectively.