Included under terms of UK Non-commercial Government License.
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Olson J, Sharp P, Goatman K, et al. Improving the economic value of photographic screening for optical coherence tomography-detectable macular oedema: a prospective, multicentre, UK study. Southampton (UK): NIHR Journals Library; 2013 Nov. (Health Technology Assessment, No. 17.51.)
Improving the economic value of photographic screening for optical coherence tomography-detectable macular oedema: a prospective, multicentre, UK study.
Show detailsIntroduction
This chapter addresses the first aim of the project, namely to investigate whether or not particular distributions and combinations of lesions (M/DHs, BHs and exudates), assessed manually or automatically, are more specific surrogate markers of MO than current grading practice, using OCT as the reference standard.
Initially statistical modelling was carried out to see if any of the current manual photographic screening strategies included everything that might be considered important. In the second part of this chapter this information is used to inform the inclusion of various eye and subject characteristics within computer-assisted manual photographic screening strategies and full automated strategies for detecting MO.
National screening programmes in the UK all agree that exudates within one DD of the centre of the fovea should be used to infer the presence of referable MO. However, there is disagreement as to how BHs or M/DHs within one DD of the centre of the fovea should be used. The value of BHs or M/DHs in relation to MO is therefore of particular interest.
The various lesions and lesion combinations within a single eye are investigated in relation to the presence of MO in that eye. Counts of the three main types of lesions are investigated in a similar way. The findings in Chapter 4, and from the analyses of single eyes, were used to guide the analyses predicting MO (in either eye) from information on the subject and the two separate eyes.
As noted in Chapter 4, the combinations of lesion types and better and worse visual acuity did not occur in this sample in the same proportions as expected from a cohort of all subjects attending retinal screening. More complex analyses were weighted so that the results might better reflect what is expected in such a cohort.
Methods for statistical modelling
Combinations of lesions (M/DHs, BHs and exudates) were considered in relation to MO. Subjects were classified into mutually exclusive categories according to their most serious feature in the relevant eye. The five categories were:
- exudates present within one DD (regardless of other lesions)
- BHs present within one DD (but not exudates)
- M/DHs present within one DD (but not exudates or BHs)
- other minor lesions (no M/DHs, BHs or exudates present) within one DD
- no lesions present within one DD.
These lesion combinations were also compared across centres by eye. Visual acuity was considered as a three category variable: better visual acuity (log-MAR < 0.3), worse visual acuity (log-MAR ≥ 0.3) and visual acuity missing.
Single eye analyses
In order to investigate whether or not particular distributions and combinations of lesions identified by manual screening (M/DHs, BHs and exudates) were valuable markers of MO, single eyes were considered separately. Chi-squared tests were used to identify potential risk markers and confounding factors. Logistic regression analyses were used on data from one eye only to identify features identified by manual screening that might be important for predicting the presence of MO within the relevant eye. Different representations of features including presence, count and distribution were considered.
Initially the presence of features within one DD (and for exudates within one to two DD) was investigated in relation to MO within the same eye. In these initial analyses, only single features were considered, the other features being ignored. Following that, analysis combinations of features were considered. The presence of exudates between one and two DD was considered as a separate variable. These logistic regression analyses considered the lesion variable alone (unadjusted), after adjusting for centre, and after adjusting for centre, demographic variables and visual acuity in that eye.
Counts of the different types of lesions (exudates, BHs and M/DHs) within one DD (and for exudates within one to two DD) were first considered singly and then in combination in logistic regression analyses for MO in the relevant eye. These analyses were also adjusted first for centre, and then for demographic variables and visual acuity in that eye. The ranges of counts of lesions were not the same for different types of lesions and there was concern that results of analyses of counts could be dominated by high counts. As a sensitivity analysis, counts of lesions were collapsed into zero, one, two, three, four, five and more than five, and this count variable was included in the logistic regression analysis as if it was a discrete count. This was to ensure that any relationships observed were robust to the influence of very high counts.
There is particular interest in the contribution of M/DHs to the prediction of MO as M/DHs are included in current manual grading practice in England, but not in Scotland. In order to investigate whether or not larger numbers of M/DHs were useful in predicting MO, the counts of all three types of lesions were collapsed even further so they could be included in the logistic regression models as ordered categories: zero, one, two and more than two. The opportunity to observe non-linear increases in the odds of MO with increasing numbers of lesions was provided by this alternative representation of counts of lesions as four ordered categories.
As noted earlier, the combinations of lesion types and better and worse visual acuity did not occur in the proportions expected from a cohort of subjects attending retinal screening and so data weighting was necessary. Weighting of sample data is common in sample surveys either where the sampling scheme uses unequal probabilities by design or if the data are known to be unrepresentative of the population,43 often due to disproportionate non-response.44 If the sample is known to be substantially different to the population in the distribution of one or more key variables then an analysis of the raw data can produce biased estimates of prevalence. Reweighting can correct for bias in the estimates, but may have the effect of increasing the variances and complicating the analysis.43 In the current study, if a statistic, such as the sensitivity of a diagnostic algorithm, differs between types of subjects, then the estimate of sensitivity based on the raw sample data could be biased towards the sensitivity within subjects who are over-represented and away from the sensitivity within those under-represented. The under-representation of subjects with only M/DHs was of particular concern. The simplest form of weight, sometimes called direct weights or post-stratification weights, was used.44–46
The weighting of subjects was performed as follows. The proportions of subjects falling into the five categories were calculated for the study sample, the expected proportions in the five categories being known from a previous study.30 Weights were obtained by dividing the proportions of subjects in the study sample by the proportions in the population and then multiplying by a factor of 3170/2845 to account for the zero weighting of 325 subjects with either no lesions or only very mild lesions in either eye. Where appropriate, the analyses described above were repeated after weighting. Ideally weighting would have been done to correspond to the population proportions of the five groups within each centre, but these were not known at each centre. This weighting was also used in the health economics analyses in Chapter 6.
Both eyes analysis
Information on features from both eyes was combined in order to look at the prediction of presence of MO (in one or both eyes) per subject. The most serious feature in either eye was given priority in determining the classification of the subject by their lesions. The mutually exclusive categories chosen were:
- any exudates in either eye
- any BHs (but no exudates) in either eye
- more than two M/DHs in either eye (but no exudates or BHs)
- exactly two M/DHs in the eye with most M/DHs (but no exudates or BHs)
- exactly one M/DH in the eye with most M/DHs (but no exudates or BHs)
- those with no M/DHs in either eye and those subjects with lesions which were not M/DHs, BHs or exudates in either eye.
Poor vision was classified as worse (if visual acuity was log-MAR ≥ 0.3 in either eye or if visual acuity was missing in one eye and log-MAR ≥ 0.3 in the other), better (if visual acuity was log-MAR < 0.3 in both eyes) or missing (if visual acuity was missing in one eye and better in the other or missing in both eyes).
Logistic regression was used to investigate this representation of the lesions in relation to predicting the presence of MO in at least one eye. The analysis was repeated first for centre and visual acuity, and then for centre, visual acuity and demographic factors. The analyses of both eyes together were repeated after weighting.
Results of statistical modelling
Presence of individual lesion types in relation to macular oedema
In Table 10 the presence or absence of a particular type of lesion in the relevant eye is considered. No adjustments were made for other lesions that might also have been present. A large difference between the percentages of subjects with MO and without MO and with a certain type of lesion present may be due to the presence of another type of lesion.
Macular oedema was roughly five times more prevalent among subjects with an exudate, a BH or a M/DH present within one DD compared with those with the same lesion absent (see Table 10). The prevalence of MO was also greater, albeit by a lesser amount, for subjects having an exudate within one to two DD, but no consideration was given whether or not there was an exudate within one DD as well. Too few eyes had CWS for a robust analysis. There were few eyes with flame haemorrhages or drusen so although there were sufficient eyes for these comparisons to be made, multivariate analyses were not considered appropriate. There did not appear to be any evidence of a relationship between the presence of drusen and the presence of MO in the relevant eye.
Subjects with worse visual acuity were about five times as likely to have MO in the relevant eye as those with better visual acuity.
Only a small number of subjects had amblyopia and, of these, very few had MO. There were no significant differences in the percentages with MO between those using and not using glitazone. In later analyses adjustment was made for glitazone use, but this was not possible for amblyopia because of the small numbers of subjects.
When mutually exclusive groups of lesions were considered, highly significant differences in the percentages with MO were found. More than 10% of subjects with exudates or with BHs (but not exudates) had MO in the same eye. The percentage with MO was just over 2% of those with just M/DHs present in the same eye, about 1% of those with lesions excluding M/DHs, BHs and exudates and < 1% with no lesions in that eye.
Visual acuity was not available for 15 left eyes, two of which had MO, and for 15 right eyes, four of which had MO.
Logistic regression analyses of single eyes
Most of the following tables in this section are for the left eye only. Analyses of the right eyes were also completed but since they showed almost identical relationships these results have not been presented. Table 11 shows the results of logistic regression analysis of a single eye with the outcome variable of MO in that eye. The odds ratio (OR) shows the increase in the odds of having MO in a subject with a characteristic compared with a subject without that characteristic. For a measured variable, such as age, the OR represents the change in the odds of having MO if that variable were to increase by one unit of measurement (1 year older) relative to the same subject without that increase. If multiple variables are included in the same logistic regression model then the OR for a single variable represents the increase in the odds of having MO if that variable were to increase by one unit of measurement, or to change to a different category, compared with a subject with all other characteristics and measurements held fixed.
In Table 11, the top category for each of the categorical variables is chosen as the reference group with OR shown as 1 and no CI displayed. The choice of the reference group is arbitrary and this does not affect statistical significance. The other ORs presented are relative to this reference group and a 95% CI is displayed in the next two columns. If this 95% CI includes 1 it shows that the OR is not significantly different to 1 (the null value) and so p > 0.05. If the 95% CI excludes 1 it indicates that this measured variable is significant for MO with p < 0.05 or that this categorical variable is significant with p < 0.05 and the relevant category is significantly different to the reference category in terms of the odds of having MO.
The first three columns of numbers show ORs and the lower and upper 95% confidence limits for each variable considered separately (unadjusted). The fourth to sixth columns show the ORs and 95% CIs for the variables adjusted for differences between centres. Ideally simultaneous adjustments would be made for centre and scanner type, but they are highly confounded with each other and so could not be included together in the same model. The feature results were similar regardless of whether they were adjusted for centre or scanner so adjustment has only been made for centre. The seventh to the ninth columns show the ORs and 95% CIs for the variables adjusted for centre, gender, age, glitazone and diabetes (type 1, type 2 and secondary/unknown). The last three columns show the ORs and 95% CIs for the variables adjusted for centre, gender, age, glitazone, diabetes (type 1, type 2 and secondary/unknown) and visual acuity.
When the mutually exclusive groups of lesions were included in the model, subjects with BHs (either on their own or BH with M/DHs) had 5.0 times the odds of having MO and subjects with exudates had 6.4 times the odds of having MO, both compared with those with only M/DHs. These ORs changed by only modest amounts as adjustments were made for centre and other characteristics to ORs of 3.6 and 6.7. Of these other variables, only centre, age and visual acuity were significant with older age being associated with higher odds of MO by 4% per additional year older (OR = 1.037; 95% CI 1.020 to 1.054). When visual acuity was included, having worse visual acuity significantly increased the odds of MO by a factor of 3.9 (95% CI 2.6 to 5.9).
An analysis of counts of lesions found that higher numbers of M/DHs, BHs and exudates within one DD significantly increased the odds of having MO when considered separately or together. The relationships were maintained after adjustment for centre, gender, age, glitazone, diabetes (type 1, type 2 and secondary/unknown) and visual acuity. Initially, counts of exudates between one to two DD appeared to be associated with greater odds of MO, but as this ceased to be significant after adjusting for exudates within one DD so it was dropped from subsequent models. It is important to note that the count of M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs and the count of exudates. This suggested that having a lot of M/DHs was still useful for predicting MO.
The OR for counts of BHs was larger than for the other lesions. This does not mean BHs were more important, but that the increase in the odds of MO for each additional BH was larger than for each additional M/DH or exudate. It was thought that this could be a consequence of there being a smaller range of counts of BHs across subjects than the range of counts of M/DHs or exudates.
The maximum numbers of exudates and M/DHs within a single eye of a single subject were 59 and 50, respectively, much higher than the maximum of 16 BHs within a single eye of a single subject. A sensitivity analysis was carried out with the counts of lesions fitted as a discrete measured variable taking a maximum value of six (so zero, one, two, three, four, five, and more than five). The ORs (not shown) were larger for each extra lesion of any type. The inferences of these results were consistent with those shown for raw counts meaning that differences in ranges of counts of different lesions were not responsible for the larger ORs for counts of BHs compared with ORs for counts of exudates or M/DHs.
See Table 42 for the corresponding data for the right eye.
A simple descriptive analysis of count data was completed representing the counts of lesions of the three main types as ordered categories: zero, one, two and more than two (Table 12). The percentages of subjects with MO increased from 1% to 2% to > 3% when the number of M/DHs within one DD increased to more than two. These results suggested that this might be a useful representation when investigating the value of M/DHs in predicting MO.
This pattern was also reflected in the results presented in Table 13, which shows that the odds of having MO increased by a non-significant factor with one or two M/DHs within one DD in comparison with having no M/DHs. However, the OR of having MO dramatically increased to 6.4 when there were more than two M/DHs. This relationship was maintained (OR = 5.2; 95% CI 3.0 to 8.9) after adjusting for the counts of other lesions represented in the same way, and also centre, gender, age, glitazone, diabetes and visual acuity. The fact that the count of several M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs, the count of exudates and visual acuity, suggested that while having any M/DHs might be of little value in predicting MO, having a lot of M/DHs in an eye was still useful for predicting MO in that eye. See Table 43 for the corresponding data for the right eye.
The unweighted analyses presented in Tables 11 and 13 were repeated using weighted analyses, the results are presented in Tables 14 and 15. These analyses had the effect of up-weighting data from subjects with M/DHs and down-weighting data from subjects with more serious lesions. This small inflation had no influence on the ORs, but made CIs narrower by a very small amount. See Tables 44 and 45, respectively, for the corresponding data for the right eye.
When the mutually exclusive groups of lesions were included in the model, subjects with BHs (alone or BH with M/DHs) had 4.4 times the odds of having MO and subjects with exudates had 6.0 times the odds of having MO, both compared with those with only M/DHs. After adjusting for centre and other characteristics, these ORs changed by only modest amounts to 3.1 and 5.8, respectively. Of these other variables, only centre, age and visual acuity were significant with older age being associated with higher odds of MO by 3% per additional year older (OR = 1.031; 95% CI 1.011 to 1.051). When visual acuity was included, having worse visual acuity significantly increased the odds of MO by a factor of 3.8 (95% CI 2.3 to 6.3).
An analysis of counts of lesions found that higher counts of M/DHs, BHs and exudates within one DD, considered separately and together, significantly increased the odds of having MO. The relationships were maintained after adjustment for centre, gender, age, glitazone, diabetes (type 1, type 2 and secondary/unknown) and visual acuity. Initially, counts of exudates between one to two DD appeared to be associated with greater odds of MO, but this ceased to be significant after adjusting for exudates within one DD so this count was dropped from subsequent models.
It is important to note here, as with the unweighted analyses, that the count of M/DHs within one DD is statistically significant after adjusting for centre and the count of BHs and the count of exudates. This suggests that having a lot of M/DHs is still useful for predicting MO.
The OR for counts of BHs was again larger than for the other lesions. This does not mean that BHs were more important, but that the increase in the odds of MO for each additional BH was larger than for each additional M/DH or exudate. It was thought that this could be a consequence of there being a smaller range of counts of BHs across subjects than the range of counts of M/DHs or exudates, but as with the unweighted analyses, a sensitivity analysis showed that this was not the case.
This pattern of the increasing prevalence of MO with increasing numbers of lesions was also reflected in the results presented in Table 13. The odds of having MO increased by a non-significant factor with one or two M/DHs within one DD in comparison to having no M/DHs and the OR of having MO dramatically increased to 5.2 when there were more than two M/DHs. This relationship was maintained (OR = 5.0; 95% CI 2.6 to 9.3) after adjusting for the counts of other lesions represented in the same way: centre, gender, age, glitazone, diabetes and visual acuity.
Some of the ORs presented in Table 15 were large with very wide CIs, particularly those for having several BHs. This was because weighting had the effect of reducing the apparent number of subjects with BHs and subjects with MO. This would tend to make the estimates of ORs and their CIs more volatile. After adjustment for centre, gender, age, glitazone use, diabetes type and visual acuity these estimates of the ORs for having BHs were more stable and had inferences more similar to the earlier unweighted analyses. The CIs for missing visual acuity were very wide and the width varied greatly because of the very small numbers with missing visual acuity.
Relationship of individual lesions in both eyes to macular oedema in either eye (analysis unweighted)
Single eye analyses are simple to interpret and are helpful in establishing factors which might contribute to a prediction model. However, they do not show how a strategy might work on a subject where information about the possibility of MO comes from both eyes. Mutually exclusive groups were defined taking information about lesions within one DD from both eyes (Table 16). The most serious lesions in either eye took priority in assigning subjects to six groups. Any exudates within one DD in either eye meant the subject was assigned to the exudates group. A BH within one DD in either eye, without any exudates, meant the subject was assigned to the BH group. For those with M/DHs in both eyes (and no BHs or exudates) or M/DHs (and no BHs or exudates) in one eye and no exudates or BH in the other, the group was defined by the number of M/DHs within one DD in the eye with the most M/DHs. A subject with no M/DHs, BHs or exudates within one DD was assigned to the ‘None/other both eyes’ group. Subjects were also classified into three groups according to visual acuity [better visual acuity (log-MAR < 0.3; Snellen 6/9.5 or better) in both eyes, worse visual acuity (log-MAR ≥ 0.3; Snellen 6/12 or worse) in at least one eye, missing visual acuity in both eyes or missing visual acuity in one and better visual acuity in the other].. The percentage of subjects with MO was calculated for each combination of visual acuity and lesions. There was a substantial two- to fourfold increase in the percentage with MO if visual acuity was worse rather than better in all of the lesion groups. There were very few subjects with MO among those subjects with better visual acuity and zero to two M/DHs within one DD in either eye.
Logistic regression of the relationship of individual lesions in both eyes to macular oedema in either eye (analysis unweighted)
Odds ratios for three models for predicting MO are presented in Table 17. From left to right these models include: only lesion type and the count of M/DHs (the unadjusted ORs for visual acuity are also presented here); lesion type and the count of M/DHs adjusted for centre and visual acuity; and lesion type and the count of M/DHs adjusted for centre, visual acuity and demographic variables. Exudates, BHs or more than two M/DHs within one DD in either eye were all predictive of having MO, as was worse visual acuity. Having one or two M/DHs within one DD in an eye did not appear to be any different to having none/other mild lesions. The odds of having MO were increased by factors of 11.2 for having at least one eye with an exudate, 5.9 for having at least one eye with a BH (but no exudates), 3.4 for having more than two M/DHs relative to having eyes with a maximum of one M/DH in either. Having a M/DH in one or both eyes was not significantly different to having a maximum of two M/DHs in either eye or having no lesions or mild lesions (not a M/DH, BH or exudate).
Having worse visual acuity was predictive of MO and appeared to increase the odds of having MO by a factor of 3.3. Having worse visual acuity increased the size of the odds of MO for all types of lesion and the size of this increase was not significantly different between lesions types.
The ORs for the fully adjusted model including lesions, centre, visual acuity and demographic variables are given in the final three columns. Other variables which were significant in this fully adjusted model were age and centre. For every additional year older the odds of having MO increased by a factor of 1.035 (95% CI 1.022 to 1.049) in other words by 3.5% per year older.
Subjects with one M/DH within one DD in one or both eyes were taken as the reference group so ORs for having MO in either eye are calculated compared with this group. It was not appropriate to take the group with no lesions/other lesions as the reference group for the analyses of both eyes because many of this group would be zero weighted in the weighted analyses.
The 95% CI for the OR of having MO in those with, at most, two M/DHs within one DD in either eye relative to at most one M/DH in either eye included 1. This was also true of the CI for no lesions or lesions other than M/DHs, BHs or exudates in either eye. This meant that there was no significant difference in the odds of having MO in these subjects compared with subjects with at most one M/DH within one DD in either eye. The fact that the count of several M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs and the count of exudates suggested that having a lot of M/DHs within one DD was still useful for predicting MO. This remained true after adjusting for other variables including visual acuity.
If the 11 subjects with missing visual acuity were excluded, the results were almost identical to those above. With this restriction it was possible to include an interaction between visual acuity and the feature category. There was no evidence of an interaction between these variables and this suggested that both lesion category and visual acuity were important in predicting presence of MO, but the strong effect of having worse visual acuity did not vary between different lesion groups.
The receiver operating curves shown in Figure 12 correspond to the three models presented in Table 17. The model that included only the lesion categories within one DD in at least one eye [at least one M/DH, two M/DHs, more than two M/DHs, at least one BH (but no exudates), at least one exudate] had a receiver operating characteristic curve made up of straight lines between the points (sensitivity and 1-specificity) corresponding to the threshold being changed to include an extra category. For example, if only subjects with at least one exudate were referred, then the sensitivity and specificity would be 0.59 and 0.70, respectively. If subjects with at least one exudate and subjects with at least one BH were referred, then the sensitivity and specificity would be 0.80 and 0.57, respectively (1 − specificity = 0.43). The other two receiver operating characteristic curves correspond to the other two models presented in Table 17 where one model included lesion type and the count of M/DHs, centre and visual acuity, and the other model included lesion type and count of M/DHs, centre, visual acuity and demographic variables. There were gains in sensitivity and specificity from choosing more complex models as would usually be expected.
Relationship of individual lesions in both eyes to macular oedema in either eye (analysis weighted)
Similar models to those in Table 17 are presented in Table 18 for weighted data. Subjects with no lesions in either eye or only lesions other than exudates, BHs and M/DHs within one DD were removed from this analysis. For the weighted data exudates, BHs or more than two M/DHs within one DD in either eye were all predictive of having MO, as was worse visual acuity (see Table 18). Having one or two M/DHs within one DD in an eye did not appear to change the odds of MO.
There were some large ORs observed in the table, again due to the down-weighting of the subjects with more severe groups of lesions so the estimates were more volatile than for the unweighted analyses. This was less pronounced after adjustment for other variables. The odds of having MO were increased by factors of 11.0 for having at least one eye with an exudate, 5.9 for having at least one eye with a BH (but no exudates) and 3.5 for having more than two M/DHs within one DD relative to having eyes with a maximum of one M/DH within one DD in either. Having a maximum of two M/DHs within one DD in either eye was not significantly different to having a M/DH in one or both eyes.
Having worse visual acuity was predictive of MO and appeared to increase the odds of having MO by a factor of 3.3. Having worse visual acuity increased the size of the odds of MO for all types of lesion and the size of this increase was not significantly different between lesions types.
The ORs for the fully adjusted model including lesions, centre, visual acuity and demographic variables are given in the final three columns. Other variables which were significant in the fully adjusted model were age and centre. For every additional year older the odds of having MO increased by a factor of 1.029 (95% CI 1.014 to 1.044); in other words by 3.0% per year older.
As for the unweighted analysis, the 95% CI for the OR of having MO in those with at most two M/DHs within one DD in either eye relative to at most one M/DH within one DD in either eye included 1. This was also true of the CI for no lesions or lesions other than M/DHs, BHs or exudates in either eye.
This meant that there was no significant difference in the odds of having MO in for these subjects compared with subjects with at most one M/DH in either eye. The fact that the count of several M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs and the count of exudates suggested that having a lot of M/DHs was still useful for predicting MO. This remained true after adjusting for other variables including visual acuity.
If the data were restricted to exclude those with missing visual acuity as well as those with no features or features other than M/DHs, BHs and exudates, the results were almost identical to those described above. With this restriction it was again possible to include an interaction between visual acuity and feature category.
There was no evidence of an interaction between these variables and this suggested that both lesion category and visual acuity were important in predicting presence of MO, but the effect of having worse visual acuity did not vary between different lesion groups.
Strategies for detecting macular oedema
Findings from the statistical analysis were used to inform the inclusion of various eye and subject characteristics that might be considered within computer-assisted manual strategies and fully automated strategies. The strategies for detecting MO modelled in this study can be categorised as follows.
- Manual grading strategies These use features, similar to those in existing national criteria, which can be identified by visual inspection by trained graders.
- Computer-assisted manual annotation grading strategies These use more detailed features obtained by manual annotation which are then combined by a software classifier to determine a likelihood that MO is present.
- Fully automated annotation grading strategies where no human intervention is required These use features determined by image analysis software. As with computer-assisted manual strategies, these features are combined by a software classifier to determine a likelihood that MO is present.
Manual grading strategies
The manual strategies used the following photographic features:
- M/DH within one DD
- M/DH between one DD and two DD
- BH within one DD
- BH between one and two DD
- exudates within one DD
- exudates between one DD and two DD.
In addition, the following feature was used:
- visual acuity worse than or equal to Snellen 6/12 (log-MAR 0.3).
Eighteen different combinations of these features were modelled as the manual grading strategies and are listed in Table 19. Manual grading strategies 1 and 2 model the current practice in the English and the Scottish national programmes, respectively. The statistical analysis in the earlier part of this chapter showed that exudates, BHs and M/DHs within one DD, as well as reduced visual acuity, all significantly increase the risk of having MO. Furthermore, the risk increases with the number of lesions present within one DD. Lesions outside of one DD, including exudates, were not found to be predictive of MO. Manual grading strategy 4 (M/DHs within one DD only), manual grading strategy 5 (M/DHs or BHs within one DD), manual grading strategy 9 (M/DHs within two DD) and manual grading strategy 10 (M/DHs or BHs within two DD) investigate red lesions alone. Manual grading strategy 6 (exudates within one DD) and manual grading strategy 11 (exudates within two DD) examine bright lesions alone. Manual grading strategy 15 considers visual acuity alone. The remainder of the manual grading strategy 16 strategies explore combinations of red and bright lesions with visual acuity.
Computer-assisted manual annotation grading strategies
A computer-assisted manual annotation grading strategy differs from manual grading in that an automated classifier (described later) is used to combine the image information. This means that improved performance can be achieved because of the association between features. Also, it is possible to include more complex features, such as the number and area of lesions, although this would require additional grading time. As shown in Table 20, the computer-assisted manual annotation grading strategy also used computer intensity measurements within the macula (as explained below), visual acuity and other patient information.
Fully automated annotation grading strategies
Like the computer-assisted annotation grading strategies, the fully automated annotation grading strategies take advantage of richer information within the image, but with all the information derived automatically from the image without any human intervention. Clearly, as the automated lesion detection system approaches the performance of the manual observer, the performance of the fully automated annotation systems should be similar to the best computer-assisted annotation grading strategies. Three fully automated annotation grading systems were tested, two of which incorporated visual acuity and other patient information (see Table 20). The automated analysis produces technical failures (i.e. images of insufficient quality for automated analysis) and these were handled by using manual grading strategy 16 as illustrated in the flowchart in Figure 13.
Automated photograph analysis
This section describes the image analysis methods used in the computer-assisted annotation manual and fully automated annotation grading strategies.
Computer intensity measurements
It was noted that in subjects with MO the macular area often appears darker than the surrounding retina (although this must be distinguished from pupillary shadows, which are a common artefact when the pupil diameter is small). This effect is even more noticeable when imaging using infrared illumination, where the wavelengths are absorbed more strongly by the oedema. To quantify this difference in the macular intensities the following measures were calculated automatically from the retinal photographs (Figure 14).
- The ratio of the mean red and mean green colour channel values within one DD.
- The ratio of the mean red and mean green colour channel values in the annular region between one and two DD.
- The ratio of the mean green colour channel in the circular one DD region and the annular region between one and two DD.
- As above, but using the red colour channel.
Note that these measures may be affected by several common features such as reflections from the internal limiting membrane in younger subjects (Figure 15c), the presence of bright exudate or drusen lesions (Figure 15d) or a shadow cast by a small pupil (Figure 15e). However, the measures may have predictive value despite these confounding factors.
Lesion detection and image quality assessment
In order to test automated grading strategies for MO detection, this project made use of software which has been validated for use in diabetic retinopathy screening30,47–49 and described in previous technical papers.50–54 A brief description of the automated methods is given here.
The black background was segmented and the image was corrected for uneven illumination. Next, the location of the optic disc and the fovea were determined.53 The elliptic shape of the temporal arcades was used as a guide to make a rough estimate of the optic disc position, which was then refined using a circular Hough transform. The position of the fovea was subject to geometric constraints relative to the arcades and the disc, and was chosen at a local maximum response of a filter matched to the expected foveal darkening.
Image clarity was assessed by performing small vessel detection within a disc surrounding the detected fovea and then measuring the total vessel length. A decrease in clarity is expected to result in reduced length of vessels being visible and the low between-person anatomical variation makes this a workable assessment. Images that have poor quality were classed as ‘technical failures’.
Lesions (M/DHs, BHs and exudates) were then detected. A non-specific morphological filter was used to determine candidate lesions by separating dot-like objects from the linear objects such as the vasculature.50 For M/DHs, the non-specific filter was applied at a single scale. For BHs and exudates, the non-specific filter was applied at multiple scales and the results combined.52,54 Dark objects were detected for BHs and bright objects were detected for exudates. The second stage of lesion detection performed more detailed analysis of the candidate lesions, measuring such features as their area, contrast and, for the dark lesions, the likelihood of it lying on a vessel. For exudates, the distance to the nearest detected M/DH was also used.
For each type of lesion, a set of images in which individual lesions had been annotated was used for training. The training results and the candidate lesion features were supplied to an automated classifier, which produced a value corresponding to the likelihood of each candidate lesion being a true lesion.
Individual candidate lesion likelihoods were combined, as follows, for the candidates within circles centred on the fovea with radii of one and two DD. For M/DHs, the likelihood was thresholded, so allowing a M/DH count to be determined within one DD and two DD from the fovea. For BHs the maximum was taken of the likelihoods of candidates within the one DD circle. For exudates, the mean of the two highest likelihoods for candidates within one DD from the fovea and the mean of the three highest likelihoods for candidates within two DD from the fovea was chosen. The outputs from the automated analysis were thus:
- computer intensity measurements
- image clarity assessment
- count of M/DHs within one DD from the fovea
- count of M/DHs within two DD from the fovea
- count of M/DHs anywhere.
- likelihood of BHs within one DD from the fovea
- likelihood of BHs anywhere
- likelihood of exudates within one DD from the fovea
- likelihood of exudates within two DD from the fovea
- likelihood of exudates anywhere.
Classifiers
The computer-assisted manual and fully automated grading strategies use a classifier to determine whether or not oedema is present given a list of features, such as those above. The classifier is an example of supervised learning. It is trained using a set of data where it is known whether or not MO is present (the MO status was based on the result of their OCT scan). Once the classifier has been trained it can be used to classify previously unseen individual cases giving a probability that oedema is present. The training phase for the classifier is often very time-consuming, but it is required to be done only once.
When evaluating classifier performance it is vital to test using a completely separate set of subjects to those used in the training phase, as recognising previously seen cases is likely to artificially boost the performance. Very often the scarcity of true-positives or true-negatives means that there are not enough cases to divide into separate testing and training sets. In such cases a form of cross-validation may be used where the data are partitioned into subsets. For instance, the data could be divided into three subsets, each containing an equal number of positive cases and an equal number of negative cases. The classifier would then be trained on two of the subsets and tested on the third. In this way the performance is evaluated on the full data set. Increasing the number of subsets will give a greater confidence in the result, but will also increase the calculation time. The limiting case where each subject is a subset is known as leave-one-out testing, as the training is performed on the entire data set minus just a single test subject.
For this study a classifier known as the Random Forest classifier was used by the computer-assisted and fully automated strategies to combine the feature information and decide whether or not MO is present. This is a non-parametric classifier that makes no assumptions about the form of the feature distributions. The Random Forest classifier is an extension of classification trees. It consists of many trees that all process the same input features and then vote on which class the example belongs to. The final class is the one which achieves the most votes from all the trees in the forest. Like many non-parametric classifiers it takes much longer to train than a classifier such as a linear discriminant analysis classifier, but classification is rapid.
The features used as inputs to the Random Forest classifier are shown in Table 21.
Results
The sensitivities and specificities for predicting the presence of MO in at least one eye are presented in Table 22 for each of the manual grading strategies listed in Table 19.
For completeness, positive and negative predictive values (percentage of those referred who have MO present in at least one eye and percentage of those not referred who do not have MO) are presented in Table 23 for the same manual grading strategies. As the calculations of sensitivity, specificity, positive predictive value and negative predictive value are based on the same four counts of subjects in a table, very similar patterns of performance of the strategies can be seen from Tables 22 and 23. For example, after weighting the greater number referred under strategy 1 compared with strategy 2 gave higher sensitivity and lower specificity of strategy 1 compared with strategy 2 (see Table 22) and also the lower positive predictive value and higher negative predictive of strategy 1 compared with strategy 2 (see Table 23).
The classifiers used in the computer-assisted manual annotation and fully automated annotation grading strategies have a numerical output on a continuous scale and hence the results for these strategies are best displayed as receiver operating characteristics curves (Figure 16). Figure 16 also shows the sensitivity and specificity for the manual strategies that were taken through to the economic analysis. These were manual grading strategy 1 and manual grading strategy 2, respectively, the current English and Scottish grading practices, and manual grading strategy 8 and manual grading strategy 16. Manual grading strategy 16 was chosen because it performs best relative to the receiver operating characteristics curves shown in Figure 16 while making a reasonable compromise of sensitivity and specificity. Manual grading strategy 8 was chosen for its closeness to 100% sensitivity in order to test, in the economic analysis, a strategy involving OCT scan of all patients with photographic surrogate markers of diabetic MO.
Of the fully automated annotation grading strategies, FA2 was chosen for economic analysis since it performs better than fully automated annotation grading strategy FA1 and was similar to fully automated annotation grading strategy FA3 (though with less input information).
In order to use fully automated annotation grading strategy FA2 in the economic analysis it was necessary to choose an operating point for this strategy. The operating point was chosen to lie on a straight line between manual grading strategy 16 and the top left corner of the weighted receiver operating characteristics curve so that it achieves better sensitivity and specificity than manual grading strategy 16. At this point fully automated annotation grading strategy FA2 achieves sensitivity 75.9% and specificity 73.7%. Further details of this operating point are included in Table 22. It should be noted that, for the above reason, only this strategy is included in Table 22.
The computer-assisted manual annotation grading strategy, CAM, was not analysed further since manual annotation of lesions was considered impractical in a screening context.
Discussion and conclusions
The statistical modelling of the data collected in the study showed that MO was strongly related to the presence of lesions and was consistently higher in subjects with exudates or BHs within one DD than those with just M/DHs within one DD. Having more than two M/DHs within one DD in an eye appeared to be of particular importance in predicting the presence of MO. However, there was no evidence of a relationship between MO and the presence or count of exudates between one and two DD after adjusting for the presence, or count, of exudates within one DD.
Having worse visual acuity was associated with higher prevalence of MO.
The performance of various grading strategies for the detection of MO using retinal photographs and other risk factors for MO were explored. Owing to lower recruitment of patients with only M/DHs in the macula than occurs in the diabetic population, it was necessary to weight subjects to bring proportions to those found in an earlier sequential study.30
After weighting, the fully manual grading strategies that model the current grading practices in England and Scotland demonstrated different sensitivity and specificity. The higher sensitivity of the manual grading system in England, at the expense of specificity, can be explained by the referral, in England, but not in Scotland, of patients with M/DH within one DD who have visual acuity greater (worse) than log-MAR 0.3.
The modelling suggested that an ideal grading strategy would be one that takes into account the presence and count of all three types of lesion within one DD and also visual acuity. The fully manual grading strategy 16, that used M/DHs (provided that the visual acuity was worse than log-MAR 0.3), BHs and exudates within one DD, demonstrated an improved specificity by approximately 4% relative to the current grading practice in England, at similar sensitivity. It differs from the current grading practice in England, in that BHs within one DD are referred regardless of visual acuity status, and exudates between one and two DD are ignored. This is supported by the modelling that showed no evidence of a relationship between MO and the presence, or count, of exudates between one and two DD after adjusting for the presence or count of exudates within one DD.
A grading strategy using automated analysis of the macula-centred image performs better than current manual grading strategies. If automated analysis, data are combined with non-image data then further improvements are obtained, largely due to the use of visual acuity. This was supported by the modelling that showed that worse visual acuity was associated with higher prevalence of MO. As visual acuity is a standard measurement performed during diabetic retinopathy screening this additional information could be added to a fully automated strategy with little effort. Therefore, the fully automated annotation grading strategy including visual acuity (FA2) was chosen for further analysis in the next chapter.
The optimum strategy in terms of area under its receiver operating characteristic curve was the computer-assisted manual grading strategy, CAM. This uses the results of manual annotation of the individual lesions in each image. This is a time-consuming procedure and so is unlikely to be considered for routine grading practice. Therefore, this grading strategy was not taken forwards for economic analysis.
- Inferring the presence of macular oedema using retinal photographs - Improving t...Inferring the presence of macular oedema using retinal photographs - Improving the economic value of photographic screening for optical coherence tomography-detectable macular oedema: a prospective, multicentre, UK study
- Evidence synthesis to inform the relative efficacy of the interventions - Adalim...Evidence synthesis to inform the relative efficacy of the interventions - Adalimumab, etanercept and ustekinumab for treating plaque psoriasis in children and young people: systematic review and economic evaluation
- Results: views of patients and health professionals about the shared care model ...Results: views of patients and health professionals about the shared care model (objective 5) - The Effectiveness, cost-effectiveness and acceptability of Community versus Hospital Eye Service follow-up for patients with neovascular age-related macular degeneration with quiescent disease (ECHoES): a virtual randomised balanced incomplete block trial
- Risk assessments and structured care interventions for prevention of foot ulcera...Risk assessments and structured care interventions for prevention of foot ulceration in diabetes: development and validation of a prognostic model
- Srl sarcalumenin [Mus musculus]Srl sarcalumenin [Mus musculus]Gene ID:106393Gene
Your browsing activity is empty.
Activity recording is turned off.
See more...