When the study was originally planned the OCT market was dominated by a single device, the Zeiss Stratus OCT, which had been available since 2001. Subsequently, a number of new scanners came on the market offering faster acquisition, lower image noise and higher resolution.
As a result it was decided not to limit the study to one scanner. Furthermore, the advantages offered by the newer scanners meant that the Stratus OCT would rapidly become obsolete and so patient recruitment would become difficult.
In order to include results from different OCT scanners, experiments were undertaken before the study began to estimate the differences in thickness between the new scanners and the Stratus OCT, and so allow corrections to be made.
This study was considered a service evaluation by the North East of Scotland Research Ethics Committee on 20 August 2007. All subjects gave informed consent after the intent of the evaluation was explained to them.
Subjects
Forty-two volunteers aged ≥ 18 years took part. Sixteen of the volunteers were male and the median age was 37 years (interquartile range 28–43 years). None of the subjects had a history of eye disease or diabetes. Each subject had both eyes scanned twice by the same operator on at least one of the four OCT scanners used in the study. Fifteen subjects were scanned on a single scanner, 17 on two scanners and 10 on more than two scanners. Pharmacological pupillary dilatation was not used as other studies have shown it has no effect on thickness measurements.32,33 There was no significant difference in the age (Wilcoxon signed-rank test) or male/female ratio (chi-squared test) between the subjects imaged on each scanner.
Measurements from left eyes were reflected left to right such that the nasal region was to the right and the temporal region to the left in both left and right eyes.
Statistical analysis
Both eyes were included in the analysis since no correlation was expected between repeat measurements in left and right eyes. This reduced the number of subjects required and also allowed investigation of temporal/nasal asymmetry. Repeatability was calculated as:
where std(D) is the SD of the repeat measurement differences.39 Confidence intervals (CIs) were calculated for repeatability as ± (tn − 1,0.05/√(2(n − 1))) × 1.96 × std(D), where n is the number of eyes and tn − 1,0.05 is the inverse cumulative t-distribution for a 5% probability.40 The intraclass correlation coefficient (ICC) was also calculated as described by Shrout and Fleiss.41 Interscanner agreement was calculated as for intrascanner repeatability using the equation above. Repeatability and ICC values were obtained from SPSS (version 17; SPSS Inc., Chicago, IL, USA). The difference between scanner measurements was plotted against the mean of the scanner measurements to detect any thickness dependence on the difference between scanners.
As not all subjects were measured on all four scanners a mixed-effects model was used in preference to a standard analysis of variance.42 It was implemented using the PROC MIXED procedure within SAS (version 9.1; SAS Institute Inc., Cary, NC, USA). The scanners were treated as fixed effects, using the Zeiss Stratus OCT as the reference scanner. Differences between volunteers' left and right eyes were also treated as a fixed effect, while the differences between volunteers were considered a random effect. The size of the differences between scanners could thus be estimated while allowing for differences between volunteers and between eyes within volunteers. Analyses were also carried out on the means of the annular inner four and outer four ETDRS regions.
Two schemes were compared for converting values between different scanners. The first applied a single additive constant, calculated as the mean difference between scanners, to all nine regions. The second used three separate additive constants for the central (region 1), inner regions (2, 3, 4 and 5) and outer regions (6, 7, 8 and 9). The mixed-model analysis was repeated in each case using the adjusted values to test whether or not the differences between the scanners remained significant.
Imaging protocols
Zeiss Stratus OCT
The ‘fast macular protocol’ was used on the Stratus OCT. It acquires six intersecting 6 mm radial cross-sections centred on the fovea. Each cross-section includes 128 A-scans of 1024 axial samples over a 2 mm depth range. The total acquisition takes 1.9 seconds. The scans were considered acceptable if (1) the SD of the central foveal measurements was < 10% of the mean central measurement, (2) the recorded signal strength was at least 4 out of a possible maximum of 10, (3) there were no warnings from the instrument regarding ‘low analysis confidence’, ‘missing data’ or ‘high variance’, (4) there were no visible boundary tracking errors and (5) the fixation point was within approximately 250 µm of the centre of the foveal pit. The scanner software version was 4.0.7.
Zeiss Cirrus OCT™ (Carl Zeiss Meditec International, Jena, Germany)
The Cirrus OCT (Carl Zeiss Meditec International, Jena, Germany) acquires A-scan data almost 70 times faster than the Stratus OCT. Scans were collected covering a 6 × 6 mm square area with a 512 × 128 raster pattern. Each A-scan consisted of 1024 samples over a 2 mm depth range. The total acquisition time was 2.4 seconds. A faster protocol is available, using a 200 × 200 raster pattern, which covers the same area in 1.5 seconds. The scans were considered acceptable if (1) the signal strength was at least 6 out of 10, (2) there were no obvious signs of movement, (3) there were no visible boundary tracing errors and (4) the fixation point was within 250 µm of the centre of the foveal pit. The scanner software version was 2.0.0.54.
Topcon 3D OCT-1000™ (Topcon Corporation, Tokyo, Japan)
Scans were acquired covering a 6 × 6 mm square area to a depth of 1.68 mm using a 256 × 128 raster pattern of A-scans. Each acquisition took approximately 2 seconds. The scans were considered acceptable if (1) there were no visible boundary tracing errors, (2) there were no obvious signs of movement, (3) < 10% of B-scan lines were missing in any ETDRS region and (4) the fixation point was within 250 µm of the centre of the foveal pit. The scanner software version was 2.11.
Heidelberg Spectralis™ (Heidelberg Engineering, Heidelberg, Germany)
The Spectralis A-scan rate of 40,000 A-scans/second is the fastest in this group. Its acquisition also differs from the other scanners in two significant respects. First, it includes an eye tracking facility which pauses the acquisition whenever the subject moves or blinks (consequently, the scan time can vary depending on subject movement). Second, each B-scan cross-section is acquired several times and combined to improve the signal-to-noise ratio and reduce speckle artefacts.
Scans were acquired for a nominal 30 × 30 degree region using a 768 × 256 raster pattern, with each B-scan cross-section acquired five times. Each A-scan consisted of 512 samples covering a 1.8 mm depth range. The scans were considered acceptable if (1) the 6 mm ETDRS region fitted within the acquired region and (2) there were no boundary tracking errors. Software allowed the regions to be translated to correct for minor fixation errors. Boundary tracings were not available for all images and so the exclusion criteria were less stringent than for the other scanners. The software version used was 3.0.
Results
Boundary measurements
Differences in how boundaries are identified by the various scanners were a significant problem. Boundary A () indicates reflections from the nerve fibre layer on the top surface of the retina; all the scanners begin their thickness measurements at this layer. However, there is disagreement about where to place the lower limit of the thickness measurement. Boundary B, probably reflections from the external limiting membrane, tends to be fainter than the three lower boundaries C, D and E. The Zeiss Stratus OCT takes the top of the bright reflective band (approximately layer C above) as the posterior limit (measurement M1). In contrast, the Heidelberg Spectralis takes the bottom of the bright reflective band (layer E, measurement M3). The Zeiss Cirrus OCT and Topcon 3D OCT-1000 use a limit between these two extremes, with the Topcon 3D OCT-1000 tending to find the lower edge of layer C and the Zeiss Cirrus OCT the top edge of layer D (measurement M2).
Optical coherence tomography cross-section through the centre of the macula showing the normal fovea (pit) at the region of highest acuity vision. As shown here, darker grey levels represent stronger signals. A number of highly reflective (i.e. dark) (more...)
Scans meeting inclusion criteria
Eyes were excluded if either of the repeat scans failed to meet the inclusion criteria. The number of eyes included from each scanner were: the Zeiss Stratus OCT (45/60), the Zeiss Cirrus OCT (55/66), the Topcon 3D OCT-1000 (17/26) and the Heidelberg Spectralis (27/28).
Four Zeiss Stratus OCT scans had visible boundary errors (), two had low analysis confidence warnings, one had a fixation error, one had missing data and two had high variance warnings. On the Zeiss Cirrus OCT, fixation errors affected 10 scans and one was excluded because of movement. On the Topcon 3D OCT-1000, 13 scans showed missing data, which in nine cases was serious enough for the scans to be excluded. One Heidelberg Spectralis scan was excluded owing to a missing thickness value in one region, probably because of a boundary tracing error.
Example of an automatic boundary detection failure. Left, Topcon 3D OCT-1000 example from a patient with MO; right, a Zeiss Stratus OCT scan from a volunteer.
Repeatability
Both eyes were included in the repeatability analysis since the repeat measurement differences are expected to be random and uncorrelated. In this study the correlation between thickness measurements in left and right eyes was strong (Pearson correlation coefficient 0.780), but there was no significant correlation in the repeat measurement differences in left and right eyes (Pearson correlation coefficient 0.014; p = 0.9).
lists the repeatability and ICCs for the nine ETDRS regions. In general, repeatability was poorest in the smallest, central region (1) and best in the inner regions (2, 3, 4 and 5). The Zeiss Stratus OCT shows the poorest repeatability, with all regions being equal to, or worse than, the newer scanners.
Repeatability measures (R) and ICC for normal volunteers from the four scanners for each ETDRS region (95% CI shown in parentheses)
Interscanner agreement
Box and whisker plots of the macular thickness in each of the nine regions are shown in . By inspection it is clear that there were differences between all the scanners and that the sizes of these differences were similar for all regions.
Box and whisker plots of macular thickness measurements for the nine ETDRS regions shown in Figure 29 for the four scanners. The plots show the median (central stripe), interquartile range (box) and full range (whiskers), excluding outliers (+) which (more...)
The mean central thickness in volunteers was greatest on the Heidelberg Spectralis (277 µm, SD 15 µm) and least on the Zeiss Stratus OCT (201 µm, SD 19 µm). The Topcon 3D OCT-1000 (230 µm, SD 22 µm) and Zeiss Cirrus OCT (258 µm, SD 21 µm) were between these two extremes. In all scanners the average thickness in the superior region was greater than in the inferior region, and likewise greater in the nasal compared with the temporal region.
lists the results of the mixed-model analysis. It shows the estimated differences between scanners calculated using scans from both eyes meeting the inclusion criteria. There were significant differences in thickness between all of the scanners in all nine regions (p < 0.001). In the central region the measurements were greater on the Zeiss Cirrus OCT, Topcon 3D OCT-1000 and Heidelberg Spectralis scanners than the Zeiss Stratus OCT by 58 µm, 28 µm and 78 µm, respectively. Compared with the central region differences, the other eight regional differences were slightly smaller on all scanners except the Topcon 3D OCT-1000, where the largest differences were in the inner regions.
Differences between other scanners and the Zeiss Stratus in each for each ETDRS region based on a mixed-model analysis
As agreement in its technical sense is proportional to the SD of measurement differences,39 any systematic difference is disregarded. The central region macular thickness agreement between the Zeiss Stratus OCT and Zeiss Cirrus OCT was 10.5 µm (n = 30; 95% CI 7.1 to 13.9 µm), between the Zeiss Stratus OCT and Topcon 3D OCT-1000 was 6.9 µm (n = 13; 95% CI 3.26 to 10.5 µm) and between the Zeiss Stratus OCT and Heidelberg Spectralis was 12.7 µm (n = 14; 95% CI 6.3 to 19.1 µm).
Interscanner thickness conversion
Using a single correction for all nine regions and rerunning the mixed-model analysis resulted in very similar measurements from all the scanners for the central region. However, statistically significant differences still remained; in the outer regions the Zeiss Cirrus OCT and Heidelberg Spectralis were up to 16 µm lower than the Zeiss Stratus OCT and the Topcon 3D OCT-1000 up to 7 µm higher. Refining the model and using three regional constants per scanner, where correction values were calculated for the central, inner and outer regions, resulted in very few statistically significant differences remaining and the average errors were now all ≤ 3 µm, with the largest errors in the inner nasal region.
The above strategy assumes additive correction alone is sufficient. The need for possible multiplicative correction was tested by plotting the difference between measurements from two scanners against the mean thickness measured on the two scanners. shows the comparison of Zeiss Stratus OCT and Zeiss Cirrus OCT measurements for the central region. If an additive constant is sufficient to correct the difference then the line of regression should have a zero gradient. However, as shows, the regression line has a gradient of −0.17, which is significantly different from a zero gradient (p < 0.001). Although only regions 1 and 9 are significantly different from zero gradient, probably owing to the small sample size, the bar chart in of estimated gradients for all nine regions shows a clear trend, with negative gradients in the central and inner regions and positive gradients in the outer four regions.
Graph of the difference between the Zeiss Stratus OCT and Zeiss Cirrus OCT measurements vs. average thickness. The dashed line shows the least squares regression.
Bar chart showing the regression coefficients (slope) for each ETDRS region for plots of difference in Zeiss Stratus OCT and Zeiss Cirrus OCT vs. mean thickness.
Conclusions
Thickness measurements differ significantly between scanners. An additive correction using three separate correction values, for the central, inner and outer regions, resulted in average errors of ≤ 3 µm.
As explained in Chapter 2, there were two criteria for judging that MO was present. The first was that the central ETDRS region thickness should be > 250 µm, or any of the inner five regions should be > 300 µm. The final decision on the presence of MO was made by examination of the OCT images for the presence of intraretinal cyst or subretinal fluid. As the thickness measurement was primarily being used to identify those images that should be subjected to a visual examination, it was judged that the accuracy provided by using the additive correction was sufficient.
