Introduction

Optical coherence tomography (OCT) is a non-invasive, non-contact imaging modality which provides high resolution, cross-sectional images of the retina and is ubiquitous in ophthalmology practice to diagnose and monitor retinal disorders1. OCT angiography (OCTA) uses moving red blood cells in the retinal vasculature as an intrinsic contrast agent to generate 3-dimensional images of retinal and choroidal blood flow2,3. OCTA is widely used to evaluate retinal perfusion in retinal and systemic disorders4, and demonstrates microvascular impairment in disorders such as diabetes mellitus5, uveitis6, age-related macular degeneration7, atrial fibrillation8, haemorrhagic shock9,10, and systemic hypertensive crisis11. As OCTA is fast, cheap, and does not risk systemic reactions (as fundus fluorescein angiography (FFA) or indocyanine green angiography do), its use is fast becoming widespread in research and clinical practice. OCTA is now used alongside OCT and FFA in the diagnosis and management of retinal diseases12.

Many OCTA platforms use proprietary algorithms to estimate and visualise retinal perfusion13,14. However, as different OCTA devices use different algorithms, comparisons of results between studies are constrained13. Further, quantitative metrics derived from the OCTA signal and images lack consistent methodology15, also limiting comparison validity16. The raw signal may be used to derive limited scaled flow information15, and additional processing before image analysis includes thresholding to create binary images from grayscale17, and skeletonization to display vessels as one-pixel width tracings18. The most commonly calculated perfusion metrics from binarised and skeletonised images are17,19:

  1. 1.

    Vessel density (VD)—the total area of perfused vasculature per unit area in a region of measurement (sometimes reported as “perfusion density”).

  2. 2.

    Vessel length density (VLD)—the total length of perfused vessels divided by the total number of pixels in the given area on the skeletonised image.

  3. 3.

    Fractal dimension (FD)—a mathematical parameter describing the complexity of a biological structure, usually applied to skeletonised images20.

  4. 4.

    Foveal avascular zone (FAZ) measurements (Supplementary Fig. 1)—a change in FAZ measurements (e.g. area and perimeter) from baseline suggests altered blood flow21.

A scoping search on the National Library of Medicine PubMed (including Medical Literature Analysis and Retrieval System Online—MEDLINE) found no existing systematic reviews or meta-analyses comparing methods of quantitative OCTA analysis. We therefore conducted a systematic review and meta-analysis with the aim of assessing which OCTA perfusion analysis method most sensitively detects pathological change between patients with disorders affecting retinal perfusion and control patients with normal retinal perfusion. Our secondary aim was to look at the stability of OCTA imaging by identifying papers that studied the test–retest variability of OCTA.

Methods

This systematic review was performed following the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA) statement22.

Inclusion criteria

Full inclusion and exclusion criteria are provided in Supplementary Table 1. We initially sought to investigate the sensitivity and stability of OCTA imaging, therefore we planned to include both studies comparing findings in normal patients with pathology and studies that included patients having repeated OCTA scans over time with or without a control group.

Prospective studies involving ≥ 50 participants were included where OCTA had been used to investigate changes to macula perfusion caused by either retinal or systemic disorders, using any one of the following analysis metrics (either on binarized or skeletonised images): VD, skeletonised VD (SVD), VLD, FD, skeletonised FD (SFD), capillary density index, FAZ measurements and; where agreement between repeated OCTA images was assessed by intra-class correlation coefficient. Included studies were limited to those with a sample size of at least 50 participants to minimise selection bias from the inclusion of small and selective case series. The year of publication was limited from 2014 to August 2021, as the clinical application of OCTA was first described in 201423. Only studies looking at foveal, parafoveal, and whole areas of the macula were included.

Papers published in medical journals and written in English were included—conference abstracts and papers written in languages other than English were excluded.

Exclusion criteria

We excluded studies investigating retinal disorders which cause focal anatomical change (e.g., age-related macular degeneration) or studies that only investigated perfusion in the choroid, choriocapillaris, or peripapillary region. Retrospective studies and studies that did not specify which region of the macula was analysed were excluded.

Search strategy

MEDLINE and Embase were searched using OVID. The applied search strategy is in Supplementary Fig. 2.

Risk of bias assessment

Two authors independently assessed the potential bias in the prospective studies using the National Institutes of Health (NIH) quality assessment tool for observational cohort and cross-sectional studies24. A consensus was then reached between the two authors to create the risk of bias table (Supplementary Table 3).

Data extraction

Retinal perfusion was compared between healthy control patients and those with defined disease states. Two independent reviewers individually reviewed all titles and abstracts retrieved from the initial search. Duplicates were removed and each reviewer decided on the study’s inclusion based on the title and abstract. Disagreements between reviewers on a paper’s eligibility were resolved by discussion, involving the senior author (RJB) if a decision could not be reached. Reference management software was used to aid the screening process as per the PRISMA flow diagram (Fig. 1). Data were extracted by two reviewers working independently, with disagreements resolved by discussion. The following variables were recorded: study information (first author, year of publication, country location of study, study design), participant information (total number of eyes, total number of patients, sex, mean age), OCTA device and imaging information (instrument manufacturer, number of a-scans, scan size), and OCTA analysis information (vessel layer, macular region, analysis metric mean and standard deviation).

Figure 1
figure 1

Systematic review and meta-analysis study flowchart.

Full size image

The outcome data (mean and standard deviation) collected included: percentage VD (VD%), SVD, VLD, FD, SFD, FAZ area, FAZ perimeter, FAZ acircularity ratio, and FAZ acircularity index. If unpublished information was required, the corresponding author of the study was contacted. If no response was received within one-month of contact, analysis proceeded based on published data. Only VD data given as a percentage and FAZ area presented as mm2 were included in the study characteristics table.

Statistical methods for the effect of diabetic retinopathy on retinal perfusion

To combine measurements of the continuous variables VD and FAZ, and estimate a value for overall common and random effects, inverse variance weighting was used for pooling. When comparisons were made between pooled standardised mean differences for different sub-analyses, statistical differences were assessed using a Z test, with p < 0.05 considered statistically significant. An overall standardised mean difference was calculated using the random effects models. A funnel plot was used to detect publication and location bias in the selection of included trials according to the method of Egger et al.25. The R statistical software (Version 4.1.1) (R Foundation for Statistical Computing, Vienna, Austria; see http://www.r-project.org) and its meta package (http://cran.r-project.org/web/package/meta) were used for these analyses.

Statistical methods for evaluating the effect of analysis methods on the assessment of retinal perfusion in diabetes without diabetic retinopathy or with mild non-proliferative retinopathy

Meta-analyses were performed on studies investigating diabetes mellitus with no diabetic retinopathy or with mild, non-proliferative diabetic retinopathy (the early stage of diabetic retinopathy in which symptoms are mild or non-existent), using Review Manager 5 (RevMan Version 5.4. The Cochrane Collaboration, 2020). Statistical heterogeneity between studies was tested for using the Q-statistic (tests the null hypothesis that all studies share the same common effect) and heterogeneity was quantified using the I2 measure of study heterogeneity (percentage of total variation across studies that is due to true heterogeneity rather than chance). A random effects model was used to address the issue of high levels of heterogeneity of results between studies.

Results

After removing duplicates, electronic searches retrieved 4543 records, of which 191 studies were included, and 88 eligible for qualitative analysis. A PRISMA flow diagram of search results is presented in Fig. 1. Excluded studies are presented in the Supplementary Table 2.

Characteristics of included studies

Study characteristics are presented in Tables 1 and 2. Of the 88 studies included, 78 were cross-sectional, six were longitudinal cohort studies, and four were case–control studies. Five papers met the original inclusion criteria but were not presented in the study characteristics table, as they did not include VD or FAZ data, instead using VLD or FD. Only summary data defined as VD or FAZ is presented as reporting of other analysis methods was too heterogenous. Some studies did not specify which macula region was analysed for VD but did include FAZ data. In these instances, the paper was included but VD data were excluded. While the baseline data were presented from five longitudinal studies that included patients having repeated OCTA scans over time, no studies reported test–retest variability.

Table 1 Study characteristics of studies included looking at patients with diabetes.
Full size table
Table 2 Study Characteristics of studies included looking at patients with diseases other than diabetes.
Full size table

Heterogeneity of assessments

The included studies recruited patients with 64 different diagnoses, used seven different OCTA systems (Table 3), defined six different volume scan densities, with four different volume scan sizes, nine different perfusion analysis methods, five different vessel layer definitions for superficial and deep capillary plexi, and examined three different macula regions, giving a total of 197 distinct methods of assessing retinal perfusion, but a potential of more than 7000 different combinations (Table 4). Heterogeneity in OCTA analysis limited data synthesis, however the most studied condition was diabetes mellitus with or without diabetic retinopathy and the most reported analysis methods were VD and FAZ area. We therefore present detailed synthesis of VD and FAZ area in diabetes mellitus.

Table 3 OCTA equipment and software in included studies.
Full size table
Table 4 Different parameters available for assessing retinal perfusion in included studies.
Full size table

Risk of bias results

A risk of bias analysis using the NIH quality assessment tool for observational cohort and cross-sectional studies (14 questions) and the NIH tool of case–control studies (12 questions) graded 23 studies as good, 47 as fair, and 18 as poor quality (Supplementary Table 3). We retained studies rated as poor quality to illustrate heterogeneity.

Bias was identified predominantly in question 6 (“were the exposure(s) of interest measured prior to the outcome(s)?”), question 7 (“Was the timeframe sufficient so that one could reasonably expect to see an association?”) and question 10 (“Was the exposure(s) assessed more than once?”) of the NIH quality assessment tools because the included studies were mostly cross-sectional and not longitudinal by design. Sample size justification was rarely given (question 5) and study population was not always explicitly defined (question 2). Funnel plots (Fig. 2) showed no evidence of publication bias.

Figure 2
figure 2

Funnel plots of the effect of diabetic retinopathy on retinal perfusion. (a) VD in patients with diabetic eye disease. (b) FAZ in patients with diabetic eye disease. (c) VD in all patients with diabetes mellitus. (d) FAZ in all patients with diabetes mellitus. VD, vessel density; FAZ, foveal avascular zone.

Full size image

Effect of non-proliferative diabetic retinopathy on retinal perfusion

Twenty-six papers were included, as they had VD% results calculated from the same vessel layer, vascular region, and used the same scan size. In comparison to healthy controls, eyes with diabetic eye disease had, on average, a smaller VD% of − 3.52% (n = 18 studies, 95% CI [− 6.71; − 0.32], p = 0.031; Fig. 3b) and a larger FAZ area of 1.50 mm2 (n = 26 studies, 95% CI [0.2999; 2.7007], p = 0.014; Fig. 3a). In comparison to healthy controls, eyes of patients with diabetes had, on average, a smaller VD% of − 1.7822% (95% CI [− 3.4935; − 0.0708], p = 0.041; Fig. 3d) and a larger FAZ area (0.7046 mm2 (95% CI [0.1826; 1.2266], p = 0.0082; Fig. 3c). Study characteristics are summarised in Table 1.

Figure 3
figure 3

Meta-analysis forest plots for diabetic retinopathy. Forest plots analysing the effect of diabetic retinopathy on retinal perfusion by comparing healthy controls with diabetic retinopathy patients for (a) VD%, and (b) FAZ and; comparing healthy controls with all diabetic patients for (c) VD% and (d) FAZ comparing healthy controls with all diabetic patients with diabetic retinopathy and without diabetic retinopathy. VD%, percentage vessel density; FAZ, foveal avascular zone; SD, standard deviation.

Full size image

Effect of analysis methods on the detection of diabetes without diabetic retinopathy

Nineteen papers were included. All included analysis methods across the different vascular plexi and retinal areas detected reduced perfusion in patients with diabetes without diabetic retinopathy compared to healthy controls (Fig. 4), although even within individual analysis methods, such as in the deep capillary plexus (DCP) with foveal perfusion assessed from a 6 × 6 mm macular volume, heterogeneity was still high (I2 = 93%, p < 0.00001). While all methods detected reduced retinal perfusion in diabetes without diabetic retinopathy (Fig. 4a–c), values for VD% in the superficial capillary plexus (SCP) and assessed in the parafoveal area tended to have the lowest heterogeneity and detected the greatest effect on perfusion (Fig. 4a). Study characteristics are summarised in Table 1.

Figure 4
figure 4

Meta-analysis forest plots for diabetes without diabetic retinopathy. Forest plots showing the effect of analysis methods on the detection of altered retinal perfusion in diabetes without diabetic retinopathy versus healthy controls measured by VD% and FAZ area, grouping studies depending on scan size, vessel layer, and macular region used to derive results. (a,b) VD%. (c) FAZ area. VD%, percentage vessel density; FAZ, foveal avascular zone; DM NDR, diabetes mellitus without diabetic retinopathy; SD, standard deviation; SCP, superficial capillary plexus; DCP, deep capillary plexus.

Full size image

Study characteristics of studies included looking at patients with diseases other than diabetes mellitus

Thirty-seven studies assessed VD and FAZ area in patients with diseases other than diabetes (summarised in Table 2), with a similar breadth of assessments as in the diabetes studies in terms of retinal area imaged, vascular layer segmented, and macular region assessed. Also similar to studies of diabetes, VD detected differences more frequently than FAZ area, parafoveal and whole macula VD more frequently than foveal VD and superficial VD more frequently than deep vessel VD.

Patients with Alzheimer’s disease (AD) and mild cognitive impairment (MCI) had reduced foveal, parafoveal, or whole macular perfusion, or increased FAZ area81,82,84,85,86,87,88,89,90,91,100, except in one study83. Similarly, patients with Parkinson’s disease (PD) had reduced retinal VD and larger FAZ area in two studies93,94, and not in one92, as did patients with atherosclerosis, stroke and cerebrovascular disease95,101,102,103, and patients with beta thalassaemia104, and diabetic patients with CKD105,106, and microalbuminurea107.

Patients with MS and NMOSD also had lower VD (in the superficial more than the deep retinal circulation) and larger FAZ area96,97,99, with patients with a history of optic neuritis having lower retinal perfusion than patients with MS or NMOSD without prior optic neuritis.

Posterior uveitis, including Behcet’s, Vogt-Koyanagi-Harada disease, and Fuch’s heterochromic cyclitis had lower retinal VD and higher FAZ area compared to patients or eyes without uveitis6,108,109,110,111,112,113,114,115,116.

Discussion

To our knowledge, we present here the first systematic review and meta-analysis of OCTA analysis methods, demonstrating very high heterogeneity of both OCTA analysis methodology employed and reported OCTA data for individual methods. Heterogeneity in analysis methods is demonstrated by the fact that there were more methods of analysing retinal perfusion (197 in this review) than included studies. The included studies varied in perfusion metric used, macula area analysed, equipment manufacturer, and retinal layer segmentation studied, although VD and FAZ area were commonly reported. Although studies investigating the effect of diabetes mellitus on retinal perfusion reporting similar perfusion metrics offered variable and heterogenous results, the OCTA data consistently demonstrated reduced retinal perfusion in patients with diabetes who had, or did not have, diabetic retinopathy when compared to healthy controls.

Reduced retinal perfusion in diabetes and diabetic retinopathy matches the known pathophysiology of the condition26, reinforcing the clinical validity of retinal perfusion measurement assessment by OCTA in these patients. However, given that early diabetic retinopathy is associated with retinal ganglion cell (RGC) loss27, and RGC degeneration is associated with reduced retinal perfusion28, it is not possible to separate primary vascular pathology from changes secondary to neurodegeneration28. Further, our meta-analysis demonstrates that not all methods of OCTA analysis reliably detected reductions in retinal perfusion in diabetic retinopathy, suggesting that different approaches have differing sensitivity and reliability. Of the approaches analysed, the superficial vascular plexus (SVP) had the lowest heterogeneity in assessment of retinal perfusion and SVP analysis detected the strongest effects, which has been previously reported in a number of conditions29, and is unsurprising given that the superficial retina suffers the fewest noise and projection artefacts19, and has the largest blood vessels, with correspondingly higher perfusion30 and greater potential for regulation of changes in perfusion30. Similarly, retinal perfusion is highest in the parafoveal area, consistent with perfusion assessment in this region detecting the greatest changes.

There are growing calls to standardise OCTA methodology16,31,32. We previously compared different OCTA analysis methods, and they concluded that the high variability between metrics and software meant that the different approaches were often not analogous15, but that VD data was the most reproducible across platforms and should be reported in OCTA studies, potentially being preferred to skeletonized metrics in the absence of software standardization. It is therefore encouraging that VD was most frequently reported in this study, although heterogeneity was still high. A different study by Rabiolo et al.33 compared different OCTA algorithms and found that, while the different algorithms all identified important differences between healthy and affected eyes, absolute values were not comparable. This lack of between-platform comparability limits the wider potential of population-level OCTA data. One meta-analysis of fractal dimension perfusion metrics34 found that heterogeneity due to different analysis methods limited comparisons, similar to our findings, and again supports calls for standardisation in OCTA protocols. In our meta-analyses we saw high levels of heterogeneity at both the macroscopic level and when analysing different individual analysis methods. One explanation for high individual analysis heterogeneity is the variety of OCTA devices used from different manufacturers that implement different algorithms to determine blood flow or segment vascular layers. A standardised method of quantifying Heidelberg OCTA of the macula and peripapillary vessels has been proposed35, although uptake may be limited when manufacturers’ own proprietary algorithms are available15.

Limitations of this study include the heterogeneity in reported OCTA methods, which limited synthesis and comparison of analysis methods and findings across different disease states, but highlights the need for standardisation. VD and PD were often used interchangeably and occasionally not defined. The definitions we include in the introduction were the most common definitions in our included studies. Due to high levels of heterogeneity, it was not possible to reliably meet the initial study aims of determining the most sensitive method of OCTA analysis. There were also no papers that studied the test–retest variability of OCTA. Papers did not routinely report reliability, stability, sensitivity, or specificity data for OCTA analyses, which are crucial for test evaluation approaches to the clinical application of OCTA. Finally, while we report increased FAZ area and decreased VD percentages in both patients with diabetes and with diabetic eye disease compared to healthy controls, we recognise that the breadth of the confidence intervals suggest uncertainty about the exact magnitude of this difference.

Currently, there are no standardised reporting guidelines for studies using OCTA—in contrast to the APOSTEL guidelines for OCT studies36—and the many thousands of possible OCTA analysis methods available limit reliable comparison of data. To ensure valid comparison of OCTA study results and robust definition of disease characteristics in which retinal perfusion is impaired, we support the suggestion that reporting guidelines and standardisation are urgently required. This would allow consistent reporting to support development of OCTA to its full potential as a ubiquitous clinical imaging modality, similar to OCT, rather than the research tool that it often remains at present37. As an initial step, we suggest reporting VD in the parafoveal area and FAZ area as a minimum dataset.

Conclusion

Analysis and reporting of retinal perfusion using OCTA is highly heterogenous, meaning that despite the myriad of published papers assessing retinal perfusion across different diseases, few direct comparisons can be made. In addition, the stability and reliability of OCTA analyses has been under-studied. We strongly support the need for standardisation of methodology along with OCTA reporting guidelines, and suggest that a minimum dataset for OCTA reporting should include parafoveal VD and FAZ area.