Use of mobile apps for visual acuity assessment: systematic review and meta-analysis

Authors: Suo L, Ke X, Zhang D.

Geographical coverage: The US, Botswana, Malaysia, Kenya and the UK.

Sector: Biomedical

Sub-sector: Treatment

Equity focus: None

Study population: People with visual impairments.

Review type: Effectiveness review

Quantitative synthesis method: Meta-analysis

Qualitative synthesis method: Not applicable

Background: Vision impairments (VIs) and blindness are major global public health issues. A visual acuity (VA) test is one of the most crucial standard psychophysical tests of visual function and has been widely used in a broad range of health care domains, especially in many clinical settings. In recent years, there has been increasing research on mobile app-based VA assessment designed to allow people to test their VA at any time and any location.

Objectives: The goal of the review was to assess the accuracy and reliability of using mobile VA measurement apps.

Main findings:

Overall, evidence gained from this study suggests that mobile app-based VA tests can be useful for on-demand VI detection.

In total, authors included 22 articles in the review. Selected studies were conducted in more than 11 countries. Six studies met criteria for quantitative meta-analysis. Among the 22 identified studies, four were population-based, 10 were observational, three were cross-sectional, four were prospective, and one was a validation study. Nine studies assessed the performance of tablet-based (i.e., iPad) VA measurement apps, and 13 studies assessed the performance of smartphone-based apps. A total of 25 mobile apps were evaluated in those 22 enrolled studies.

Among the 22 identified studies, the average number of participants per study was 1,648, ranging from 43 to 10,579 participants. The age of the participants varied significantly from 3 to 89 years. Five studies included children aged younger than 18 years, 10 studies included middle-aged (19-55 years) adults, and the remaining seven studies included older adults (aged 55 years and older). Most of the 22 selected studies were considered high quality by the authors.

For the age group of 3 to 5 years, the authors reported a pooled sensitivity of 0.87 (95% CI 0.79-0.93; p =.39) and a pooled specificity of 0.78 (95% CI 0.70-0.85; p =.37) for visual acuity (VA) app tests compared to clinical VA tests. For those aged 6 to 22 years, the pooled sensitivity was 0.86 (95% CI 0.84-0.87; p <.001) and the pooled specificity was 0.91 (95% CI 0.90-0.91; p =.27) for VA app tests versus clinical VA tests. For the group aged 55 years and above, the pooled sensitivity was 0.85 (95% CI 0.55-0.98) and the pooled specificity was 0.98 (95% CI 0.95-0.99) for VA app tests compared to clinical VA tests. The group examined by non-professionals (AUC 0.93) demonstrated higher accuracy than the group examined by professionals (AUC 0.87). In the group using iPad-based apps, the pooled sensitivity was 0.86 and the pooled specificity was 0.79 for VA app tests versus clinical VA tests. In the group using smartphone-based apps, the pooled sensitivity was 0.86 (p <.001) and the pooled specificity was 0.91 (p <.001) for VA app tests compared to clinical VA tests.

To increase the generalisability of review findings, authors note the need of more studies with longer follow-up periods. Future research should explore how to determine abnormal changes in eye anatomy and functions.

Methodology:

A study was deemed relevant by the authors if it (1) evaluated visual acuity (VA) using a smartphone or tablet app; (2) utilized an acceptable VA reference standard; (3) was written in English; and (4) was published post-2008. The exclusion criteria were: (1) studies where the count of participants with visual impairment was less than 10; and (2) literature reviews, commentary articles, short communications, or case reports. The authors conducted a search on PubMed, EMBASE, Cochrane Library, and Google Scholar for pertinent articles on mobile apps for VA assessment published between January 1, 2008, and July 1, 2020. Two researchers independently reviewed and selected the relevant studies. Two authors and a research assistant extracted information from the identified studies. Two researchers specialising in eye care independently reviewed each selected article and assessed its quality by using the Quality Assessment of Diagnostic Accuracy Studies-2 scores (QUADAS-2) tool.

The study used Chi-square and I² values to assess heterogeneity in sensitivity, specificity, likelihood ratio tests, and diagnostic odds ratio (DOR). Heterogeneity was determined by the Cochrane Q-test (I2 value) and was considered present when p >.10. A fixed-effects model was used for I2 results ≤50%, and a random-effects model was used otherwise. The DOR values ranged from 0 to infinity, with 0 indicating no test discrimination and higher scores indicating better discrimination. Sensitivity, specificity, positive and negative likelihood ratios, and DOR were calculated for each subgroup (age, examiner, and mobile device type) using a random-effects model due to expected high heterogeneity. Meta-regression was performed to investigate if heterogeneity could be explained by methodological factors or study sample characteristics. A summary receiver operating characteristic (SROC) curve was constructed using the Moses constant of linear mode model. Meta-DiSc software was used for meta-analysis and Review Manager for paper quality assessment. Data was synthesised by creating forest plots of sensitivity and specificity.

Applicability/external validity: Authors note a number of limitations that reduce the applicability of their study, including the fact that some age groups were not covered in meta-analysis; and limited coverage of different technologies.

Geographic focus: The review encompasses a variety of high-, low-, and middle-income countries, yet it does not evaluate how the efficacy of mobile devices might differ across these diverse settings.

Summary of quality assessment:

There were several limitations in the approaches used to identify, include and critically appraise studies. The search was restricted to material published in English and there was no evidence of reference lists of relevant articles being checked. We were also unclear as to the rationale of the cut-off of 2008 for the search. While the approach to data analysis was generally robust, there did not appear to be any attempt to undertake separate analysis for the small number of included studies that were at a higher risk of bias. For these reasons, we attributed a low confidence in the findings of this review.