The use of the plusoptiX photoscreener for vision screening

Aim: Auto-refractors have evolved into tools that can be used for vision screening by detecting uncorrected refractive errors and eye misalignment in less than a second. By comparing the measured levels of refractive error to pre-set criteria and assessing ocular alignment, the device will advise a ‘Pass’ or ‘Fail’ result for each child screened. The aim of this review is to determine if the plusoptiX photoscreener provides a viable alternative to traditional screening methods. Methods: A literature search of databases was performed, focusing on publications from the last 5 years. Results: The ability of the device to detect the presence of a referral factor (sensitivity) ranges from 47% to 99%. The ability of the device to correctly identify that referral factor (specificity) ranges from 49% to 100%. Low sensitivity resulted in up to a 6.3% rate of under-referral by the device (false negative) with low specificity resulting in 35% of those screened being referred unnecessarily (false positives). These ranges can be changed by adjusting the referral criteria within the device, with the level of hypermetropia detected the greatest cause of underor over-referral. Conclusion: By setting the appropriate criteria for referral within the device, a reasonable level of sensitivity and specificity have been demonstrated. The device could provide a useful tool for screening, offering a good level of accuracy for the detection of amblyogenic risk factors in children.


Introduction
The report Health For All Children (1989Children ( , revised 2006, recommends that vision screening in the UK should be undertaken to identify three key target conditions: 'amblyopia, strabismus and any incapacitating vision defect'. 1 This document provides a number of recommendations regarding the timing (age 4.5 years), profession of the tester (orthoptist-led service) and type of assessment (crowded linear visual acuity (VA) test). However, there is controversy regarding each of these recommendations as they all have implications on the efficacy of the programme. These include an impact on the sensitivity and specificity in detecting the conditions, a potential delay in diagnosis, subsequent treatment, and financial implications. While each component of the screening programme design contributes to its efficacy, the aim of this review is to focus on assessment methodology where the plusoptiX photoscreener device has been used in place of traditional screening methods. This review will evaluate available evidence relating to the viability of the device as an alternative to traditional screening methods.
VA tests are used widely in vision screening, but technological advances now provide an alternative methodology. For example, a photoscreener, developed in 1994 2 offered a method of rapidly determining refractive error. However, this device is coming to the end of its working life, due to non-manufacture and instant development film being no longer available. With continuing advancements in technology, not only have photoscreener devices evolved to incorporate automated interpretation of refractive errors, but also the ability to rapidly detect manifest strabismus allows their use by those without specialist ophthalmic training. The speed of assessment and ability to test a wide age range of children provides the opportunity for an alternative to VA tests, with the aim of detecting the same target conditions. This raises the question of its efficacy to detect the target conditions and whether it could be used within vision screening programmes. Support for this method of screening is increasing, with the American Association of Pediatric Ophthalmology and Strabismus (AAPOS) releasing a statement in support of the use of photoscreeners. 3 However, within the context of the American healthcare system and screening programme, children are typically screened as a quick additional test at the paediatrician's office. Therefore support for the use of photoscreeners may not be directly applicable to use in screening programmes in the UK.
The plusoptiX device is a photoscreener which can measure refractive error. Its accuracy has been discussed elsewhere [4][5][6][7][8][9] . However, the requirement of the device for screening purposes is to detect the risk of having the condition and not to measure the refractive error. Therefore this review will consider the efficacy of the plusoptiX at detecting the target conditions and amblyogenic risk factors, in comparison to current gold standard methods, and its potential contribution to an effective vision screening programme.

Search strategy
A search of Medline, Scopus, Web of knowledge, ProQuest, CINHAL, Cochrane Library, PhycInfo and The Orthoptic Journals and Conference Transactions Search Facility provided by the Directorate of Orthoptics and Vision Science at the University of Liverpool 10 was performed using free text terms with Boolean operators to structure the search around the terms: 'Vision', 'Visual', 'Screen*', 'Refractive Error*', 'Ocular Motility Disorders', 'Disorder*', 'Test*', 'Amblyopi*', 'Strabism*', 'photo*'.
Using the above search terms, 18 papers were retrieved and appraised. The most appropriate methodology to use for the comparison of a new method to an accepted 'gold standard' is a diagnostic test study. The features of these are described in the Critical Appraisal Skills Programme (CASP) guidelines for diagnostic test studies. 11 Of the papers appraised, three met the following criteria for use in the review: . A study of diagnostic accuracy determining the ability to detect amblyogenic risk factors . Subjects are from an unselected representative population, to replicate the intended purpose of mass screening . Independent, blinded comparison to a valid reference standard . The majority of those photoscreened should receive the reference standard and only those children who received both should be statistically analysed The initial intention was to only include those papers that met the strict criteria of the CASP guidelines. However, given the limited number of papers that met all criteria it was decided to expand the review to include an additional four studies which met all but one of the criteria. The only criterion they did not fulfil was that the subjects were not from an unselected representative population, but instead drawn from a patient population. While this results in a higher prevalence of amblyopia, resulting in spectrum bias, 12-14 they do provide information comparing test results on larger numbers with the conditions. These studies are all summarised in Table 1.
All the studies identified used the plusoptiX S04 (plusoptiX GmbH, Nürenberg, Germany), a handheld device which simultaneously measures refractive error, interpupillary distance, pupil size and eye alignment. The device is connected to a computer which, when used for screening, reports either a 'Pass' or 'Refer' response, with 'Refer' a result of detection of one of the target conditions, or when no reading was possible.

Impact of age on assessment type and efficacy
The key elements of the debate regarding the optimal age for vision screening are aimed at maximising detection at an age where (i) conditions are present, (ii) testability rates are high and (iii) the impact of the disease can be minimised. In support of screening at an early age, it has been demonstrated that when amblyopia is treated in the first three years of life, the occurrence of the disease in the population at the age of eight years is lower than if detection or treatment commences after age five. 22,23 In addition, treatment efficacy is better if undertaken at an earlier age, potentially reducing the time of occlusion by up to 2 months. 22 The arguments against screening prior to 4.5 years are the difficulty in ensuring that the child will attend the screening appointment, and the reduced reliability of any test results gained. It has been shown, for example, that whilst the ability to assess VA using Lea symbols is 76% at three years of age, this increases to 95% at age four. 24 Photoscreeners provide an alternative approach to  16 271 Ophthalmologist Ophthalmic examination identified 29 children with amblyopia or amblyogenic factors. PlusoptiX device identified 34 cases of which 11 were false positives. The device failed to identify 6 children with amblyogenic factors. Matta et al. 17 105 Ophthalmologist Ophthalmic examination identified 16 children to be referred with amblyopia or amblyogenic factors. The plusoptiX referred 17 children in total, including 2 false positives. The device failed to refer 1 case with amblyogenic factors. Matta et al. 18 153 Ophthalmologist From within a patient population ophthalmic examination identified 80 children with amblyopia or amblyogenic factors. The device suggested referral of 93 patients including 13 false positives. The device failed to detect 1 case with amblyogenic factors. Matta et al. 19 109 Ophthalmologist From within a patient population ophthalmic examination identified 75 children with amblyopia or amblyogenic factors. The plusoptiX device referred 78 including 4* false positives. The device failed to detect 1 case with amblyogenic factors. Matta et al. 20 151 Ophthalmologist From within a patient population ophthalmic examination identified 98 children with amblyopia or amblyogenic factors. The plusoptiX device referred 99 patients including 2 false positives. The device failed to detect 1 case with amblyogenic factors. Ugurbas et al. 21 182 Ophthalmologist From within a patient population ophthalmic examination identified 58 children with amblyopia or amblyogenic factors. The plusoptiX device referred 118 patients including 59 false positives. The plusoptiX device failed to detect 6 cases with amblyogenic factors.
*Personal correspondence with the study author.
screening and have been shown to provide repeatable results in young children (5.5 years) with a mean spherical equivalent of 0.03D (À0.62D to 0.68D) variation between two different examiners. 5 The quick acquisition of the measurements (less than 1 second) makes it useful in uncooperative children. 7

Referral criteria
The 'Pass' or 'Refer' response generated by the device is based on criteria relating to anisometropia, astigmatism, level of refractive error and anisocoria. Incidental findings such as strabismus, media opacities or ptosis can be detected by the device as a result of not acquiring a measurement, thereby prompting a referral. Misalignment of the eyes is displayed on the device as a series of red dots, noted manually in older versions, 25 whereas the latest (un-reviewed) version encompasses a 'symmetry of corneal reflex' assessment as part of the refer/pass criteria. 26 The referrals for strabismus and other problems in the studies were based on 'no reading possible -refer' and on refractive error. [15][16][17] The device will also fail to give a reading when refractive error is outside a range of À7.00DS to þ5.00DS. The photoscreener device referral criteria are based on a set of amblyopia risk factors, created by AAPOS. The refractive error components of the criteria state that the following are significant risk factors for the development of amblyopia in preschool children: 27 . Anisometropia (spherical or cylindrical) 1.50D . Hyperopia 3.50DS in any meridian . Myopia magnitude 3.00DS in any meridian . Astigmatism 1.50DC at 90°or 180°. Astigmatism 1.00DC in oblique axis (10°eccentric to 90°or 180°) The criteria for referral are customisable within the device with a number of variants in use: 25 the 'manufacturer' and 'Matta' criteria, which vary according to subject age, and the 'Arthur' and 'Barry & Konig' criteria (see Table 2). More recently, an article comparing refractive readings of the device to cycloplegic autorefraction/retinoscopy theoretically improved the sensitivity of the test whilst minimally impacting specificity (by adjusting the cut-off for hypermetropia to þ1.87DS and astigmatism to 1.12DC rather than 1.50DC). 8 However, this cannot be applied to the device, as the photoscreener has an accuracy of AE0.250DS. The impact of these different referral criteria was investigated comparing referral levels to a consecutive cohort and found a trend that high sensitivity resulted in low specificity. 25 The lack of consensus, as demonstrated in Table 2, illustrates better agreement for anisometropia and astigmatism, but with at least 2.00D variability for myopia and hypermetropia in the screening-age population. The relationship between the degree of hypermetropia and VA is variable, therefore resultant variability in referral criteria is unsurprising. However, there is a clear inverse relationship between increasing degrees of myopia and reducing VA, 28 but it is not clear what the justification is for differences of up to 2.50D in the referral criteria and how this contributes to the device efficacy. While the referral criteria are age based, no data were found which analysed the efficacy of the plusoptiX by age group.

Hypermetropia
The UK National Screening Committee specifies that the target conditions for vision screening encompass all refractive errors, including hypermetropia, but without defining magnitude. 30 While a VA test is accurate in identifying the effects of the majority of the target conditions, including myopia and astigmatism, 28 there is the potential for children with significant degrees of hypermetropia to remain undetected. This has been demonstrated recently in a study where levels of hypermetropia 5þ3.50DS or astigmatism of 1.50DC were not detected by a cut-off criterion of VA better than 0.200 logMAR, in children aged six to seven years using a linear logMAR test. 31 Given what is known about the developmental course of refractive errors 32 it is likely that the level of hypermetropia at age 4.5 would have been no smaller than at six or seven years of age and so, if it was not detected by a cut-off of 0.200 logMAR at that time, it could be assumed that the level of hypermetropia would not be detrimental to visual functioning. Evidence to support the inclusion of hypermetropia as a target condition, in the presence of good distance VA, suggests that not correcting hypermetropia may result in a detrimental impact on the child's educational attainment. However, this evidence is limited with a number of flaws such as non-consensus on the definition of hypermetropia, ethnic and social biasing and the type of cycloplegia used. 32 The counter-argument is that if the level of hypermetropia is not symptomatic or causing a detectable reduction in visual performance, it is unimportant and does not require identification. In addition to this lack of clarity regarding whether to prescribe, there is no consensus regarding the correction of hypermetropia, whether to prescribe the full amount or to undercorrect as full correction may impede the emmetropisation process. 32 The detection of hypermetropia of 54.00DS at age one and over, would indicate the necessity for refractive correction to be prescribed. 32 Therefore, with the ability to detect the presence of high amounts of hypermetropia, even without the accuracy of diagnosis, the plusoptiX would suit the needs of a screening programme. Based on the current evidence it is difficult to determine whether hypermetropia without amblyopia should be a specified target condition for vision screening. If the evidence did support its inclusion there is still the challenge of defining the referral criteria, specifically the minimum level of hypermetropia. The lack of consensus is clearly illustrated in Table 2, and the referral criteria of any study evaluating the efficacy of the plusoptiX must be determined when evaluating the outcomes.

Diagnostic accuracy
The performance of the studies is described in Table 3, with varying values of sensitivity and specificity to amblyogenic risk factors found. The majority of studies screened children aged three to seven years. However, one study 18 investigated an age range of 0.5 to 16 years with the comparator MTI study 2 screening children aged 0.5 to 5 years. The efficacy of the studies including younger age groups is not different to those centring on children aged three to seven years, suggesting age does not have a significant impact on device performance, due primarily to the non-invasive and rapid nature of the assessment. The ability of the device to correctly detect the presence of one of the referral criteria (sensitivity) ranges from 47% to 99%, the lower end of which resulted in 18 out of 34 cases being missed by the device, which were identified by orthoptic screening. 15 The ability of the device to correctly identify the absence of any of the referral criteria (specificity) ranged from 49% to 100% giving from zero to 63 (out of 124) false positive referrals, 21 where children were referred unnecessarily.
While the plusoptiX does not miss many cases with amblyogenic risk factors (as demonstrated by the low false negative rates in Table 3), the most common cause of a false negative result is from not detecting hypermetropia greater than 3.00DS. 25 The underestimation of hypermetropia detected by the device in comparison to cycloplegic retinoscopy, is most likely due to a myopic shift caused by accommodation. 4,5,7,9,33 Attempts to improve the accuracy of the device by eliminating the influence of any accommodation through the use of þ3.00D lenses or distance fixation, reduces the device's ability to take a reading. In relation to a screening programme the precise quantification of the refractive error is not essential, but the aim is to detect cases of reduced VA. From the published evidence it is known that the false negative cases had hypermetropia, 25 but it is not known whether the VA was within normal limits, and therefore it is difficult to extrapolate from these findings and apply to a screening programme.
From the studies listed in Table 3, the highest rate of false positives was 35%. 21 This high number (63) of false positives may be due to the demographic of subjects screened, as the population was those with intellectual disability. The use of this population may have biased the findings of the study and the results can not directly be considered in comparison to a population unaffected by spectrum bias. Discrepancy in sensitivity and specificity levels between studies could be due to a number of reasons such as the criteria used for referral, the person operating the device, the accuracy and content of the gold standard and the bias of the population being screened.
An additional study, 34 not included in the table, presents a very high rate of false positives (200 out of 260 referrals) that would undoubtedly be considered as 'harmful' in a screening programme A large number of children (n ¼ 996) were photoscreened; however only 38% of these received a 'gold standard' assessment, biased towards those children that had failed photoscreening. Therefore, while the finding of such a high rate of false positives suggests that the device would provide a high rate of over-referral, weaknesses in the methodology suggests that the data from the papers presented in Table 3 provide a more accurate reflection of the true accuracy of the device.

Maximising sensitivity and specificity
Only a small amount of evidence was identified that fulfilled the CASP inclusion criteria, 11 with only three studies replicating the screening environment. Based on this limited evidence, it is difficult to determine the efficacy of the device for a large scale vision screening programme. However, from the data available, the ability of the device to detect the presence and absence of the condition can be as high as 98% using the 'Matta'  35 However, just one false negative within the 399 who did not receive the gold standard would reduce the sensitivity from 100% to 96%, with a change in prevalence level of þ0.2%. This gives a slight rise in the study's prevalence level to 4.7%, which is much lower than in photoscreening studies, but with similar levels of sensitivity and specificity. A limitation of basing comparison on the prevalence levels of amblyogenic factors presented in Table 3 is that the prevalence data are not of a population-based study. The levels are much higher than those previously reported for the population (3-5%) and so the accompanying sensitivity and specificity levels may be reduced with a lower prevalence and an increase in false negatives and positives. 36 The sensitivity and specificity of any screening programme are affected by the choice of referral criteria, where an increase in sensitivity would result in a reduction in specificity. So what are acceptable levels of sensitivity and specificity for a vision screening programme? The balance between missing cases of disease and avoiding too high a number of false referrals is reliant on this. The ability to detect the presence and absence of disease by the 'gold standard' ophthalmic examination is assumed to be 100% in each case. As demonstrated previously, this assumption may be flawed. 15 However, there were no confirmed cases of a condition positive detected by the photoscreener that the clinician did not detect, suggesting that traditional screening is a suitable 'gold standard' for comparison. The figure of 100% sensitivity to amblyogenic factors could appear desirable; however a low specificity equates to many false referrals, increasing the demand on hospital time, finance and also causing undue anxiety or inconvenience for the patient/parent. Therefore, referral criteria need to be determined that balance the sensitivity and specificity, maximising detection while minimising over-referrals.

Application
The evidence presented here suggests that the plusoptiX device may have a role in vision screening, used under the same conditions as VA assessment. 6 Depending on the requirements of the screening programme, the photoscreener could replace the VA assessment element of the screening only, thus decreasing time requirements and allowing a greater number of children to be screened. When other tests, such as the cover test, are used in combination with the photoscreener, the sensitivity to detect strabismus is likely to increase, 37 validating its use as a tool during orthoptic vision screening rather than a replacement. Although the Vision In Preschoolers (VIP) study utilised an autorefractor, rather than a photoscreener, Paff et al. 7 have demonstrated similar rates of sensitivity and specificity between the two devices; therefore the results are applicable here. However, from the VIP data it is unclear which types of strabismus were undetected by the photoscreener. It could be surmised that if the response to the photoscreener indicated no strabismus, then it could be an intermittent deviation, and if asymptomatic is referral required? However, this is pure supposition and no conclusion can be inferred without the additional information about the strabismus types associated with normal VA.
As the device requires no specialist knowledge to use, and referral criteria can be set by the orthoptic lead, the photoscreening can be carried out by non-ophthalmic professionals, or the school themselves. One of the studies included in this review 16 used dental technicians to perform the photoscreening during dental health checks in schools. There were no noticeably large differences between the sensitivity and specificity figures of this and the other studies where the device was operated by ophthalmologists, orthoptists or orthoptic technicians.
In comparison to VA testing alone, with a reported positive predictive value (PPV) of 32%, 35 photoscreening alone of an unknown population has demonstrated a PPV between 73% and 100%. This suggests the probability that the photoscreener will detect a true positive result is higher than VA testing alone. This modelling of the diagnostic accuracy of the Snellen chart and crowded logMAR tests 35 suggests that to achieve a level of sensitivity and PPV equivalent to the photoscreeners' performance in the study by Ugurbas et al. 21 (47%), a crowded logMAR test would require a cut-off of 0.200 logMAR. The modelling suggests that to achieve a PPV of 100% using crowded logMAR to detect amblyogenic factors, a cut-off of 0.500 logMAR would be required, but this would reduce sensitivity to just 18%. This demonstrates the benefit of using the plusoptiX photoscreener over VA testing in isolation.
The plusoptiX manufacturer's website 26 is advertising the plusoptiX S09 screener with a new algorithm for detecting corneal reflection misalignment but no published research for this evolution of the device is currently available. A quote received in 2012 for NHS use of the S09 photoscreener was close to £9000. In areas with no screening programmes, the device may prove cost effective as a primary screening tool to detect amblyogenic risk factors in a school age population. While the device is simple to use, the referral criteria need to be devised and monitored, and referrals reviewed for accuracy, which, as advocated by the Hall report, 1 is best achieved by an orthoptist-led service.

Conclusion
There are a number of issues relating to vision screening, and therefore referral criteria, which apply to all vision screening approaches when determining the appropriate target conditions. Although there is no consensus on the optimum method for detecting amblyogenic factors, the plusoptiX photoscreener appears capable of reasonable sensitivity and specificity, if appropriate criteria are set. The data from the studies reviewed suggests the plusoptiX device offers a better balance between sensitivity and PPV than that suggested for a crowded logMAR test alone. Using the referral criteria set by Matta et al. 19 appears to offer the best balance between sensitivity and PPV and therefore the best performance for the detection of amblyogenic factors.