Visual screening: a comparison of the Keeler crowded logMAR and Snellen tests

Aim: To assess the effectiveness of the Keeler crowded logMAR visual acuity test in comparison with the Snellen test and to establish the optimum visual acuity threshold, using the Keeler crowded logMAR, for referral from the orthoptic secondary screening service to the ophthalmologist. Methods: A retrospective audit was carried out of community and hospital records of all referrals from the secondary vision screening clinics over the 2-year period from 31 July 2007 to 1 August 2009. In total, notes of 1035 patients were reviewed for this audit. The age at referral, visual acuity, vision test and primary diagnosis were recorded. Data were excluded if the patient: had records that were incomplete, had poor cooperation during assessment, had no formal visual acuity testing, or had been tested using the Cardiff Acuity Test or Kay Pictures. All Snellen visual acuities were converted into a logMAR equivalent to allow for direct comparison. Frimley Park Hospital’s visual acuity criterion for referral from the secondary screening service was a uniocular Snellen equivalent of poorer than 6/6 (equivalent) vision. Since the introduction of the Keeler logMAR tests, the criterion has been set at less than 0.100 for the crowded logMAR. Results: The false-positive referral rate was 15.094% (n = 32) for the Snellen test whereas for the crowded logMAR it was 17.949% (n = 7) at the established referral level of visual acuity. Conclusion: Such a high false-positive rate is attributable to our high visual acuity threshold referral criterion. Analysing the data available using different threshold levels of acuity for referral, it was found that accuracy can be improved, whilst maintaining maximum sensitivity, by adopting 0.175 on the crowded logMAR test as the visual acuity referral criterion and also ensuring that all patients with an intraocular difference of 0.100 logMAR are referred.


Introduction
Following the introduction of crowded and uncrowded logMAR visual acuity tests into the orthoptic secondary screening service in September 2008, an audit was conducted to see whether, when using the crowded logMAR test, the visual acuity threshold used for referrals was appropriate or too stringent. A logMAR threshold of less than 0.100 was defined as the appropriate level for referral to the ophthalmology department for routine refraction, fundus and media check followed by orthoptic assessment 6-8 weeks later.
In the UK, the National Screening Committee produced a report in 2006 1 and made a series of recommendations on vision screening. The Committee agreed with the findings of the Hall Report, confirming the optimum age for conducting screening for visual defects is at 4-5 years old due to the ability of this age group to 'perform visual acuity tests more easily than 3-year-olds'. 2 The service run by the orthoptic team at Frimley Park Hospital assesses children ranging in age from 1 month to 12 years. These children are referred from general practitioners, health visitors, school nurses and paediatricians because of parental concern, suspected strabismus, family history of eye problems, failed primary school screening tests and learning difficulties. This means that a wide range of patients are seen within the service, not all of whom are able to undertake letterbased tests. Because of the nature of this audit, only children able to undertake Snellen or crowded logMAR tests are included within the results. This makes our patient group similar to that suggested within the Hall Report.
A literature search was conducted to establish the recommendations of previous studies. Table 1 outlines these.
The aim of the audit was to determine whether the threshold level of 0.100 logMAR for further ophthalmology assessment, following vision screening, was appropriate, and to compare Snellen acuity chart and crowded logMAR test referrals.

Methods
Frimley Park Foundation Trust provides an orthoptic secondary screening service, allowing health visitors, school nurses, occupational therapists and paediatricians to refer patients with suspected strabismus, parental concerns, family history, reduced vision at school screening or optometrist appointment, and some children with learning difficulties. The children seen within the screening service who have learning difficulties are mainly seen for assessment with coloured overlays, but they do have a full orthoptic assessment and therefore have been referred if other problems are identified. If a need for further assessment is established the patient will be seen by an ophthalmologist at the local hospital ophthalmology department.
Prior to the introduction of the Keeler crowded logMAR test into the screening service, the letter-based test used was the Snellen acuity chart. The 'pass' rate for the Snellen test was 6/6 equivalent, for all children. Due to the multitude of screening locations used, room length varied from 3 m to 6 m, meaning that the pass criterion had to be set at 6/6 'equivalent'. The visual acuity pass rate threshold set out by the orthoptic department for use with the crowded logMAR test was better than 0.1 logMAR. Children would need to achieve this visual acuity level uniocularly, with no significant intraocular difference (no greater than 0.075 logMAR) to be deemed to have no significant vision problems. It must be added that any child who achieved the required visual acuity threshold but had been found to have a large phoria, squint, ocular motility abnormalities, inability to overcome a 20 dioptre prism consistently, or other ocular abnormality (anisocoria, ptosis, etc.) would still have been seen within the ophthalmology department.

Audit design
This audit looked at all the screening referrals from 31 July 2007 to 1 August 2009. The notes of all patients who met our inclusion criteria were retrospectively reviewed. The age at referral, vision test conducted, monocular visual acuity at referral and at three subsequent follow-up appointments, primary diagnosis and any treatment conducted were recorded.
To standardise the data used for audit purposes, inclusion and exclusion criteria were set out.

Inclusion criteria
The following patients were included in the audit: . Those referred from the Frimley Park Hospital secondary vision screening service. . Those referred on for further assessment from the ophthalmologist (refraction, fundus and media check). . Those tested using one of the following tests: crowded logMAR or Snellen visual acuity chart.

Exclusion criteria
The following patients were excluded from the audit: . Those whose visual acuity met the threshold required in each eye. . Those referred to a different Hospital Trust. (Due to the wide range of locations where secondary screening is conducted, some patients are sent to other hospitals in the Surrey area which are not run by Frimley Park Foundation Trust.) . Those initially referred directly to the hospital eye department by general practitioners, paediatricians and opticians. These patients will undergo full orthoptic assessment prior to being seen by the consultant ophthalmologist, but will not have been seen in the screening clinic. . Those tested using the Sheridan-Gardiner, uncrowded logMAR, Cardiff Acuity Test, Kay Pictures and those who did not have a formalised visual acuity test. . Those who were unable to cooperate when being screened. . Those who were subsequently discharged due to nonattendance. . Those for whom relevant information in the notes was lacking.
To enable statistical analysis, all vision test results from the patients included within the audit, from both screening and hospital visits, were converted into a logMAR equivalent, and put into a Microsoft Excel spreadsheet. The data were anonymised for the analysis. More detailed analysis was then done using the program SPSS.

Results
Of the 1035 cases referred from the secondary vision screening service, a total of 251 were suitable for analysis for the purpose of this study. The cases analysed comprised 131 males and 120 females with an age range of 3.428 years to 11.433 years. The mean age of participants in this study was 5.552 years. All the cases suitable for analysis included vision tested with either the Snellen or Keeler crowded logMAR test. The remaining 784 cases were excluded for the following reasons, as outlined in our exclusion criteria: . 43 cases (5.485%) where the notes were either unavailable or the patient had not attended their ophthalmology appointment; . 406 cases (51.786%) referred to another local trust; Due to the crowded logMAR test being used only within the last 9 months of the study period, only 15.538% (n = 39) of patients were assessed using this test, whereas 84.462% (n = 212) were assessed using a Snellen test.

Analysis of referral criterion
Based on the referral criterion of 0.1 logMAR the rate of false positives was 15.094% for the Snellen test and 17.949% for the crowded logMAR test ( Table 2). If 0.200 logMAR is defined as the referral criterion, as per the Hall Report, the percentage of false positives using the Snellen test would have reduced to 7.075%, i.e. n = 15 out of 212 cases tested, and for the crowded logMAR test to 7.692%, i.e. n = 3 out of 39 cases. However, as the visual acuity referral criterion at Frimley Park Hospital was set at such a high standard it was possible to analyse the data from this retrospective audit to ascertain those with a visual problem that would have been missed (false negative) had a referral criterion of 0.200 logMAR been used. This showed that at this new threshold level the Snellen test produced a higher rate of false negatives (19.340%) and the crowded logMAR test a much lower rate of false negatives (10.256%), as shown in Table 3.

Determining the level of accuracy by adapting the visual acuity referral criterion
In this audit, by using the Keeler crowded logMAR test and by altering the visual acuity level of the referral criterion, the highest level of accuracy can be achieved using 0.175 logMAR, as illustrated in Table 3. Note that the false negatives were derived using the original data from the audit, by determining how many visual defects would have been missed if the visual acuity referral level had been changed.

Discussion
This retrospective audit was instigated to assess the impact of our change in practice.
Obviously referrals from the orthoptic secondary screening service were not solely based on the visual acuity level, as a full orthoptic screening assessment was performed. Assumptions have had to be made in order to allow a statistical analysis to be performed. The first assumption is that the screening assessment was thorough and accurate, as it included case history, visual acuity, cover test, ocular movements, convergence, stereotest and either the 20 or 15 dioptre test. It was also assumed that there were no false negatives among those discharged on their screening appointment.
For the purpose of this audit we have concentrated on the visual acuity levels of all those referred to the ophthalmologist. By altering only the visual acuity level referral criterion, we were able to ascertain the optimum visual acuity threshold for referral. Due to the study design, there is a skew in the numbers in the crowded logMAR cohort. This is due to this test being introduced to the screening service at Frimley Park Hospital in September 2008.
The results indicate that the optimum visual acuity for use as a referral threshold is 0.175 logMAR, which supports the findings of Stewart et al. 3 and corroborates the Hall Report. Using a logMAR level of 0.175 is less stringent than the criterion currently used for the Frimley Park secondary screening service (0.100 logMAR), and would explain the relatively high numbers of false positives referred by the orthoptic team. To further refine this criterion, the audit found that as well as referring all of those not attaining 0.175 logMAR, referring any patients with an intraocular difference of 0.100 logMAR or more, no matter what the visual acuity level in the worse eye, significantly improved the sensitivity of the test from 0.906 to 0.967. It must be stated that as Stewart et al.'s study was prospective in design it therefore had more ability to control the variables, as well as being able to be specifically designed to overcome issues that this audit was affected by. Stewart et al.'s research looked at a screening service run by school nurses, whereas Frimley Park Foundation Trust's vision screening is run by the orthoptic team only.
There are clear weaknesses in our audit design. Firstly the nature of retrospective studies means that data are not as strictly controlled as they would be in a prospective design. The results were collected by a number of orthoptists, over a 2-year period in multiple locations, which will have allowed variation in not only the room the tests were conducted in, but also the illumination level of the room. Ingram et al. 4 showed that lack of standardised conditions affected the specificity of visual acuity measurements. Visual screening: a comparison of the Keeler crowded logMAR and Snellen tests A change in policy also occurred in January 2009, as a result of which a maximum of two visits could be arranged in the screening clinic prior to a decision being made to refer or discharge. This has meant that a number of patients will have been referred into the ophthalmology department due to poor cooperation/concentration or behaviour. Some of these children will have been screened out by our exclusion criteria, but if the orthoptist has not documented behavioural issues on the screening notes, then these patients will have been included within the cohort.
Due to the large variation in the lenses prescribed, those patients who were not prescribed glasses at any point in their treatment have been classed as false positives. In cases where the patient was advised no current glasses prescription was required, a vast majority failed to attend their subsequent orthoptic appointment. Therefore true analysis of whether their visual acuity subsequently increased to a 'normal' level is unclear.

Conclusion
Our audit considered the appropriate threshold level when making a vision assessment using a crowded logMAR test, and at what level further orthoptic and ophthalmic assessment is required.
Our results support those of Stewart et al., 3 with a visual acuity threshold of 0.175 logMAR improving accuracy of referrals from our secondary screening service. Our results also showed that any child who has an intraocular difference of 0.100 logMAR or more also needs referral, even if their visual acuity in the worse eye is greater than 0.175 logMAR.
Following the results of this audit, the Frimley Park Hospital secondary screening service will be changing the threshold for referral to uniocular visual acuities of less than 0.175 on a crowded logMAR test.