The overall aim of the I-TRAC study was to determine the feasibility and acceptability of digital technologies to monitor glaucoma at home and inform the possible need and design of a definitive evaluative study. Through a multiphase mixed-methods study design, I-TRAC sought to answer four interlinked research objectives, and the data addressing this have been presented in the preceding chapters. This chapter brings together the key learning from the study in the form of a statement of feasibility for a future evaluative study. We will draw largely on the empirical data generated from the study findings; however, we will also include important ‘lessons learned’ captured as notes and reflections throughout the duration of I-TRAC.
Feasibility of a future evaluative study of clinical and cost-effectiveness of digital technologies for home monitoring of glaucoma
The data from across all phases of the I-TRAC study were mapped to the AdePT framework in order to establish the feasibility of a future evaluative study of the clinical and cost-effectiveness of digital technologies for home monitoring of glaucoma. The AdePT framework contains 14 methodological ‘issues’ to be evaluated in feasibility studies or pilot trials.51 The results of mapping the data from I-TRAC to the 14 methodological ‘issues’ in AdePT are presented in Table 28.
Summary of findings against the AdePT 14 methodological issues for feasibility studies
We did not design I-TRAC to inform sample size calculation, randomisation procedure, or masking procedures – the latter because masking would not be possible in a future study due to the nature of the intervention. The issues relevant for I-TRAC will be discussed below.
Eligibility
Findings from the survey (see Chapter 2) and interviews and focus groups with clinicians (see Chapters 3 and 4) highlight a lack of agreement on which patient population would be most suitable for using digital technology at home to monitor their glaucoma. While generally there was agreement that high-risk patients would not be appropriate for home monitoring as a replacement for face-to-face examination, there was support for home monitoring with digital technology of low-risk patients. The possible benefits of adding home monitoring to current face-to-face clinic visits in high-risk patients were mentioned by some clinicians. This variation in agreement among clinicians was also evident from the way in which recruitment was approached across the three sites involved in the I-TRAC intervention study, with some staff approaching higher-risk patients for inclusion while others only approached those who are already being monitored through virtual clinics. It is also worth considering that different populations may be best suited to one or other of the interventions. For a future trial to be feasible, further clarification and definition of the patient population, and whether home monitoring would be used as a replacement for or an addition to the current model of hospital-based eye care, would be required such that there was agreement among the clinical community.
Findings from the qualitative interviews with expert glaucoma clinicians (see Chapter 3) also highlighted concerns in relation to patient characteristics that may influence eligibility and/or opportunities to participate. These included: accessibility, language, education, and technical abilities, which could lead to ethical consequences; how to select and prioritise patients for home monitoring (particularly where there will be resource constraints); impact upon those who are not selected for home monitoring; and the risk of creating a two-tier system (as some may have to contribute financially).
Recruitment
In-home Tracking of glaucoma: Reliability, Acceptability, and Cost recruited 93% (42/43) of its proposed sample size within the original recruitment window (1 October 2021 – 31 August 2022). The recruitment approach varied across each of the three sites. Originally it had been proposed that each site would recruit 15 patient participants each across three home monitoring cohort periods, with 5 participants in each of the three cohorts. However, given delays in approvals for one of the sites, recruitment was predominantly across two sites with the third recruiting to the last two cohorts. Site staff reported in interviews that I-TRAC was easy to recruit for due to significant patient buy-in to the interventions. However, staff did report perceptions about the samples recruited not being representative of the typical glaucoma population. This was predominantly linked to disease stage, age and ethnicity.
The representativeness of patients in DHT trials was raised in the external researcher team interviews; they believed that trial populations often over-represented individuals who were white, younger, highly educated and had previous access to technology and the technological confidence to participate (see Chapter 5). Linked to this, it is important to note that 94% of patient participants in I-TRAC identified as white and only two (5%) reported not being a current user of a smartphone or tablet. Any trial evaluating home monitoring technologies should use strategies to target eligible patient groups that are not currently recruited to these trials.
Consent
Two centres prescreened participants before approaching, which resulted in 100% of participants consenting to participate. In the third site, there was a much lower proportion of participants who consented (16%). This was likely due to this site not prescreening but rather approaching all potentially eligible glaucoma patients in clinic waiting areas – which may be the case for a future large evaluative study. Where provided, reasons for non-participation were linked to a lack of confidence or interest in technologies. This was supported by findings from external researcher interviews which highlighted low digital literacy and a lack of social support as barriers to participation in trials of DHT more broadly (see Chapter 5). Patient participants reported reasons for participating in I-TRAC to be based on recommendations of their clinician but also were often motivated by their own history of glaucoma and interested in engaging with new technology and/or the research project. These motivations may also translate to decisions to participate in a future evaluative study; however, the influence of randomisation and how it may modify those decisions should also be considered.
Adherence to intervention
Patient participant adherence to the interventions was good overall, with 67% (n = 28) and 63% (n = 26) adhering to the tonometer and app, respectively (i.e. > 80% of measurements completed), and 55% (n = 23) adhering to both interventions. In the interviews, patient participants stated that remembering to conduct the monitoring was not difficult and they incorporated it into their established routines, pairing it up with taking medications or self-care activities. As the monitoring period progressed, they individualised the process (e.g. location to ensure light levels, or aid to enable steady motion) such that they could reliably achieve reading in less time or with fewer attempts. The ability to access the readings from the devices may also have enhanced adherence due to the positive feedback on the behaviour. A familiarity with technology, prior experiences in monitoring health, and control over their health were factors that influenced engagement with the intervention. As part of the study design, I-TRAC included reminder e-mails or texts (for which the patient participants expressed their preference) to be sent weekly to participants to remind them to monitor to further enhance adherence. In some cases, the e-mails and texts were being filtered into spam folders or blocked by providers. This would need to be addressed for a future evaluative study.
The overall adherence rate is especially positive given the high prespecified definition of adherence that the study implemented: > 80% of all weekly measures conducted across the 3-month monitoring period. Definitions of adherence for complex interventions such as these DHTs may be expected to be lower than our prespecified definition of adherence. Further consideration of what may be an appropriate adherence level for interventions of this type is critically important. For any future evaluative study, it would be important to consider the likely trade-off between duration of monitoring and intervention adherence. Patient participants suggested that to minimise anxiety through frequent measurement, the monitoring should be restricted to the necessary requirements – which they deemed to be once a week.
Acceptability of intervention
Overall affective attitudes of patient participants, site staff, and expert glaucoma clinicians in relation to beliefs about the interventions were positive; they cited reduced demand on the hospital-based clinical service as a main benefit. From a patient participant perspective, 71% stated they were highly or very satisfied with the home monitoring. In interviews, they reported that overall, they were able to engage with the new home monitoring technology and appreciated its potential impact on their health and the eye healthcare system. A notable subset of participants had persistent difficulties with the device (the tonometer in particular) that limited their engagement and resulted in some of them doubting the feasibility of implementing home monitoring going forward, whereas engagement with the app seemed more acceptable, reportedly due to familiarity. Additionally, there was a large proportion (48%, n = 20) of the intervention study participants who required at least one additional contact with the site staff to resolve issues that largely related to app access issues or difficulties using the device. Linked to the difficulties using the devices, both patients/participants and site staff interviewed raised the issue of a need to improve training, both of staff and of patients, for any future study. Yet on the whole, relevant stakeholders were cautiously optimistic about the interventions and their potential for benefit.
It is worth noting that the broader global context for I-TRAC was its delivery within the COVID-19 pandemic and related lockdowns. This meant that many patients were being advised to stay away from hospitals but were also becoming more familiar with technology. This may have also enhanced perceptions of acceptability of this type of intervention as a mechanism, to support recovery of the health service post lockdown.
A key consideration for a future trial is to further define the intervention. Within I-TRAC, the digital technologies were considered collectively as the intervention. However, findings from the qualitative data suggest there were preferences for one of the technologies regarding ease of use and readiness for use in practice. Some reported that only one intervention should be evaluated in a future study, which may have been linked to the OKKO app not being exactly fit for purpose for glaucoma. Other aspects relating to intervention maturity were linked to concerns about the volume of data this type of activity will generate and how this will be managed. The use of AI to support systems that can manage this volume of information and identify troubling readings and trigger additional alerts would also be required. Linked to this, obtaining clinical agreement to define the ‘trigger’ would also be required.
Cost and duration of intervention
In-home Tracking of glaucoma: Reliability, Acceptability, and Cost did not conduct an economic evaluation within the intervention study; however, it did develop a conceptual framework for the future economic evaluation through a review of existing models and combined this with data collected from participants and staff on resource use and patient preferences. Overall, given the complex care pathway and insufficient studies in the literature on glaucoma home monitoring, we recommend using a step-by-step approach leading to a meaningful economic evaluation in a future study. Categories of intervention costs of glaucoma home monitoring identified include equipment cost, patient training, ongoing patient support during home monitoring, potential spill-over costs such as hospital visits triggered by high IOP readings, costs of data integration to the existing medical records, and evaluation by AI. Glaucoma is a chronic condition, and relatively high initial costs can be balanced by long-term benefits. Therefore, long-term follow-up or modelling extrapolation beyond the evaluative study follow-up should be considered.
Perceptions of intervention cost-effectiveness were also explored in interviews with expert glaucoma clinicians (see Chapter 3). These clinicians had mixed views: some believed the interventions would be cost-effective but a majority believed this approach would not be good value for money, with concerns about initial capital cost of equipment and associated maintenance and support (e.g. staff time for training) costs. While these concerns were raised within the context of service use, they are also valid for a future evaluative study.
Outcome assessment
We did not attempt to analyse in detail data on clinical outcomes, as our goal was to focus on feasibility and acceptability issues. The majority (40/42, 95%) of patient participants attended follow-up data collection at the final clinic visit at 3 months. Travel costs for patients to attend this follow-up visit, which was not part of routine clinical care, were provided and this may also have acted as an incentive to return. For any future trial it is worth considering the duration and timing of follow-up, and any necessary co-interventions that may be required to maintain data completion and return. I-TRAC asked patient participants to monitor their glaucoma at home using the digital technology for 3 months. This allowed assessment of acceptability and feasibility. It would be likely that a future trial may expect a longer duration of follow-up, with 6- or 12-month or longer time points to complement the clinical guidelines for in-hospital monitoring of glaucoma.4
Selection of most appropriate outcomes
The objectives of I-TRAC were to assess the outcomes of feasibility and acceptability regarding digital technology for home monitoring glaucoma. The clinical outcomes collected at baseline and 3-month follow-up centred on IOP and VFs. We collected IOP and visual function data weekly during the home monitoring period. Clinical outcome data were collected and reported for descriptive purposes only, with no statistical testing undertaken. One of the site staff interviews felt that the outcomes being collected are of benefit to different subgroups of glaucoma patients and those who would benefit from both were in the minority. This links back to eligibility and who the patient population are. It is worth noting that IOP is the most frequently selected primary outcome in glaucoma trials, with over 90% of all glaucoma trials registered reporting this outcome as the primary end point, and is considered a core outcome.82,85
It is also worth considering the secondary outcomes required for a future evaluative study. The qualitative research generated data from clinicians that identified a range of important additional outcomes, including: improved management from increased data collected, detecting progression quicker and preventing sight loss (either directly from monitoring or indirectly due to clinicians spending more time with high-risk patients in clinic); patient anxiety; and reduced travel. In addition to the resource use categories such as time for patient training, ongoing patient support, and healthcare contacts triggered by information from the home monitoring devices, a future study should consider health-related (e.g. QALYs) and non-health-related QoL outcome measures. Patient participants stated convenience as a benefit of home monitoring and this is a possible source of non-HRQoL utility; however, they also mentioned concerns about the potential lack of reassurance from clinicians under home monitoring. These are examples of possible sources of utility that should be explored further when deciding the outcome measures for an economic evaluation within an evaluative study.
Retention
The I-TRAC study retained 39 (93%) patient participants across the 3-month monitoring period. One of these participants had withdrawn from completing the monitoring but completed all data collection. Two participants withdrew from all study procedures due to events external to the study (one had a stroke and was unable to continue to complete the study and the other Bell’s palsy). As mentioned previously in relation to intervention adherence, if the monitoring period were to be extended, it is important to consider the trade-off between duration and completion of follow-up. In I-TRAC patient participants were asked to return to the clinic (a research visit not part of standard routine care) at 3 months at which they had clinical measurements taken and completed additional follow-up data collection. As such, timing and mode of follow-up require further consideration for implementation in a future evaluative study. In addition, it would be important to consider any negative affect from the ‘control’ group and how this would need to be mitigated during design to ensure there is no differential retention between trial arms.
Logistics of a multicentre trial
All three I-TRAC sites were keen and engaged, and this would need to be replicated for the scale-up of implementation of I-TRAC across multiple centres. External researcher interviews (see Chapter 5) highlighted that a lack of evidence of effectiveness of technology can contribute to a lack of staff buy-in at sites. Therefore, it would be important to ensure that any evidence that is available on efficacy of interventions is made available to sites to support buy-in. Having a DHT champion with experience of trials of this type, and selecting sites with confidence in technologies, would also be ideal for supporting delivery.
Suggestions for improvement to training that would be relevant for a multicentre trial included: identification of a dedicated experienced clinical site trainer; more detailed training for site staff such that they could better address patient concerns; multisite training days for staff to get together to complete training; and for patients, group training as an efficient approach for sites but also to provide social opportunities for patients which may be motivating. Having a technical support line available for patients such that they do not have to use site staff time with technical queries about the interventions was also suggested as an improvement.
One of the key logistical considerations for a future trial relates to the large volume of data that would be generated from a weekly (or similar) data collection schedule. Consideration would be required for how to review and act on the data (i.e. what measurement would trigger a patient to be seen in hospital) received from the devices in a future trial. In addition, the time required for review of data processing etc. would also need to be factored in. It was suggested that it would be valuable to have an automated algorithm that would highlight abnormal findings, to minimise the need for clinicians’ time to review data. The additional and often complex movement and storage of patient data which frequently occurs in digital home monitoring studies can provide challenges, with each geographic regional approval body having different information governance, research, and development processes which required factoring into delivery timelines and resource requests. The impact of differing policies regarding regulatory approvals did cause delays in opening sites and additional administration workload for the I-TRAC study team. For a future evaluative study, exploring local policies before recruiting sites is essential. These challenges with approvals, and specifically slow systems not built for DHT trials, were also noted in the external research team’s interviews (see Chapter 5). Solutions proposed were the need for more agile regulatory systems with the use of standardised approaches for ethical issues posed by DHT trials, and institutional digital support.
All components of the protocol work together
There were a number of protocol amendments during the study, and these are documented in Appendix 6. Overall, the I-TRAC study as delivered was fit for purpose and, as site staff reported in the interviews, the components worked well together. The study was perceived by site staff as low burden for both staff and patients, and largely easy to run and deliver. There were, however, some aspects that could be improved. The main one of these, a recurring theme, was the training of staff and patients to use the interventions. Additional opportunities for site staff to learn about the technologies and how to train patients in their use, possibly through a ‘see one, do one, teach one’ model, would be favourable.
Several key features of the interventions also arose. The first, and most notable, was the change in the app-based intervention from the MRFs VF technology to the OKKO visual function app. This was due to the MRF not being CE marked at the time of study approval (it has since obtained a UKCA mark) and as such being deemed a medical device by the MHRA, timeline for approval of which would have threatened the overall delivery of I-TRAC. Therefore, after a review of existing app-based methods for measuring VF or visual function (see Appendix 3), the CE-marked OKKO app was deemed fit for purpose. Our clinical experts suggested that the OKKO app would not be suitable for monitoring disease progression in glaucoma as it is measuring central visual function. Clinically, the MRF or another app designed to measure peripheral vision is the better suited technology, and while I-TRAC did not specifically assess acceptability, it did assess the general acceptability of tablet-based apps (which is also the MRF mode of delivery). In addition, findings from the patient participant interviews suggested they perceived the OKKO app to be less relevant to their glaucoma, which may also explain the slightly lower adherence to the OKKO app compared to the tonometer, which was reported as harder to use.
Additional unforeseen intervention issues with delivery also arose related to delays in iPad supply due to chip shortage and a surge in demand (due to remote working) which also resulted in an increase in cost. In one site, chargers for devices were lost, and due to not having replacements immediately available this led to delays in recruiting patients. Finally, there were two issues in relation to use of batteries. The first related to batteries running out quickly in the tonometers, which then required a specialist screwdriver to open the battery compartment (that had to be ordered from the distributor), and only specific branded batteries could be used. The other issue was mailed devices being returned due to batteries being included inside the device that were required to be sent separately, again leading to slight delays with recruitment.
As highlighted in interviews with the external research teams (see Chapter 5), I-TRAC also faced some minor challenges with commercial partners. At study outset, establishing realistic expectations regarding timely access to data and whose responsibility it was within the partner organisation to provide would have been helpful.
These lessons will be incorporated into any future evaluative study design.
Research findings in context: relevant evidence published since I-TRAC was commissioned
Since the commissioning of I-TRAC in 2019, a small number of relevant papers reporting on acceptability testing of digital technologies for home monitoring of glaucoma have been published. Unlike I-TRAC, these studies were not designed to explore the feasibility of a future evaluative study; however, the studies’ findings were mapped against the AdePT issues to allow any additional insights to supplement the findings from I-TRAC and inform future decision-making. The studies were conducted in four geographical settings: Slovenia, UK, Australia and USA.37–39,86 They used a range of methods to determine acceptability and feasibility of measurement for a single intervention across various technologies measuring IOP (iCare ONE HOME) and VF (Virtual Field, Eyecatcher, MRFs), with only one of the studies evaluating both IOP and VF technologies. One study did not require patients to monitor at home but rather assessed acceptability within a hospital setting.37 Details of each of these studies are provided in Table 29. Other studies investigating the accuracy and reliability of home monitoring technologies for glaucoma have also been published.87–90 These studies are not discussed here as they did not explore patient and/or clinician acceptability.
Findings from recent glaucoma DHT studies compared against AdePT issues
Studies were broad in their eligibility criteria, including patients with POAG, OAG, OHT, and glaucoma suspects. Sample sizes varied, ranging from 20 to 117, with no information provided about number of eligible patients approached or reasons for declining participation. Studies reported participant age and sex, but not ethnicity. Durations of home monitoring included 1 week, 14 days, 6 weeks, and 6 months. Adherence to the home monitoring interventions under investigation was good, ranging from 77% up to 100%. Retention to the studies was also good, with the lowest level being 81%.86
The UK-based study monitoring VF at home reported that patients were widely accepting of home monitoring, and some found it empowering to be involved in their care.38 Home-based VF monitoring in glaucoma patients was tested and indicated many potential benefits. This included improved patient focus due to calmer home testing environments, increased compliance with testing, and burden removal of attending appointments.38 However, there was an associated concern with performance pressure and anxiety surrounding constant awareness of symptoms.38 The study which explored the use of the MRF app did not directly explore acceptability, but it did investigate reasons for non-adherence. These included reasons related to: the technology being too difficult to use; logistical IT reasons; too much effort to participate; no interest or motivation to participate; a deterioration in health; and competing life demands.86 Hu et al., conducted a mixed-methods feasibility study in which the acceptability of home tonometry and home perimetry was assessed within glaucoma care in the USA.39 They demonstrated that the majority of patients found the devices easy to use and acceptable.39 Patients in the study reported a range of potential personal advantages such as convenience, enhancing the relationship with physicians, disease prevention and empowerment. Patients also recognised personal advantages of the intervention, such as the feedback to ensure appropriate use; and advantages for the overall service, such as additional data for disease management.39 However, there were also challenges or concerns in relation to using the home monitoring technologies, such as concerns about accuracy, technical difficulties, test fatigue and frustration and, more broadly, the impacts on the relationship with their physician. Patients also raised that, when considering future application, there may be patient groups who are particularly suited to home monitoring, such as those who are underserved, older, glaucoma suspects, or children.39
In addition to these studies exploring acceptability, a systematic review of glaucoma home monitoring interventions was published in 2022.91 The review indicated that self-led monitoring has promising potential for the future but found that current literature provides insufficient relevant evidence to fully support a home monitoring model – indicating room for further work.91 These findings were primarily due to a small total sample of conducted studies that were performed with a highly specific target patient group, decreasing the overall generalisability of results. Additionally, technical standardisation issues regarding home monitoring measurements, such as control of screen distance from patient and calibration requirements, were identified across numerous studies, adding a supplementary barrier to overcome for generalisability. This review demonstrated that patient-led home monitoring may not replace clinic care but may provide great value in certain situations, such as pandemics.
There are additional studies worth highlighting with findings that are also relevant when considering the overall results of I-TRAC. Firstly, Ballouz et al. explored whether glaucoma screening could be conducted remotely in two Michigan community clinics.92 The study found that patients from poorer demographics identified transportation, cost, trust, and lack of personal knowledge as barriers to glaucoma care.92 This was particularly prominent among those from ethnic minority groups. These findings from glaucoma screening research may also translate to a home monitoring setting, and further support the need to explore acceptability among the glaucoma populations not represented in I-TRAC and who are underserved by research. It is also important to note the MONARCH study, which has evaluated the diagnostic accuracy of self-led monitoring within the management of neurovascular age-related macular degeneration, another chronic ophthalmology condition.93 The study team introduced two digital (iPod)-based home monitoring tests and one paper-based journal to patients. In addition, the MONARCH team conducted interviews with participants, close family members, and healthcare professionals to explore acceptability and adherence.94 As with the home monitoring studies in glaucoma, the findings indicated that patients were largely accepting of home monitoring in this context and viewed it as relatively straightforward and low burden, citing the benefits to reduce clinical visits as a motivator. There was also recognition of the essential requirement for effective training and ongoing support to ensure those less digitally adept were supported.94
Finally, it is worth highlighting evidence from a recent overview of qualitative reviews that synthesised evidence from studies of digital health experiences reported in chronic disease management.95 Evidence was synthesised from 22 systematic reviews from across mental health, cancer, diabetes, chronic obstructive pulmonary disorder, chronic pain, cardiovascular disease, irritable bowel disease and combinations of chronic disease reviews. Several common themes across the conditions were identified and categorised through nine domains:
Participation and engagement – references strong usability and engagement balanced against a reluctance to use digital technologies when the former is not considered.
Trust, confidence, and competence – users felt reassured, but for those with technology illiteracy there was a perceived lack of control.
Perceived value, perceived effectiveness, transaction cost – in respect of gains afforded by digital technology but could also be lost due to burden of data entry requirements.
Perceived care quality – which required tailoring and motivation.
Barriers and threats – of the technologies’ risks and challenges.
Health outcomes – improved capabilities about self-management.
Relationships – improved healthcare professional relationships, but interpersonal aspects of in-person care lacking.
Unplanned benefits – patient empowerment.
Diversity of experiences – highlighting condition-specific experiences or ambivalence of experiences.
Overall, this overview of reviews highlights the need for digital technologies to be developed through a co-design model, ensuring the ‘consumer’ has meaningful involvement in the planning and design phases of products and developments.95 This could also extend to the planning and design of evaluative studies of these technologies, ensuring patients are actively involved in the design of trials that plan to evaluate these interventions.
Findings from these primary studies and literature reviews echo many of the findings from I-TRAC and suggest that home monitoring of glaucoma is largely acceptable, adherence is adequate, and delivery within the context of the study was feasible. However, there are several unknowns in relation to scalability and sustainability for a future evaluative study. A recently developed conceptual framework to support implementability of healthcare interventions recommends starting with assessing acceptability (across a range of stakeholders), then moving to explore fidelity (which can include adherence but also whether the intervention was delivered as intended), and then assessing feasibility of delivery.96 These factors should be investigated, iteratively, with stakeholders during intervention development and early evaluation. Then, when initial evidence of effectiveness has been established, scalability and sustainability should be considered. It is important to note that acceptability, fidelity, and feasibility are dynamic concepts that likely require reconsideration when scaling to different settings (e.g. research to healthcare practice) or populations (e.g. adults to children) over time.96
Strengths and limitations
One of the main strengths of I-TRAC was its ability to recruit across three geographically distinct locations, with varied patient pathways and routes to patient recruitment. This allowed ‘testing’ of study processes in a range of contexts, providing greater reassurance of acceptability of study processes when considering scale-up to a larger study. The inclusion of perspectives from a range of stakeholders (expert glaucoma clinicians, patient participants, site staff, and external research teams with experience in DHT) was also a key strength to the study – and not evidenced in the literature to date. Encouraging a plurality of perspectives should ensure the main opportunities and challenges of future research and delivery of DHT for glaucoma are evidenced and acted upon. Lastly, the use of guiding theoretical frameworks (TFA and the TDF) to ensure interview questions were comprehensive in assessing possible barriers and facilitators to engaging with home monitoring was also a strength. Responses were also coded using this same framework to produce deductive themes centred around the behavioural domains thought to drive behaviours, such as engagement with health monitoring, and provided rich sources of data to help feed into overall assessments. This will facilitate the development of future behaviour change strategies to help address the key challenges identified.
A number of key weaknesses of I-TRAC stemmed from study design. Firstly, the lack of assessment of the original VF
MRF app was not ideal. While we would anticipate that – given the MRF, like the OKKO app, is also delivered on a tablet – many of the findings on acceptability would be transferable, this remains to be demonstrated. Secondly, there are some limitations in design of the survey described in Chapter 2, albeit the accounts and responses across the study were insightful. Within the survey of expert glaucoma clinicians, we did not specify whether home monitoring would be in addition to or a replacement for existing services. This means some responses may have considered service replacement and others addition of a service, which makes interpretations of the findings less clear. However, this might explain why there was considerable lack of agreement across the four clinical scenarios. It is also important to acknowledge that the survey represents only a sample of expert glaucoma clinicians’ views on which glaucoma patients are most appropriate for home monitoring. Understanding the views and opinions across the wider profession, including community optometrists, would also add value to this work.
Equality, diversity and inclusion
Another important limitation of the study was the patient population: largely white, experienced with technology, and generally research motivated. While we did not capture data on socioeconomic status, education or health literacy, given the types of patients who tend to participate in research it is likely that some categories of these other characteristics were also under-represented. This is of particular importance when considering that some ethnic groups are at much higher risk of advanced vision loss after a glaucoma diagnosis (e.g. six times more likely in black patients than white patients) and as such, monitoring may be more relevant for these higher-risk patients.97 In addition, given many studies have reported that disadvantaged groups who experience health inequalities are also more likely to be digitally excluded, it will be critical to ensure people from these groups are not excluded from future research or evaluation.98 In order to ensure representativeness across the sample, any future trial should use the INCLUDE Ethnicity framework (plus broader INCLUDE frameworks) and associated guidelines to help inform recruitment and retention of underserved populations.99–101
Recommendations for future research
Given the evidence generated from the I-TRAC study, we believe there are key unknowns that need to be addressed before moving to an evaluative study. These are outlined below using the population, intervention, comparator, outcome (PICO) framework. Once these unknowns related to the PICO framework and the specification of the research question have been elucidated, decisions about type of study design could be determined.
Population
Determining precisely which patient group would be most suitable for home monitoring requires further attention; for example, whether home monitoring using digital technology in patients with high risk of progression or those with low risk of progression should be the priority. Ensuring research teams engage with underserved communities to support recruitment to future studies such that there is broad representation across future populations is also required. Different populations may benefit from home monitoring if either or both tests are used. Linked to this, considering potential unconscious bias at sites with regard to participant selection should also be explored and mitigated against for any future study.
Also linked to population is whether home monitoring is considered as an additional service (i.e. in addition to routine monitoring through HES) or as a replacement service (i.e. patients would not attend HES and instead would be monitored at home). This may be directly linked to the eligible population, as high-risk patients may require an additional service whereas low-risk patients would be able to use home monitoring as a replacement.
Intervention
A single test or a combination of tests may potentially be useful for home monitoring of glaucoma. The interventions for home monitoring are complex interventions, and need to be conceptualised as such to ensure effective assessment and implementation.102 Understanding how these complex interventions operate within the complex system is also key, and using a systems perspective during evaluation would be important. The complexity could also extend to consideration of these interventions as behaviour change interventions, and findings from the behavioural theory-informed components of I-TRAC could be used to develop behaviour change techniques to embed within patient information, training etc. to enhance intervention uptake and engagement.
Given the trade-offs, mentioned earlier in this chapter, that may be required in intervention adherence, duration, and follow-up for implementation and evaluation in a future study, it is also important to base these adjustments on evidence. This could be achieved through conducting a DCE to determine design and delivery components of the home monitoring interventions. There are examples in the literature of how DCEs have been used to inform the design of complex interventions, which could in turn improve user uptake and adherence.103 In addition, this DCE approach has been used to develop a digital self-management intervention for chronic kidney disease.104 Further exploratory work to assess intervention acceptability would then be required, again ensuring engagement from all appropriate populations.
Lastly, further research to obtain agreement from clinicians around what level of adherence would be deemed acceptable for these home monitoring interventions would also be important both for future evaluative studies and for clinical practice.
Comparator
While the comparator in a future evaluative study would likely be standard care, there is variation across (and within) the devolved nations in the UK regarding patient care pathways. For example, there is an increasing but variable use of: (1) ‘virtual clinics’ where patients attend the HES and have a series of glaucoma tests, and then a clinician reviews and reports the findings at a later date; and (2) community optometry-based glaucoma clinics for patients at low risk of visual impairment, for example those with OHT. Understanding these contextual differences would be important for planning and analysis of any future study to identify how the intervention would be situated and any differences in results from a site or country setting.
Outcome(s)
Further research is needed to determine the most appropriate outcomes for evaluation of digital technologies for home monitoring, which could consider structure, process and outcome outcomes. The DCE approach to intervention development, mentioned earlier, could also be used to identify which outcomes are highly valued by patients, healthcare professionals, service managers, and commissioners of services.
In addition to the glaucoma-specific outcomes, a future evaluative study may also need to consider whether there are overarching outcomes of importance that could be informative for other trials of home monitoring technologies. There may be scope for a generic core outcome set (defined as the minimum set of outcomes that should be collected and reported)105 for digital technology interventions used to monitor chronic conditions at home. There are examples of core outcome set development for telehealth in particular clinical contexts which could provide a starting point for the development of linked sets.106
Patient and public involvement
Research to ‘improve early diagnosis of sight-threatening glaucoma’ has been identified as a top priority (for patients and clinicians) for funding (James Lind Alliance Priority Setting Partnership).107 Therefore, patients have been directly involved in identifying and prioritising the broader research question. We have worked closely with two people living with glaucoma (patient partners DS and BL) and have also had representation of a patient advocacy group (Glaucoma UK) throughout the duration of the study to ensure consideration of wider members of the community. One patient partner, DS, was engaged at grant development stage, and, through discussion with their local clinician and the Chief Investigator, agreed that I-TRAC was aiming to answer an important research question and agreed to act as patient co-applicant (having input into the development of the applications and specifically the plain language summary) and member of the SSC. DS has provided input into the overall plan of activities. Through another clinical collaborator, we identified one further patient partner (BL) who also contributed to the study as a member of the SSC.
Support for engaging in activities and meetings was provided by the I-TRAC Research Fellow (CS) and the Chief Investigator (KG) and ensured patient partners had received the necessary information, in preferred mode (e.g. paper over electronic) to support their contributions to meetings. The two patient partners were reimbursed for their time in line with NIHR recommendations.108
Our patient partners actively contributed to all SSC meetings, inputting to study decisions (including the change of digital technologies and decision to apply for an extension for the final cohort of participants), and providing feedback on provisional findings. In addition to the standard inputs into patient-facing documentation such as patient information leaflets and questionnaires, activities where specific patient input has been sought during I-TRAC included: contributing to the content and writing of the training materials for participants, ensuring they were clear and covered a range of possible problems that participants may encounter; and patient partners helped to develop the topic guide for patient participant interviews through reviewing the initial draft, making recommendations for improving the questions and structure of the topic guide, and reviewing and approving the revised version. The patient partners also made important contributions to the plain language summary and contributed to a discussion of a dissemination strategy for communicating findings to people living with glaucoma. Having the benefit of insights from persons with a lived experience of glaucoma had a positive effect on patient-facing materials, in terms of both content and display, with our patient partner representatives identifying poor word choices and poor layout that may have deterred volunteers from participating. This likely contributed to the positive feedback from patient participants as to the clarity and ease of use of study materials. There were no obvious negative effects of their contributions, but important lessons were learned in terms of how to ensure patient partner input is as effective as possible; that is, patient partners actually being familiar with the technologies and having lived experience of learning to use the technologies.
Reflecting critically on the process of involvement, two key items arose. Firstly, the involvement of I-TRAC patient partners in developing training manuals without them having lived experience with the technologies was not ideal. One way to have combatted this would have been to have sent the technologies to the patient partners and asked them to use them in line with the training manuals, as this would have highlighted critical aspects. The other area that proved challenging for our patient partners (on occasion) was joining the SSC meeting online using MS Teams. The Chief Investigator’s host institution only supports the use of MS Teams as a platform for online meetings. Even with support from the I-TRAC Research Fellow in advance of meetings to join the Teams call, on one occasion this meant a patient partner was not able to join (which was resolved through an opportunity to meet after the fact).
Before submission of the final report, we invited our patient partners to complete a feedback form. The feedback was largely positive. They felt information about the study was communicated well and that meetings had a clear purpose, were well managed, and were of an appropriate duration. When asked what they would have changed, suggestions were: being provided with a tonometer to monitor their own glaucoma at home and reducing the use of unfamiliar abbreviations. Overall, patient partners felt involved, valued, and that their involvement made a difference to the study.
Conclusion
The I-TRAC study has demonstrated ‘cautious optimism’ when considering patients’ and healthcare professionals’ views on the acceptability of digital technologies for home monitoring patients with glaucoma. The study also evidenced sufficient fidelity, good adherence to the interventions, and feasibility of delivery of both the interventions and the study processes, but this must be considered with reference to the narrow patient population included. However, I-TRAC also highlighted several unknowns relating to the PICO framework of a future evaluative study that require addressing before progression to a RCT. The I-TRAC study has also considered the wider ecosystem challenges of running digital health technology trials through evidencing the views of external research teams experienced in DHT delivery. Given the high system demand for digital solutions, in a space where innovation happens at pace, generating evidence to evaluate digital health technologies is challenging. Yet the potential promise for the health system more generally, and HES more specifically, provides justification for further research in this area.
