There was a requirement to identify key challenges and considerations for the delivery of a future large-scale evaluation of digital technology for home monitoring of glaucoma. In order to answer this research objective, we engaged with research teams involved in carrying out research studies or trials of digital home monitoring technologies (DHTs) for both ophthalmology and a range of other health conditions.
Methods
Study design
Online semistructured qualitative interviews with researchers who had been involved in conducting evaluative studies or trials of DHTs across a range of clinical specialties, including ophthalmology but also other health conditions.
Sampling and recruitment
Our sample size was informed by the five key principles of information power: broad or narrow study aim; dense or sparse sample specificity; application or not of established theory; quality of the dialogue; and finally, whether case or cross-case analysis.49 Informed by literature and our Project Management Group, it was agreed that three to four studies and six to eight participants would be sufficient to address our research objectives.
Chief Investigators and Trial/Study Managers of study/trial teams that have been involved in evaluating digital technologies for home monitoring of eye disease or other diseases (either as feasibility, pilot or full-size trials) were invited to participate in an interview to discuss their experiences of running such studies/trials. We first sampled purposively by contacting Chief Investigators of studies known to the authors or identified in the literature, with attempts to match study types to those identified in the literature review in Chapter 6. We invited Chief Investigators to participate, and then identified additional participants through snowball sampling by asking them to nominate the relevant Trial Manager/research fellow who may also provide important insights. We aimed for two representatives per trial and a total of six participants. We invited several research teams from within ophthalmology (n = 3 teams) and outside of ophthalmology (n = 3 teams) to consider challenges common to DHTs, and others which may be unique to ophthalmology. All participants provided verbal consent prior to the interview.
We had originally planned to also interview IT staff (n = 3) within NHS organisations to explore organisational IT issues in relation to DHT. However, evidence of issues relating to IT that are relevant for DHTs was generated through insights from existing stakeholder groups. Therefore, we chose to interview additional participants from research teams involved in delivering evaluative DHT studies or trials to provide further evidence relating directly to the broad issues of future evaluation challenges.
Data collection
Interviews were conducted online (Microsoft Teams)52 by CS and transcribed verbatim. Demographic data were collected at the start of the interviews. Discussion was guided by a semistructured topic guide developed within the team and included input from a researcher with expertise in digital technology, to cover broad questions about challenges and solutions for running trials of digital technology for home monitoring. Participants were also asked about their opinions about how easy it is to scale up DHT studies or trials, either to definitive trials or for implementation into the real-world setting. We had originally proposed that these interviews would be guided by the Consolidated Framework for Implementation Research (CFIR). However, following further review of the CFIR and through I-TRAC team discussions (supported by the SSC), it was deemed to be more appropriate to design less directive topic guides to capture all relevant data around the pragmatics of running evaluative studies of digital technology for home monitoring.
Data analysis
The six-phase Braun and Clarke approach to thematic analysis was adopted to analyse and code the data.53 The transcripts were reviewed several times by two reviewers, noting underlying points, ideas or feelings being conveyed. Transcripts were then transferred to NVivo 12 Pro46 for further analysis. The underlying points, ideas or feelings noted in first review were noted as codes. These codes were reviewed for themes – broader ideas linking the individual codes. Themes were constructed inductively (e.g. what types of challenges and solutions were reported) and then deductively (e.g. what did participants say about a given facilitator or challenge), noting potential relationships between themes, until a clear framework of inter-related themes, that could explain much of the data, was proposed. A coding dictionary describing the themes was developed and updated throughout the analysis (see Appendix 7).
Results
Sample characteristics
Twelve researchers were invited, with eight researchers interviewed (66.7% response rate): three Chief Investigators, four Study/Trial Managers and one Study Coordinator (who worked under one of the recruited Trial Managers). Of the four who either declined or did not respond, three were Chief Investigators and one a Trial Manager. All researcher participants had experience in DHT studies or trials within the UK, with some researchers working across a range of disease areas. The trials they worked on involved using digital technology to home monitor a range of health (mostly long-term chronic) conditions, including diabetes, hypertension, chronic obstructive pulmonary disease, COVID-19, age-related macular degeneration, glaucoma, alcohol addiction, and skin cancer. Table 21 shows further demographic information.
Researcher demographics (n = 8)
Findings
Despite the challenges researchers reported they encountered during the trials, the majority (n = 7) had positive attitudes towards digital technology research. Researchers emphasised the possible solutions to address the challenges they encountered rather than focusing on the challenges themselves.
All researchers described a number of challenges they had experienced in setting up and conducting DHT home monitoring studies or trials. However, they also presented examples of good practice (solutions) that helped, or could help, mitigate these challenges. These challenges and solutions fell under six major themes: stakeholder acceptance of technologies; understanding the devices and predicting potential technical problems; resource planning; peripheral infrastructure – institution ethics and funding; committed and supportive commercial partners; and digital exclusion. These themes, and the associated challenges and solutions, are reported in detail below and illustrated in .
Challenges and solutions for DHT trials.
Stakeholder acceptance of technologies
Similar to trials evaluating other interventions, most researcher participants (n = 7) reported challenges recruiting and/or retaining participants. This was described in relation to recruiting both patients and site staff/clinicians and concerned the acceptability of the digital technologies being studied by these central stakeholders. The key challenge for DHT studies or trials related to recruiting and retaining representative samples. Researchers frequently reported non-representative samples, over-represented by individuals who were highly educated and had previous access to technology and the technological confidence to participate.
[T]here was positivity from a lot of participants and a willingness to take part, but when we look at our demographics we’re obviously looking for evidence of inequalities and the ones that you would expect are there, sort of higher education, white, British as opposed to those that chose not to take part and in particular with respect to exposure to previous technology, about 66% had a smartphone, about 85% had internet in the home, so all of those for that demographic are much higher than the national average.
P021, Chief Investigator
Patient acceptability
Low digital literacy skills, anxieties about using technologies, difficulties understanding how to use technologies, and no or low support at home to assist participation were commonly reported by researchers as reasons for patients choosing not to engage with DHT studies or trials.
So we got quite a lot of, ‘No, sorry, I can’t’, or, ‘Sorry, I tried but I can’t manage this, it’s just giving me too much stress and hassle’, or, ‘Sorry, I need my son to help me but he’s not around because I can’t use a computer’. So there was a lot of… I would say probably, you could probably say about a third of them were very happy and capable and able, a third were willing and needed assistance and the other third were probably like you don’t even ask.
P028, Trial Manager
Researchers described how these issues reflected low acceptability of DHT from the patients’ perspective, resulting in a lack of willingness to participate in studies or trials of this type. Several researchers reported how DHT study or trial populations were over-represented by younger, more educated, white participants – a finding that, while not dissimilar to issues across research fields, researchers felt was more difficult to overcome in evaluating DHTs due to generational differences in ability and perceptions of technologies. A further impediment to recruiting patients was the assumptions made by clinicians or site staff about patients’ willingness and/or ability to take part in DHT studies or trials, often making judgements based upon age (i.e. perceiving their patient group as too old to use technologies and anticipating barriers to participation prior to discussing with patients). Researchers frequently reported how sites varied in their attempts to offer DHT study/trial participation to as a great a number of participants as possible. Some researchers highlighted selective approaches and offers, stating that many of their patients would not be able to use technologies or would not be interested.
. . . well identifying patients that were eligible wasn’t necessarily a problem because that’s irrelevant whether or not they wanted to take part in the study or not, it’s I think the sites, some sites were more selective with who they approached depending on how they felt the patient would cope with it, and some sites were… would approach anyone who was eligible and try get them on board. In a lot of ways the latter was better because you get a better picture, but we still had a lot of rejection because of the technology.
P023, Trial Manager
Clinician and site staff acceptability
It also became apparent that there were a number of barriers put up by clinicians in agreeing to host DHT research at their site. Researchers discussed how a lack of supportive evidence on the effectiveness and benefits of the technology being tested contributed to the lack of staff/clinician buy-in. In some circumstances, clinicians challenged the benefit of the intervention, questioning what value the digital intervention had for them personally, and at times for their patients.
And the other beauty was that we were able to search the records to look at the number of face to face consultations, and we were able to demonstrate that there was a significant fall in face to face consultations in the group that were being managed that way, a 25% reduction in all face to face consultations. Once you have this sort of data and you can show this to people, they start to get interested…’
P026, Chief Investigator
If clinicians did not see that the digital technology would make their life easier – that is, if they perceived the technology to be cumbersome, time-consuming and not leading to a clear improvement for the patient – they would be reluctant to participate.
. . . the main thing is that the equipment works. That’s really, really important that it works and is reliable. If you’re a busy ophthalmologist, a busy GP and it just doesn’t work a couple of times, you’ll think, ‘Stuff this’, and you just won’t use it, you know? Even an enthusiast, after a while, they might like you and say, ‘We’re doing our best here but I’ve got a busy life and I can’t be bothered with this’. It’s really important that it works.
P026, Chief Investigator
Researchers reported that many of these acceptability issues, believed by the researchers to inhibit recruitment and retention, could be enhanced in DHT studies or trials through several mechanisms. Ensuring the technology has a clear purpose and is as simple to use as possible, with minimal steps and actions, and that this is communicated clearly to clinicians, can aid buy-in. The importance of simplicity was also echoed in relation to improving the acceptability of DHT for patients.
For the [study name] the patient population, the average age was, I think it was 74 and, so that hugely impacts how that patient population should be approached for the trial because the main reason for not being interested in taking part was being put off by the technology. So that… you will need to consider how best to approach the patient population without putting them off. We had training sessions for an hour long where the site would demonstrate the technology, go through it, but it’s still a lot to take in and obviously we weren’t just testing one device, we had a paper journal plus two apps, plus a MiFi device, it was a lot to take on. So I feel like yeah, considering the patient population is really important, and also not to overwhelm them. I appreciate if a trial’s being set up and you want to measure as much as possible, but keeping it to one device or one test is the easiest way from a patient perspective, yeah.
P023, Trial Manager
Exploring the needs of stakeholders early in the development of the technology, and building these into the design, was considered important. Many researchers felt using technologies they (clinicians and patients) are already familiar with aids acceptability. However, this issue may be a double-edged sword, linking with ‘digital exclusion’, discussed later in this chapter. Also, many believed that recruiting patients face to face in a clinical setting, demonstrating the technologies, helps with training coherence. In relation to the training to be delivered, multiple pilots of training materials were considered a good approach to ensuring the final training programme is effective and acceptable to patients and clinicians. Contextually, many considered that the COVID-19 pandemic helped with both clinician and patient acceptability through increased interest, motivation, and confidence in using technologies. The value of digital technologies was also perceived by researchers to have increased for patients and clinicians; DHTs have offered a lifeline for remote monitoring when hospital visits were not possible or were undesirable due to COVID-19 risk: ‘So basically have a system where you only have one click option so they can’t click on the wrong thing, that’s ultimate necessity’ (P028, Trial Manager).
Some researchers reported how selecting a research site experienced in running DHT studies/trials may be beneficial. One participant described this as identifying research champions, those who are motivated and enthusiastic enough to see new DHTs through the early testing period where many problems are anticipated. These champions were less likely to give up at the first hurdle and set the path for future implementation. This was considered good practice for facilitating feasibility or pilot studies of DHT. As was stated, busy clinicians often drop challenging studies at the first hurdle. Additionally, researchers reported that the involvement of researchers from communities and ethnicities that were under-represented in recruitment helped to balance the representation.
It’s really good if you can get champions, so if you can get a few people who are tech enthusiasts who will get this thing going, who don’t mind if it’s going to be difficult in the first – there’ll be bumps in the road. We always did this, we would pick a few practices, for example, where we knew the people there were reasonable folk that understood technology, that understood that this was not perfect, you know? We would say that to them at the start, ‘You are helping us develop this. We know there are going to be bumps in the road and the reason we have chosen you is because you’ve got that approach and that you will be prepared to put up with this and you will feed back. You know that we will take what you say and make changes as a result of that’. I think that’s really, really important. Try to as much as possible involve the early people in this as much as you can in the actual design and running of this, continually feed back to them. We always kept feeding back to people, ‘Has anyone come up with an idea? We’ve heard that this works, this is another way of recruiting people’, you know, this sort of stuff. ‘Here are the numbers, these people have managed to recruit this number of people. Let us know what trouble you’re having, we’ll see if we can sort it out’, and respond quickly every time… .
P026, Chief Investigator
Understanding the devices and predicting potential problems with the technologies
A number of problems related to the technologies researchers had tested were reported, typically in relation to hardware, software, accessories (e.g. chargers), and technical issues (e.g. passwords).
There were some issues with passwords which we thought we had solved at the start. There were some of the laptops that had password renewals that came into play after the first six laptops that were issued. We had to give instructions about how they could change their password. So although we thought we had turned that off we hadn’t done so, so for the final 14 or so we were able to ensure that they didn’t have that password notification come up on their laptops.
P025, Trial Manager
While problems are specific to individual technologies, a number of issues are likely applicable to many DHT studies/trials. Common issues reported by researchers included compatibility of software with different devices, outdated applications requiring frequent updates, managing mass number of remote devices, and connectivity issues (internet, Bluetooth to submit and receive data).
[T]here was an Apple update then that it messed with the apps and then there would be a period when they couldn’t use them and that sort of tech stuff, there was a few issues throughout the study follow up time where there was just periods where the participants weren’t able to test due to tech problems.
P021, Chief Investigator
Researchers reflected on how a number of these technical problems encountered could be mitigated through utilising familiar devices and simplifying technologies (as discussed in Stakeholder acceptability above), such as minimising opportunity for error by reducing the number of options participants can choose. However, a recurring subtheme was the lack of foresight as to what DHT studies/trials would involve to set up and deliver.
I think when we first… we got the iPods and we thought it would be a case of just turning them on, downloading two apps and giving it to the patient, but it turns out it’s not as simple as that.
P023, Trial Manager
It became evident across the interviews that researchers felt they had opened DHT studies or trials without really understanding how the technologies worked.
I think anything that’s tech based has to be very closely checked first of all for the potential for breakages or software errors or problems with the device… I would think that from a tech versus non tech perspective you would have to have all your options covered for the unknowns which are more so than other non tech based studies.
P025, Trial Manager
Several statements from researchers reported that they felt many issues would have been prevented, or resolved more efficiently, if they, or another on the research team, had more knowledge of technologies. This led to a suggestion from most that research teams need to have digital expertise within the team from the very beginning, concept development stage, whether provided internally (within the institution) or externally (such as commercial partners).
In terms of leadership I think it would be very useful to have someone with experience in the trial group, some who are experienced in setting up devices and all that because the trial team consists of a trial manager, a statistician, a database manager, a qualitative researcher and none of these people are particularly clued up on how to set up Apple devices for example. So I think yeah, you would want to have someone like that within the trial team.
P023, Trial Manager
It was also suggested that feasibility studies have an important role to play in ensuring all stakeholders understand the technology and that potential problems with the digital technologies are identified and resolved at a very early stage, prior to progressing to trial. Below is a quote from one Trial Co-ordinator discussing this when reflecting upon the numerous problems they encountered with the devices being studied in their trial.
I think some kind of pilot study or feasibility study would’ve been very useful in identifying some of the issues and they’re addressed before we go into a full trial or a full study; hindsight.
P023, Trial Manager
Resource planning
Across researcher accounts, the under-resourcing of DHT studies/trials was repeatedly mentioned. The complexities of setting up and distributing technologies placed significant time demands upon trial managers and research fellows who were often unfamiliar with technologies, which likely worsened the time-consuming nature of these activities.
The intervention took a lot of setting up, you know? I think we were probably under resourced, I think we probably needed more resource than we actually had to set up the technical infrastructure and architecture of the intervention. So the complexity in setting that up.
P027, Chief Investigator
For some, these activities were delegated to sites; however, research site staff were frequently unable to accommodate this around their routine clinical research roles and often pushed back to the central research site. Another significant demand upon trial managers’ time was the provision of support to both site staff and participating patients.
[W]e had a single mobile phone included to act as a patient helpline and the calls to that were considerable and that was a huge massive resource but I don’t think we had fully anticipated how much it would be used and how necessary it was.
P021, Chief Investigator
Most reported feeling as though they were running a support helpdesk, which in addition to time demands was at times frustrating for study/trial staff as they generally felt unprepared to provide this support. Researchers reported feeling as though they did not know the technologies any better than the site staff or patient participants, and resources for ensuring adequate support were required.
As a trial manager you’re quite busy and then to then have to deal with a technical support helpline, it was actually quite time consuming and difficult because you’re trying to resolve issues over the phone with elderly patients and it can be quite difficult to resolve issues in that way. So I think if you’re implementing a digital technology you have to have factored in support from the start and that can be with a helpline, with regular calls going to patients. We tried to… every now and then we would try and call patients for whom we haven’t received any data after a certain amount of time, and, so that was another thing that we hadn’t particularly considered but was time consuming, and also support to the sites, they’re the ones training the patients and, so they also needed technical support. Then from the trial management point of view, I personally didn’t set up the devices because they were based in [city name], but there was someone there who’s actually the qualitative researcher, the responsibility fell to them to set up the devices, which was completely out of their job description. So yeah, factoring in costs and resources for device set up and device support.
P023, Trial Manager
The challenge of providing this support depended upon the study/trial population. For example, it was difficult to provide support to technologically inexperienced users remotely over the phone.
I think sufficient support is really important. It depends on I think obviously the patient population and my experience is with age related disease so they’re an elderly bunch and, so they need technical support, well a lot more than the normal population, and also providing support to the sites. I think yeah, any trial using digital technology will require some level of support, some more than others, so I think that’s really important.
P023, Trial Manager
Balancing providing this support with time demands for addressing unexpected technical issues (as discussed in Understanding the devices and potential problems) and more routine study/trial management activities was a frequent challenge. In addition to the time and human resource demands, several issues were reported in relation to the financial resourcing of DHT studies/trials. Researchers often reported unexpected costs connected with the technologies, many of which were related to lack of researcher experience with technologies (as discussed in Understanding the devices and potential problems). For example, the need for multi-device management licences – which, while beneficial for the study/trial as they allowed important functions like sending out updates automatically, were an unexpected significant cost which went beyond planned budgets – led to the use of less optimal alternatives.
. . . so the Apple, the fact that we were using Apple devices so they needed multiple device management system, which we hadn’t costed for in the grant, so we then had to use the standard Apple one, which had particular constraints
P021, Chief Investigator
Factoring in requirements for spare equipment, particularly accessories (such as replacement chargers) that were expected to get lost or broken, was also cited by researchers. A further complexity of resource planning was reported as the unpredictable changes in technology costs and availability. One researcher highlighted that while technology costs are expected to reduce over time, this is not always the case, ‘I think technology gets cheaper over time but I guess costing is a factor because certain things get cheaper but certain things get more expensive’ (P025, Trial Manager).
Researchers recommended that future DHT studies/trials carefully plan resource requirements, especially in relation to the time and human resource to manage technical support required by participants and sites. There was also a belief among researchers that more resource-efficient ways of delivering technical support be explored. For example, apps could have inbuilt help sections that are easy to access and understand.
I’m just trying to think of ways around the support without burdening the clinicians further, it’s having patients help themselves, it’s self-taught. A little practice or a little demo that’s in there, so again the app itself, make a game of it; take people through and show them, okay, you want to enter these data here. Where do you go? It’s got a little button flashing and boink, and we reiterate, now where do you go? But this time the button doesn’t flash and it’s all about inbuilt training of getting people used to the layout and where the steps are. Little videos, for the trials we often try and have little, short ones, almost like Twitter snapshot training videos. Anything from little cartoons with a voice-over to handheld actual physical videos. Then maybe testimonials from patients who have used it. Top tips from people who have actually used the blooming thing.
P022, Trial Manager
Ensuring adequate feasibility testing of technologies, as suggested in Understanding the devices and potential problems, was reported as one approach to improving resource planning for DHT pilot studies and full-size trials in the future, allowing researchers to monitor resource needs on a smaller scale, prior to scaling up.
Peripheral infrastructure: institution, regulation and funding
Researchers discussed several challenges imposed by, or made more problematic through, insufficient support from peripheral infrastructure systems such as academic institutions, regulatory bodies (e.g. research ethics) and research funders. For example, several researchers reported that their academic institutions were not set up to provide the technical expertise required to run DHT studies/trials.
[W]e worked with the computer scientists and result of enthusiasm about getting that going, but the actual support provided to those of us working in digital health, there’s not a great deal about actual practical go-to sought-out support. We saw… this sounds a bit moany but we visited a unit in [city name] that were developing digital applications and, to be quite honest, the actual applications that they were developing were no better than anything I’ve seen around here. They’ve just got that team of their own programmers that they can access and can help them with that sort of stuff.
P027, Chief Investigator
The additional and often complex movement and storage of patient data which frequently occurs in DHT studies/trials posed challenges when navigating regulatory bodies. For example, several researchers reported having ethics approval difficulties due to the monitoring devices collecting data in patients’ homes and transmitting data via third parties, potentially breaching patients’ safety, data security and confidentiality.
GDPR, the data protection impact assessment, the risk assessment around personal data was a huge thing for our sponsor… and so it took a long time to show them that’s it’s okay… because of the personal data aspect it’s all being collected on a widget that’s outside of their control, your control, it’s in a patient’s home
P022, Trial Manager
This was reported as becoming even more problematic for multisite studies, where each geographic regional approval body may have different information governance, research and development processes for studies/trials involving DHTs.
The IT governance thing is a bit of a nightmare, you know? And it’s absolutely bizarre that a country like Scotland, how you have to go through this 13 or 14 times, everyone has different questions, everyone has different ideas, everyone has different ways of using the software. We did this recently with a piece of work we did on tele Covid, which is a method of monitoring Covid at home. Oh, it took forever, you know? The epidemic was almost over by the time we got permissions, you know?
P026, Chief Investigator
A fundamental problem for DHT studies/trials expressed by researchers was the clash between the fast pace of technologies and the inflexible and time-consuming research infrastructure, which resulted in lengthy delays awaiting approval for changes, and as reported by some, contributions to research waste. As reported by two researchers, the duration between research grant application, research grant award, regulatory approvals and opening a trial could be several years, by which time the technologies proposed for evaluation are no longer current.
I think the other challenge was, again, the technical side of things, it wasn’t a particularly mature intervention that we were trying to get set up, it was kind of out of date by the time we were setting it up and it needed to be updated…
P027, Chief Investigator
Another reported contribution towards waste in DHT research is the lack of stopping rules, which are common in other trials (e.g. drug trials). These rules give clear criteria to stop when there either appears to be no benefit, or in the event that a better treatment becomes available. One researcher argued that this should also apply to DHT studies/trials if a new and better technology becomes available. The solution proposed by several researchers is to develop a more agile and responsive funding and regulatory system for DHT studies/trials, one in which research protocols can be changed and implemented quickly and where minor changes in technologies no longer necessitate significant resource to approve and implement.
So I think there’s bound to be a more efficient way of getting the answers that you need quickly and making sure that research resources et cetera, et cetera, aren’t squandered in the process and people’s time and effort and all of that. So yeah, that’s kind of… and the amount of time that it takes from an HTA… ours was a commissioned call and it went through various iterations and then by the time contracting(?), all the desperately long drawn-out processes that are necessary in a big NIHR trial it’s just the complete opposite of how this kind of digi-health thing works.
P021, Chief Investigator
If Apple decide to make a change and therefore the provider of the app has to make a change in accordance with Apple’s change, then do you have to go back to the start again? These questions need to be sorted out. Your phone is updating itself every three months, you know? You can’t be going back to the MHRA every time.
P027, Chief Investigator
A more streamlined approach to approving data management and assessing data risks and security is also necessitated. From an academic institution perspective, more networking of DHT researchers alongside IT and technical specialists was proposed as an important step towards improving the quality of DHT studies/trials and would help prevent and resolve a number of technical issues (as discussed in Understanding the devices and potential problems).
Effective relationships with commercial partners
For researchers involved in delivering DHT studies/trials with commercially developed technologies, relationships with commercial partners led to several challenges. These were, namely, delays in receiving technical help to resolve problems, delays in access to data, removal of support following a company decision to withdraw a product from market, and support withdrawal following the commercial partner being acquired by another company.
[O]ne of the companies got bought by another company… it made things a bit complicated for a while. The other app, the company that owned it discontinued it. So they continued running it for our trial but support was absolutely minimal because they were discontinuing it.
P023, Trial Manager
One researcher also raised the challenges of maintaining good relationships with commercial partners when results fail to demonstrate evidence of benefit of the technology under evaluation.
But in COPD [chronic obstructive pulmonary disease], it was very clear it didn’t work at all, it just doesn’t work, you know? Didn’t reduce the number of admissions to hospital, it increased workload, you know? It was no good. I’ll tell you, you’re never very popular when you come out with a result like that, particularly with tech companies, they are really cheesed off.
P026, Chief Investigator
For researchers, these problems emerged due to having poor relationships and communication plans with commercial companies, and a lack of, or poorly termed, contractual agreement with commercial partners. For researchers who felt commercial partnerships had been beneficial, clear contracts, enthusiasm, willing support and investment from the commercial partner were critical components of this relationship. It was also highlighted by some researchers that large commercial partners have significant human resource dedicated to supporting DHT studies/trials, not just in relation to technical support, but on the regulatory support. For example, one researcher reported how their commercial partner facilitated all ethics and R&D approvals, using their own experience and expertise to navigate this efficiently and freeing up the research team to focus on their own expertise, trial design and set-up.
. . . just even from working more closely with them you’re getting more of an insight of how a health type start up works and the processes and the agile situation that they have and ways of doing things quickly and efficiently.
P021, Chief Investigator
One researcher reported how she felt commercial technology companies are more motivated than pharmaceutical companies, as often the product is their main focus for business development due to their limited portfolio of products.
They seem to be much more engaged and enthusiastic and interested than, I don’t know, a thumping great pharmaceutical company that seems to begrudge giving their tablet for an academic site to go and test. But I think some of it is also that the company relies on the outcome of this. That’s where it’s different to a thumping great pharmaceutical company who, they’ve got loads of products, they’ll keep the money rolling in, whereas this company… I think they do have other apps and things, but this is the one that’s their newest I think, they want to test it properly.
P022, Trial Manager
The relationship the research team has with commercial partners appears to be an important one, and one in which researchers reported that having a clear contractual agreement outlining agreed expectations and requirements from both parties was key in reducing some of the burdens they encountered.
. . . some form of agreement with the third parties or the app developers to make sure that if certain things need to be changed that they are there… they will be willing to provide support and make certain, obviously they can’t change everything, but to make some minor changes.
P023, Trial Manager
Digital exclusion
Frequent concerns were raised by researchers about study or trial design leading to digital exclusion of participants, which was perceived to lead to recruitment difficulties and unrepresentative samples that fail to reflect the disease population being targeted. Among researchers interviewed, this arose from participants lacking personal devices such as smartphones, and/or lacking access to the internet.
So again, it’s that thing of then how you think hard about preventing the digital exclusion because if I was redoing [study name], there is part of me would say you don’t even attempt to give people new devices, you just use their own smartphone or, and then you cut out so much of the… there isn’t issues about somebody with familiarisation to a new device. But yet then on the other hand then you still then have that whole big problem of the digital exclusion. So I don’t have an easy answer as to how that works or whether you just accept that there is a batch of people that you can remove from your follow ups or see less and that will help everybody else and just make sure that the people that then aren’t part of the home monitoring thing are still being seen by a speciality. So yeah, it’s a tricky balance with that.
P021, White, Chief Investigator
It struck me, it was actually our Steering Committee, so our Chair there pointed out that 80% of the problem with alcohol related liver disease is in the 20% lower socioeconomic group, and who’s less likely to have a smartphone? You can’t say that they won’t, but it’s just that on average, so there is something about still being able to reach everybody, our NHS, free at the point of care for everyone and I know that… because we did discuss providing smartphones if some didn’t have their own but that was quickly vetoed because people need to be familiar with the technology.
P022, Trial Manager
While researchers agreed that device familiarity was important and allowing participants to use their own devices as far as possible was seen as helpful for engagement, the concern about excluding those who did not possess such technologies was evident. Providing devices was the obvious and frequently reported potential solution. However, as reported by many, lack of familiarity replaces the problem of lack of technology, and many doubted whether it would help. This option was also reported as including significant financial costs.
. . . because if they’ve got the tech then they’re used to that tech and they know how to use… and we’re all the same, I know how to use my phone, it’s an iPhone, whereas if you give me your Samsung I’m lost, I don’t know what to click, what to do with it. So if you’re signing me up to the study and then you give me a new laptop or a new tablet on top of that I’m like, ‘Okay, firstly, I don’t know what I’m doing, and secondly now you’ve given me something else new to play with’, which again, makes matters worse, so I don’t think that’s going to help in that aspect. The ones that didn’t have the tech, I don’t think giving them the tech would help because I would say they don’t have the tech for a reason, as in they’ve chosen not to…
P023, Trial Manager
Access to the internet, so often required in DHT studies, can also be a problem. One researcher reported how they tried to overcome this by providing a second device allowing the primary device to connect to the internet regardless of what the participant has available. However, rather than reduce digital exclusion, this seemed to increase it; the second device only added more complexity and resulted in a number of participants dropping out of the study. The agreed problem of digital exclusion, as yet, remains without recommendation.
. . . even with us trying really hard, because we provided not just the device… but we also provided a MiFi device with a mobile contract for internet access, so that was our attempt at trying to avoid the digital exclusion, but you can see that it’s still there. So I think one of the biggest challenges is addressing that in whatever way you do a bigger study.
P021, Chief Investigator
For some, digital exclusion was anticipated to be less of a problem in the future.
Well I think as we move forward in time it’ll be a lot easier for a patient to do it on their own device and as the population becomes more technologically literate that will be more and more acceptable, and I think patients will be more open to that.
P023, Trial Manager
For now, the challenge of overcoming digital exclusion remains, without solution, for many of the researchers.
