Introduction
Mobile applications and devices intended to monitor and promote health (“mHealth apps”) are becoming ubiquitous, and the amount and scope of data they collect are constantly expanding.1 These data can be valuable for an array of health-related research (“mHealth research”), including research conducted outside traditional academic settings.Reference Rothstein, Wilbanks, Brothers and Dorsey2 However, ethical uncertainties arise when mHealth data are collected and/or used in research that is beyond the scope of federal regulations intended to protect human research participants (“unregulated research”).Reference Rothstein3 Confronting these challenges is essential to ensuring that end users (individuals who ultimately use an mHealth app/device)Reference Downing, Covington and Covington4 are protected against the kinds of risks and harms that such regulations address, while supporting the conduct of potentially beneficial research.
To help inform these issues and contribute to the development of ethical policy and practice in mHealth research, we conducted in-depth qualitative interviews with experts from key stakeholder groups. We explored their perspectives on two hypothetical scenarios involving unregulated research using health, behavioral, and other data originally collected by commercial mHealth apps for non-research purposes.
Methods
Participants
We conducted in-depth qualitative interviews with experts from four key stakeholder groups:
Patient and research participant advocates (“Advocate”)
Researchers who use mHealth technologies in their studies, including independent researchers and citizen and community scientists (“Researcher”)
Regulatory and policy professionals (“Regulatory”)
Mobile app and device developers (“Developer”)
We identified potential participants based on leadership positions in prominent organizations, institutions, and studies; authorship of influential papers; and nominated expert sampling.Reference Namey, Trotter, Guest and Namey5 We used stratified purposive sampling to interview at least six experts per group, the minimum expected to reach saturation.Reference Guest, Bunce and Johnson6
Instrument Development
Based on our knowledge of the issues and in consultation with the larger research team, we developed a semi-structured interview guide centered around hypothetical scenarios (Box 1) involving two commercial mHealth apps collecting health, behavioral, and other data that may be shared for various purposes including research:
“MoleStar,” an app designed to support people diagnosed with or at high risk for melanoma
An app designed to predict, detect, and prevent relapse during recovery from substance abuse (“Substance Abuse app”)
Box 1 Hypothetical Scenarios
Scenario 1: MoleStar App Footnote *
Health Apps, Inc., has developed a comprehensive smartphone app called “MoleStar” for people at risk for melanoma or who have been diagnosed with melanoma. MoleStar includes this functionality:
Educational information about melanoma, such as basic information about prevention, diagnosis, and treatment
Image capture tools to help users monitor changes in moles over time
Disease management, such as tools to track appointment times and locations, lab values, and medications. It also has tools to log daily physical and psychosocial wellbeing
Social support/networking, such as tools to share information and communicate with others through social networks, and forums to communicate with other MoleStar users. It also enables optional participation in Health Apps. Inc, surveys about melanoma and MoleStar.
The primary purpose of MoleStar is to help people at high risk of melanoma to monitor moles between dermatologist visits, and also to provide support for people who have been diagnosed with melanoma.
All information captured by the app is automatically transmitted to Health Apps, Inc. and stored in a database to support further product development. Health Apps, Inc. also sells the data it collects to third parties for marketing, research, and/or product development purposes. Information about these uses is available via a link displayed at download; prospective users must click “I agree” in order to continue. The data use policies can also be accessed from MoleStar’s “About” page.
Michael Lee is a computer engineer whose spouse recently died from melanoma. Eager to help others, he contacts Health Apps, Inc. to purchase the images captured via MoleStar along with the disease management information. His goal is to create a machine learning algorithm that can identify via the images early melanoma as well as moles at high risk of becoming cancerous.
Mr. Lee fills out an online form with basic information about himself and his intended use of the data. He signs an agreement saying he will use the data only for that purpose, and that he will not give or sell the data to others. After paying the data access fee, Mr. Lee’s request is granted. Health Apps, Inc., provides him access to the images and data after removing direct identifiers (such as name, address, phone number) and replacing them with a code (which Health Apps, Inc., can link back to identifiers).
Scenario 2: Substance Abuse App Footnote **
Imagine now that Health Apps, Inc., made a different app that has a number of features intended to predict, detect, and prevent relapse in recovery from substance abuse, including:
Connecting with others for support, e.g., through discussion groups with other app users, video chats with counselors
GPS tracking to detect when an individual is near a high-risk location (such as a liquor store). When near a high-risk location, the app causes the phone to ring and a number of recommended coping strategies display.
A “panic button,” which sends a text message to support prompting a response for assistance
Like the MoleStar app, Health Apps, Inc., captures and stores the data transmitted by the Substance Abuse app, and sells it to third parties for marketing, research, and/or product development purposes.
* Based on J.L. Bender et al., “A Lot of Action, But Not in the Right Direction: Systematic Review and Content Analysis of Smartphone Applications for the Prevention, Detection, and Management of Cancer,” Journal of Medical Internet Research 15, no.12 (2013): e287; G. Nasi, M. Cucciniello, and C. Guerrazzi, “The Performance of mHealth in Cancer Supportive Care: A Research Agenda,” Journal of Medical Internet Research 17, no. 2 (2015): e9; B. Odeh et al., “Optimizing Cancer Care through Mobile Health,” Support Care Cancer 23, no. 7 (2015): 2183-2188.
** Based on D. Gustafson, A-CHESS: Developing and Testing a Computer-Based Alcohol Use Disorder Recovery System, available at <https://center.chess.wisc.edu/research-projects/view/achess-developing-and-testing-a-computer-based-alcohol-use-disorder-recovery-system> (last visited September 23, 2019).
Following pilot testing and refinement, the final guide (Appendix A) comprised questions about benefits and risks of the apps, approaches to notification/permission for research use, data access procedures, new primary data collection, offering individual research results, and data sharing and dissemination. We also asked questions about expectations for independent oversight, responses to which are reported elsewhere in this issue.Reference Beskow7
Data Collection and Analysis
Prospective interviewees were invited by email to participate. Prior to the interview, we provided a study information sheet and a description of the Mole-Star scenario. Interviews were conducted by telephone between October 2017 and February 2018 by two research team members and, with permission, audio-recorded and professionally transcribed. Interviews lasted approximately one hour and participants were offered $100 compensation for their time. The Vanderbilt University Institutional Review Board deemed this research exempt.
We uploaded transcribed interviews into NVivo 12 and used standard iterative processes to code and analyze the data.Reference MacQueen8 See Appendix B for additional methodologic details. Narrative segments presented below are exemplary of frequently mentioned ideas unless stated otherwise; see Appendix C for additional examples.
Results
We interviewed 41 experts (Table 1) representing a range of demographic characteristics and holding diverse views on the basic level of risk associated with using MoleStar (Table 2).
Table 1. Participant Characteristics (n = 41)
* Many of our interviewees have multiple areas of expertise and could have been recognized as belonging to two or more categories of stake-holder groups; this table reflects the primary perspective for which we identified them as experts.
^ Reflects >1 degree per interviewee, as applicable
Table 2. Participant MoleStar Ratings and Opinions (n = 41)
Table reflects responses to direct interview questions.
Views on Notification/Permission Models for MoleStar
general notification
We asked about General Notification (i.e., a brief, broad disclosure) as a way to let people know that data collected by MoleStar could be shared and used for research. Interviewees discussed disadvantages and advantages for both end users and researchers. Over half noted that such notifications are often overlooked or ignored:
When you put notices of any sort in connection with a cell phone app, people just click through them. People don’t read them. People don’t understand them. It’s not an effective way of giving people notice. (09_Regulatory)
Many felt that, even when users read these notifications, they contain insufficient detail about what users are agreeing to or the risks involved, particularly regarding who may be conducting research and on what topics:
People would have a top-level idea, ‘it’s going to be research,’ but they may not understand all of what that entails. They may think, ‘This is the company I’m giving it to and they’re in control of it,’ rather than, ‘It’s going to be sold and resold and resold to multiple other companies who I have no relationship with. I’m going to lose track and control of it.’ … You don’t know what types of research. You may assume that it’s going to be research on melanoma, and not research on something totally unrelated… So, you may be giving consent for something that you didn’t understand. (27_Regulatory)
Still, some emphasized the value in letting people know that their data may be shared for research, even if this disclosure is limited or may be disregarded:
It’s kind of like the general notification on a cigarette. ‘Cigarettes are harmful to your health.’ People don’t really pay that much attention to it, but it’s important that it is included.
(11_Advocate)Interviewees also identified low burden on end users as an advantage, describing General Notification as standard and easy to navigate:
People are very familiar with the ‘click here to consent’ concept … that model of ‘here’s something, read it, click here to go read more’—that’s become very familiar to all of us.
(07_Regulatory)Some remarked that General Notification is also efficient for researchers because, in addition to not constraining future uses, it likely increases the amount of data available:
Most people will not think very much about the particular harms and risks. So the main strength is that they’ll get a really lot of people agreeing to share their data. And many of those people—even if they knew all the [information], thought about it in more detail—would still probably agree. So, it’s a strength in terms of getting your research numbers up in a very efficient way.
(13_Advocate)After discussing advantages and disadvantages, slightly less than half of interviewees indicated General Notification was acceptable for MoleStar and an equal proportion said it was not acceptable (Table 2). The remainder said acceptability depended on content and/or presentation, i.e., the disclosure must be prominent, comprehensible, and sufficiently detailed for end users to understand what they are agreeing to.
most appropriate approach to notification/permission
Interviewees expressed a wide range of opinions when asked about the most appropriate approach to notifying people that their data could be shared for research (Table 3). Some supported the use of General Notification, though often suggested the addition of subsequent reminders or optional supplemental information. Others advocated for a more active approach that would require some increased attention or additional action in response to the notification. Some suggested offering a broad yes/no choice, making research participation optional (i.e., not a barrier to using the app for its commercial purpose). Others went further, stating that providing multiple yes/no choices for various categories of data and researchers would be optimal. A few felt end users should be contacted and offered a choice about each specific research use.
Table 3. Most Appropriate Approaches for MoleStar: Illustrative Quotes
Some interviewees did not describe a particular approach, but instead made other suggestions that could be applied to any approach, such as a tracking system to make transparent who has accessed end users’ data:
In an ideal world I would include that a company, when they share and sell the data, would need to have a site that users could access to see with whom their data has been shared.
(13_Advocate)Examples of other suggestions included the use of educational modules, quizzes, and ongoing two-way communication, leveraging the interactive nature of mHealth apps:
Apps are designed to keep our attention, to maintain our engagement. [The most appropriate approach would be] to design consents and notices that are like that as well—real-time, updated, frequently communicating with you and letting you know not only how your data is going to be used and how it will be protected privacy and security wise… I think a consent-information type notice should happen regularly [and] keep you engaged in understanding the continued use of this data.
(20_Regulatory)Views on Data Access Procedures for MoleStar
coded data and data use agreement
In discussing MoleStar’s data access procedures, interviewees again identified several advantages and disadvantages for end users, app developers and companies, and unregulated researchers. For end users, many commented favorably on the privacy protection afforded by replacing direct identifiers with a code in data shared with researchers, but anticipated that techniques to re-identify data would continue to evolve. A few noted that photos uploaded to the app may be identifiable…
What if the melanoma is on somebody’s face?
What if the melanoma is on a part of the body that has really recognizable features that could link somebody to that image? We know that image recognition has gotten pretty sophisticated and the image, even without the name, address, and phone number, could be linked to individuals.
(39_Researcher)…or that an app may be “accidentally collecting more data than it should” (38_Developer), undermining efforts to conceal identity:
De-identification is really hard to do, almost impossible if there’s enough content. [For example], does their app scrub the latitude and longitude coordinates that are put into each photo? If not, then the photos themselves, even though they were scrubbed, can still give away the location of the person … and the developer may not even know.
(24_Developer)With respect to data use agreements, advantages included the ability to set clear boundaries and expectations for what may and may not be done with the data. Interviewees commented that a formal agreement between an app company and an unregulated researcher may be helpful in clarifying appropriate uses of data in a manner that is “a bit more of a legal process as opposed to just a handshake or a hand-off in the hallway” (03_Regulatory). Still, many observed that the effectiveness of such agreements varies depending on the terms:
Sometimes they’re really good and really protective. Sometimes they’re very minimal and more about protecting proprietary interest than requiring privacy and security protection. So, it would really depend on what was in that agreement.
(20_Regulatory)Interviewees also noted weaknesses associated with having to rely on the parties to understand, uphold, monitor, and enforce such agreements:
The limitations are that there’s no vetting of who Mr. Lee is. He could be anyone. Just because he put some information in an online form and pays a fee and clicks ‘Yes, I will only use it for algorithms.’ There’s no checking who Mr. Lee is. Whether he’s actually a real person, whether he’s the person he says he is and what his background is for even understanding what he’s signing, how committed he is to fulfilling this agreement… If this is just a person out there with no background in this, he might not even realize the implications of what he’s signing and … what the risks are for the people that donated the data. That is a huge risk.
(39_Researcher)Thus, interviewees recognized that MoleStar’s data access procedures offered valuable protections, though not a panacea:
It is important to have access procedures generally, so that the data aren’t just wide open that anyone could use. Having a data use agreement that a researcher has to abide by is important. You can imagine I’m skeptical about total anonymization, because that’s very, very hard to do, and somebody who really wants to re-identify people probably can. But, making it harder is valuable, because you’re reducing the risk that someone is going to re-identify and do something wrong. If the data use agreement also punishes that sort of thing legally, that can be helpful as well.
(30_Developer)Regarding the effect of MoleStar’s data access procedures on app developers/companies and unregulated researchers, some interviewees were concerned that removing identifiers from data and imposing overly-restrictive data use agreements may constrain potentially beneficial research:
You can’t link records over time, or different people, or whatever it happens to be. The researchers get a fixed set of data and that’s gotta make the research harder… The weakness is that as we learn more and more about health outcomes, identifiers are really important to integrate the information that’s needed to ultimately develop effective approaches.
(09_Regulatory)A few were concerned that high standards for removing identifiers may over-burden app developers/companies:
It’s not trivial cost in staff time to be able to actually make the data available, to clean the data, to code the data, so that they are anonymized, to maintain and enforce the [agreements], possibly to train the researchers on how use and interpret the data. It’s a real operation, and, for a small start-up company, it’s probably not feasible.
(30_Developer)After discussing their views of MoleStar’s data access procedures, just over half of interviewees said they were acceptable (Table 2), and about one-fifth found them unacceptable; the remainder believed that acceptability depended on other factors, such as the specific terms of the agreement and technical security measures.
most appropriate approach to data access procedures
When asked about the most appropriate data access procedures, interviewees identified several key attributes (Table 3). Many emphasized the importance of technical security measures, including the use of multiple methods to remove identifiers, special attention to latent identifiers (particularly in images), and alternative approaches to storing and transmitting data aimed at decreasing and detecting unintended access and/or use.
Interviewees also suggested additional terms beyond what was described for the MoleStar data use agreement, such as requirements to follow standards for regulated research, report any protocol deviations to the company, and dispose of the data in a particular way (e.g., return, destroy) upon completion of the project.
Several described stewardship functions — such as vetting applicants, monitoring recipients, and enforcing the data use agreement — as essential. More generally, interviewees emphasized the importance of transparency and building and maintaining trust with end users:
Google, Amazon, they all have my data. I have no idea what they’re doing with it. I just give it to them every day I’m on the computer, or when I’m using my phone, but I have no idea… So I think there’s a lack of trust in general, and I think the way to close that trust is to be as transparent as possible.
(12_Researcher)Views on Developments in Research Using MoleStar Data
new primary data collection
We also asked experts to consider what, if anything, end users should be told if Mr. Lee (an unregulated researcher) requested that additional questions, specific to his research purpose, be added to the periodic surveys that are part of the basic functionality of Mole-Star. The majority believed that users should receive some information about such questions (Table 2), primarily echoing the themes of transparency, trust, and choice. Some further mentioned specific details of what end users should be told. Most said users should be informed of the particular research purpose, with some also suggesting disclosure of researcher’s identity and/or qualifications, which they emphasized as particularly important in unregulated research:
I feel like I’d wanna know, because my choice to participate in research that I consider more rigorous versus less rigorous — to me, it would matter. I wouldn’t waste my time with something that’s just some random person playing around… I don’t wanna be anti-open science, but I’m struggling with how I feel about just anybody having access to data.
(14_Researcher)Some interviewees described how these questions should be presented. The majority proposed the MoleStar app should, at a minimum, emphasize that these questions are optional and participation is voluntary. Some went further, suggesting users must give express permission for primary data collection for new unregulated research:
People would have to be notified that these are research questions, and now you’re getting into the point where you probably do need informed consent. I think you have to start from the drawing board again at this point. You can imagine, if I told my IRB, ‘I’m just adding new questions to the study, it’s going to be great, don’t worry about it,’ I think they would turn pitchforks on me.
(31_Researcher)The few interviewees who believed users did not need to be alerted to these questions argued that end users were previously notified that any data — including survey responses — may be shared and used for research:
Any survey questions, whether original or new, could be used for research… It’s not practical to think you’re going to pick out two questions and say, ‘Oh, these two questions are for research and the others are just for the company — and, well, they might be used for research, too, at some point if somebody really wants to pay for it, or it might be used for marketing’ and so on.
(37_Regulatory)Some interviewees answered, “it depends,” saying that whether end users should be informed of new research-specific questions could vary based on the nature and content of the initial notification regarding potential research use, the new research purpose and the sensitivity of the data to be collected, and the potential for interfering with the app’s primary purpose (e.g., notification fatigue).
offering individual research results
We also asked interviewees to imagine that Mr. Lee believes his algorithm is highly accurate in identifying moles at risk of becoming cancerous and wants to provide his research results to individuals in the data-set whose images show such lesions. About one-fourth of experts believed that such results should be offered to end users (Table 2). Among them, most discussed notions of fair exchange and decency:
Mr. Lee’s research would not have been successful without the people who provided their data… That’s like a reciprocity, not to mention just being a good human being.
(05_Developer)Other common themes included a general right to information about oneself, as well as the prospect of providing direct health benefit:
It may not be a guarantee that this will happen, but one of the key issues in a disease such as this … is early identification and spurring people to action. Melanoma is probably one of the cancers that kills a lot of people, I would imagine because they aren’t aware of it and don’t act early enough… So, if he has an algorithm that’s more than 50% accurate, it’s imperative that he let the individuals be aware.
(11_Advocate)Several interviewees made suggestions for how to offer results, emphasizing the importance of explaining the uncertainty of the results and protecting end users’ privacy.
About forty percent of experts believed that individual research results generated in unregulated research should not be offered. Most expressed strong reservations about the likelihood that unregulated research would be conducted with sufficient rigor, validation, expertise, or skills. Some were primarily concerned that the algorithm and results had not been verified:
Having Mr. Lee, who is not a card-carrying researcher, if you will, make diagnoses and share them back would just scare people, and there may not be any basis for his conclusions unless there’s some kind of further review of the quality of his work and the conclusions that he’s drawn.
(09_Regulatory)Others questioned unregulated researchers’ qualifications to interpret and communicate health information, noting that “[Mr. Lee] is not a physician, so he’s not qualified to offer clinical diagnoses, treatment, or prognoses” (41_Researcher).
Some interviewees highlighted the lack of upfront consent; as one interviewee stated, offering individual research results should not happen “unless the user explicitly opted in when using the app—and even then, the risks of giving them insights when [the algorithm is] not validated at scale is potentially very damaging” (04_Developer).
A few characterized offering results as an unwar-ranted invasion of privacy, given that MoleStar users were likely already under the care of a medical professional:
It’s just not Health Apps’s business or Mr. Lee’s business to invade people’s privacy and tell them that they could be dying, when we’re already pretty sure they’re seeking medical attention and they’re already monitoring their health on this particular issue. I don’t think that’s appropriate at all.
(26_Regulatory)About one third of interviewees believed that whether individual research results should be offered would depend “on what the people were told, and what they said they did or didn’t want to know, and if any of those things came into the consent process” (27_Regulatory), the unregulated researcher’s qualifications, the reliability and validity of the algorithm/findings, whether the results were independently verified, and additional logistical considerations such as “if information is going to be relayed back, [by] who and how will that be done?” (03_Regulatory).
Views on Substance Abuse App
When discussing the Substance Abuse app, many interviewees described ways it differed from MoleStar. Some perceived these particular end users as “a vulnerable population” (25_Researcher) and expressed overarching concern about marketing use of the data: “I don’t know what’s to stop them from selling the information to liquor companies so they can send discount liquor ads to all these people cause they know they’ll be good customers” (09_Regulatory). More generally, interviewees characterized the data as more sensitive:
The risk of somebody finding out that you have potentially cancerous mole is not going to impact your job prospects or your relationships. Somebody finding out that you might have a substance abuse problem could have some pretty serious social side effects.
(05_Developer)They were particularly concerned about GPS information…
That location data and GPS data is incredibly sensitive information about people. Once you have that, you can pretty much paint a very detailed portrait of a person, where they go throughout their day.
(20_Regulatory)…and the associated potential for legal jeopardy:
Suppose we’re having a custody fight over kids, or some kind of divorce, or some kind of family issue, information about where you’ve been … you spend hours a day hanging around a liquor store, or in an outdoor drug market, what have you, that information can be used against you. It also can be used by police… This is a source of information that could be used by a variety of people if they knew it was there.
(09_Regulatory)Even so, over half of interviewees believed the most appropriate approach to notification/permission would be the same as or substantially similar to what they said would be most appropriate for MoleStar, although some suggested providing additional details would be ideal:
Considering that substance use and abuse is very fraught with stigma and also legal ramifications, this has to be a much more detailed description of the user’s rights and responsibilities in terms of understanding what the company plans to do with their information.
(36_Advocate)A few advocated for a different approach to notifications/permissions from the one they described as most appropriate for MoleStar. A common theme was the increased sensitivity and identifiability of the data:
I think this one is really problematic because people get fired from jobs, they get massively stigmatized, they can get arrested. There’s all kinds of problems users can get into if they are identified either personally or by their location. There’s a huge amount of trouble that this app can get people into… It could potentially do some great good, but the amount of harm that it could do is so substantial that people would have to be really aware of what they’re getting into.
(13_Advocate)Another was concern about undue influence and end users’ capacity to understand and agree to terms of use:
You’re talking about someone who’s probably in really bad shape and they’re looking for all the help they can get in the world… That person who’s dealing with substance abuse, they don’t care about your research. They care about staying clean and sober… Research is the last thing on their mind.
(19_Advocate)Responses were generally similar regarding data access procedures. The majority of interviewees advocated for the same approach as they had for MoleStar: “I think the same general principles apply — certainly this is more invasive, but I think the same protections apply regardless” (31_Researcher). Some suggested additional or heightened technical data security measures and more stringent data use agreement terms:
The company needs to be much better educated on the sensitivity and identifiability of data and data security management, especially the GPS data. I think they have an obligation to hire experts and consultants to make sure they do a good job understanding which data they’re sharing with whom and whether the data … has enough information redacted… If they’re giving people access to sensitive data, they may want stronger contractual agreements. Sometimes that involves only wanting to share the data with people who have organizations to back them up, for example, people who are in academic institutions. It sucks that we have that divide between the citizen scientists and the institutions, but part of the reason it does exist is because those institutions are there to try to enforce more precautions and vetting and resources to enable good practices.
(38_Developer)Only a few believed the most appropriate approach to data access procedures for the Substance Abuse app would be substantially different from what they described as most appropriate for MoleStar, citing potential interest by parties other than researchers:
It may put people at greater risk if there are no legal protections other than a data use agreement between the data provider and data user. That by itself can’t protect you against all the legal jeopardy that’s out there.
(09_Regulatory)Views on Sharing Data from Unregulated Research
A large majority of experts said unregulated researchers should seek to make their data and/or aggregate results available to others. Over half emphasized the importance of dissemination for advancing science:
Why were the researchers interested in this in the first place, if it’s not to draw conclusions, expand the scientific body of knowledge, generate new information? Why were they doing it if it’s not to disseminate the results?
(03_Regulatory)Some focused especially on the importance of efficiency, referencing the value to the field of “having access to all the research, not just selected research” (09_Regulatory), to avoid unnecessary duplication and build on what others have learned.
Many interviewees cited ethical considerations, describing dissemination of data and aggregate results as “the right thing to do” (08_Advocate) and “a basic ethical and civic responsibility that people have to share what they learn when they’re learning it based on engaging the public” (32_Advocate).
Some experts who believed that unregulated researchers should seek to share their findings nonetheless noted competing considerations that may act as disincentives:
They may want to hide their results — they may want to profit from their results if they find something that’s exploitable, they may want to hide their failures, they may want to hide their incompetence. There are reasons why people would not want to publish research.
(09_Regulatory)Regarding publication in particular, views expressed ranged from having a dedicated “citizen science section, so everyone knows exactly what we’re looking at” (18_Researcher) to the expectation that all researchers should meet the same standards:
If you’re doing science, then that would be the goal: to have it subject to the scrutiny that science is subject to. I don’t think we can reinvent science for a group of people who wanna circumvent standards. If you have something that you’ve found that’s awesome, then yes, I think the only way that this can be considered legitimate is to submit it as a paper and publish it.
(14_Researcher)Only a few interviewees did not believe that unregulated researchers should seek to make their data and results available to others. These experts doubted that unregulated research would be appropriately designed, conducted, or validated; thus, they foresaw risks arising from sharing data and/or results from research lacking ethical or scientific rigor:
We’ve seen it historically: unregulated or unfounded research claiming things really makes societal impact in a negative way. We find out five to ten years later that it was all bogus. It is tremendously important that in this day in age of fake news, and how fast things spread, regulation on these types of things are more and more critical. Not validated? Should not be published.
(02_Developer)Another theme was concern about stifling innovation:
Mr. Lee, if he knew he would have to make his data available, is probably not going to invest in this because he wants to make money off it. He apparently wants to save the world from melanoma, but he has to make a living. If he’s investing in this, he probably doesn’t want to give his data away for that reason… On a voluntary basis, do I think it’s a good idea? Sure. Do I think the government should come down and tell them they have to do it? I have a harder time with that.
(27_Regulatory)About one-fifth believed that unregulated researchers’ ethical obligations regarding dissemination would depend on contextual factors such as competing obligations to stakeholders, limited resources, and privacy considerations.
Discussion
Although data collected by mHealth apps and devices may be valuable for research,Reference Dorsey9 the use of mHealth data in research that is not subject to federal regulations for the protection of human research participants raises pressing ethical, legal, and social questions. Answering these questions is essential to ensuring that end users are protected against the kinds of risks and harms that federal regulations are intended to address, without overly restricting the conduct of potentially beneficial research.
The role of empirical data is to inform the development of ethical policy and practice. Qualitative, descriptive studies such as ours do not provide definitive answers, but rather illuminate critical issues from multiple perspectives. The interview results reported here suggest several key points to consider in the development and implementation of ethical approaches to unregulated mHealth research.
First, there are several possible approaches to informing end users about the potential for research use of their data, ranging from models that simply notify them, to those that allow for a broad yes/no choice, to those that provide for more detailed choices or even full informed consent for each specific research use. Consistent with prior recommendations,Reference Vayena, Tasioulas and Lynn10 experts in our study identified a range of factors that should be considered in selecting the most appropriate approach, including the level of risk involved (e.g., identifiability, sensitivity of data), burden on end users, practicability for research, and the effectiveness of the approach in informing end users.
Regardless of the approach selected, promoting and maintaining transparency and trust are essential to protecting end users as well as the research enterprise. Potential strategies include designing processes to actively call end users’ attention to information about research use, developing easy-to-read disclosures that focus on key details most important to end users, and making additional information about research use available elsewhere for those who might be interested.
Second, when an unregulated researcher seeks to use data gathered originally by an mHealth app for non-research purposes, a variety of data access procedures could be used to help protect end users, such as data use agreements and careful removal of identifiers from datasets, as well as requiring independent oversight of the proposed research.11 Designing and implementing the most appropriate combination of procedures should account for the ability of all parties to understand, uphold, monitor, and enforce the terms of data use agreements, and the associated need to meaningfully vet potential researchers’ capabilities and resources.
Third, if an unregulated researcher were to request new data collection through an app whose primary purpose is not research, app developers/companies and researchers must decide what information (if any) should be disclosed to end users about this activity (e.g., the particular research purpose, the researcher’s identity and/or qualifications). Important considerations include what end users were initially told about potential research uses of their data and the sensitivity of the new data to be collected.
Fourth, when determining whether or not — and, if so, how — individual results from unregulated mHealth research might be offered, vital considerations include what end users knew about and/or gave permission for when they downloaded the app, the validity and reliability of the results (particularly given the unregulated context), researchers’ ability and expertise to accurately assess and communicate any health-related implications of the results, and end users’ potential claims to information about themselves.
Finally, stakeholders should consider unregulated researchers’ responsibilities with respect to sharing their results (e.g., making data available through centralized databases, publications). To make a positive contribution to generalizable knowledge, careful attention must be given to the rigor, expertise, and validity brought to the conduct and interpretation of the research.
The results of our study are subject to some limitations. Given the qualitative nature of our study and our interviewees’ multiple areas of expertise, we did not attempt to analyze similarities or differences between stakeholder groups. The prevalence of and rationales for potentially differing perspectives between stake-holder groups may be an area for future research. We conducted these interviews in 2017-2018 with experts throughout the United States. While we believe many of our findings reflect fundamental ethical considerations, mHealth technologies, research using mHealth data, and privacy expectations among individuals and groups are rapidly and constantly evolving; ongoing attention to these issues over time is essential.
Acknowledgments
Research on this article was funded by the following grant: Addressing ELS Issues in Unregulated Health Research Using Mobile Devices, No. 1R01CA20738-01A1, National Cancer Institute, National Human Genome Research Institute, and Office of Science Policy and Office of Behavioral and Social Sciences Research in the Office of the Director, National Institutes of Health, Mark A. Roth-stein and John T. Wilbanks, Principal Investigators.
Appendix A. Interview Guide
Did you have a chance to read over the study information sheet?
Did you have a chance to read over the hypothetical scenario?
Any questions about either?
Is it okay to audio record the interview? ☐ Yes ☐ No
☐ App/device: Before we get started, tell me about your experience or involvement in health-related research using mobile devices or apps.
☐ Regulatory/policy: Before we get started, tell me about your experience or involvement in health-related research using mobile devices or apps.
☐ Researcher: Before we get started, tell me about your experience or involvement in health-related research using mobile devices or apps.
☐ Patient/participant Advocate: Today we’re going to talk about health-related research using mobile apps/devices—is that anything you’ve thought about for your advocacy organization, or you think your constituency group would be interested in?
[Scenario 1: MoleStar App]
2. Imagine that a close family member or friend is thinking about using the MoleStar app.
2a. Considering the various features of the app, how would you describe to him/her the main benefits of using it?
2b. Considering the various features of the app, how would you describe to him/her the main risks of using it?
2c. Thinking about the benefits and risks you just told me about, how worried should people be about using this app?
→ If participants ask for clarification regarding “worry”: Consider the benefits you’ve identified, along with the probability of the risks you identified occurring, and the magnitude of harms if the risks did occur.
2d. Where would you put that on a scale from 1 to 5, where 1 is “not at all worried, by all means, use the app” and 5 is “very worried, think long and hard before using the app”?
---1----------2----------3----------4----------5---Other: __________
In the hypothetical scenario, we described several different steps that were being taken to protect human subjects. One was general notification to inform app users that their data could be used for research. A second set of protections were about data access procedures, including a data use agreement between the company and the researcher, and the researcher getting only coded data. So, I’d like to ask you about each one of those separately.
3. Let’s start with general notification of potential research use:
3a. In your opinion, what are the strengths of general notification as a way to let people know their data could be used for research?
3b. What do you think are the limitations of general notification as a way to let people know their data could be used for research?
3c. How reassured do you think your family member/friend should be by general notification as a way to let people know their data could be used for research?
3d. Where would you put that on a scale from 1 to 5, where 1 is “not at all reassured” and 5 is “very reassured”?
---1----------2----------3----------4----------5---Other: __________
3e. Okay, thank you. Now, for this next question, let’s set aside strict regulatory requirements (or what IRBs typically expect or do), and just focus more generally about informing people that their data could be used for research.
Given the strengths and weaknesses you just described, do you think general notification is acceptable? In other words, regardless of whether you think this is best approach or not, is it an acceptable approach?
3f. Let's continue to set aside strict regulatory requirements (or what IRBs typically expect or do). What do you think would be the most appropriate approach to let people know that their data could be used for research?
3g. So, if we think about putting that into actual practice, tell me how you would see the pros and cons of that approach.
4. Now let’s talk about the data access procedures, which included a data use agreement between the company and the researcher, and the researcher getting only coded data.
4a. In your opinion, what are the strengths of these procedures for protecting human subjects?
4b. What do you think are the limitations of these procedures for protecting human subjects?
4c. How reassured do you think your family member/friend should be by these data access procedures?
4d. Where would you put that on a scale from 1 to 5, where 1 is “not at all reassured” and 5 is “very reassured”?
---1----------2----------3----------4----------5---Other: __________
4e. Okay, thank you. Now, again, let’s set aside strict regulatory requirements (or what IRBs typically expect or do), and just focus more generally on data access procedures.
Given the strengths and weaknesses you just described, do you think the procedures described are acceptable? In other words, regardless of whether you think this is best approach or not, is it an acceptable approach?
4f. And continuing to set aside strict regulatory requirements (or what IRBs typically expect or do), what do you think would be the most appropriate approach to data access?
4g. So, if we think about putting that into actual practice, tell me how you would see the pros and cons of that approach?
Now let’s talk about a few potential developments in Mr. Lee’s research.
5. Let’s say that Mr. Lee believes his algorithm would very likely work better with additional health information that is not currently captured by MoleStar. To help Mr. Lee test his theory, Health Apps, Inc. agrees to gather the information he needs by adding a few questions to one of its periodic surveys.
What, if anything, do you think users should be told about the purpose of these new questions?
→ Probe: Tell me more about the reasons why it’s important or necessary that users be told that?
6. As another possibility, let’s say Mr. Lee believes that his algorithm seems to be highly accurate in identifying moles at risk of becoming cancerous. He becomes extremely concerned about individuals in the data set whose images show such lesions. He feels strongly that his research results should be conveyed to them so they can take action to avoid his wife’s fate.
In your opinion, should results be offered?
[Scenario 2: Substance Abuse App]
7a. What do you think would be the most appropriate approach to letting people who use this substance abuse recovery app know that their data could be used for research?
7b. What do you think would be the most appropriate approach to allowing researchers to access data from this substance abuse recovery app>
Appendix B. Consolidated Criteria for Reporting Qualitative Studies (COREQ)
Appendix C. Additional Selected Quotes
1. Level of concern about MoleStar app
2. MoleStar’s use of General Notification
3. Most appropriate approach to disclosure/permission in MoleStar
4. MoleStar’s data access procedures
5. Most appropriate approach to data access procedures in MoleStar
6. Should app users be told the purpose of new questions on periodic surveys?
7. Offering individual research results
8. Most appropriate approaches for Substance Abuse app
* Relative to what interviewee said would be the most appropriate approach for MoleStar
9. Should unregulated researchers seek to make their data widely available?
