Artificial Intelligence (AI) in Research
Stay up to date on IRB expectations around the use of AI technology in human research at the University of Louisville.
Sidebar
Overview
It is vital to develop effective guardrails around the use of AI technology in human subject research. These guardrails must prioritize the protection of human participants, research integrity and innovation, while minimizing risks to participants and their data. In order for the IRB to conduct an ethical review of the AI technology, the IRB must have a clear understanding of its use and how participants and their data will be protected. This guidance was created to assist researchers in navigating the process for conducting human subjects research with AI and providing a resource for best practices. It is important to note, the AI landscape is ever evolving. As new information arises, the resources and guidance shared here will be updated.
What is AI?
Artificial intelligence combines computer science, data and algorithms to enable machines to solve problems, make predictions and classify information. - IBM, “What is Artificial Intelligence (AI)?”
AI technologies may be used to collect, process or analyze data as well as generate outputs that inform study activities or participant interactions. A research protocol may include more than one example of AI technology. It is important to understand what type of AI technology is involved in the study so that the research team and the IRB can work together to anticipate potential risks and identify appropriate privacy, bias and safeguards needed to conduct compliant research.
Understanding AI Systems
Traditional AI
Traditional AI relies on preprogrammed rules and algorithms to perform specific tasks. It has clear goals. It does not generate new information or engage in creative problem solving or predictions. It excels at repetitive tasks.
Examples: Google translate, basic email spam filters
Machine Learning
Machine learning is a subset of artificial intelligence. It involves sophisticated algorithms which can be trained to sort information, identify patterns and make predictions with large data sets without being explicitly programmed to do so.
Examples: fraud detection, customer service chatbots, virtual assistants like Alexa or Siri
Deep Learning
Deep learning is a subset of machine learning. Deep learning models train artificial neural networks on large amounts of data to learn patterns and representations. Once trained, the model can make autonomous decisions/predictions.
Examples: facial recognition, healthcare image interpretation
Generative AI
Generative AI is a subset of deep learning. Generative AI models are a system of algorithms that can create novel output in text, images or other media based on user prompts. These are created by programmers who train them on large data sets. The AI learns by finding data and can provide novel outputs to users' queries based on its findings.
Examples: ChatGPT, Microsoft Co-Pilot, Anthropic's Claude
Researchers must understand risks associated with using AI systems to interact with research participants and their data. As the AI system becomes more interactive with participants and/or drives decision making, risk increases.
Efficiency | Inform | Drive |
|---|---|---|
| The technology is intended to complete tasks for researchers, administrative staff, and other users such as collecting and collating information, writing short computer programs, or filling in paragraph bodies from thesis statements. | Research related decisions are made (and confirmed) without the technology, but the technology can support the decision. | The technology is intended for use as an autonomous system, alerting the user of identified risk, etc. Decisions are made by the AI model. |
AI Technology Use
IRB Review Considerations
The UofL IRB must ensure research involving AI adheres to the fundamental ethical principles described in the Belmont Report: Beneficence, Justice, and Respect for Persons. These principles are incorporated into the guidance documents linked below. These guidance documents should be used to enter information into the protocol and consent templates.
Protocol Guidance Consent Guidance
Responsibilities of Researchers Using AI Technology
Using AI technologies in human subjects research raise new and additional considerations not found in traditional research protocols. The principal investigator (PI) is ultimately responsible for understanding these risks and ensuring adequate protections are in place in the research design to protect participants and their data from harm. These risks may include issues related to data privacy and confidentiality, accuracy and reliability of outputs, bias and data quality and the level of transparency or human oversight.
It is required that investigators ensure compliance with regulations (e.g. HIPAA, FERPA, FDA, etc.), local laws and institutional policies (e.g. privacy, information security, data sharing, vendor vetting, etc.).
Researchers have a responsibility of disclosure, discretion and verification.
Disclosure | Discretion | Verification |
|---|---|---|
| Document and disclose the use of AI in all aspects of a research process. For example, disclosure to participants, IRB, funders, journals, etc. | Do not share confidential, sensitive, proprietary and/or export controlled information with non-University approved AI tools. This may violate legal/regulatory requirements (e.g., HIPAA, FERPA), contractual requirements and/or institutional policies. | Verify the accuracy and validity of the AI results. |
| Disclose the use of AI (in any capacity) in the IRB application and research protocol. This includes, but it not limited to, disclosing AI use for data collection/storage, formatting, analysis and participant interaction. | Follow UofL ITS Policies related to data privacy and security. | Confirm diversity in the source and assure bias is limited. |
| Disclose any connections of AI to crucial systems, such as the electronic medical record. | Utilize UofL approved vendors with sensitive proprietary research data. | Check for unintended plagiarism. AI can generate copies of existing work. |
| Disclose the use of AI in the informed consent, including potential risks and limitations of the AI use. | Limiting the amount of data that is uploaded into AI to the minimum necessary in order to answer the research question. | Apply ongoing monitoring to assure AI is working as intended. |
Clinical Research Investigations
When do FDA regulations apply to AI research?
AI technologies may be considered a medical device if they are “…intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease…” If AI technology meets the definition of a medical device AND the protocol investigates the safety and or effectiveness of the technology, the protocol is an FDA regulated clinical investigation.
The FDA has established an interactive decision tool to help determine whether a product's software functions are potentially the focus of the FDA's oversight. This tool is designed to walk through each step to determine where a research project falls within the FDA requirements. Additionally the FDA has resources and information available for researches on the FDA Device Advice page.
