Presenting Authors: Travis Smith, PharmD, MBA, Senior Manager–Research Pharmacy, Mayo Clinic, Rochester, MN; Alan Yee, PharmD, MS, Research Pharmacist, Mayo Clinic, Rochester, MN
Co-Author: Camille Walters, PharmD, Research Pharmacist, Mayo Clinic, Rochester, MN
BACKGROUND: Clinical research protocols are often complex, intricate, and lengthy documents and require the development of department-specific summaries to operationalize at the study site. Creating these summaries requires significant pharmacist time and effort. Incorporating artificial intelligence (AI) into this process may significantly decrease administrative burden and time while maintaining clinical accuracy.
OBJECTIVES: To assess the time required to complete pharmacy-specific study summaries manually versus AI-assisted and to assess the accuracy of documents created with AI assistance.
METHODS: As assessed by a scoring system, moderate- or high-complexity trials were eligible for inclusion. The time required for the manual preparation of pharmacy study summaries through usual workflow by the lead pharmacist for the study was recorded. Microsoft Copilot was prompted to complete a study summary template using the study protocol and pharmacy manual. The resulting AI-assisted summary was reviewed by an investigational drug service (IDS) pharmacist who was naïve to the study protocol. The time taken by an IDS pharmacist to complete the document was compared with the time taken by an IDS pharmacist assisted with AI. A mean difference of 20 minutes was defined as significant a priori. The number of corrections and grades of error were recorded for each study.
RESULTS: A total of 4 protocols were summarized manually and AI-assisted in parallel. The mean time to complete a study summary with AI assistance was significantly lower than manual summarization (35 minutes vs 167 minutes, respectively; P=.0296). There was an average 74% reduction in the time to complete the study summaries using AI-assisted workflow. Each initial prompt output required an average of 4.5 corrections.
DISCUSSION: In this case, the burden of time-consuming creation of documentation was mitigated by utilizing predesigned prompts to create standardized document output. Although manual reprompting and quality control were required to correct major errors, improvements in the prompt model and automation may further reduce manual effort.
CONCLUSION: The integration of AI into the clinical trial workflow significantly reduced the time to synthesize a study summary document when compared with traditional, manual processes.