ASSESSMENT OF SAFETY AND ACCURACY OF AI CHATBOTS IN BIOCHEMICAL REPORT INTERPRETATION AND DRUG RECOMMENDATIONS

Background: Artificial intelligence (AI) chatbots are increasingly explored as clinical decision-support tools. However, their accuracy and safety in biochemical report interpretation and pharmacotherapy recommendations remain insufficiently evaluated. Objective: To assess the accuracy, safety, consistency, and error patterns of AI chatbots in interpreting biochemical data and providing pharmacological recommendations. Methods: This cross-sectional, case-based analytical study evaluated 100 standardized clinical scenarios across diabetes mellitus, dyslipidemia, renal dysfunction, and liver disorders. Five AI chatbots were assessed using a uniform prompt. Responses were compared with gold-standard references and scored using a composite system (maximum score = 9) evaluating interpretation accuracy, therapeutic accuracy, safety, and justification. Statistical analysis included Kruskal–Wallis and Chi-square tests. Results: Significant differences were observed among chatbots (p = 0.003). Chatbots A and D demonstrated superior performance, with excellent responses in 68% and 60% of cases, respectively, compared to 28% in chatbot E. Interpretation accuracy was consistently higher (91% in chatbot A) than therapeutic accuracy (87% in chatbot A) across all models. Domain-wise performance was highest in diabetes (91%) and lowest in renal dysfunction (78%). Safety analysis revealed that 75–92% of responses were safe, although moderate-to-severe safety concerns occurred in up to 13% of cases in lower-performing models. Consistency was highest in chatbots A and D (85–88% identical responses), while chatbot E showed greater variability (16% different drug recommendations). Drug selection errors (up to 20%) and safety-related errors (up to 18%) were more frequent than interpretation errors. Conclusion: AI chatbots demonstrate strong performance in biochemical interpretation but exhibit limitations in therapeutic decision-making, safety, and consistency. Their role should remain adjunctive, with clinician supervision essential for safe clinical integration.