Joanna Gotlib-Małkowska

Can ChatGPT Provide Patient-Friendly and Reliable Information on Cervical Cancer Screening? A Study of ChatGPT-Generated Information in Polish

BACKGROUND Cervical cancer (CC) mortality remains a global health problem, and women's awareness of the need for regular CC screening is insufficient. In the era of rapid development of artificial intelligence (AI), large language models (LLMs) such as ChatGPT may become an important source of information about health, including women's health and screening. The aim of the study was to assess the quality of ChatGPT-generated medical information on cervical cancer screening (MICC_GPT). MATERIAL AND METHODS A total of 196 MICC_GPT items were assessed. Objective readability was assessed on a scale of 1 to 7 using Jasnopis.pl programming. Subjective assessment of the MICC-GPT on a scale of 1 to 5 was made by 4 independent expert reviewers. RESULTS The average difficulty level of the MICC_GPT was 5.314, and full understanding of the MICC_GPT requires the target audience to have completed at least 15 years of formal education. In the opinion of the expert reviewers, most of the MICC_GPTs (71%) met the criteria of relevance, accuracy, completeness, clarity, coherence, and appropriateness. CONCLUSIONS ChatGPT-generated information in Polish on the topic of cervical cancer screening is accurate, detailed, comprehensive, clear, coherent and appropriate for the target audience, confirming the great potential of ChatGPT in health education, as well as promotion of screening tests and cervical cancer prophylaxis. To reduce the level of difficulty and increase the accessibility of the MICC_GPT for less educated end-users, the MICC_GPT needs to be simplified, especially in terms of specialized medical terminology.

Effectiveness of ChatGPT in simplifying information for Polish patients on cervical cancer screening using advanced prompting

ChatGPT can generate reliable medical information in gynaecology and obstetrics,but the content is often difficult to understand for patients with lower educational levels. The aim of the study is to evaluate the impact of Audience Persona Prompting on the simplification and readability of ChatGPT-generated medical information on cervical cancer screening (MICC_GPT) in Polish. 392 MICC_GPT were analyzed, with 196 generated using Zero-Shot Prompting (STANDARD) and 196 generated using Audience Persona Prompting (EASY). The Audience Persona prompts included instructions to simplify the content: 'Explain as if to an average Polish woman with only primary education' (8 years of formal schooling). Readability was assessed using 24 objective linguistic indicators available at Jasnopis.pl. Statistica 13 (StatSoft, Poliand), the Brunner-Munzel test, p < 0.05. The average difficulty level of STANDARD output was 5.32 (at least 15 years of formal education), while EASY output averaged 4.15 (12 years of formal education). Of the 24 indicators, 21 showed statistically significant improvements in the simplification of EASY output (p < 0.05). While ChatGPT significantly simplified MICC_GPT, the readability levels remained too high for patients with only primary education. ChatGPT shows promise in tailoring medical information on cervical cancer (CC) screening for the needs of Polish patients with varying educational backgrounds, with the use of advanced prompt engineering techniques. However, further research is required to refine prompt engineering methods and develop effective strategies for generating information on cervical cancer screening that is accessible to individuals with only primary education.