X Seminário Internacional Inovações Curriculares

Introduction of large language models in scientific methodology must be accompanied by intensive training in critical thinking

Not scheduled

20m

Auditório da Faculdade de Ciências Médicas (Universidade Estadual de Campinas)

Pôster presencia Projetos Pedagógicos com incorporação de competências digitais

Speaker

Konradin Metze (FCM/UNICAMP)

Description

Artificial intelligence (AI) is of growing importance in science and education. Large language models (LLMs) have become very popular, since they are freely available and user-friendly.
Recently, it has been demonstrated that prolonged use of AI may be harmful, since it diminishes critical thinking and other cognitive skills. Therefore, when introducing LLMs in the classroom, this should be accompanied by training in critical thinking.
Here we present a strategy that was integrated in a post-graduate course on “Scientific methodology in biosciences”. After an introduction into the history of critical thinking (enlightenment, critical rationalism) the technical principles of LLMs were explained.
Then, tasks related to biomedical methodology were given. Homework questions were answered according to the following protocol: first the student had to write a solution without using LLMs. Then a prompt was elaborated and submitted to 3 different LLMs from a list of 20 chatbots and their replies were verified by external sources and mutually compared. A final conclusion was created by integrating the LLM answers with the human opinion. The tasks varied from simple bibliographic search to more complex questions such as: indication of an examination method and the correct statistical test, search of hidden structures in data, detection of errors in study planning, data collection and interpretation, detection of contradictions and explanation of paradoxical results. The different chatbots presented varying styles of reporting and wording, sometimes in a confusing manner. Hallucinations were present, sometimes difficult to be detected.
All answers for the homeworks were discussed. Thus, participants could learn that LLMs, although helping with some tasks can provide misinformation and therefore should always be closely supervised by the experienced researcher.
Supported by FAPESP (Bios project), CNPq and BAS (grant UNICAMP)