After AI help, participants skipped tasks without chatbot more often — Live Science

A study presented at a language modeling conference in San Francisco found that participants who used a chatbot to solve simple tasks were more likely to skip exercises or give incorrect answers after losing access to it. A total of 1,222 people took part in three randomized controlled experiments, Live Science reports.

The researchers examined how offloading mental work to a chatbot affects performance on arithmetic and reading-comprehension tasks. Participants in the different experiments did not overlap, and each test lasted from 13 to 15 minutes.

Experiments with fractions

In the first test, 354 people were divided into two groups. One had access to a chatbot and could receive ready-made answers or hints for 12 fraction problems, while the other did not. Access to AI was then turned off without warning, and all participants were given three more similar exercises.

While the chatbot was available, its users solved problems more often and skipped them less often. After the assistant was turned off, their results worsened: they were more likely to skip tasks or fail to complete them. Participants were told that there would be no penalty for incorrect answers, and the correct answer was shown after an error.

More current news is available on the UA.News Telegram channel Telegram.

Similar result in three tests

The second experiment, involving 667 people, repeated the basic design but included a preliminary test. It made it possible to exclude people who could not solve the simplest tasks. The results were similar to those of the first test, especially when the chatbot provided ready-made answers rather than hints.

In the third test, 201 participants were given text-comprehension questions. The group with access to AI first answered five questions, after which the chatbot was turned off and everyone was given three more tasks. The results also indicated lower persistence after AI assistance ended.

The study’s author, Carnegie Mellon University machine learning doctoral student Grace Liu, linked this to offloading mental work onto technology. According to the researchers, after becoming accustomed to instant answers, people may perceive a method of solving problems that takes more time as inefficient.