AI is increasingly prompting a ready-to-date response where a person would have to admit a lack of knowledge, and, as it was recently revealed, such assistance can reduce the reliability of conclusions. Experts from universities in France and Italy found that access to the advice of a language model makes people less likely to answer “don’t know”, even when the AI offers a erroneous answer.
Participants were asked questions about subtle details from films that are rarely found in training data. One group had to respond on their own, and the other could turn to the Step 3.5 Flash model. The authors specifically chose the AI system, which was more likely to make mistakes on such tasks to check the reaction of people to unreliable clues.
Without the help of AI, 44% of participants admitted that they did not know the answer. But after the appearance of clues, the share of such answers fell to 3%. At the same time, the correctness of such answers has also decreased. In the first group, 27% of the participants correctly answered, and in the second only 9%. Some people could first choose the right option, but changed the decision after the misguided advice of the model.
Confidence has grown in the opposite direction. Without AI, the correctness of their response was highly appreciated by 30% of the participants, and with tips already 76%. Thus, access to the model not only worsened the result, but also created a false sense of reliability of the answer in people.
Right now, someone's hunting you. You don't know about it, it's in vain.
The monetary reward for accuracy mitigated the effect. The share of answers “I don’t know” increased from 3% to 8%, and the right answers from 9% to 16%. Both indicators still remained noticeably lower than the results of the group that worked without AI.
The authors believe that the problem is not limited to questions about cinema. Constant access to ready-made answers can prevent people from assessing the boundaries of their own knowledge and maintaining a critical attitude to clues.
To reduce the risk, according to experts, will help to train work with AI, in which the answers of the models are checked, and recognition of uncertainty is considered a normal part of reasoning.
Participants were asked questions about subtle details from films that are rarely found in training data. One group had to respond on their own, and the other could turn to the Step 3.5 Flash model. The authors specifically chose the AI system, which was more likely to make mistakes on such tasks to check the reaction of people to unreliable clues.
Without the help of AI, 44% of participants admitted that they did not know the answer. But after the appearance of clues, the share of such answers fell to 3%. At the same time, the correctness of such answers has also decreased. In the first group, 27% of the participants correctly answered, and in the second only 9%. Some people could first choose the right option, but changed the decision after the misguided advice of the model.
Confidence has grown in the opposite direction. Without AI, the correctness of their response was highly appreciated by 30% of the participants, and with tips already 76%. Thus, access to the model not only worsened the result, but also created a false sense of reliability of the answer in people.
Right now, someone's hunting you. You don't know about it, it's in vain.
The monetary reward for accuracy mitigated the effect. The share of answers “I don’t know” increased from 3% to 8%, and the right answers from 9% to 16%. Both indicators still remained noticeably lower than the results of the group that worked without AI.
The authors believe that the problem is not limited to questions about cinema. Constant access to ready-made answers can prevent people from assessing the boundaries of their own knowledge and maintaining a critical attitude to clues.
To reduce the risk, according to experts, will help to train work with AI, in which the answers of the models are checked, and recognition of uncertainty is considered a normal part of reasoning.
