Why your team trusts AI too much, or too little
Some people copy every AI answer without checking; others give up after one mistake. What research says about trust in AI, and how to get your team to the right level of trust for each task.
- reading time
- 3 min
- sources
- 5 · 4 peer-reviewed
- published
- language
- Lees dit in het Nederlands
- After one AI mistake, people give up faster than after the same mistake by a colleague. That's algorithm aversion.
- At the same time, we readily copy polished AI answers without checking. That's automation bias.
- The goal isn't more or less trust, but the right trust for each task. Three questions help.
You see it in almost every team. One person pastes everything into ChatGPT and takes whatever comes back. Someone else tries it once, spots a mistake and never touches it again. Both are very human reactions, and both cost your team time and quality.
Too little trust: one mistake and you’re out
Researchers call this algorithm aversion. In a series of experiments, Dietvorst, Simmons and Massey (2015) asked people to choose between their own forecasts and those of an algorithm. Once people had seen the algorithm make a mistake, they were far less likely to choose it, even when it outperformed them on average.
Sound familiar? A colleague asks AI for a summary, finds one wrong number and decides it “doesn’t work”. We’d never write off a colleague for a single slip like that.
Too much trust: it just looks so good
The opposite happens too. Logg, Minson and Moore (2019) found that in some situations people are more likely to follow advice when they think it comes from an algorithm. And people who work with automated systems check less carefully the more reliable the system seems (Parasuraman & Manzey, 2010). That’s automation bias.
With language models the effect is even stronger. An AI answer is always well phrased, confident and neatly structured. None of that tells you whether it’s right.
Why AI always sounds convincing
A language model doesn’t know anything in the everyday sense of the word. It keeps predicting the most likely next word, based on vast amounts of text. That’s why its answers always read smoothly, even when the content is wrong.
Once you understand that, you start asking a different question. Not “does this sound right?” but “can I check this, and do I need to?”
The right trust for the task
So the goal isn’t for your team to trust AI more, or less. It’s for the level of trust to fit the task. A study with consultants shows why that matters: on tasks well suited to AI, they worked faster and better. On a task outside that range, they were more likely to get it wrong than colleagues without AI (Dell’Acqua et al., 2023).
Three questions that help:
- Can I check the answer? For an email you’ll read through anyway, the risk is small. For figures, sources or legal wording, it isn’t.
- What happens if it’s wrong? The bigger the consequences, the more checking you need.
- Do I know enough about this myself? AI is at its best when it helps with work you could judge on your own.
Make it a team conversation
You can’t fix trust in AI with a manual. It’s shaped in conversation: what went well, what went wrong, what do you watch out for? Teams learn best when people feel safe sharing doubts and mistakes (Edmondson, 1999), and that applies here too. Whoever dares to say “I don’t trust it” or “I just copied it without checking” helps the whole team move forward.
Start small. In your next team meeting, share one example where AI helped and one where it went wrong. You’ll find that everyone has a story.