Imagine using an AI agent as your first line of customer support. When a customer contacts you, they are greeted by an LLM-based chatbot that answers common questions. A perfectly reasonable solution that saves time for both the organization and the customer, who doesn't have to wait for an available agent.

But what happens when the chatbot makes a mistake? When it provides incorrect information that has consequences for both the customer and the company. Who is then responsible? The customer? The developer? Or is it the company that the AI agent represents?

This dilemma became very real for Air Canada in 2022. Their chatbot informed a customer that he could get money back on a trip after the fact because he was entitled to a discount. The problem? The chatbot was hallucinating. The discount only applied if requested. before the journey.

Air Canada offered a cashier's check, but the customer refused and sued the company. The court ruled in his favor: the company was liable for the chatbot's mistake. The airline had to pay the full amount plus legal costs.

The case illustrates how complex the liability issue becomes when we let AI act on our behalf. It’s easy to think that Air Canada should have taken responsibility right away and just paid the discount. But if we draw the lines, the issue becomes much more difficult:

• Who is responsible if an AI agent deletes all customer data (something that actually happened in a high-profile case)? CASE with the development tool Reply)?  
• Who is responsible if a self-driving car causes an accident?
• Who is responsible if an autonomous weapons system commits a war crime?

These are questions that researchers in ethics and law are grappling with around the world.

Unlike classical code, AI models are non-deterministic. We cannot follow a string of code and see exactly why the model gave a certain response, which also makes them difficult to test. Therefore, we cannot consider AI errors as:

• a manufacturing defect
• a bug
• a childhood illness

Even though the models are getting better, they will always to be able to make mistakes — just like humans. At the same time, AI models are trained to appease the user, without any real ethics, morals or context. They can sound confident even when they are wrong. That makes the issue of liability unique. 

To take advantage of AI's enormous benefits, we must weigh:

• The gain in efficiency and cost savings
  against
• The risk of harm if the AI makes a mistake

At one end of the spectrum: an incorrect discount. At the other: something that could threaten the entire business.

Therefore, there should always be someone who is responsible – whether it is a person or a company.

My recommendations in brief:

• Clarify responsibilities — a person should be able to stop, correct or approve decisions.
• Introduce multiple levels of review in sensitive cases — important decisions should have multiple pairs of eyes on them.
• Let the risk guide you — is it a test environment or a live operation with customers? Adjust the level of control accordingly.
• Ensure that you have control over your AI agents. You need to ensure that the agents do not have the rights, in practice do not have the ability, to do things that you do not want. As experience has shown, you cannot trust them to refrain just because you have instructed them not to.