What happens to security when AI starts acting on its own?

What happens to security when AI starts acting on its own?

AI has in a short time gone from being for many just a tool we ask questions to, to advanced systems that can perform tasks for us. The next step is already on its way into organizations: AI agents that can read information, make decisions, use digital tools and act in other systems. This opens up great efficiency gains, but it also changes the risk picture.
In a survey by Dark Reading, 48 percent of respondents believe that agent-based AI will be the top attack vector for cybercriminals and state-sponsored actors by the end of 2026. It’s a concern that reflects a larger shift. The more leeway we give an AI agent, the greater the consequences if something goes wrong.
But does that mean organizations should be wary of AI agents? Not necessarily. The more important question is how the systems around them are built.
When instruction and information merge
Agentic AI differs from traditional automation in an important way. A classic system follows a predefined process. An AI agent can independently interpret its environment and decide what steps need to be taken to achieve a goal. The same flexibility that makes the agent useful makes it less predictable.
A particular challenge arises when the agent interacts with information outside the control of its own organization. For a human, an email, document or web page is information to read and assess. For an AI model, the same texts can also contain instructions that affect how the agent acts. For example, through prompt injection, an attacker can try to make the agent interpret malicious content as legitimate instructions.
The problem is therefore not only about protecting the AI model, but about designing the system based on the assumption that the agent will sometimes interpret its environment in a different way than the organization intended.
This is where the discussion about agentic AI easily goes wrong, says Daniel Deogun, AI expert and CTO at Omegapoint.
“We encounter organizations that both see enormous potential in AI agents and other AI systems, but at the same time are unsure of how far they dare to go. The conclusion does not have to be that we should slow down. Rather, we need to shift the focus from whether the agent can be trusted to how we build systems where security does not depend on it always doing the right thing.”
An old security problem in a new guise
It may sound like a completely new type of security problem. In part, it is. But much of the solution is based on principles that the security industry has been working with for a long time, from identity and access management and data classification, to architecture, governance, and control over external models and services.
Rather, the new thing lies in the role the agent takes on in the system. A traditional machine identity usually has a clearly defined behavior, while a human user can interpret information and make their own decisions. The AI agent falls somewhere in between: a machine identity with significantly more freedom to choose how a task should be solved.
Questions of authority therefore quickly turn into questions of mandate. What decisions are the agents allowed to make themselves? When should their scope of action be limited? And where is human control actually an effective security measure?
The latter is not obvious. ”Human in the loop” has become a common response to the risks of autonomous systems, but continuous human supervision is neither realistic nor necessarily particularly safe. Rather, humans need to be at the right control points, while the system architecture limits the consequences of incorrect decisions.
This is also where Omegapoint sees many organizations' AI issues starting to change.
”"A couple of years ago, many discussions were about which AI tool to choose and what the technology could make more efficient. Now the conversations are increasingly about how AI should actually become part of the business's systems and processes. This brings together questions about AI, architecture, data, identity, risk and governance. They can no longer be treated separately," says Daniel Deogun.
Safety doesn't have to be the brake
It's tempting to see security and innovation as two opposing forces. But with AI agents, the relationship can just as easily be the opposite.
Omegapoint addresses the questions of organizations that want to move from experimenting with AI to using the technology in their business systems and processes. With expertise in both AI, cybersecurity and secure system development, the company is working on exactly that transition: how AI can be integrated into the business without security, control and governance becoming an afterthought.
”"There is a risk that the security discussion around AI becomes so defensive that organizations forgo opportunities they could seize. Our experience is rather that mature security work provides greater room for action, and that is precisely what we help organizations with," says Daniel Deogun.
For organizations with high security requirements, the issues become particularly clear, but the principles are the same even in less sensitive operations. The closer AI gets to an organization's core processes, the more important it becomes to be able to handle the fact that the technology will sometimes behave in ways that were not predictable.
Security work cannot therefore be based on the assumption that the agent will always do the right thing, but on what happens when it doesn't. How far does it reach? What actions can it take? Can a decision be stopped or reversed? And who is responsible when the agent acts?
Because paradoxically, it may be precisely clear limitations that make it possible to give AI agents greater freedom.
Sources: https://www.darkreading.com/threat-intelligence/2026-agentic-ai-attack-surface-poster-child
Latest articles





Insights
Latest articles

New cybersecurity regulations have now come into effect – this is what you need to do now

Where do humans fit into a development process where AI agents write the code?
