Anastasiya Kazakova, Cyber Diplomacy Knowledge Fellow and Geneva Dialogue Project Coordinator, DiploFoundation.
The email looked like it came from IT. A password reset, routine maintenance — the kind of message that arrives on a Tuesday and gets clicked before the second coffee. The contractor clicked it. By the time anyone noticed, the attacker had been inside three energy networks for weeks, moving quietly, touching nothing, learning everything.
The tool that made this possible was not exotic. It was sold commercially, marketed to security teams who wanted to test their own defences against exactly this kind of attack. The AI monitoring system had flagged the anomaly within seconds. The alert reached a human analyst hours later. The attacker had left by then.
The operators moved to contain the breach. And then the ground shifted in a different way entirely. A military strike in a distant region took out a data centre — unrelated, in any formal sense, to the energy networks or the intrusion. Except that the data centre happened to run the AI monitoring platform the operators depended on to detect threats. Overnight, their visibility dropped to forty percent. Three analysts, doing manually what the platform had done automatically, could not keep up.
No one had planned for this. Not the operators, not the regulator, not the vendor. The breach and the blackout had different causes, different geographies, different legal frameworks. And yet someone had to decide what to do next.
The scenario is fictional. The dilemmas it surfaces may not.
In May 2026, the Geneva Dialogue on Responsible Behaviour in Cyberspace — an international multistakeholder initiative that examines what agreed UN cyber norms mean in practice, for the companies, technical communities, academia, and civil society that operate cyberspace as much as for the governments that negotiate its rules — placed around 40 experts inside this scenario during Geneva Cyber Week. Participants came from national cybersecurity authorities, international organisations, civil society, the humanitarian sector, academia, technical operations, and policy. Three parallel breakout groups, two rounds of escalating pressure, Chatham House Rule throughout.
The objective was not to find solutions. The goal was specific: to stress-test the agreed UN cyber norms against operational reality — and to examine how stakeholders actually make decisions about responsible behaviour when AI tools are dual-use by design, when the boundary between a cyber incident and a military one has dissolved, and when no single actor can see enough of the system to be fully accountable for it.
What the scenario revealed
What the exercise surfaced was not a list of missing policies. It was a set of structural tensions — places where the logic of the governance framework and the logic of operational reality pull in opposite directions.
1/ When a tool is designed to serve both attack and defence, ‘responsible use’ is not a property of the tool. It is a decision made by a human actor — and the existing framework provides limited guidance on how that decision should be made. Weeks into the incident, with visibility still at forty percent, the vendor made an offer. The platform could now model how an adversary would approach the operators’ specific infrastructure — map vulnerabilities, predict attack paths, generate a working proof of concept. Defensive, the vendor said. A way of seeing the attack before it arrives. Someone asked the question that had been in the room since the offer arrived: if the platform can model how an adversary would attack their infrastructure, what stops it from modelling how to attack someone else’s?
This is the sharpest version of a problem that runs through the entire scenario. The AI phishing tool at the centre of the original intrusion had not changed between its last legitimate use and the moment it stole a contractor’s credentials. What changed was the context. Across all three groups, participants converged on the view that where dual-use is a design feature rather than an accident, responsibility cannot be delegated to documentation. The harder question — who bears that responsibility, and how is it allocated across developers, users, and operators — produced three distinct positions, each with genuine grounds and structural limits. The developer who puts a capable tool on the commercial market carries ongoing obligations that do not end at the point of sale. The operator whose staff were not equipped to recognise a sophisticated AI-generated phishing attempt carries a separate failure. The user who deploys the tool offensively probably bears primary culpability — but remains, in most scenarios, the hardest actor to reach. The current governance frameworks do not clearly allocate these obligations in a way that produces accountability in practice.
Human oversight of AI security systems is necessary, agreed upon, and, in its current form, routinely satisfied on paper while failing in practice. All three groups converged independently on the principle: AI systems operating in critical infrastructure environments should not take autonomous policy-affecting actions without a human in the decision chain. That principle held without dissent. The harder question arrived immediately after. In the scenario, the first alert was processed by the AI monitoring platform in seconds and reached a human analyst hours later, after the attacker had already left. The oversight model had been formally satisfied. One participant identified three distinct failures nested inside that formal compliance: a capacity problem, where the analyst queue simply could not be covered in real time; a competence problem, where nominal oversight by someone without the technical knowledge to challenge the system’s outputs is oversight in name only; and a governance design problem, where oversight of what an AI system does operationally and oversight of how the system itself is trained, updated, and accessed by the vendor may require entirely different mechanisms — and conflating them risks leaving both inadequately governed. Another participant invoked a historical parallel: in 1983, a Soviet engineer named Stanislav Petrov chose to treat a missile early-warning alert as a false positive rather than escalate. That decision — independent, technically informed, made under pressure — is precisely what meaningful human oversight requires. The question is whether current governance frameworks create the conditions for that kind of judgement, or whether they design it out in the name of efficiency.
2/ The blurring of cyber and kinetic attacks is not an edge case. It is becoming a structural feature of conflict — and the normative framework has not caught up. The scenario’s second round introduced a disruption that no cybersecurity policy had anticipated: a military operation in a distant region knocked out a data centre that happened to host the AI security platform three energy operators relied on for monitoring. No cyberattack had occurred. No one had targeted the operators. The effect — partial blindness at a critical moment — was indistinguishable in its consequences from a deliberate intrusion. Participants agreed on the legal analysis: the kinetic disruption falls under international humanitarian law, not peacetime cyber norms. Norm F of the UN GGE framework, which prohibits states from conducting or knowingly supporting ICT activity that damages critical infrastructure, does not apply. This is not a drafting gap. It reflects a genuine boundary condition. But the practical consequence — that an energy operator can lose its security monitoring as collateral damage in a conflict it has no part in, with no applicable governance framework assigning responsibility or requiring remediation — was one participants found genuinely unresolved. One described the current normative environment as frameworks designed for a stability context being applied to a geopolitical reality that no longer resembles it.
3/ Distributing responsibility across the supply chain without allocating it produces accountability for no one. Veridian, an energy provider, did not know its security monitoring platform ran on infrastructure in a region that could become a military target. The vendor had not disclosed it. The national cybersecurity authority doesn’t necessarily have a mechanism to require disclosure. Each actor in the chain had principled grounds for pointing at the others: the operator should have asked; the vendor should have told; the authority should have set standards. And each position had a structural limit that good faith alone could not resolve. One participant put it directly: where adequate domestic alternatives do not exist, an operator cannot be held fully responsible for dependencies that the market has made unavoidable. The broader question — whether critical security functions should depend on infrastructure the state cannot reach — is a sovereign-level policy question, not an operator-level compliance question?
Where cyber norms help, where they fall short, and where they encounter structural limits
The agreed UN GGE/OEWG framework of responsible state behaviour in cyberspace proved directly relevant in several places — and encountered identifiable limits in others. It is useful, as it was in the first thematic cycle and scenario, to distinguish between two types of gaps.
The first is a specificity gap: norms that are sound in principle but insufficiently detailed to guide operational decisions. Norm J, which encourages states to promote responsible vulnerability disclosure, is operationally important in the scenario but provides no guidance on timelines, sequencing, or the cross-border coordination that makes disclosure workable in practice. As with the open source software cycle, industry practice has not merely implemented this norm — it has filled the space the norm left largely empty. Vendors have developed detailed coordinated disclosure policies; bug bounty platforms have established standardised channels. The normative framework could usefully recognise, codify, and harmonise these existing practices rather than develop parallel guidance from scratch.
Norm G, which calls on states to protect critical infrastructure, sets a clear objective but does not define what ‘appropriate measures’ look like when the CI operator depends on a commercially procured AI platform running on infrastructure the state cannot audit. The norm assumes that appropriate measures can be identified and executed. The scenario demonstrated that the chain of dependencies through which the state would need to act runs through private vendors, cloud providers, and data centres in multiple jurisdictions, none of which the norm directly reaches.
The second type of gap is structural — a mismatch between what the norm assumes and what the domain actually looks like. Norm F encountered this harder limit in the scenario’s second round. The kinetic disruption of the data centre was not an ICT activity, was collateral to a military operation in an active armed conflict, and was not intentionally directed at the energy operators. International humanitarian law governs this situation, not peacetime cyber norms. That legal consensus is genuine. But the grey space it leaves — hybrid conflict, military operations with significant civilian cyber effects, state-affiliated actors using commercial tools — is governed by incomplete consensus and increasing operational frequency.
A deeper structural issue runs across all dilemmas the scenario surfaced. The agreed norms address state behaviour. The actors with the greatest operational proximity to the problems the scenario exposed — AI security vendors, CI operators, cloud providers, security researchers — are non-state actors whose conduct the eleven norms do not directly reach. In several areas, these actors are already implementing elements of normative expectations in practice: through product design decisions, procurement standards, disclosure policies, and incident response procedures. But this implementation is informal and uneven — and because the normative framework does not recognise it, there is no mechanism to strengthen it, harmonise it, or hold anyone accountable when it fails. The actors with the deepest technical knowledge of what a given AI security capability can do — and what it can do beyond its stated purpose — currently carry the fewest formalised accountability obligations for how that capability enters the market and how it is used.
What comes next
The questions the scenario raised did not resolve. How responsibility for dual-use AI tools should be allocated across developers, operators, and states. What human oversight of AI security systems actually requires in terms of capacity, competence, and governance design. How civilian critical infrastructure should be protected when the threats to it arrive not through a network but through a conflict on another continent.
These are not questions the Geneva Dialogue can answer alone. But there are questions that the non-state actors closest to the problem — the vendors, operators, researchers, and foundations who build and run the systems the norms are meant to govern — are already navigating, informally and unevenly, every day.
The Geneva Dialogue’s next step is to translate these structural dilemmas into practical guidance for non-state actor responsibilities, feeding into the future Chapter 3 of the Geneva Manual. The stress-testing continues.
We invite you to join the conversation.
