Reassessing Vigilance in Security: A Need for Automation Over Reliance on Human Attention
An air traffic controller faced with numerous aircraft can easily overlook a critical one. This phenomenon, known as the vigilance decrement, indicates that sustained focus wanes under pressure—a challenge aviation has addressed for decades.
Today, professionals in various fields find themselves overwhelmed, juggling multiple communications and tasks, with efficiency often celebrated despite the cognitive strain. Your inbox, flooded with messages, becomes the new air traffic controller's scope, where critical alerts can easily slip through the cracks.
How Aviation Tackled Vigilance Issues
The aviation industry didn’t expect controllers to merely concentrate harder. Instead, it developed advanced systems that can monitor situations independently, establishing protocols where these systems take precedence over human input.
Consider the tragic incident in 2002 over Überlingen, where an overloaded air traffic controller inadvertently commanded a passenger jet to descend while its onboard collision-avoidance system directed it to ascend. The resultant collision led to 71 fatalities. This incident prompted a crucial aviation rule: when machine and human guidance conflict, the machine’s directive prevails. Notably, these systems don't interpret intent but rely solely on data analysis of converging paths.
The Security Sector’s Counterproductive Path
In stark contrast, the security domain has historically depended on human judgment as its last line of defense, relying on individuals to recognize irregularities. This method has become increasingly inadequate, especially with the advent of AI.
AI has diminished the tell-tale signs of phishing attacks significantly. Research has shown that AI can drastically reduce the cost of these campaigns while maintaining impressive click-through rates, making old-school detection methods nearly obsolete.
Furthermore, the shift towards AI has overwhelmed users. The fragmented focus demanded of knowledge workers today mirrors the strained attention of air traffic controllers who struggle to catch critical threats amid their busy workflows.
Identifying the Most Pressing Threats
Not every threat can be scrutinized equally—it's crucial to prioritize those that pose the greatest risks. Data indicates that business email compromise (BEC) attacks, despite being fewer in number, account for a massive share of financial losses. In 2024, BEC incidents reported to the FBI resulted in nearly $2.77 billion in reported losses amid over 859,000 total complaints.
The lesson here aligns with aviation's protocol: implement stringent measures that act automatically during high-stakes moments, rather than relying on human interpretation of ambiguous situations. For instance, mandatory callbacks for changes in banking information, dual approvals for wire transfers, and predefined thresholds for payments that automatically trigger alerts can mitigate risk.
While these protocols used to back up email filters designed to catch obvious phishing attempts, as the recognizable signs of deception fade, their preventative role becomes critical. These systems act impartially, responding to direct actions rather than subjective interpretations.
Still, challenges remain. Cyber adversaries can craft fraudulent requests that appear legitimate in every detail, complicating verification processes. This discrepancy between intent recognition and the need for decisive action eerily mirrors the aviation system's challenges.
The Limitations of Current Technologies
For now, aviation's rules on obeying machinery cannot fully transpose to the cybersecurity context. Machines that assess threats in ambiguous scenarios need a framework to establish credibility. Erroneous actions could inadvertently halt legitimate transactions, prompting hesitation among finance teams.
A noisy system simply shifts the burden of vigilance onto another overloaded user. Reliance on an infallible machine is premature until it can consistently prove its reliability in discerning high-risk situations.
Contemplating the Path Forward
The aviation industry revamped its approach following serious consequences with fatal outcomes. In security, however, the stakes manifest quietly—fraudulent transactions and identity theft that go unnoticed until too late. As we lean on human operators to identify these rare but impactful events, evidence suggests they're increasingly unfit for the task, especially as AI becomes more adept at masquerading as legitimate communications.
Security has its own rudiments of machine-driven controls—actions triggered automatically without relying on human intervention. But to gain acceptance, intelligent systems that read intent must provide the same reliability as their more straightforward counterparts.
In the final analysis, the critical question shifts from whether employees are paying attention to whether any effective barriers exist against threats when actions are about to be taken. The lesson from aviation is clear: it’s time for security to evolve beyond the fallibility of human oversight before the next breach becomes inevitable.
