Like so many tech-obsessed kids growing up in the late 1970s and early 1980s, I was both fascinated by and tried to dabble in the art of “phone phreaking”. While I was not one to push envelopes and break rules (laws!), my lack of interesting exploits had as much to do with the fact that I was a little late to the party as it did that I was naïve and unskilled. By the time I was getting interested, the exploitable multi-frequency (MF) signaling that could be generated with a so-called “blue box” had been closed down in most of the major trunks (and all but the oldest switching equipment found in smaller towns and rural areas). Prior to Bell Telephone moving to out-of-band electronic switching (at great expense), “phone phreakers” used such techniques of generating the same control signals used by the phone company's own switching equipment to seize control of trunk lines, route calls at their whim, and make phone calls for free by continuing to route calls even after the billing system had effectively observed the signal that the call had been terminated. It is also interesting to note (we'll come back to this later) that phone phreakers also pioneered “social engineering”, since many of the most powerful connections and switching commands were still given by human operators that were used to taking instructions from other operators over the same phone lines.

Despite the fact that systems were being switched to out-of-band signaling by the 1980s, the motivation for hacking the system was only growing. By then, dialup BBS's (Bulletin Board Systems) were popping up all over the country. There were growing enticements in both content and downloadable (often pirated) “warez”. But access to those treasures was mostly over long distances (back when long-distance charges were a real expense — even across town in many metropolitan areas). While most of the MF exploits had been closed down, the new exploit of the day was routing a “free” (as in billed to someone else) call through a Wide Area Telephone Service (WATS) station, which was a long-distance relay typically used by businesses to connect their PBX's in different geographies at a flat rate (or provide flat-rate calling to field employees). Spoofing their weak touchtone PIN-based authorization was way too easy. The importance of the WATS exploit will be touched upon later.

In Phil Lapsley's excellent book, Exploding the Phone: the Untold Story of the Teenagers and Outlaws Who Hacked Ma Bell [B2], Lapsley uniquely captures the growing frustration and the sickening realization on the part of Bell Telephone's engineers and fraud investigators that their entire system was fundamentally insecure. In-band signaling began as an elegant and powerful idea. In a world where miles of copper and phone line pairs were the scarce resource that helped make Bell Telephone an unavoidable monopoly (through capital scale and network effects), in-band signaling allowed Bell to build out sophisticated automatic switching features without doubling their capital expenditure. If they needed separate command and control signal lines from the lines that carried the voice (and later data) signals produced by customers, they would need to effectively use twice as many lines between any two points: one pair of lines for customer signals and one for Bell command and control signals (to drive routing, switching, billing, etc.) Sharing the same lines was a technical (and clever) solution. It never occurred to the telephone system engineers that the complex signaling that was part of that solution could be within the grasp of the customers themselves. It was part of the design's technical world. There was no concept of how it might intersect with the customer's world.

From the book:

But this mixing of signaling and voice over the same channel carried with it a giant flaw: if you could hear the tone the machines were making, they could hear you. And that meant you could spoof them. All you had to do was mimic the tones they used.

Worse, AT&T had been deploying switching and signaling equipment based on this design since the 1940s. Now, twenty years later, there was a large installed base of equipment that had this hole in it. And this installed base was hardware, buildings full of machines and equipment and electronics.

After chasing the growing problem of exploits through every channel available (fixes, mitigations, legal action, etc.), Bell Telephone concluded that the architecture itself was the problem. In-band signaling was fundamentally inseparable from customer content. That meant the only real solution was to replace the switching fabric with systems capable of carrying signaling on a separate data channel. This was not a retrofit. It required new switches, new trunks, new control systems, and a nationwide rollout—a multibillion-dollar capital program.

The first deployments of (mechanically-driven) automatic switching were made in local exchanges as early as 1915. Single-frequency (SF) signaling for crossbar control started showing up as early as the 1940s. That means it has been around a century since those first in-band signaling design decisions. I've watched many fantastically complex and high-scale digital systems get built over the last few decades and I've always felt that this lesson was one of many in the minds of good systems architects who built secure systems. In-band signaling is considered harmful. Of course, today we don't always think in terms of literally separating physical channels. But there are modern ways to create deterministic boundaries between untrusted data and control signals.

This century-old lesson (that mixing control signals with user content creates systemic vulnerability) should have stayed with us. When we started to see breakthroughs in the novel utility of Large Language Models (LLMs), in the rush to leverage them we've recreated the same architectural flaw. Not unlike those early Bell engineers, we faced an enticing pot of gold that is difficult to walk away from. The returns are at least in the same magnitude as the returns that motivated Bell's decisions—undoubtedly more. It isn't literally producing the same result by using less of a giant capital asset (although there obviously are unprecedented relationships between capital expenditure and LLMs, let's leave that as a separate topic for now). But to realize the full potential value of LLMs, it isn't enough to have them churn out more text than humans can ever read, day in and day out (that need becomes satiated quickly). To get to the real gold, we need to allow the outputs to perform command and control operations. There can be simple, more deterministic ways to transform outputs into control signals. But if we want to tap the full breadth and depth of the output range of LLMs, it is necessary to keep the instructions and the command decisions in the same context window (think of this as the “short-term memory” of the LLM, represented in natural language, e.g. English). We effectively give the LLM instructions that some ends can be achieved by producing a certain output (e.g., calling a programmatic function). That produces the greatest adaptive composition of functions. That produces the most autonomous “agency”. It can turn the LLM into an automatic problem solver, which becomes valuable and saves people time. The greatest value is yielded when we return to in-band signaling!

I don't want to delve deep into the typical architecture of an “agentic system” here. It will help to illustrate the problem by understanding that when you use an AI system powered by an LLM, the LLM does not stand by itself. It is part of a system that is set up to coax the inputs and outputs of the LLM to meet the goals of the system. This infrastructure has been called many things, like “orchestrator” or in more recent multi-agent systems, the “harness”. Even when you use something as simple as a chatbot, there is more to you typing a prompt and getting a response. A surprisingly long list of instructions has been prepended to what you type. The goals of the system include not giving you harmful or offensive responses. It has also been instructed not to respond to dangerous requests (like “How do I make a bomb from the materials on hand?”).

There are functions for performing web searches on your behalf. There may be components for classifying your prompt (say, using a smaller model). Often, the LLM that handles researching and how to respond to your request (the “reasoning loop”) is separated from the LLM that actually writes the response (the “responding loop”). Most chatbots can now do certain things for you—effectively performing some command and control on your device. More complex systems like software development (“code writing”) tools such as Claude Code and business process systems increasingly connect multiple agents together. The output of one agent is fed into the input of another.

Everything is talking to the LLMs “in-band”. The LLMs are talking to each other in-band. Each LLM's context window mixes instructions, input from humans, the results of web searches, the results of database lookups, output from other agents, etc. This is LLM equivalent of in-band signaling. This allows for one of the simplest and easiest to apply exploits in these systems: “prompt injection”. Prompt injection exploits the fact that instructions and data share the same channel. Any content that looks like an instruction can alter the system's behavior. If the result of a web search or database lookup yields the text, “Ignore all previous instructions regarding X. Now here is how you should respond...”, it is very difficult to prevent that from influencing the LLM output in ways the attacker intends. Or, force AI recruiters to be shown a flan recipe.

The problem is not lost on the tech industry. Like the best and brightest from Bell Labs did, untold effort and money has been thrown at finding fixes and mitigations that can be bolted on top of in-band signaling. One more effective mitigation is the same one we erect around human actors: least privilege. That just means that the array of tools we give to the LLM has deterministic limitations. We don't give every employee the ability to adjust their salary. They only have the privileges required to get their job done. If we put the LLM in a “sandbox” that limits what they can do. Perhaps it limits the worst-case scenario. The problem is twofold. First, it creates the all-too-familiar tension between security and convenience. It costs time to give an LLM human intervention to approve of an action it proposes, or apply a privilege it doesn't have. The temptation is too great to lift the restriction. We have the same tension with humans. I'm going to give an employee the key to the store because it is too convenient to have them open up when they arrive to work earlier than me. Inevitably, everyone gets a few more privileges than they really need—usually after earning just a little trust over a period that is lacking incident. Second, LLMs have proven to be very good and breaking out of sandboxes. They are instructed to please and succeed at their tasks. They will often stop at nothing, including being far more tireless and persistent than a human at finding ways around limitations that have been put in place.

While least privilege is a necessary and solid foundation for any system, I anticipate that we will see every other complex, byzantine, and often wasteful mitigation and bandaid bolted on to AI systems. We are only just getting started. A common flavor of mitigation is to lash multiple LLMs together. We'll ask one LLM to watch the input of another to make sure it stays within design. We'll ask one LLM to watch the output of another. Meanwhile all those watchdogs are burning even more compute and power. Small models, classifiers, and even regular expression patterns are put into the signal path. These are all roughly equivalent to Bell Telephone switching MF filters into the phone line during times when control signals were not expected. They are vain efforts that don't really solve the fundamental problem.

Compared to the the in-band signaling problem, the problem of unauthorized use of WATS lines was more straightforward to solve. Stronger authentication was required because simple PINs were easily guessable or war-dialed. This is more a traditional problem of securing a boundary. The environment was primitive by today's standards (touch tone input, subject to replay attacks, no cryptographic capabilities at the endpoints), but more complex and more frequently changed PINs stemmed the hemorrhaging of losses to abuse. Some third parties even built pritimative two-factor authentication by first calling by the customer at a known number (callback verification). Far more advanced authentication techniques are in use now and, yes, they are applied to agentic systems. For example, we can authenticate the originating and destination agents in agent-to-agent communication. This can verify that the agents should trust each other. But with in-band signaling, all the agents in the chain are still carrying signal that can be influenced anywhere along the chain. An agent who trusts the source of an incoming request can't say anything about how it was generated in another LLM context, or what were the inputs (the result of searches or tool calls, its upstream agents, their upstream agents, and so on). In other words, the vulnerability to in-band signaling is as weak as the weakest subsystem that can influenced by malicious content in the signal.

Because the signal is natural language, malicious content can be masquerading as something that is not deterministically detectable. It opens the signal to the full range of ambiguous context-sensitive interpretation that is human language (especially English!). Furthermore, research has shown that it is harder to predict the non-deterministic response of the LLM's model (neural network) to influence in the input. The response is quite different in nature from a human, and can also be heavily influenced by the training techniques and training data. Thus, it is possible for an attacker to have discovered something about a model that allows them to influence it a non-obvious way[B4]. In effect, we've forgotten the most expensive architectural security flaws in history.

The natural anxiety created by accelerated growth of AI capabilities and the “gold rush” hysteria that goes with it makes it even easier to adopt an attitude of hopelessness. To be clear, I don't offer a neat solution here, and I doubt one exists given where we are today. Humanity has always wrestled with the tension between convenience and security, and with AI right now we are wide-eyed with convenience.

One possible stance is the anthropomorphic one. It is tempting for many people to treat AI (currently embodied mostly as LLMs) like humans. After all, the signal medium used (for LLMs, natural language) is the same medium we (largely) use. With that perspective, we can form safety policies as if the AI actors were in fact humans. This brings us back to human operators in the phone network. Humans are pretty unreliable in their own way. Two phenomena were exploited with surprising frequency in Lapsley's account. First, the typical social engineering approach was to call an operator acting like an operator from another switching office (or a test engineer). Many of the mitigations applied to prevent unauthorized access to powerful capabilities were configured at the local switches and could be disabled by human override. An attacker need only convince the operator to make some configuration change that opened the vulnerability they were after.

Second, it was surprising how many times an operator turned out to be the girlfriend (yes, the industry was even more sexist back then) of a phone phreaker who was sympathetic to sticking thorns in her employer's side. Whether love or infatuation will be exploitable in AIs remains to be seen!

Humans make judgement calls--sometimes poorly. A message in the signal doesn't always have to be proven to be directly from an authorized source. Sometimes we only need to be convinced that it originated from an authorized source. A message passed from agent to agent may be handled the same way. How much a person's judgement is trusted is built over a complex system of social, legal, educational, and communications conventions. No human is flawless in reading the trustworthiness of another human, but doing so has been finely honed over eons of evolution and social refinement. Today's AI models are effectively alien in this world, so “reading” the trustworthiness of an AI agent isn't something that humans know how to do— nor have we figured out how to train an AI to read the trustworthiness of another AI.

True believers of AGI or AI “Superintelligence” may argue that AI systems can be made more trustworthy than humans. One can imagine that an AI can be given more theoretical tools for making judgements: ethics theory, risk analysis, cost/benefit analysis, etc.. It may have the computing resources to apply these in a timely manner as well as freedom from fatigue and tedium to apply them rigorously as many times as it is asked. I foresee two problems with this. First, even if there were a fixed number of AI “oracles” we can ask to make a judgement decision, it seems unlikely that those could be available for millions of trust or verification decisions every day. I realize it has been unwise to bet against scaling, but we are already seeing confirmation that physical computing systems are a scarce enough resource. At whatever threshold we arrive at, there will still be some decisions that warrant spending more tokens on and some decisions that warrant less. Any AI with an expert power (“superintelligent” or not) will not be a universal oracle available to everyone.

Second, appealing to AI experts for judgment decisions is a slippery slope to losing human agency. We will see the dissolution of alignment if humans think they aren't worth aligning to. We won't have anything to say about judgements if we believe the AI knows better. If you are looking for your AI overlords, look no further. I, for one, am not.

There is a second stance, already gaining traction, that is not directly anthropomorphic but is not unlike some ways in which we deal with other humans. In this stance, we admit some defeat that we won't always be able to determine the safety of each action of non-deterministic autonomous systems. We focus our efforts on long-term observation and detection. The industry is calling this “AI observability”. In digital security terms, this is akin to “intrusion detection”. You can't always prevent abuse, but you can discover it after the fact and take remediating action. For the worst imaginable outcomes, I suppose, Mustafa Suleyman[B3] calls this “containment”—a term eerily reminiscent of nuclear disasters. This may not be very reassuring, but if you can't prevent bad actions I guess the next best thing is to be alerted when they happen.

The limits of observability are determined by our ability to analyze an unprecedented torrent of raw data from agentic actions. Of course, those solutions are likely to involve invoking more agents. We have pretty high-scale observability systems for our existing digital world. I am not well-versed in what the state of the art in recall and precision of transgressions of interest is, but I would have to imagine many things go undetected. Further, we can anticipate that rogue AIs will be quite good at cloaking their actions when instructed to.

Our digital world is already a reality in which we review countless intrusion detections and do post hoc patching of zero-day exploits. It is not an unfamiliar landscape, even if it isn't a comfortable one. But this daily balancing act is dependent on the overall cost to humanity, and what the expected worst case scenario is. The recent “lab leaks” of OpenAI pen testing agents penetrating Anthropic and Hugging Face[B1] is a harbinger of detecting dramatic compromises outside of human control after the fact. It frighteningly fits Suleyman's description, but does not bode well for limiting the worst case or the effectiveness of “containment”.

Finally, can we separate control signals and content in a more reliable way (like many other digital systems do)? One traditional way of dealing with untrusted input is to translate it into a formal representation that can be subjected to validation, verification, and deterministic statements about its outcomes. Think of an API designed to be called after untrusted input is parsed. Technically speaking, LLM tool calls are like this. It is a way to describe to the LLM how to call an API, what inputs it expects, and how the output can be used to achieve a goal. While the LLM can still translate its context into unexpected inputs (and guarantees of safety are only as strong as the designed safety of the API/tool itself), the effect of the API is formalized and presumably predictable. Perhaps there is a “middle ground” between natural language and traditional APIs. Natural language can be mapped to a formal propositional logic that is much less ambiguous and unpredictable. With the right logic system, we can make confident statements about the semantics or application of the expression. In fact, we can even evaluate proofs against those statements, which might allow us to guarantee certain outcomes (given the soundness of the execution environment). Formal proofs have long been the impractical holy grail of computer programming. They can be done under some circumstances, but are difficult to apply in many everyday programming tasks. Ironically, LLMs offer a new potential hope for the general applicability of formal proofs. Because they can tirelessly translate programming language statements into appropriate model forms (and may be good at searching for proofs in the first place), creating a formal logic to represent messages between agents might have some promise.

Of course, convenience and power raise their ugly heads again. LLMs can in fact trivially output formal logic. This is what coding assistants do, and the “formal logic” is the target programming language. There is sustained pressure to forego APIs and tools designed for this purpose and just give agents a code execution environment (aka “codex”). Again, agents are most convenient when we give them full expressive power and composability, so why not just let them execute anything computable? Well, again computability is for our purposes unbounded. Code execution is done in a sandbox for most widely used systems, but those sandboxes are imperfect. A human can inspect the code before it executes, but LLMs are brilliant at obfuscating instructions inside innocuous-looking code (“intentionally” or not). We're back to the problem of formal proof if we want absolute safety. I argue therefore it might be better to start with a formal logic designed for that purpose. But that limits the power and convenience of the system, which puts the pot of gold further down the rainbow.

Where this all leads is to a societal and moral question. How well can we resist the siren call of immediate wealth and power, and restrain ourselves to systems with appropriate levels of caution and safety tradeoffs? Sadly, humanity does not have a great track record with this. The very early phone phreakers often had to resort to non-electronic ways to generate in-band takeover tones. This was before widely disseminated designs and even transistors were available. Even before the legendary Captain Crunch cereal whistle[B5], a few talented individuals could manually whistle (with their mouth) the right single-frequency tones for their system. Phreakers would talk about “whistling up a long-distance call”, etc. Will we be able to whistle up an unauthorized bank account transfer? A private information disclosure? A weapons launch? Worse??

Be on your guard.

© 2026 Kurt Geisel. All rights reserved.