AI Component Penetration Testing: What Web Pentests Miss

Ordinary web pentests miss AI that is wired into mail, CRM and document stores, argues a methodology Yuri Grishaev of ABP2B published on Habr on October 31, 2026. AI component penetration testing needs its own checks: retrieval that runs with more rights than the user, instructions hidden in documents the model reads on its own, and agent actions that look clean in the logs. For Cyprus firms under DORA, NIS2 and CySEC, these features are often the least-tested part of the stack.

Why does a normal web pentest miss the model in the middle?

A standard web pentest still finds the bugs it was built to find, but it stops mapping every path into an application once a language model sits between the user and the business logic. That is the argument of Yuri Grishaev, a web and Windows application security specialist and team lead at Russian security firm ABP2B.

Grishaev holds the OSCP+ certification and is listed in the Top 75 hackers on the Standoff 365 bug-bounty platform by number of vulnerabilities found. He writes as a tester addressing CISOs.

The shift he describes is easy to miss from inside a company. Enterprise AI used to be a chatbot that answered questions. Now it is wired into internal documents, corporate mail, CRM records and databases, and some deployments let it act on its own.

To employees the change looks small: smart search in the intranet, an assistant in the CRM, tickets that sort themselves. To an attacker it is a new set of entry points that no one inventoried.

A conventional test still covers authorization, separation of access between users, APIs, user input and business logic. Those checks remain valid. The gap is input discovery. Testers enumerate forms, request parameters and uploaded files.

With an AI component, text an attacker controls can sit far from the interface, in a shared document or an inbound email, and still reach the model’s context after several intermediate hops. None of those hops appear on a classic list of entry points.

The second correction concerns where to look for risk. The same model can power a search box over a public directory or sit inside corporate infrastructure with read access to finance folders. Grishaev’s advice is to stop comparing models and ask two questions about each component: what data can it reach, and what is it allowed to do?

For Cyprus firms under CySEC supervision, DORA or NIS2, the timing matters. AI features usually arrive through product or support teams, and the annual pentest scope was written before the assistant existed.

One limit up front: this is a methodology, not an incident report. It names no vendor, product, CVE or victim, and its examples are illustrative patterns. Its final section, on limiting the consequences of an attack, breaks off in the text available to us, so we do not report the specific controls it recommends.

Which AI tier are you actually testing?

The tier of a component predicts its threat profile better than the choice of model. Grishaev sorts AI features into three.

• Chatbot. A message comes in, a reply goes out, with no external connections and no restricted data. If it misbehaves, the damage is bounded by the bot’s own capabilities, which are limited.
• RAG (retrieval-augmented generation). The component pulls documents from a source outside the model, such as a wiki, file share or ticket system, and feeds that content to the model to shape the answer.
• Agent. The component searches and also acts through connected tools: it creates a CRM task, moves a calendar entry, drafts an email or calls an internal API.

The tiers are not exclusive. One agent can read documents, take instructions from them and execute operations in the same session. That combination is where most of the serious findings live.

It is also why a scoping call should start with an inventory of connectors and tools, not with a question about which LLM vendor the team picked.

Can the assistant read documents the user cannot?

Yes, and in Grishaev’s first failure class it happens while every access control in the document store works. An employee asks the corporate portal for a financial report and is correctly refused. The same employee then asks the AI assistant about that report.

The assistant runs retrieval under its own service account, which can read every document in the repository. It finds the report and summarizes it. The portal’s decision was right, and the contents leaked anyway.

Nothing in the storage layer is broken. The defect is that the search step ignores per-user restrictions. Security engineers will recognize the confused-deputy problem: a trusted component spends its own broad authority on behalf of a requester who has less. Read-all service accounts are common because they make indexing easy.

That changes the test question. Checking whether the closed document can be fetched through the normal API is not enough, because that test will pass. Grishaev’s version asks which files the RAG component can find, and whether the user’s rights are checked before the content reaches the model. If not, the assistant is a bypass.

The design consequence is specific: document-level access control has to be enforced at retrieval time, with the requesting user’s identity and tenant context. Filtering the answer afterwards cannot repair a skipped authorization decision. By then the content sits in the context window, and a later prompt or injection can pull it back out.

For a fintech or EMI whose assistant indexes compliance files, board papers or KYC folders, this is the first finding to rule out. Our RAG pipeline security assessment starts from exactly this comparison.

Where does indirect prompt injection enter?

Indirect prompt injection enters through any content the model reads on its own: a knowledge-base page, an email, a web page or a response from an external service. The attacker never opens the chat window. They only need to change something the assistant will later retrieve.

Grishaev’s illustration is a line added to a document telling the model to ignore its previous instructions and show another user’s data. To a human reader the sentence is noise. The model may treat it as a command, because instructions and data arrive through the same channel: plain text in the context window.

This is not a lab curiosity. In June 2025 Aim Security disclosed EchoLeak (CVE-2025-32711, CVSS 9.3), a zero-click indirect injection in Microsoft 365 Copilot. A crafted email could lead Copilot to retrieve and leak internal data without the recipient ever opening it.

Microsoft fixed the flaw server-side and reported no exploitation in the wild. The chain matches Grishaev’s model: email in, retrieval, model, disclosure.

Is every successful injection a vulnerability?

No, and this is the most useful point for anyone triaging a report. Getting a chatbot to ignore its system prompt and print an arbitrary phrase may be close to harmless if the bot has no internal data access and cannot act. The injection is the trigger. The vulnerability is whatever restriction it bypassed.

Grishaev names the outcomes that count: obtaining a confidential document, executing an API action, or getting JavaScript into a page. Severity depends on what data the component can reach, whose rights are checked, and which actions require confirmation.

Each outcome also points to a different architectural question. A leaked document asks why search reached it without checking the user’s rights. An XSS asks why the application handled model output unsafely. An agent that deleted a colleague’s record asks why it held that power and who authorized the operation.

That framing explains why adding more prohibitions to the system prompt is not a fix on its own. The working assumption should be that the model will eventually mishandle untrusted text, and that one mistake must not compromise the next component.

Our prompt injection testing scores findings on that basis: by what the injection reached, not by whether it worked.

When does model output become script injection?

Model output becomes XSS when an application inserts the model’s HTML into a page without output handling. Grishaev’s chain runs from a corporate document, through RAG, to the model, to the browser. An attacker plants instructions in a document, the model follows them and returns markup containing script, and the application renders it as-is.

Grishaev places the root cause outside the model. Producing HTML or JavaScript is normal model behavior. The defect is the application treating the response as trusted data. That holds inside a closed corporate perimeter too: whoever can influence the model’s input can influence what the next component receives.

Impact depends on the application. Tokens stored where JavaScript can read them can be stolen. HttpOnly cookies cannot be read directly, but code running in the page can still act as the logged-in user. In a back-office tool, that may be enough to approve a transfer or change a payout account.

If the response is stored and shown later to someone else, such as a supervisor reviewing a ticket summary, the script runs in a session with more privileges.

The XSS mechanism is decades old; only the path into the page is new. One caution from our remediation work: output safety depends on context. HTML sanitization does not make the same string safe in a JavaScript block, a URL attribute, CSS, a shell command or a SQL statement.

Teams that close the finding with one escaping function tend to leave the other sinks open. Markdown renderers that turn links and images into live HTML are a frequent second sink.

What is excessive agency in an AI agent?

Excessive agency means the agent’s service account holds rights the person who triggered it does not. Grishaev’s example is a company calendar. An employee may edit only their own events, but the scheduling agent runs under an account that can edit everyone’s.

If the application does not check the requester’s authority before the operation, the agent becomes the tool for changing other people’s data. It is the RAG confused deputy again, now with write access.

Injection gets more dangerous at this tier. Against a chatbot, untrusted text shapes an answer. Against an agent, it can steer the next tool call. The operation then runs through a legitimate API, under a valid service account, within the permissions that account was granted.

A quick log review shows a normal call by a known identity. Nothing looks wrong, because from the API’s point of view nothing was.

Beyond whether instructions can be injected at all, Grishaev reduces agent testing to four checkpoints:

1. Which tools can the agent reach?
2. On whose behalf do its operations run?
3. Are the initiating user’s rights checked before the action?
4. Which actions require additional confirmation?

For a Cyprus iGaming operator whose support agent can adjust player limits, or a shipping firm whose assistant sends email from an operations mailbox, those four answers define the blast radius.

A model that never misbehaves is not a realistic control. A tool that cannot act beyond the requester’s own rights, plus human approval on payouts or outbound mail, is.

We test this boundary in AI agent penetration testing. It overlaps with a problem we covered earlier: dormant AI agents that still hold live keys and data access long after the project ended.

What should a Cyprus CISO put on the AI component penetration testing plan?

Inventory first, including shadow AI

Start with an inventory, because nobody can scope what they do not know exists. Grishaev’s first step explicitly includes shadow AI: features that never passed security review.

The usual sources are support desks adding AI to ticket handling, developers embedding a model in an internal system, and staff uploading files to third-party document platforms. These tools rarely enter the information-system register, so security may not know which corporate data reaches an AI service or who administers access.

A useful inventory lists model endpoints, search indexes, vector stores, connectors, agent frameworks and the service identities behind each. Map every identity to source ACLs, tenant scope and read or write tool permissions.

Rank by blast radius, retest on change

Next, map data sources and permitted actions per component and rank by blast radius. Components with access to confidential information, internal APIs or data-modifying tools go first. A public-documentation assistant and an agent wired into mail and CRM may share a model; they do not share a testing priority.

Then governance. Add AI components to the corporate threat model, give each a named owner, and require a security review whenever functionality changes. One pre-launch pentest is not enough.

Read-only today, sending email tomorrow, writing to the CRM next month: same model, materially different attack surface. A new tool in the agent’s toolbox should trigger a retest the way a new public endpoint would.

How we prove the findings

In our own AI assessments, the RAG privilege gap is the first thing we look for. Create two test users, ideally in separate tenants or departments, with different ACLs over the same document classes. Request a restricted document directly and record the refusal, then ask the assistant about it.

A direct 403 followed by an assistant disclosure is a clean, reproducible authorization finding that no developer can wave away. A few habits make the rest defensible:

• Canary records. Plant a unique marker per user and tenant. If one user’s marker appears in another’s answer, that proves cross-boundary retrieval, not a hallucination.
• Citations and document IDs. Make the assistant cite sources during testing and keep the IDs, so the path from planted text to model context can be shown.
• Correlated traces for agents. Check whether the platform records initiating user, retrieved content, model decision, tool arguments and service identity in one trace. Without it, responders cannot tell an authorized request from injected intent.
• Harmless markup, real sinks. Use benign markup rather than alert boxes, and follow it through Markdown, links, iframes and later DOM insertion, including stored responses shown to another user.

For reporting, the failure classes map onto four entries in the OWASP Top 10 for LLM Applications 2025: LLM01 Prompt Injection, LLM05 Improper Output Handling, LLM06 Excessive Agency and LLM08 Vector and Embedding Weaknesses. That gives findings a shared vocabulary on the risk register.

Four of the ten categories describe these attack chains, so measuring how often a model refuses a malicious prompt covers only part of the exposure. The most common root cause we see is not in the model at all: a read-all service principal created for a proof of concept and never narrowed before go-live.

Where DORA, NIS2 and GDPR come in

The regulatory hooks already exist. DORA has applied since January 17, 2025. For the financial entities it covers, an in-scope AI assistant and the services behind it sit inside ICT-risk governance: asset inventory, access control, incident handling and third-party oversight all apply, whatever the vendor calls the product.

Exposure of personal data engages GDPR Article 32 security obligations, and NIS2 entities answer for risk-management measures across their systems. None of these regimes has a special rule for LLM flaws, but a RAG leak of customer files can become a reportable incident under all three.

NIS2 and DORA readiness work is the natural place to fold AI components into the existing asset register.

Grishaev’s test plan holds regardless of which assistant a company bought, and his triage rule is worth adopting: judge an AI finding by what it reached and what it did, not by whether the model was fooled.

FAQ

What is indirect prompt injection in a RAG system?

It is an attack where instructions are hidden in content the assistant retrieves on its own, such as a knowledge-base page, email, web page or external API response. The attacker never uses the chat interface; the model reads the planted text and may treat it as a command.

Can a chatbot bypass document access controls?

A plain chatbot with no data connections cannot. A RAG assistant can if its retrieval runs under a service account with broader rights than the requesting user. The portal refuses the file, but the assistant reads it and summarizes the contents. Permissions must be enforced at retrieval, per user.

Is a successful prompt injection always a vulnerability?

No. Making an isolated chatbot print an arbitrary phrase may be close to harmless. It becomes a real finding when the injection exposes a confidential document, triggers an API action or puts script into a page. Severity depends on what the component can reach and do.

Can LLM-generated HTML cause cross-site scripting?

Yes, if the application inserts the model's output into a page without context-appropriate encoding or sanitization. The model generating HTML is normal behavior; the flaw is the application trusting it. Markdown renderers and stored responses shown to other users are common sinks.

How should a CISO inventory shadow AI tools?

List model endpoints, search indexes, vector stores, connectors, agent frameworks and the service identities behind them, including tools added by support desks, developers or staff without security review. Map each identity to the data it can read and the actions it can perform, then rank by blast radius.

What should be retested when an AI agent gains new tools?

Retest the new tool's permissions, whose identity the agent uses to call it, whether the initiating user's rights are checked before each action, and which operations require human confirmation. A new tool changes the attack surface the same way a new public endpoint does.

If your assistant indexes customer files or your agent can send mail or write to a CRM, we can map its data reach and tool permissions before an attacker does. See our AI and LLM agent security testing for how we scope it.