Imagine the following: You visit a webpage with a lot of text you don’t want to read and ask your AI assistant for a summary. A few moments later, the AI assistant has extracted one of your emails and sent it to an attacker without you ever knowing.
In October 2025, we found exactly this vulnerability in Firefox’s AI chatbot integration1.
Firefox offers a summarization, explaination and proofread AI feature. When a user makes use of one of these features, Firefox pastes a prompt into the sidebar AI chat including the page title, the selected text (or, if the whole page is summarized, a selection is being made by Firefox) and an instruction on how to process the provided text. The sidebar AI chat is essentially an IFrame of a third-party chatbot (Claude, Copilot, …).
Over the last few weeks, I have had a very productive exchange with
Christoph Klaassen
on the impact of AI on security governance and compliance. In this post, we
summarize our thoughts.
AI agents are here, there, and everywhere. Smarter, faster, and more skilled,
they gain greater autonomy and trust. We trust their capabilities to do many
tasks much faster and sometimes better than we can. We trust them as they
usually demonstrate their eagerness to please us and fulfill our commands. Isn’t
that too good to be true, and we might be dealing with a double-edged sword
here? Can attackers use the same capabilities of the AI agents to attack their
own users? Can they exploit their eagerness to please their users to fulfill the
attackers’ intentions? And most importantly: what’s the worst that could happen
if you fully trust some random AI Agent?
With the rise of AI assistance features in an increasing number of products, we
have begun to focus some of our research efforts on refining our internal
detection and testing guidelines for LLMs by taking a brief look at the new AI
integrations we discover.
Alongside the rise of applications with LLM integrations, an increasing number
of customers come to ERNW to specifically assess AI applications. Our colleagues
Florian Grunow and
Hannes Mohr analyzed the novel attack
vectors that emerged and presented the results at
TROOPERS24 already.
Dennis and I already published blog posts about our research project dealing with vulnerabilities in parcel tracking implementations at DHL and DPD. At the Winterkongress (winter congress) in Winterthur, Switzerland, we had the great opportunity to give a talk about the matter. The talk was recorded and can be watched here.
DigiGes held the Winterkongress, which took place in Winterthur on 01.03. till 02.03.2024. The main topics are ethics, threats, and opportunities of IT. This year, many talks looked at AI in some way.