AI assistants are vulnerable to eavesdropping despite being encrypted
Researchers have discovered a security flaw that allows hackers to decipher responses from AI assistants with alarming accuracy. According to Yisroel Mirsky, head of the Offensive AI Research Lab at Ben-Gurion University in Israel, this passive attack can occur without the knowledge of OpenAI or its clients, and exposes the content of encrypted messages. While Mirsky referred to OpenAI, the vulnerability affects all major AI chatbots except Google Gemini.
Deciphering encrypted AI assistant responses
The attack technique involves exploiting a side channel in AI assistants, refining the raw results through large language models (LLMs), primarily trained for the task. The outcome is that an individual with a passive adversary-in-the-middle position can deduce the exact topic of 55% of all recorded responses with a high level of word accuracy. The attack can deduce responses with perfect word accuracy 29% of the time, revealing sensitive information despite encryption measures.
Real-world implications of the security flaw
As stated previously, the security flaw affects all major AI chatbots, except Google Gemini. For instance, encrypted responses from ChatGPT and Microsoft Copilot can be inferred with high accuracy. Although the exact wording may not be perfect, the overall meaning of the inferred sentence remains highly accurate. This vulnerability exposes users to potential eavesdropping attacks on sensitive topics discussed via these AI assistants.
Role of tokens in AI assistant issues
The vulnerability lies in the tokens that AI assistants use when responding to user queries. Tokens, similar to encoded words, are sent on the fly to enhance user experience. However, this real-time transmission exposes a previously unknown side channel called the "token-length sequence." Despite encryption, this token-by-token delivery system allows attackers to exploit the side channel and decipher encrypted responses.
Understanding side channels in AI assistant vulnerability
A side channel is a method of obtaining secret information indirectly or from unintended sources. In this case, the side channel resides in the tokens used by AI assistants. By carefully monitoring these tokens, attackers can gather enough information to recover encrypted responses. This discovery highlights a significant security flaw in the encryption methods used by most major AI chatbots.
