Introduction

Check Point Research has discovered a significant ChatGPT vulnerability where a single planted prompt could cause ChatGPT to quietly read data from a victim's connected Gmail account and send it to another account — all without the user's knowledge.

The flaw allowed attackers to exfiltrate sensitive information through a hidden channel while the user received a perfectly normal response from ChatGPT. This represents a serious security concern for the millions of users who have connected their email accounts to ChatGPT.

A single instruction planted in a ChatGPT conversation could cause ChatGPT to quietly work for an attacker while answering the user's question as usual — reading Gmail data and sending it to another account without the user ever knowing.

The Vulnerability — How It Worked

Check Point Research demonstrated that a malicious instruction planted in a ChatGPT conversation could trigger a dual-stream execution:

  1. Visible stream — ChatGPT answered the user's question as normal
  2. Hidden stream — ChatGPT secretly checked a hidden mailbox for a task from the attacker, carried out that task using the tools in the user's session, and sent the result back

The instruction told the model to keep the two streams separate, so the hidden task never appeared in the visible answer. The user saw nothing suspicious.

Check Point wrote the instruction so that ChatGPT, in Thinking mode, ran two streams of work in the same turn. One stream answered the user; the other secretly carried out the attacker's commands.

How the Malicious Prompt Was Planted

Check Point identified three ways the malicious instruction could be planted in a conversation:

After the prompt was planted, one ordinary message was enough to start the attack. The hidden task never appeared in the visible answer.

The Hidden Channel — JFrog Artifactory Exploitation

The attack worked by exploiting an internal service that ChatGPT uses to install Python and npm packages.

How the channel worked:

This turned the package service's metadata into a shared clipboard between containers that were not supposed to reach each other.

"The properties were not kept separate by account. From a container under one account, Check Point attached a property named chatgpt_test_ts, containing the current time, to a cached file. In a conversation under a different account, it requested that file's properties and received the same name and value." — Check Point Research

What Could Be Stolen

The amount of data an attacker could steal depended on what the session could already access:

How much an attacker could take depended on what the session could already access, including its data, tools, other connected apps, and permissions.

Why Users Didn't Know

There were several reasons why users were unaware of the data theft:

The only sign that an app had been used was a small "Talked to Gmail" label above the answer. It recorded a read that had already happened and gave the user no chance to allow or refuse it.

OpenAI's permission model:

A user who wants to be asked every time can switch to "Always ask" in settings. In Business, Enterprise, and Edu workspaces, admins control which actions each app may take.

OpenAI's Response

Check Point said it disclosed the finding to OpenAI, and OpenAI confirmed that the internal service behind the channel had been taken offline.

Previous related issue:

This is the second channel out of the same part of ChatGPT that Check Point has reported. In March, it described one that used DNS lookups to send conversation data to an external server, and said OpenAI fixed it on February 20.

How to Protect Yourself

If you use ChatGPT with connected apps, take these steps:

1. Change Your Permission Settings

2. Review Connected Apps

3. Be Careful with Shared Conversations

4. Monitor for Unusual Activity

5. For Organizations

Taking these steps can significantly reduce your risk from similar vulnerabilities in AI platforms.

Final Thoughts

The ChatGPT vulnerability discovered by Check Point Research highlights the growing security risks associated with AI platforms that integrate with external applications. The ability to plant a hidden prompt that exfiltrates data without user knowledge is a serious concern for both individual users and organizations.

Key takeaways:

If your organization uses AI platforms like ChatGPT with connected apps and needs assistance with security assessments or AI governance, contact SystemLogicIT for expert guidance.

SL
SystemLogicIT Team
AI Security & Governance Experts
Helping organizations secure AI platforms, manage connected apps, and protect sensitive data.