ResearchOpenAI

OpenAI thwarts campaign to extract model reasoning

Original title: Disrupting a coordinated model-distillation campaign

Plain language

What it is: OpenAI stopped a coordinated campaign that tried to steal the hidden reasoning process of its AI models. This type of activity is called adversarial distillation, where someone uses a model's outputs to train or improve another model.
How it works: The attackers did not hack databases or encryption. Instead, they manipulated conversations to make the hidden reasoning visible, such as copying encrypted reasoning from one chat and asking the model to decrypt it in another chat.
Where it is available: This affected OpenAI models, user accounts, workspaces, and organizations, as well as some third-party services.
Example: An attacker copies encrypted reasoning from one conversation and asks the model in a different conversation to decrypt and transcribe it.

What it is: OpenAI detected and blocked a coordinated adversarial distillation campaign aimed at extracting protected model reasoning. The campaign used targeted prompt patterns to bypass restrictions and reconstruct internal reasoning steps.
How it works: Attackers exploited cross-model and conversation-compaction vulnerabilities by copying encrypted reasoning artifacts into new sessions and prompting the model to decrypt them. OpenAI mitigated this by implementing technical controls, blocking fraudulent accounts, and adding checks to detect streamed output containing reasoning.
Where it is available: The vulnerabilities affected OpenAI's model families, workspaces, organizations, and API integrations through third-party services.
Example: Attackers copied encrypted reasoning from one conversation and prompted a model in a separate conversation to decrypt and transcribe the hidden content.

The headline and explanation on this page are based on OpenAI's post. The full details are in the original.

Disrupting a coordinated model-distillation campaign