A mode collapse is the tendency of generative systems to converge on a small set of high-probability patterns even when many valid alternatives exist. This happens because those patterns are strongly represented in their training data and are generally rewarded by user preferences. The result is that many different prompts can end up producing outputs that feel surprisingly similar.
Why It Happens
The underlying mechanism involves statistical attractors—regions of possibility space that satisfy multiple constraints simultaneously. Certain ideas occupy a “sweet spot” where they’re familiar, coherent, genre-appropriate, and broadly appealing. Given a vague prompt, a generative system is naturally pulled toward those regions.
The Core Pattern
Generative systems tend to converge on culturally reinforced defaults.
graph LR A["Human culture develops conventions."] --> B["Training data captures them."] B --> C["Models learn them."] C --> D["Generic prompts activate them."] D --> E["Outputs converge."]
Escaping Mode Collapse
Several strategies can create friction and push outputs away from high-probability attractors:
- Add unusual constraints
- Exclude common tropes
- Generate many alternatives
- Combine unrelated domains
- Identify the cliché answer first, then avoid it
A meta-prompt
«Before answering, identify the most likely cliché response. Do not produce that response. Generate something meaningfully different while still satisfying the request.»