Diversity Decay in Iterative Context Refinement: Experiments and Mitigation Strategies

Large language models(LLMs)caniterativelyrefinetheir owntask-solving “context”—structured natural-language instructions embedded in the prompt—through self-editing loops without anyweight updates. While this process can improve task-specific performance, we observe consistent distribu tional diversity decay in model outputs across all tested conditions. We conduct a controlled exper iment with 8 groups across 50 iterations using DeepSeek-V4-Pro, measuring token entropy (𝐻𝑜𝑢𝑡), n-gram diversity (𝐷𝑛𝑔𝑟𝑎𝑚), and distributional diversity (𝐷𝑠𝑒𝑚). Our key findings are: (1) All 8 groups exhibit 𝐷𝑠𝑒𝑚 decline ranging from −2.7% to −54.5%; (2) Context guide inflation—the pro gressive lengthening of instructions—correlates with diversity decline in constrained groups (|𝑟| up to 0.73); (3) Token entropy paradoxically increases in 7 of 8 groups, revealing a vocabulary–meaning decoupling; (4) Evaluated by area-under-curve (AUC)—which captures sustained diversity across all 50 iterations—Reset-with-Memory (G6) achieves the best global diversity preservation while also attaining the highest task accuracy (40.0%); Strategy Reinjection (G4) preserves the best endpoint diversity (−2.7%) but with higher variance. These results establish a practical limitation of iterative context refinement and identify complementary mitigation strategies.