Peter Chang
translateEN
All publications

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang, Hsin-Hsi Chen

Conference on Language Modeling · 2026

Abstract

Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments remains a key challenge. Existing memory-augmented approaches retrieve individual past examples as direct references, but do not explicitly synthesize actionable strategies from them, causing the same types of errors to recur.

We propose Dynamic Retrieval-based Policy Generation (DRPG), a framework that integrates memory-based retrieval with a dynamic policy generator, leveraging historical data and environment feedback to produce task-specific policies for continual LLM improvement. We evaluate DRPG across six benchmarks spanning text-to-SQL, question answering, medical diagnosis, and Python programming, using seven LLMs from both proprietary and open-weight families.

DRPG outperforms strong baselines across most datasets and models. Further analysis demonstrates that DRPG's policy generation is robust to retrieval strategy, operates effectively without prior policy continuity, and can leverage smaller or cross-family models as cost-efficient policy generators. We also find that the benefit of policy-level guidance depends on task characteristics, offering practical insights into when and under what conditions this mechanism is most effective.

COLM 2026

COLM 2026First author

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang, Hsin-Hsi Chen

TL;DRLets an LLM turn its own history into strategies — continual gains, no retraining.

arXivDOIPDF
BibTeX
@inproceedings{chang2026smarter,
  title     = {Smarter by the Moment: Environment-Driven Dynamic Policies for Continual {LLM} Improvement},
  author    = {Chang, Ting-Wei and Chen, Po-Chun and Huang, Hen-Hsen and Chen, Hsin-Hsi},
  booktitle = {Proceedings of the Conference on Language Modeling ({COLM})},
  year      = {2026},
  eprint    = {2609.16800},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi       = {10.48550/arXiv.2609.16800},
  url       = {https://arxiv.org/abs/2609.16800}
}