Natural Language Reinforcement Learning

Key figures & insights

• NLRL achieves 85% board evaluation accuracy vs 61% for baseline LLMs in 5x5 breakthrough game
• Win rates improve from 0.4 to 0.9 (125% increase) in tic-tac-toe against stochastic opponents
• Language TD estimation with 8 variations and 3 look-ahead steps reduces average reward from -27.29 to -11.19 in maze navigation
• Traditional RL suffers from Chain-of-Thought degradation, producing meaningless reasoning after training

Implications

• Enables active learning vs passive policy gradient methods that rely on chance sampling of good actions
• Language Value Functions provide interpretable rationale for decisions, addressing traditional RL's "what but not why" limitation
• Framework applicable to any LLM-based agent system using reinforcement learning

Required action

• Consider NLRL for applications requiring explainable AI decisions in sequential environments
• Evaluate framework for trading algorithms, risk management systems requiring audit trails

About the authors

• Multi-institutional collaboration led by researchers from UCL, NUS, Brown University, and Shanghai Jiao Tong University
• Published as preprint on arXiv, focuses on foundational research bridging natural language processing and reinforcement learning