强化学习NeurIPS 20252025
KTO Revisited: Improved Alignment with Human Feedback
Soheil HumanStanford
摘要
We revisit Kahneman-Tversky Optimization (KTO) for aligning language models with human feedback. Our improved KTO variant introduces better loss functions and training dynamics that more effectively capture human preference patterns, leading to more natural and helpful model behavior.
KTOhuman feedbackalignmentloss functiontraining dynamics
技术细节
数据集
模型骨架
LLaMA-2 7B + Improved KT Loss
编码器
Transformer Encoder (LLaMA-2)
解码器
Improved KT Decoder (32 layers)