Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models
Published in The Thirteenth International Conference on Learning Representations, 2025
Diffusion models have made substantial advances in image generation, yet models trained on large, unfiltered datasets often yield outputs misaligned with human preferences. Numerous methods have already been proposed to fine-tune pre-trained diffusion models, achieving notable improvements in aligning generated outputs with human preferences. However, we point out that existing preference alignment methods neglect the critical role of handling unconditional/negative-conditional outputs, leading to a diminished capacity to avoid generating undesirable outcomes.
Recommended citation: Fu-Yun Wang, Yunhao Shui, Jingtan Piao, Keqiang Sun, Hongsheng Li. (2025). "Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models." The Thirteenth International Conference on Learning Representations.
Download Paper