Improving Image Generation with Better Captions

ABSTRACT

Current text-to-image models often struggle to faithfully follow detailed or complex prompts, frequently ignoring specific attributes or object relationships. We propose that this issue stems from the noisy and inaccurate alt-text captions found in standard training datasets. To address this, we train a highly capable image captioner to re-caption our entire training dataset with extremely dense and accurate descriptions. By training a diffusion model on these highly descriptive captions, we introduce DALL-E 3, a model that demonstrates unprecedented prompt adherence, spatial reasoning, and text-rendering capabilities without requiring complex prompt engineering.