Inference-Only Prompt Projection for Safe Text-to-Image Generation

ABSTRACT

Text-to-Image (T2I) diffusion models enable high-quality synthesis, but deployment demands safeguards. We formalize the tension between safety and alignment through a total variation (TV) lens, yielding a principled Safety-Prompt Alignment Trade-off (SPAT). We propose an inference-only prompt projection framework that selectively intervenes on high-risk prompts via a surrogate objective with verification, mapping them into a tolerance-controlled safe set while leaving benign prompts unchanged. Across four datasets, our approach achieves 16.7–60.0% relative reductions in inappropriate percentage (IP) versus strong model-level alignment baselines while preserving alignment on benign COCO prompts.