Researchers from Tongji University and other institutions have published a paper on arXiv proposing DiT-ST, an innovative framework that uses Split-Text Conditioning to significantly improve diffusion Transformers' performance on text-to-image generation.

Technical Breakthroughs:
Traditional text-to-image diffusion models suffer from incomplete-text understanding flaws — text length limits, softmax competition, and positional bias lead to attribute-binding errors and semantic confusion. DiT-ST addresses these challenges through three key innovations:

  1. Text Parsing: Uses a large language model (LLM) to parse complex text into semantic primitives (objects, relations, attributes) and build a hierarchical graph.
  2. Hierarchical Input: Converts complete text into simplified split-text inputs, reducing grammatical complexity.
  3. Incremental Injection: Based on the denoising stage's sensitivity to different semantic types, injects semantic information at different timesteps in an object-relation-attribute priority order.

Performance Gains:

  • 69% overall accuracy on the GenEval benchmark, close to SDv3.5 Large (71%).
  • CLIPScore of 34.09 on the COCO-5K dataset, about 4.1% above SDv3.5 Large.
  • More robust on complex long text, solving existing models' text-length sensitivity.

This work not only improves generation quality — more importantly, it reveals diffusion models' intrinsic rules of sensitivity to different semantic types at different denoising stages, offering a new perspective on understanding how diffusion models semantically build up. This semantic-decomposition-based approach is expected to push multimodal generation further.