DIFFLET: A Fast Inference Engine for Diffusion Transformer Models on AWS Trainium
The authors present Difflet, a serving engine that accelerates diffusion model denoiser steps by fusing attention and reusing earlier outputs.
The authors present a novel serving engine that combines multiple model pipelines and reuses earlier outputs to accelerate diffusion model denoiser steps.
Before reading this…
Applications
- →image generation
- →video generation
To understand this paper, make sure you know these concepts first:
- diffusion modelsfind papers →
- attention mechanismsfind papers →
Abstract
More Like ThisOn AWS Trainium2, every component of a diffusion model runs as a program compiled ahead of time for fixed shapes, while pipelines differ across models, prompt lengths across requests, and caching decisions across denoising steps. We present Difflet, a serving engine in which five model pipelines share one compilation backend, joint attention is split across cores by attention head, and a host controller decides at each denoising step whether to run the denoiser or reuse its earlier output. A small probe program computes the caching signal on the device and returns one number per step, and a resident model process keeps every program loaded across requests. Across five image and video model families, fused attention accelerates denoiser steps by 1.2-4.3x over the same framework using PyTorch/XLA attention, and at 32.8k visual tokens it is the only attention that compiles. On four of the five models, resident serving is 1.5-3.4x faster than standalone runs that reload weights from a warm file cache for each generation.