Small Vision Transformers: Shortcut Models, Few Steps without a JVP

A shortcut model learns big steps from pairs of small ones, with no JVP: FID-10k 22.5 at 4 calls and near 16.8 from 16 to 128 calls, against MeanFlow.

On this page

The MeanFlow post showed one-step images from the same small DiT. MeanFlow pays for that with a Jacobian-vector product (JVP) in every batch, and with a target that can blow up if the time input has the wrong scale. Shortcut models (Frans et al., 2025) aim at the same goal with a simpler trick.

The idea: one big step equals two small ones

A shortcut model gets one more input than a flow-matching model: the step size d. Its output s(x_t, t, d) is the direction for one step of size d, from time t to t − d. At d = 0, that’s the ordinary flow-matching velocity.

For d > 0 there’s no exact target. So the model teaches itself. Two steps of size d should land at the same place as one step of size 2d. The target for the big step is the average of the two small-step directions:

s(xt,t,2d)≈12(s(xt,t,d)+s(xt−d,t−d,d))

The two small steps run without gradients, so the target is fixed. That’s two extra forward passes, and no JVP.

How I trained it

The DiT is the same as in the MeanFlow post, with a second time input for the step size and 9,937,420 parameters.

  • Two parts per batch: 75% of each batch is plain flow matching with d = 0 and uniform t. The other 25% is the self-consistency part.
  • Step sizes: d comes from {1/128, 1/64, …, 1/2}, so the big step 2d goes from 1/64 up to 1. The time t sits on a grid of multiples of 2d, so the big step never goes past the image at t = 0.
  • Finest level: when d = 1/128, the two small steps use the plain velocity (d = 0 input). That’s what ties the whole ladder of step sizes to flow matching.
  • Guidance at sampling: unlike MeanFlow, the targets have no guidance in them. Guidance (scale 2) comes at sampling time, so each step costs two model calls.

The data, optimizer, EMA and number of epochs (200) are the same as for the other methods.

Results

NFE Steps Shortcut FID-10k MeanFlow FID-10k (same NFE)
2 1 40.5 17.5
4 2 22.5 16.6
8 4 18.2 16.1
16 8 16.9 15.0
32 16 16.8 17.7
64 32 16.7 21.0
128 64 16.8 22.8
256 128 19.0 n/a

At equal model calls, MeanFlow wins up to 16 calls and the shortcut model wins from 32 calls on. From 16 to 128 calls, the shortcut model is flat at about 16.8.

FID-10k against NFE for DDPM, flow matching, MeanFlow and the shortcut model on log axes. The shortcut curve drops from 40.5 at 2 calls to about 17 at 16 calls and stays flat until it rises to 19.0 at 256 calls.

Per step, rather than per call, the gap is smaller. With one step, the two are close: 39.9 for MeanFlow and 40.5 for the shortcut model. With two steps, MeanFlow is still ahead (17.5 against 22.5). Part of MeanFlow’s lead at equal calls comes from its guidance being part of training, so its step costs one call and not two.

Why 128 steps is worse than 64

With 128 steps, each step has size 1/128. That value never appears as the step-size input in training: the flow-matching part uses 0, and the self-consistency part uses 2d, which is at least 1/64. So at 128 steps, the model sees an input it never learned, and FID rises from 16.8 to 19.0. That’s my explanation from the code; I didn’t run a test that isolates it.

The samples at 8 calls

Four panels of samples from the same noise, each with 8 model calls. DDPM with 4 steps: soft and blurred. Flow matching with 4 steps: sharper but still soft. MeanFlow with 8 steps and the shortcut model with 4 steps: sharp, clear objects with fine detail.

At 8 model calls, the two flow-map methods give sharp samples, and DDPM and flow matching are still soft. The same noise gives the same layout in every panel. At this size, the MeanFlow and shortcut samples look alike to me.

The cost

Flow matching Shortcut MeanFlow
Time per epoch 25.9 s 31.0 s 50.5 s
Training time (200 epochs) 86.2 min 103.3 min 168.4 min
Peak VRAM 3,595 MiB 3,599 MiB 3,599 MiB

The shortcut model adds 20% to the time per epoch of flow matching. MeanFlow adds 95%. That’s the price of the JVP and of the extra pass for the guided target.

Limits

  • I trained each method once, with one seed, so I can’t say how much FID changes from one seed to another.
  • I computed FID from 10,000 samples (FID-10k). FID from fewer samples is biased upward, so these values are higher than the FID-50k (50,000 samples) that papers report. Compare them with each other, not with published results.
  • The guidance scale is fixed at 2 for every method and every step count.
  • The shortcut model and MeanFlow use different training-time distributions (uniform against logit-normal), as in their papers. So this compares the two recipes, not only the two targets.

Run it

  1. Train the shortcut model.
  2. Compute FID-10k for 1 to 128 steps.
uv run python scripts/train_diffusion.py --method shortcut
uv run python scripts/evaluate_fid.py --method shortcut --steps 1 2 4 8 16 32 64 128

The code is in the repo.

What I take from this

  • Self-consistency between step sizes is enough to get good few-step samples, with no JVP and 20% more training time than flow matching.
  • In one step, the shortcut model and MeanFlow are close. Per model call, MeanFlow wins up to 16 calls, partly because its guidance costs nothing at sampling time.
  • Don’t sample with a step size the model never saw in training.
Written by Sina Fathi-Kazerooni

I write about reinforcement learning, transformers, multi-agent systems, and the tools I build along the way.