AdaLN
fm/diffusion 常用
1flow step t -> t_emb -> [MLP] -> [Linear] inside Ada -> split as γ,β2x = (1.0 + γ) * layer_norm(x) + βCD
一致性蒸馏的目的是让 student 具有 从任意噪声时刻直接一步映射到干净样本 的能力,而 teacher 通常需要多步迭代去噪才能得到干净样本。
方法则是 “通过相邻时刻的一致性约束”.
1s, t = sample_adjacent_times() # t < s. time=0 is noise => time=1 is clean.2x_t = add_noise(x0, noise(), t)3x_s = x_t + (s - t) * teacher(x_t, t) # 教师做一步 Euler 求解4target = stop_grad(ema(x_s, s))5loss = distance(student(x_t, t), target)6optimize(student, loss); update_ema(ema, student)