How to?

models README

Jan 1, 2025
papersmodels
1 Minutes
150 Words

AdaLN

fm/diffusion 常用

1
flow step t -> t_emb -> [MLP] -> [Linear] inside Ada -> split as γ,β
2
x = (1.0 + γ) * layer_norm(x) + β

CD

一致性蒸馏的目的是让 student 具有 从任意噪声时刻直接一步映射到干净样本 的能力,而 teacher 通常需要多步迭代去噪才能得到干净样本。

方法则是 “通过相邻时刻的一致性约束”.

1
s, t = sample_adjacent_times() # t < s. time=0 is noise => time=1 is clean.
2
x_t = add_noise(x0, noise(), t)
3
x_s = x_t + (s - t) * teacher(x_t, t) # 教师做一步 Euler 求解
4
target = stop_grad(ema(x_s, s))
5
loss = distance(student(x_t, t), target)
6
optimize(student, loss); update_ema(ema, student)
Article title:models README
Article author:Julyfun
Release time:Jan 1, 2025
Copyright 2026
Sitemap