14 posts
Refine, Don’t Re-roll: Few-Step Generation by Iterating in Data Space
Our work on few-step distillation without a timestep schedule: the generator refines its own image prediction instead of re-noising it.
LLM 都会画画了，还需要训练视觉生成模型吗？中文
从一只小猫开始讲起。
So, What is Jev?
Is it really something new?
Mixture of Experts: How Routing Becomes GPU Computation Colab notebook
Why fewer active parameters don’t automatically mean faster models.
RL, OPD, VSD, and GANs: One Density Ratio, Different Updates
How a log-density ratio connects policy gradients, Gaussian mean MSE, diffusion scores, and discriminator feedback—and where the equivalences end.
Scaling Long-Context Attention I: Exact Blocks, Rings, and Sparsity Colab notebook
A unified view of block attention: FlashAttention, Ring Attention, and sparse attention share the same blockwise computation but obey different contracts.
Scaling Long-Context Attention II: From MHA to MLA Colab notebook
How MHA, GQA, MQA, split-KV, Paged Attention, and MLA reshape the KV cache and its decode kernels.
Scaling Long-Context Attention III: Recurrent and Learned Memory Colab notebook
How Linear Attention, chunkwise training, DeltaNet, TTT, and LaCT replace a growing token history with recurrent and learned memory.
Distributional Distillation I: From a Marginal KL to Two Scores
A first-principles derivation of distributional distillation, from marginal KL objectives to two-score and f-divergence gradients.
Distributional Distillation II: What the Methods Actually Estimate
A method-by-method map of what SDS, VSD, Diff-Instruct, DMD, DMD2, and Phased DMD actually estimate.
Distributional Distillation III: From Scores to Particle Flows
From score differences to particle motion, connecting DMD, Drifting Models, and the Sinkhorn-based W-Flow through Wasserstein gradient-flow geometry.
Theoretical Analysis of ConsistencySolver (CVPR 2026 Highlight)
A numerical-analysis view of ConsistencySolver, deriving its learnable update rule from PF-ODEs and linear multistep methods.
Theoretical Analysis of Rectified Diffusion (ICLR 2025)
An optimal-transport analysis extending rectification, path straightening, and O(1/K) convergence from linear flows to general diffusion schedules.
On the Equivalence of Consistency Models and MeanFlow
A continuous-time derivation showing that consistency training reduces to MeanFlow, with an extension from one-step CM to multi-step CTM.