🌈 MoG: Motion-Aware Generative Frame Interpolation

Guozhen Zhang^1,2, Yuhan Zhu¹, Yutao Cui², Xiaotong Zhao², Kai Ma², Limin Wang^1,3

¹State Key Laboratory for Novel Software Technology, Nanjing University  ²Platform and Content Group (PCG), Tencent  ³Shanghai AI Lab

arXiv Code Hugging Face Demo

🌱 Introduction 🎬 Demos 💥 Comparisons

🌱 Introduction of our MoG

MoG is a generative video frame interpolation (VFI) model, designed to synthesize intermediate frames between two input frames.

MoG is the first VFI framework to bridge the gap between flow-based stability and generative flexibility. We introduce a dual-level guidance injection design to constrain generated motion using motion trajectories derived from optical flow. To enhance the generative model's ability to dynamically correct flow errors, we implement encoder-only guidance injection and selective parameter fine-tuning. As a result, MoG achieves significant improvements over existing open-source generative VFI methods, delivering superior performance in both real-world and animated scenarios.

🎬 Demos produced by our MoG

Input frames	Interpolation results	Input frames	Interpolation results

Input frames	Interpolation results	Input frames	Interpolation results

Input frames	Interpolation results	Input frames	Interpolation results

Input frames	Interpolation results	Input frames	Interpolation results

Input frames	Interpolation results	Input frames	Interpolation results

Input frames	Interpolation results	Input frames	Interpolation results

💥 Comparisons with existing generative VFI methods

Real-world scenes

Input frames	GI	DynamiCrafter	MoG (Ours)

Input frames	GI	DynamiCrafter	MoG (Ours)

Input frames	GI	DynamiCrafter	MoG (Ours)

Input frames	GI	DynamiCrafter	MoG (Ours)

Input frames	GI	DynamiCrafter	MoG (Ours)

Animated scenes

Input frames	GI	ToonCrafter	MoG (Ours)

Input frames	GI	ToonCrafter	MoG (Ours)

Input frames	GI	ToonCrafter	MoG (Ours)

Input frames	GI	ToonCrafter	MoG (Ours)

Input frames	GI	ToonCrafter	MoG (Ours)