🌠ArbInterp: Arbitrary Generative Video Interpolation
🌱 Introduction of our ArbInterp
ArbInterp is a novel generative video frame interpolation (VFI) paradigm that enables efficient interpolation at any timestamp and of any length.
ArbInterp proposes Timestamp-aware Rotary Position Embedding (TaRoPE), which modulates temporal RoPE to align generated frames with target continuous timestamps between input frames, overcoming the inflexibility of traditional fixed interpolation.
To mitigate the discontinuities in motion and appearance across segments, ArbInterp also proposes a novel appearance-motion decoupling conditioning strategy, ensuring seamless spatiotemporal transitions.
📷 Comparison of demos produced by ArbInterp and other methods on MultiInterp Benchmark
Case 1: a driving car.
| 2x Interp. | 8x Interp. | 16x Interp. | 32x Interp. | |
|---|---|---|---|---|
| Input frames | ||||
| DynamiCrafter | ||||
| TRF | ||||
| GI | ||||
| ArbInterp |
Case 2: a surfing person.
| 2x Interp. | 8x Interp. | 16x Interp. | 32x Interp. | |
|---|---|---|---|---|
| Input frames | ||||
| DynamiCrafter | ||||
| TRF | ||||
| GI | ||||
| ArbInterp |
🎬 Comparison of videos produced by ArbInterp and other methods on StreamInterp Benchmark
| Input frames | DynamiCrafter | TRF | GI | ArbInterp |
|---|---|---|---|---|