Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

Cet article n'a pas de fiche

Statut : exclu au tri — filtre mots-clés : aucun terme du groupe « tache ». Seuls les articles retenus au tri et dont le PDF est accessible sont lus en entier.

Résumé des auteurs

Video and audio are perceived together, yet most generative models treat them in isolation. We examine methods that model the two modalities jointly, generate one from the other, or edit them in a coupled manner, organized around a single question: how is the output kept coherent across modalities in time and semantics? A unified formulation casts joint generation, cross-modal generation, and joint editing as three problems defined on a single distribution over audio-visual pairs, and a taxonomy compares methods along five design axes. To our knowledge, this is the first overview to systematically taxonomize joint audio-visual editing, which we map as nine edit categories spanning 28 edit types. We describe methods, datasets, and metrics for each setting and close with the open problems we view as most consequential.