Night at the Museum: Text-Driven Motion Generation via Semantic Gaussian Bones
Chenyang Xu, Zeyu Jiang*, Haoran Li*, Shichen Zhang*, Juexiao Zhang, Guangzhao He†, and 3 more authors
2026
We introduce AniMuse, a two-stage framework for text-driven animal mesh animation directly from raw meshes, without predefined skeletons, joint names, or manual rigging. At its core are Semantic Gaussian Bones (SGBs), a compact skeleton-free deformation representation decoded from a globally shared learnable query book and trained through explicit linear blend skinning with topology-aware mask-gated weights. The shared query book yields stable cross-instance bone slots, providing a mesh-native control space for text-conditioned generation and SGB-slot motion inpainting. A diffusion model generates per-bone trajectories from text and geometric bone latents, while allowing users to clamp selected SGB slots and inpaint the remaining full-body motion. On DeformingThings4D, our rig reduces bidirectional CD-L1 by 39% over the best neural skeleton baseline, and a forward-only variant achieves the lowest CD-L2 overall. In a 50-participant user study on the out-of-domain AnimalML3D benchmark, participants preferred AniMuse over the strongest skeleton-free baseline 74.6% of the time overall, and 84.0% on whether the motion matched the prompt.
