PBDyG: Position Based Dynamic Gaussians for Motion-Aware Clothed Human Avatars
Shota Sasaki1, Jane Wu2, and Ko Nishino1 1Kyoto University 2University of California, Berkeley
We introduce PBDyG, a novel clothed human model that learns from multiview RGB videos to recover both body and cloth movements with physical accuracy. PBDyG, which stands for Position Based Dynamic Gaussians, incorporates a physics-based approach to simulate “movement-dependent” cloth deformation, unlike previous methods that rely on “pose-dependent” rigid transformations. Our method holistically models the clothed human by treating clothing as 3D Gaussians attached to a skinned SMPL body. The SMPL body captures the movement of the person, and its articulation drives the physically-based simulation of the cloth deformation, allowing the avatar to adapt to new poses. To simulate realistic cloth behavior, physical properties such as mass and material stiffness are estimated from the input RGB videos using Dynamic 3D Gaussians. Experiments show that PBDyG not only achieves accurate body and clothing appearance reconstruction but also enables the realistic modeling of highly deformable garments, such as skirts or coats, which have been difficult to simulate using existing techniques.
PBDyG: Position Based Dynamic Gaussians for Motion-Aware Clothed Human Avatars
S. Sasaki, J. Wu, and K. Nishino,
[ arXiv ][ video ][ project ]
Video
Overview
We introduce PBDyG, a novel avatar reconstruction method that models not only pose-dependent cloth deformations but also non-rigid deformations driven by movement. Given an RGB video captured from multiple viewpoints, PBDyG achieves avatar reconstruction in two steps. In the first step, PBDyG tracks the positions of the Gaussians in each frame using Dynamic 3D Gaussian. In the second step, Delaunay Triangulation is applied to establish the connectivity between the reconstructed Gaussians and the set of SMPL vertices. This connectivity allows the Gaussians to be driven by the deformation of the SMPL model. The driving of the Gaussians is done using a physics-based model called Position Based Dynamics (PBD). By optimizing the PBD physical parameters, the simulation results are adjusted to reproduce the tracking results obtained in the first step, enabling avatar reconstruction that reflects the physical properties of the clothing in the video.
Comparison with Pose-Dependent SOTA
We compare our method with Animatable Gaussians[12], one of the state-of-the-art (SoTA) existing techniques. For the evaluation, we use 0188 from DNA-Rendering dataset. The results of Animatable Gaussians show a failure in preserving the shape of the skirt, while our method successfully reconstructs the correct geometry.
Next, we conduct a same experiment on 0007 from DNA-Rendering dataset. Compared to Animatable Gaussians, our method is able to accurately capture the non-rigid motion of the sleeves.
Please see the video for more animation results.
