Software & Data Downloads — AssemblyBench

Physics-Aware Assembly of Complex Industrial Objects for grounded multimodal assembly reasoning with CAD and instruction manuals.

Assembling objects from parts requires understanding multimodal instructions, linking them to 3D components, and predicting physically plausible 6-DoF motions for each assembly step. Existing datasets focus on simplified scenarios, often overlooking shape complexity and assembly trajectories in industrial assemblies. We introduce AssemblyBench, a synthetic dataset of 2,789 industrial objects with multimodal instruction manuals, corresponding 3D part models, and physically plausible 6-DoF part assembly trajectories. We also propose AssemblyDyno, a transformer-based model that uses the instruction manual and the 3D shape of each part to jointly predict assembly order and part assembly trajectories. AssemblyDyno outperforms prior works in both assembly pose estimation and trajectory feasibility, with the latter evaluated using our physics-based simulations.

AssemblyBench is designed to facilitate research that bridges instructional manual understanding and the execution of assembly steps, serving as a benchmark for developing next-generation assembly algorithms. In this release, we publicly share the AssemblyBench dataset and our Python implementation, including code for data generation, training the AssemblyDyno model, and physics-based evaluation.