Table Assembly Process
We designed a process to automatically generate step by step assembly instructions simply by recording an expert doing it. We used our high quality motion capture system to record the expert assembler as they move the parts into place and attach them.
For this demo, we used a coffee table from IKEA. We chose this table since it uses standard screws and many people have assembled a piece of furniture from IKEA.
Step 1: Preprocess 3D table model
Each separate part of the coffee table is held together with machine screws. Use the 3D CAD model to define the relationships between the parts.
Step 2: Motion capture the assembly process
Record the motion of the person, the tools, and each part of the coffee table.
Step 3: Generate step by step instructions
Process the mocap data to automatically extract the assembly steps. The sequence of steps are printed in plain language.
Step 4: View assembly steps
Each step is annotated with extra information for visual display. View the instructions as a document, as a step by step 3D visualization, or as an interactive VR trainer.
How it works
1. Preprocess 3D table model
We modeled the coffee table in Autodesk Fusion 360 CAD/CAM software. Fusion 360 supports assembly constraints and joints.
We use the parametric data from the 3D CAD model to help automatically generate the step by step assembly instructions.
2. Motion capture the assembly process
We use a hybrid mocap approach consisting of our Shadow inertial mocap suit synced with an optical motion capture system. We track the full body motion of the assembler, the impact driver, and the four parts of the coffee table.
The motion capture data is used to automatically generate the step by step instructions. The steps are sequenced in time. The assembler may put together the table in any order and the steps update.
We recorded the data with our Shadow software. All of the measurements are streamed at 100 frames per second.
3. Generate step by step instructions
We use the motion capture data to automatically extract the sequence of assembly steps. How the person moves, assembles the parts together, and operates the tools are all used in this process.
We defined a limited vocabulary, or taxonomy, of possible actions. For the coffee table we only need to pick up and move parts, align screw holes, and put in screws.
Move
Pick up one of the parts.
Place
Line up two screw holes.
Fasten
Use the driver to put in a screw.
The steps are time coded in seconds. The coffee table took 23 steps over 2 minutes to assemble.
| Time (sec) | Instruction |
|---|---|
| 10.09 | move Left Leg Frame |
| 11.53 | place A12 on A4 |
| 13.00 | place A13 on A5 |
| 15.32 | move Shelf |
| 30.15 | fasten A13 to A5 with M5x10 Machine Screw using Impact Driver |
| 32.03 | move Right Leg Frame |
| 35.33 | place A15 on A9 |
| 42.43 | fasten A12 to A4 with M5x10 Machine Screw using Impact Driver |
| ... |
4. View assembly steps
The step by step instructions include much more information than just the time and action. We use this metadata to create more advanced visualizations of the assembly process.
These views are dynamic and will update given a new assembly sequence.
Step by step visualization
See all of the parts in a 3D scene
Step through the assembly process to view the instructions
Interactive VR trainer
Assemble the table in virtual reality
See the instructions, placement hints, and complete the required steps
Videos
Interactive VR trainer
See the entire assembly process in virtual reality. The training application uses the steps automatically generated from the motion capture session
Motion capture the assembly process
Full recording the motion of the assembler, the driver, and the parts of the coffee table. See the 3D motion preview along side the video
Step by step 3D visualization
This video steps through the entire assembly process, viewing all of the parts and step instructions in a 3D scene