This project answers the following question: Can the floor plan of a building be reconstructed using only inertial measurement unit (IMU) sensor measurements collected from humans navigating within the building? Our research aims to explore this prospect and to contribute to the development of more data-driven, accurate and versatile floor plan estimation methods. Current systems lack the integrated ability to simulate realistic walking trajectories, translate them into multi-room floor plan layouts, and predict structural elements. The herein-introduced pipeline consists of three modules:
- trajectory sampler generating paths including drift and sampling of human activities
- image-to-image translation network transforming trajectories into floor plan layouts
- multi-task learning network predicting structural elements, including boundaries (walls and doors), room types, furniture locations and labels.
The system was evaluated with two synthesized data sets. Our results show that it is possible to reconstruct entire floor plans and recover semantic information based on IMU sensor data alone.
The code of this project is organized in five different folders:
- The trajectory sampler is located in trajectory_sampling
- Image-to-image translation training and testing scripts for floor plan layouts are located in pix2pix
- Multi-task learning of structural elements is located in tf2deep
- Real-world trajectory generation and evaluation scripts (FID and LPIPS) are located in trajectory_testing
- Application to record activities (under development) in IMUrecorder
For reference, an old implementation of the Pix2pix cGAN is located in cgan with the DataLoader and Dataset class for the Structured3D dataset.