A Stereo-Panoramic Telepresence System for Construction Machines

Working machines in construction sites or emergency scenarios can operate in situations that can be dangerous to an operator both for direct control of the machine or teleoperation in co-presence. Remote operation has been typically hindered by limited sense of presence of the operator in the remote of the environment to due the reduced field of view of cameras. Starting from these considerations we are introducing a novel real-time panoramic telepresence system for construction machines. This system does allow fully immersive operations in critical scenarios while keeping the operator in a safe location at moderate distance from the construction site. An omnidirectional stereo vision head mounted over the machine acquires and streams data to the operator with a streaming technique that focuses on the current direction of site of the operator. According to the motion of the operators head, the telepresence system will determine the involved cameras to use and the appropriate region of interest in cameras. The operator uses a head-mounted display to experience the remote site also with the possibility to view digital information overlaid to the remote scene as a type of augmented reality. The paper addresses the design and architecture of the system starting from the vision system and then proceeding to the immersive visualization