Automotive DNN-Based Object Detection in the Presence of Lens Obstruction and Video Compression

Recent advances in sensing, processing, machine learning, and communication technologies are accelerating assisted and automated functions development for commercial vehicles. Environmental perception sensor data streams are processed to generate a correct and complete situational awareness. It is of utmost importance to assess the robustness of the sensor data pipeline, particularly in the case of data degradation in a noisy and variable environment. Sensor data reduction and compression techniques are key for higher levels of driving automation, as there is an expectation that traditional automotive vehicle wired networks will not be able to support the needed sensor datarates (i.e. more than 10 perception sensors, including cameras, LiDARs, and RADARs, generating tens of Gb/s of data). This work proposes for the first time to consider video compression for camera data transmission on vehicle wired networks in the presence of highly noisy data, e.g. partially obstructed camera field of view. The effects are discussed in terms of machine learning vehicle detection accuracy drop, and also visualising how detection performance spatially varies on the frames using the recently introduced metric, the Spatial Recall Index (SRI). The presented parametric obstruction noise model is generated to emulate real-world patterns, whereas compression is based on the well-established AVC/H.264. While Deep Neural Networks’ (DNNs’) performance is stable with lossy compression (up to 70:1) of ‘ideal’ data, when noise is added there is a significant accuracy degradation, in the range of a 7%-90% decrease. The proposed compression and noise tuning of the DNN training improves the performance up to 35%, enhancing the noise and compression robustness of the system. However, in the presence of compression combined with extreme levels of noise (i.e. more than 80% of pixels affected), DNN performance significantly degrades, up to a 90% decrease, even with re-training. This issue needs to be carefully considered in the design phase of perception and communication networks used to transmit sensor data.