As developments in robotics, autonomous driving, and spatial computing proceed to unfold, a rising variety of Pc Imaginative and prescient and Machine Studying (CVML) algorithms are incorporating three-dimensional knowledge into their frameworks. Debugging these 3D CVML fashions typically requires going past conventional efficiency analysis strategies, necessitating a deeper understanding of an algorithm’s conduct inside its spatio-temporal context. Nevertheless, the shortage of acceptable visualization instruments presents a big impediment to successfully exploring 3D knowledge and spatial options in relation to key efficiency indicators (KPIs). To deal with this problem, we discover the appliance of Immersive Analytics (IA) methodologies to reinforce the debugging means of 3D CVML fashions. By way of in-depth interviews with eight CVML engineers, we establish frequent duties and challenges confronted through the improvement of spatial algorithms, and set up a set of design rules for creating instruments tailor-made to spatial mannequin analysis. Constructing on these insights, we suggest a novel immersive analytics system for debugging an indoor localization algorithm. The system is constructed utilizing internet applied sciences and integrates WebXR to allow fluid transitions throughout the reality-virtuality continuum. We conduct a qualitative examine with six CVML engineers utilizing our system on Apple Imaginative and prescient Professional, observing their analytical workflow as they debug an indoor localization sequence. We focus on the benefits of using immersive analytics within the mannequin analysis workflow, emphasizing the position of seamlessly integrating 2D and 3D visualizations throughout various ranges of immersion to facilitate simpler mannequin evaluation. Lastly, we replicate on the implementation trade-offs and focus on the generalizability of our findings for future efforts in immersive 3D CVML mannequin debugging.
- †Harvard College, Cambridge
