Explainable, Safe, Contact-Aware Planning and Control for Heavy Machinery Manipulation and Navigation

Earth-Moving, Forestry, and Urban Logistics are sectors where increased autonomy can spur drastic economic growth along with addressing some core societal (e.g. address labor shortage) and environmental problems (e.g. minimize soil damage, fuel consumption). Yet there are persisting challenges related to variations of tasks/environments that are intricately linked to the terrainmachine contact encountered during navigation and manipulation. For example, an Excavator machine used in Earth-Moving needs to adapt to different types of terrain (ground) underneath (loose soil, rocks of different shapes and sizes), for scooping. Such task and environment adaptation require machines to modify their “perception-to-action” mapping based on online observations from different sensing modalities. XSCAVE will leverage the exceptional representation and approximation capabilities of deep neural networks to automatically learn the terrain/specific adaptation of excavation, forwarding, and navigation strategies from data. The overall objective of XSCAVE is (i) to develop capabilities for learning performant (high-speed), safe (stable, contact-aware), and explainable perception-to-action models for terrain adaptive excavation and navigation strategies and (ii) demonstrate step-change in autonomy for Excavation, Forwarding and Navigation tasks prevalent in Earth-Moving, Forestry and Logistics industries. To this end, XSCAVE aims to re-imagine deep-learned models as neural networks augmented with parameterized structured priors derived from physics, optimization, and classical search to bring domain knowledge into the learning pipeline. The fundamental innovations at the algorithmic level will translate to unprecedented ability for the machines to plan, control and adapt their actions depending on the task and terrain contact conditions. The end-results will be demonstrated in partnership with Novatron (earth-moving), Komatsu (forestry), and Clevon (outdoor logistic vehicles).