Thesis - Smart Monitors exploiting differential video sensors & neuromorphic computing

Increased automation in automotive mobility is one of the grand challenges of our society. The goal is to drastically reduce road fatalities (vision zero), while increasing efficiency and productivity. SAE J3016 defines steps towards highly automated and autonomous vehicles (HAV). However, beyond level 3 this turns out to become tough. Several challenges of engineering safe and secure highly automated vehicles have been identified (arXiv:2103.03544), and approaches to overcome those have been suggested. Monitoring represents an important element. Smart monitoring components can provide reliable information about the status of a vehicle's safety, reliability and security. This includes continuous monitoring, if the vehicle observes its safety clearance, abides applicable traffic rules, etc. In this master thesis (done in collaboration with an ongoing HorizonEurope research project), a new approach based on event-based Dynamic Vision Sensors (DVS) and neuromorphic computing shall be analyzed. Based on simulators and HW prototypes from research partners, a prototypical smart sensor shall be implemented, and validated regarding energy consumption, accuracy and robustness. YOUR RESPONSIBILITIES: State of the art analysis of approaches to perform ML on DVS streams Select a model, then fine-tune it if required for a typical automotive scenario alternatively: build a simple own model by applying multi-stage learning using an existing perception system as feature extractor Port model to run on neuromorphic computing system (or simulator) Evaluation in typical driving scenarios on AVL's proving ground