Overview
Safety, in the form of protection from harm, is non-negotiable. However, the way we assure safety is changing.
Just as AI represents a disruptive technological shift, it requires a corresponding shift in safety engineering. This lecture with Prof. Mario Trapp outlines the key challenges posed by AI systems, including their data-driven nature, limited predictability and new types of failure modes. These properties challenge established assumptions in safety engineering.
Against this background, the talk argues that safety needs to be re-interpreted. Traditional approaches based on complete specification and upfront guarantees are no longer sufficient and must be extended to address the characteristics of AI-based systems. Finally, the lecture introduces three steps toward safe AI, providing a structured perspective on how safety can be addressed in this new context.
Speaker bio
Prof. Mario Trapp holds the chair of Engineering Resilient Cognitive Systems in the School of Computation, Information and Technology at the Technical University of Munich.
Dr. Trapp is also executive director of the Fraunhofer Institute for Cognitive Systems IKS in Munich, and was recently appointed Vice Chair of the Board of Directors of Fraunhofer USA. He is also Chair of the EWICS Association, EWICS established the leading safety conference, SafeComp, and still acts as the steering committee for SafeComp.
Prof. Trapp’s research focus is on the engineering of resilient cognitive systems. He combines his many years of experience in the field of model-based safety engineering with the engineering of self-adaptive software systems. In addition to basic research, successful transfer to industrial practice is particularly important to him. These days we are at a cross-roads in safety – AI presents opportunities while also raising the likelihood of safety breakdowns if we do not use it with appropriate guardrails. Mario Trapp is at the forefront of working out ways we can do this.
He is a member of the Bavarian State Government’s Council on AI (Bayerischer KI-Rat) and the Bavarian State Ministry of Economic Affairs, Regional Development and Energy’s AI — Data Science (KI — Data Science) expert panel.