"It works really well!": verification in theory and practice

“It works really well!” is the expected conclusion of academic publications, technical reports, and project outcomes that seek to demonstrate a designed system (i.e. robot, autonomous vehicle, controller) is functionally sound, safe and/or trustworthy. In this workshop, we dissected and examined under a critical eye each component of the phrase:
- the “It”, namely how the system-under-test has been defined for verification and validation purposes, and how this meets the needs for robotics in sectors such as autonomous vehicles and medical robots;
- the “works”, namely the definition of the system’s tasks or behaviours, including systems that might learn and at industrial scale;
- the “really well”, namely the formulation of metrics, evaluation tools, reproducibility and replicability frameworks, and extensibility to an “industrial grade”; and, furthermore, the ability to openly report “failures”. The workshop will bring together experts in robotics, verification and validation, end-user experience, regulations and standards, formal methods, assurance and certification, and industries including healthcare, transportation, aviation, military, and automotive.