I had the honor of speaking at a working at MIT yesterday on the testing of robotics systems. My own talk was heavily based our experiences testing our robotic boat, robot plane and underwater vehicles. The field trials we do involve a range of challenges in terms of experiment planning, but also including logistics, safety and team management.
The workshop overall was motivated by a project called PATFrame that deals with understanding and optimizing test planning, especially for large military-sponsored robotics projects. Other participants from the US government and military spoke about the challenges of testing adaptive systems that both change over time, and also exhibit performance which depends on (variable) features of the environment. In short, it is essentially impossible to cover al possible conditions when a system is so versatile and complex.
As intelligent systems gradually approach the complexity and richness of humans, testing them definitively will become as tricky as testing people. Even when you think you know a person very well, they can exhibit surprising behavior and there is no way to preclude the same phenomena from robotics systems. This poses a challenge not only for testers, but for society at large.
Robot Attack: did we test it enough
The time robots will truly behave in an interesting unexpected way we will have made a big leap in robotics (and AI!). If you are talking about bugs, I agree with you. Bugs in robotics (either software or hardware) are tricky, as most of them depends on some non-repeatable-outside-world input.
But if a bug (!?) will turn out to be an unexpected behaviour, something the designer will have to devote time to investigate... well, that's the topic or a whole research project.
And maybe a film! (Short circuit?)
Lorenzo, I think it is fair to say that today we are seeing robot systems in the field that exhibit both bugs as well as other unexpected behaviors that may or may not count as bugs. These other behaviors include phenomena known as "emergent behaviors" that result from the combination of two or more algorithms or behaviors to produce a new one that was not clearly anticipated. Even the Roomba robot exhibits simple forms or emergent behavior. In practice, with robotic systems in extensive use (for example thousands of explosive disposal (EOD) robots in the ground in Iraq) this is a real issue here an now.
Clearly predicting such emergent behavior is very hard and a research problem, but that does not eliminate the need to estimate and determine risk factors NOW. Even in cars this problem needs to be done every day despite the fact that some phenomena cannot actually be verified analytically.
There are no trackbacks.
