Picking interesting pictures

We submitted a paper on automatically extracting interesting pictures from robot navigation video to the Robotics and Automation (ICRA) conference. Normally I don't blog about real work, or write about unpublished results, but this time the conditions are right. This work is part of John-Paul Lobos' thesis research and it extends the thing dubbed the "Vacation Snapshop Problem" in the work by Eric Bourque and me a few years ago (pdf paper). The idea of the Vacation Snapshop Problem is to select a few descriptive "Vacation Snapshots" from the potentially large set of pictures you (or a robot) might take while navigating along a route (i.e. a video that is a so-called navigation record). In this work we looked that what defines the most appropriate "interesting" photographs that summarize what a robot saw.

An aspect of the approach involves building clusters of images that are depict about the same content as one another (even though the pixel-to-pixel content of these images may vary quite a bit). The figure here illustrates the kind of results we get fully automatically, but there are a lot of details that determine what kind of pictures are selected.