Automatically assisting human memory: A SenseCam browser

Abstract

SenseCams have many potential applications as tools for lifelogging, including the possibility of use as a memory rehabilitation tool. Given that a SenseCam can log hundreds of thousands of images per year, it is critical that these be presented to the viewer in a manner that supports the aims of memory rehabilitation. In this article we report a software browser constructed with the aim of using the characteristics of memory to organise SenseCam images into a form that makes the wealth of information stored on SenseCam more accessible. To enable a large amount of visual information to be easily and quickly assimilated by a user, we apply a series of automatic content analysis techniques to structure the images into “events”, suggest their relative importance, and select representative images for each. This minimises effort when browsing and searching. We provide anecdotes on use of such a system and emphasise the need for SenseCam images to be meaningfully sorted using such a browser.

Keywords:

• SenseCam
• Lifelogging

ACKNOWLEDGEMENTS

This material is based on works supported by the Science Foundation Ireland under Grant No. 07/CE/I1147. The authors also wish to acknowledge the support of Microsoft Research (grant number 2007-056). We would also like to extend our gratitude to our friends in Tampere, Utrecht, and Duke Universities for their feedback. In addition to this we very much appreciate the insightful feedback comments from our reviewers. An open source version of the browser described here is available at: http://sensecambrowser.codeplex.com/

Notes

¹GPS (global positioning system) is a mechanism to automatically locate a person's location anywhere on earth.

²One week would take 35 minutes to review, with 1 year taking over 30 hours to look back on using RSVP alone!

³Users still have access to all the images in the given event. For display purposes a single keyframe image is displayed to the user to summarise all the (broadly similar) images in the given event.

⁴In the cited work the keyframe was based on the most visually representative event image. A worthwhile future investigation would see if this technique selects the most powerful memory trigger too.

⁵ http://www.omg3d.com/html/IPLicenseagreement.html

⁶ http://sensecambrowser.codeplex.com/