Intelligent system design for episodic memory on mobile daily life

Conventional way of recording the daily life is to write a diary which can help the people recall what they did, but it is based on the human's limited memory. Recent advances in pervasive and mobile technology allow us not only to log user's daily activities (location, call log, SMS, and proximity information with Bluetooth) but also to store personal multimedia information (photo, video and documents), which contains private information to reveal user's daily events. Exploiting the information in a full scale would give us a chance to produce innovative applications in accordance with personal preference and contexts. Automatic diary generation from user's log (explicit or implicit) is one of hot research topics. Sumi et al. designed comic diary to summarize the user's conference tour in a comic form. Eagle et al. tried to develop a diary system based on the log information collected from cellular phone. Nokia's LifeBlog provides a way to store and manage user's photo, multimedia and SMS in a chronological manner. In this talk, I will present the trend of the research to summarize user's daily life based on the information collected from mobile devices, and report on the design experience of an intelligent system that collects the available information from Smartphone, extracts the landmarks in a daily life, and generates the summary of comic-style diary automatically. Landmark means very relevant or novel events that are useful to recall a sequence of events, and the organization must be similar to the human memory structure. From the logged information such as GPS, call log, SMS, MP3 play lists, battery level and photo viewer usage, an ensemble of Bayesian networks are used to infer the landmark events, and the most probable candidates are dynamically selected based on the relevance. Finally, they are converted to the comic cartoons by composing the comic image components.