An Empirical Study of Tagging for Personal Information Organization: Performance, Workload, Memory, and Consistency

Abstract

Tagging allows users to organize their information and retrieve it later with multiple, freely chosen keywords, which is impossible with categorical folders. The need to organize information for personal later retrieval has been found to be one of the most important motivations for tagging. Despite the popularity of the concept, more empirical evidence is still required to verify the real benefit of tagging for information organization and retrieval. Furthermore, the problems of inconsistency in tagging hamper the usefulness of tagging as an effective organization tool. The current study aims to investigate users' motivation, performance, and workload when they use tagging to organize personal information and how the system design could improve the process. First, a pilot study combining think-aloud and interviews was conducted to obtain insights on why and how users select tags. Then, the first experiment with 40 participants was conducted to empirically compare the performance and workload difference in information organization and retrieval tasks between categorization and tagging interfaces. The results show that tagging users reported a significantly higher level of mental demand and frustration when performing organizational tasks and a significantly higher level of temporal demand and error rate when performing retrieval tasks compared with categorization. However, tagging users tend to have better memory of the organized content. The second experiment aimed to study how individual tagging consistency can be improved by the proper visualization of tag suggestions. The impact of frequency visualization by font size and semantically clustering was studied with 40 participants. The results show that semantically clustered tag clouds improve tagging consistency significantly; when a semantic clustering effect is presented, frequency visualization by font size can significantly alleviate the physical demand perceived by users.

Notes

¹Yahoo! MyWeb was shut down in February 2009, after Yahoo!'s acquirement of del.icio.us.