CHI 2012 Papers and Notes Reviews of submission #434: "A Field Inverstigation of Individual Differences in Personal Task Management" ------------------------ Submission 434, Review 4 ------------------------ Reviewer: primary Your Assessment of this Paper's Contribution to HCI This paper presents a study on academics' use of Personal Task Management systems. Findings suggest three clusters of PTM users: adopters, make-doers, and do-it-yourselfers and the implications of these clusters on the design for e-PTM tools. Overall Rating 3.0 - Neutral: Overall I would not argue for accepting or rejecting this paper. Review Expertise 3 (Knowledgeable) The Meta-Review A lengthy discussion took place, and unfortunately, no consensus was reached. To start, the reviewers, as far as I can tell, did not have any problems with the methods used and thought the study was well reported. Here is the main point of contention: the specific population studied and their representativeness. Given that current PTM tools are not designed for academics, is it appropriate to study academics? Can what we find with academics really help guide the design of general PTM tools? If not, just how important is this specific market niche? R3 thinks that despite this limitation, this paper still provides sufficient contribution to be accepted for CHI, while R1 and R2 disagree. For the rebuttal, the authors would need to motivate their rationale for focusing on academics in this work. Even if the reviewers end up agreeing that this paper offers sufficient contribution, the authors will need to tone down their claims...focusing more on "A Field Investigation of Individual Differences in *Academic's Personal Task Management" as opposed to general PTM. Reviewers have all also provided specific suggestions to make the paper stronger. In particular, R2 points out that the three user-types could be better substantiated by using the wealth of existing research. Has prior research also found similar clusters? Associate Chairs Additional Comments The Review Areas for Improvement ------------------------ Submission 434, Review 1 ------------------------ Reviewer: external Your Assessment of this Paper's Contribution to HCI The paper contributes findings about personal task management (PTM) behaviors and proposes three types of person in terms of PTM strategy; adopters, DIYers and make-doers. A number of detailed observations are given about the specific behaviors of each strategy type, which are the basis for arguing that they require different types of support tool. Overall Rating 2.5 . . . Between possibly reject and neutral Review Expertise 4 (Expert ) Areas for Improvement This is a nice preliminary study, well reported too, but it has three serious flaws that need to be addressed in future work, which the authors apparently plan to do. One is to study a more representative population; the participants in this research were 17 students, a doctor and a professor, which is hardly representative of the type of person that might want to purchase a task management system. Because of this, the PTM types presented are highly suspect and cannot be used as a basis for arguing for developers to develop different systems to support each of the types. Second, the types are a nice way to describe observed behaviors, but the research provides no way to know if these are clusters of behaviors that really go together without a lot of overlap, or rather the authors' mental short-hand for representing a lot of diverse behaviors. The sample size could not possibly allow for a reliable cluster analysis here, even if the authors wanted to do that. So the PTM types could be illusory. It is not uncommon to posit categories of people (e.g., the well-known Whittaker and Sidner types of email user), but if the paper then goes on to argue that they represent some kind of strategic market grouping that developers should pay attention to, without providing more quantitative evidence, it pushes too far for my liking. investigating which of the existing tools can better solve the problems of our participants . Third, the paper does not consider all of the many tools that are out there which could solve the problems of the people studied. The authors are arguing for developers to design *more* tools in this already overcrowded space. I think the real problem here is that people are unaware of which existing tools would best suit their needs and unwilling to risk wasting time by systematically trying them one after another until they find the one that works for them (at least for a while until their situation changes and drives a need for a different type of tool). In fact I would argue that a really good study for the authors to pursue as a next step should include an attempt to determine which of some of those tools is good for what kind of task management strategy, or best for most users. Perhaps this could be built into the authors' future study of a broader-based population. To sum up, there is promise to this line of work and the possibility to add a substantial contribution beyond past work, given a more representative study sample, validated characterizations of types and consideration of existing design solutions. I encourage the authors to pursue this direction further. ------------------------ Submission 434, Review 2 ------------------------ Reviewer: external Your Assessment of this Paper's Contribution to HCI The authors present a study of the task management habits of roughly 19 students, 17 of whom are from the CS department of the authors' institution. The main contribution of this paper is its segmentation of users into adopters, "DIYers", and "make-doers" that the authors use as the basis for their design implications. Given the limited participant population and the existing body of work in this area, I don't think it's a strong enough contribution to warrant acceptance as a full paper. Overall Rating 2.0 - Possibly Reject: The submission is weak and probably shouldn't be accepted, but there is some chance it should get in. Review *Methods* The authors use focus groups, interviews, and grounded theory affinity analysis for this study, which seem perfectly appropriate given that their research questions are exploratory in nature. *Novelty* The authors are working in a crowded space; the wealth of research on tasks, activities, calendars, email, and time management in the field covers a lot of the findings, and the authors note this in their submission. The contributions offered by the submission, in my mind, are the user segmentations developed by the authors' affinity analysis as well as the design implications that follow from them. What the authors have provided is a segmentation based on CS graduate students that fits their data well. This brings me to my main criticism of the submission. The authors' stated goal was very broad: to understand individuals' task management behavior The sample was strangely confined to mostly CS students that, despite using a range of tools, are likely providing a small glimpse into a much larger range of management strategies. In addition, given that we have a lot of existing research surrounding tasks as they are managed within email, calendar, office work, etc, I'm curious as to why the authors didn't incorporate this data into their own segmentations and design implications. Instead, they seem to be focusing on setting their work apart from these other studies. Do adopters, DIYers, make-doers apply to the work of, say, Bellotti et al? Do the design implications also apply? The design implications themselves seem to follow general guidelines for personalization (flexibility, extensibility, integration with existing tools). I think these guidelines will run up against the classic problem of encouraging users to customize at all. It also begs the question of whether CS students are more willing to do this customization because of their more technical bent. Expertise 4 (Expert ) Areas for Improvement The writing is good and easy to follow. The initial figure on page 1 does a great job of showing the diverse range of task management strategies used by participants, and Table 1 summarizes the findings well. I'd also recommend citing some of the home calendar research done by Carmen Neustaedter and AJ Brush, which also supports some of the findings of this study in terms of family task management. ------------------------ Submission 434, Review 3 ------------------------ Reviewer: external Your Assessment of this Paper's Contribution to HCI This paper describes a field study of Personal Task Management in which the reasons for using particular tools, needs around these tools, and specific activities pertaining to task management were examined through semi-structured interviews with 12 academics. Although definitely not the first task management study, this remains a very interesting area of investigation, as this study makes clear that people are _still_ _not_ using ePTMs effectively, and suggests some reasons why this might be. Thus, the results of this study will be very much of interest to PIM researchers and practitioners alike. Overall Rating 4.0 - Possibly Accept: I would argue for accepting this paper Review With respect to directions for improvement, I believe the paper could greatly improve the "discussion/conclusion" to amplify the findings that most distinguish this study, particularl for other PIM researchers familiar with prior work in this space. What's new here, and what does it mean for the design of task management The third comment below focuses on this issue. Comment 1. On classifications 1: Environmental vs personal vs tool-related: In the conclusion, the paper claims that their findings suggest that adoption of ePTM systems has failed for 3 kinds of reasons -- environmental, personal, and tool-related. While easily understandable, this characterization is rather obvious (it is sort of like 'earth', 'fire', 'water'...) Since the _core_ findings of this paper concern uncovering the reasons, it would be nice if the paper provided several other characterizations of these reasons, in addition to this basic characterization. For example, a common theme cited by participants pertained to reasons of access and availability -- e.g., the accessibility of a tool when it was needed, while other reasons seemed to pertain to it's effectiveness at periodically priming one's memory (e.g., participant who wanted her task list to be visible at all times.). Is this an "environmental" or "tool-related" constraint -- doesn't seem to quite fit in either. Another finding seems to be that people just prefer simpler interfaces / avoid the want to avoid the complexity of having to learn or use baroque task-list specific interfaces. (Is that a personal preference?) A second comment pertains to the characterization of participants as "adopters", "DIYers" and "make-doers". Comment 2. On classifications 2: adopters vs DIYers vs make-doers
 Rather than being distinct categories, might it be more natural to reflect this on two axes? One (adopters vs make-doers), : the degree to which people adopt dedicated task list management tools vs use general purpose tools. The second dimension is "DIY-ness", or the degree to which people extend/appropriate these tools in various ways? Also there seems to be a distinction between DIYers who craft task list tools out of general ones (e.g,. spreadsheets and word processors) from those who overload the affordances of some other tool and bend it to its will- e.g., the participant who overloaded the fields of the Calendar entry. Comment 3. Conclusions and discussion --- 3a. - On "Explore with users' changing design needs" -- This implications for design prescriptively advises PIM practitioners to "support evolving needs" without really specifying exactly how such needs might evolve or why this is being prescribed. This section should be fortified with specific, solid examples witnessed during the study of how exactly PIM practices were perceived to have changed. The singular example given "early adopters" is not a good one, as this may or may not have been an issue of changing needs -- it may have also been an issue of having taken time to realize that a new tool didn't fit well in a person's workflow. This is particularly true for core PIM tools such as task list managers, which seem to go through a "trial assimilation process" where the individual tries out the tool in various ways to figure out how it best fits PIM practice. Thus I see no direct evidence provided in the study that supports this supposed need to support "evolving needs" (although I would believe it true in practice.) Thus it would be nice if the authors would back it up. 3c. "Utilise unique functionality" - I think this section almost hits the nail on the head - and could really be drawn into a separate section of the paper altogether. By "unique functionality" the authors really speaking of affordances that people used to accomplish particular aspects of their task-management activities. What are these affordances and the needs behind them? Clearly reminding is a need to which the example of 'starring' to mark something as un-read (so that it will be happened across later) is the affordance. Explicitly enumerating these affordances and latent needs as inferred from the study would greatly strengthen this conclusion Perhaps more importantly, it's clear that it may be several factors combined that are causing DIYers to avoid use of particular tools - what are these flaws (or "anti-affordances") and can they be rectified? A few are touched upon in the description of results, and should be summarized together. Finally, why the individual variation in task tools? It high tolerance for such flaws that makes adopters adopters and others not? Or is it that peoples' needs are genuiunely different so different that one set of affordances becomes another's flaws? Tolerance or adapting to the tool. Adoption seemed to be a combination of high tolerance Adopters seem to adapt themselves better to the way the tool re Expertise 4 (Expert ) Areas for Improvement All recommendations included in review above. Utilize unique functionalities of everyday tools in support of PTM. Some general tools are used for PTM because they support reminding especially well, and some other tools for their ease of task recording. For example, when a task is received by email, it is easy to star it or mark it ÒunreadÓ, and it is easy to leave a web page open to signify there is something left to do on that page, and it is easy to jot down a thought on a piece of paper at hand; these are all easier than explicitly recording a to-do in a separate structured task list. One way to utilize each toolÕs unique affordances with respect to recording tasks is to have a PTM tool that supports distributed task recording; it would ÒpullÓ in the tasks recorded in other tools into a central repository to provide an overview of all tasks. For make-doers, however, who showed little interest in adding yet another tool to their tool-set for managing their tasks, we propose a slight modification for tool integration. Instead of using an additional tool for pulling in distributed tasks, some general tools should allow tasks to be recorded and copied to a designated frequently accessed tool, such as email; this would allow make-doers to have an overview of all their tasks in one of their frequently accessed tools. We acknowledge that at least one existing PTM tool, Things, has already made some advances in this direction by providing support for recording tasks in a limited number of applications. Affordances: