Activity River: Visualizing Planned and Logged Personal Activities for Reflection

Traditional “self-tracking tools” (like diaries, calendars, and to-do lists) contain rich contextual information about people’s daily experiences which provide opportunities for deeper self-reflection. Keeping a diary, beyond the artefact itself, is a psychological process that involves putting together what we remember, what we perceive, and what we anticipate for the future. The diary author reflects on their daily activities, considers how these fit with their self-identity, and plans activities to address unmet goals. Over time, the collected writings represent an evolving account of their self-identity. Diaries can help expedite the process of constructing what McAdams describes as the “narrative of the self” or “a special kind of story that each of us naturally constructs to bring together the different parts of ourselves and lives into a purposeful and convincing whole.” (McAdams, 1993, p. 12)

While this implies that reflection is a long-term process, the model of lived informatics suggests that reflection also occurs alongside data tracking, incorporating both short and long-term goals (Rooksby et al., 2014; Epstein et al., 2015). This can be observed in the practice of keeping calendars and to-do lists, which involves recording data that can inform individuals about their day-to-day activities (Haraty et al., 2012; Kamsin et al., 2012). In a study of calendar use, Payne found that calendars support prospective memory—remembering future events (Payne, 1993). Payne proposed that calendars engage people’s prospective memory by first helping them to set the intention to perform an action in the future, then helping them to recall the intention, and finally, in so doing, helping them to plan how to realize the action. He also found that a small majority of participants in his study also used calendars as an archive for report generation. Archived reports support retrospective memory, or recalling past events.

To support reflection, Li et al. suggested that designers must account for the two phases of reflection: discovery—where individuals learn about their behavioural influences, and maintenance—where individuals keep up with their goals. These phases rely on informing individuals of their status, history, goals, discrepancies, context, and factors affecting their behaviours (Li et al., 2011). Current PI tools provide individuals with some information such as status and history. However, most do not provide the contextual information that is necessary for people to reflect and understand their behaviours (Choe et al., 2017). For example, Fitbit’s dashboard provides a timeline where individuals can see their activity levels, but it does not provide context on why certain hours have more activity than others which is important if an individual wants to know how to adjust their activity.

Other tools like Toggl (Toggl, 2020) are designed to support more manual activity logging. These tools enable individuals to track specific tasks and the time they spend on them. Through this contextual logging, these tools provide reports that individuals can use to examine their time use and productivity. However, such tools require diligent manual data-logging and do not integrate well with planning tools such as calendars.

Design challenge: Self-reflection tools should present historical data with an appropriate level of context.

Researching how visualizations can help individuals in their everyday lives has received growing attention from the visualization community, prompting the research areas of Casual Infovis (Pousman et al., 2007) and Personal Visualization (Huang et al., 2015). One of the goals of personal visualizations is to help individuals reflect on personal data. For example, Huang et al. suggested the use of on-calendar visualizations to contextualize physical activity data (Huang et al., 2016). Their tool’s main view is a calendar superimposed with an area graph representing a person’s activity level. Enabling data exploration with this visualization helped people gain a clearer understanding about the context of their activities. Thus, they were able to reflect and give reasons as to why their activity levels would fluctuate. Choe et al. offer a preliminary explanation for how individuals could use such visual exploration for self-reflection (Choe et al., 2017). They found that through overviews and timeline representations, their participants were able to recall past behaviours by identifying trends and peaks in the visualization.

One factor still lacking in personal visualization research is support for goal setting and planning, where individuals assign concrete goals and plan how they can achieve them. Conway and Pleydell-Pierce suggested that knowledge from life-logged data can be used to find discrepancies between expectations and actual performance, which can then be used to determine personal goals, and in turn, create plans for achieving those goals (Conway and Pleydell-Pearce, 2000). In this regard, Baumer’s concept of breakdown, or violating one’s expectations, can also lead to reflection. In essence, tools can help individuals reflect on what behaviours they need to change by uncovering their inaccurate assumptions about themselves based on their activities in context with their plans.

Design Challenge: Self-reflection tools should facilitate comparison and show discrepancies between goals and achievements.

While visualizations like maps, bar charts, scatterplots, etc. can be adapted to support individual reflection, personal visualization researchers suggest considering factors such as aesthetics, playfulness, and pleasure. Huang et al. identified several works in PI and persuasive technologies which positively affected people’s behaviours through playful, non-standard visualizations (Huang et al., 2015). Such qualities may contribute to long-term use of a visualization tool.

Artistic abstractions shown in aesthetically appealing ways may be particularly appropriate for self-reflection. For example, Consolvo et al. used an artistic abstraction of flowers to represent activity and fitness levels (Consolvo et al., 2008). A variety of other work also suggests that aesthetically pleasing visualizations can be more engaging (Cawthon and Moere, 2007; Fan et al., 2012) and more memorable (Bateman et al., 2010). Furthermore, people make reliable first impressions of infographic visualizations based on aesthetics (Harrison et al., 2015). As with aesthetic computing (Fishwick, 2008), the delightfulness of a visualization can elicit an emotional response from individuals, and it can also enable social engagement through shareable artefacts (Thudt et al., 2015). While we scope our work to digital forms of data abstraction, other forms may also be appropriate for supporting reflection. For example, the process of creating data physicalizations (such as a piece of jewelry representing personal data) can also help individuals effectively self-reflect (Thudt et al., 2018).

Design Challenge: Self-reflection tools should provide delightful, pleasing, and engaging visualizations.

Early life-logging research focused primarily on providing tools for supplementing human memory (Kalnikaite et al., 2010). Consequently, these early tools were designed to automatically record data, creating an ongoing record of what has transpired in one’s life. A more recent approach considers active participation from individuals to complement or as an alternative to automatic logging altogether (Sas et al., 2015; Thiry et al., 2013). These authors emphasize the value of involving individuals in authoring their life-logs, enabling them to select meaningful events and contextualize the process, thereby enriching their records. Being able to identify personally meaningful events from a larger set of mundane ones has been shown to support positive acceptance of life-logging tools (Sas et al., 2015). In addition, Thiry et al.’s work suggested that individuals often wish to add more personalized accounts like events with their family when recording their timelines (Thiry et al., 2013).

Design Challenge: Self-reflection tools should provide ways for individuals to author their life-logs.

Individuals’ use of life-logging tools vary widely and have differences in how they manage their tasks and goals. Allowing individuals to author their personal goals in addition to life-logs can give them agency. In a study about personal task management, Haraty et al. proposed adding flexibility in task management tools to accommodate these differences (Haraty et al., 2012). This can allow individuals to set their own goals and achieve them as they see fit, which can help them adapt a new tool to their life-logging methods (Ayobi et al., 2018).

Design Challenge: Self-reflection tools should provide agency to individuals through flexible means of managing their goals.

Our approach to address DGcontext is to visualize data through a timeline which allows people to see their current status, and how this changes over time. We developed a visualization which we call a timeline stream as the main visual component of Activity River. Timeline stream is a modified version of a streamgraph, where flowing stacks represent multiple time series (Byron and Wattenberg, 2008). A timeline stream is composed of three components: activity waves, duration bins (vertical axis), and time slices (horizontal axis).

• Activity Waves. Each coloured band or activity wave shows times during the day of planned or logged activities. Multiple waves can be stacked, and clicking on a wave pulls it to the baseline to ease readability (Figure 2.d-e).

• Vertical Axis: Duration Bins. The vertical axis shows the duration (in minutes) that an individual spent on, or allotted for, a specific activity during a given hour. Bins above the horizon baseline show the amount of logged time spent on an activity. Bins below the baseline show the planned times for each activity.

• Horizontal Axis: Time Slices. The horizontal axis contains the hours of a single day, starting at midnight on the left and continuing right. However, unlike a linear timeline (Aigner et al., 2007) where the unmarked spaces in between time markers imply progression (e.g., there are minutes implied in between 1:00 and 2:00), the time markers in the horizontal axis of the timeline stream are discrete and do not signify progression. They are akin to the discrete time markers in ThemeRiver (Havre et al., 2000) where, for example, the space between a given year and the next does not imply continuity.

A timeline stream combines two distinct graphs (one for planned and another for logged activities), mirrored across the central horizontal baseline. This juxtaposition allows individuals to compare their data through quick assessments of symmetry. The more symmetrical their visualization appears, the closer they are to adhering to their plan.

We recruited 10 university students (six female and four male) who already life-log or use diaries, calendars, and/or other apps to plan and record their daily activities. All of them used calendars and to-do lists to a varying degree—seven had used calendars for more than a year, while the rest had used them for about a year. Only three of the participants were actively keeping diaries/journals at the time of the study, while two had kept diaries in the past. Most used such tools to organize their schedules and daily tasks.

For privacy purposes, we employed a role-playing approach where we asked our participants to perform tasks using Activity River while playing the role of an imaginary student. We designed structured scenarios to guide the role-playing exercises and to encourage realistic planning, logging, and reflective behaviours. During the study, an investigator narrated and facilitated the flow of the scenario, while the participant interacted with the tool and discussed their thoughts and actions using a think-aloud method.

This approach simulated the experience of using the tool for everyday planning and reflection. Role-playing allowed participants to engage in a wide variety of behaviours—including planning and altering schedules in response to changing events—which otherwise might not happen in a given time-span. Thus, our approach can be thought of as a discount method that can supplement or lay the groundwork for more involved longitudinal studies (Harrison et al., 2014; Simsarian, 2003). Moreover, when participants role-play a character that they can relate to, they may be more open to discuss situations that they would otherwise withhold for privacy reasons.

Participants completed the study individually, with each session running for about an hour. We asked them preliminary questions about their demographics, the tools they use to plan and/or log their daily activities, how they use those tools, and the types of activities they track. We then introduced them to Activity River and explained how to read the timeline stream visualization.

During the planning task at the beginning of the day, all participants planned for the activities that were on their character’s fixed schedule (including courses that they needed to attend). All participants also looked at their free times and started filling them with other activities that they thought were useful for achieving their character’s goals. They also used the timeline stream to identify plans they could change to get closer to their character’s goals.

As they went through the scenarios, all participants tried to adhere to their plans. They all used the visualization showing their planned activities as a reminder of what they still needed to do and when they planned on doing it. All participants agreed that seeing their plans visually helped them stay on track as their scenarios unfolded. Half of the participants claimed that they would have most likely overestimated the time it took to finish some activities if they were not able to see their plans.

Beyond using Activity River to simply make short-term decisions, participants used the visualization to uncover insights about their characters based on their timeline streams. All participants used the timeline streams to make decisions about what they should change in order to get closer to their character’s goals. We observed two specific instances when the visualization triggered seemingly intuitive insights in our participants. By seeing the comparison between their planned and actual activities, they made unprompted personal connections between their character’s data and themselves. For instance, after seeing her activities at the end of the scenario, Participant 10 said “[As this character,] I think I need to do more studying and working out because I [was not able to do any]. I need to invest on ‘me’ and prioritize myself. That’s the best investment for life.” Similarly, Participant 6 said “I think my plan [as this character] shouldn’t be too strict. I need to add flexible times in-between activities” after getting a disruptive life card that required him to alter his schedule extensively. This is reminiscent of non-rational reflection in which environmental cues trigger creative interpretations (Korthagen, 1993).

In comparison to their methods of activity planning and logging, six participants appreciated how Activity River visualized their entire day at-a-glance so they know exactly what had happened. Those who use calendars and to-do lists said that they would like to be able to use Activity River along with their tools because it would help them know whether they were able to complete the activities that they had planned to do. For example, Participant 6 mentioned that Activity River was more motivating than his calendar because it gave him insights on whether he was achieving his planned activities or not. Participant 4 said “I use Google Calendar, but that doesn’t really visualize your day. It also doesn’t tell you whether you actually did or did not perform an activity. I actually come back to my Google calendar right now and edit/delete activities that I didn’t get to.” On a similar note, Participant 3 appreciated the fact that she could see the amount of time she needed for an activity. She also suggested this might help her allocate the right amount of time and not over-plan, as she had done before with her agenda.

Letting individuals define the activities they wish to track was one goal for Activity River’s initial design (DGauthorship). While our participants defined the activities for their characters and not necessarily for themselves, we still asked them about their thoughts on self-defining activities. All participants stated that the ability to define their own activities is important. One common rationale participants gave was a need for more or less specific categories that better matched their activities. They also acknowledged that not everyone is the same and some individuals may wish to track completely different activities from what they have been given. For example, Participant 7 felt that activities such as “leisure” and “study” were too broad. Because he wanted to know exactly what activity he was spending his time on, he envisioned using more specific names. Furthermore, Participant 4 felt that arbitrary activities such as “study” did not imply the same meaning as “studying for an exam” or “studying for homework,” hinting that individuals do in fact add personal meaning to the names of their activities.

All the participants gave at least one or two activities a colour which they claimed best represented the activity. For example, Participant 5 said “I will choose this dark shade of blue to denote Sleep because it’s like night.” and Participant 6 said “Leisure will be this ‘lemony’ colour because that is the colour of [beer].” Some participants also used colour coding to group similar activities together. For example, Participant 1 said “School is blue because it makes me blue. Studying is [light blue] because it’s similar to School.” Later in the study, participants also used the overall colour tone of their streams to get a feel of whether they have accomplished their character’s goals. There were also isolated cases where our participants had prior associations of importance to colour and used that idea to rank the activities according to their perceived importance. For example, Participant 9 said “Study is red, because red is a very important colour. Work is orange, not as important as red, but still important.”

Activity River supports planning of activities at specific times. This prevents individuals from planning too many activities in a limited time span. However, it also pushes them to specify times for some activities for which looser specifications, such as those in a to-do list, might be more appropriate. These are activities that can be interrupted and/or have non-specific start and end times (and thus the time when participants do certain activities does not matter as long as they get to do them). These non-fixed activities are typically the ones that our participants looked for and shifted during our study when their schedules were interrupted. Participants were more concerned with the amount of time spent on such activities rather than when they happened. For example, Participant 4 planned for a total of 5.5 hours of studying, and while she did not study at the exact times she had planned, she still managed to study for the full 5.5 hours and a bit more (Figure 6).

These can be thought of as bankable activities, in which individuals first set an amount of time as their goal and then work towards that amount whenever they have time to do so. We consider bankable activities as a class of activity where identifying start times may be difficult or altogether inappropriate. For these activities, it is more important to show how much time has already been spent on the activity rather than when it happened. To improve our current design goal (DGflexible) and properly handle bankable activities, applications should allow individuals to plan some activities in list-form. However, to prevent over-planning, the application should also account for the duration these listed activities should take. The application could then suggest times for completing these bankable activities (for example, adding visual cues to the visualization which highlight free times). Tools could also keep ongoing counts of bankable activities and signal when the individual has met their planned duration. Nevertheless, the tool should give the individuals flexibility to plan/log bankable activities as there are times when setting a fixed time for this type of activity becomes necessary (such as when they plan to do it with another person).

Our participants rated the aesthetics of Activity River highly and were interested in personally using it for more than a week in order to see how their data would look. This suggests that the aesthetics of timeline streams, with its flowing symmetry and vibrant colours, sparked curiosity in our participants. While not necessarily an emotional investment, curiosity is a motivator which can initiate further exploration of the data (Arnone et al., 2011), and encourage continued use of a tool. While some participants needed detailed instructions to be able to read the visualization, sparking individuals’ curiosity to engage with their data likely encouraged more investment in that learning process (DGengagement). Future work that evaluates the effects of visual aesthetics in tools for reflection is needed to better understand this trade-off.

Activity River’s current manual logging of data is a burden that could deter individuals from using it long-term. However, many people already use apps such as Toggl (Toggl, 2020) and Optimized (Optimized-App, 2019), as well as diaries, to-do lists, and calendars—all of which involve manual logging of daily events at similar granularities. Although tedious, manual logging has the beneficial side-effect of providing an opportunity for people to be more involved with their data, and thus reflect on it more deeply.

The majority of our participants requested to integrate Activity River with tools that they already use. For instance, Participant 4 said she would like to connect it to her digital calendar because she wanted to see the timeline stream of her actual activities, but still wanted to plan using the calendar’s interface. Alternatively, linking with geo-tracking tools could also allow Activity River to detect an individual’s location and prompt them to record their activity. This semi-automated enhancement could help reduce the burden of manual logging (Li et al., 2010), but still keep the person in the loop to handle the very personal task of identifying and categorizing activities.

While evaluating tools like Activity River in-the-wild is an important future step, in-the-wild studies are time-consuming and expensive. Before undergoing such a venture, approaches like our role-playing study can be used to reveal important issues and opportunities, and better ensure tools are ready for deployment. Our role-playing approach highlights how a controlled, simulated setting can still provide a sense of realism and trigger frank discussions about familiar activities. This controlled approach also reduces the risk of violating participants’ privacy—which can complicate the design of real-world evaluations of personal informatics tools and may lower participants’ willingness to take part.

Our choice of familiar, yet simulated activities provided us with considerable insights, and participants indicated that they saw value in the tool for themselves. However, we acknowledge that we cannot assume whether individuals would use this tool over a longer period of time. Furthermore, in letting our participants choose the speed at which their scenarios advanced, they were able to always log their activities which may not be the case in real-life. If privacy concerns can be addressed, then we could explore how people use the tool for themselves with a field study. This could give us more insight into the utility of self-defined activities and how the tool works in real-life situations.

In this work, we reported on how timeline streams supported six visual planning patterns and how our participants used them to manage their goals. However, these visual explorations may also be relevant for many other types of planned and logged data. Future work could investigate how timeline streams’ comparative visuals could help people identify discrepancies between budgets and actual spending. For example, in examining discrepancies in software release planning budgets or in tracking medication adherence. Furthermore, we saw how our participants used Activity River to communicate and describe their thoughts (for instance, by pointing at interesting parts of the visualization). Thus, future studies could also investigate how timeline visualizations could help collaborators (managers and employees, coaches and athletes, etc.) communicate about shared planning tasks.