InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks

Abstract

Computational notebooks have become increasingly popular for exploratory data analysis due to their ability to support data exploration and explanation within a single document. Effective documentation for explaining chart findings during the exploration process is essential as it helps recall and share data analysis. However, documenting chart findings remains a challenge due to its time-consuming and tedious nature. While existing automatic methods alleviate some of the burden on users, they often fail to cater to users’ specific interests. In response to these limitations, we present \system, a mixed-initiative computational notebook plugin that generates finding documentation based on the user’s intent. \system allows users to express their intent in specific data subsets through sketching atop visualizations intuitively. To facilitate this, we designed two types of sketches, \ie open-path and closed-path sketch. Upon receiving a user’s sketch, \system identifies the sketch type and corresponding selected data items. Subsequently, it filters data fact types based on the sketch and selected data items before employing existing automatic data fact recommendation algorithms to infer data facts. Using large language models (GPT-3.5), \system converts data facts into effective natural language documentation. Users can conveniently fine-tune the generated documentation within \system. A user study with 12 participants demonstrated the usability and effectiveness of \system in expressing user intent and facilitating chart finding documentation.