Implicit collaboration with a drawing machine
through dance movements

Mixed-Initiative Interaction (MII), is a concept first coined by Horvitz (Horvitz, 1999) and is a paradigm that assumes that interactions are both initiated by technological artifacts as well as users—none of these two are consistently leading or following the interaction. Mixed-initiative interactions have been studied initially in the context of user interfaces (Horvitz, 1999), and later also robotics (Jiang and Arkin, 2015).

Our research investigates what happens when mixed-initiative interaction gets brought to creative machines. Since the presence of initiative brings agency to a machine, such as a pen plotter, we can consider that machine to be a non-anthropomorphic robot. This makes it worthwhile to consider interaction with mixed-initiative creative machines a topic in the field of human-robot interaction.

In creative activity, the activity itself can be understood as a conversation between the maker and material (Schon, 1983). Initiative from the machine itself, then, adds a third party to the interaction—it becomes a shared interaction between a person, a machine, and the material as an ongoing reflection. When an interaction is collaborative, there is a need for communication between the interacting parties. Past human-robot interaction research has looked into the similarities between animation and robotics and the benefits of applying animation principles to the design of robotic interactions (Takayama et al., 2011), as well as ways to design for this type of interaction. Due to the collaborative nature between creative machines and people in a mixed-initiative setting, the person is not just an observer—we propose that the interaction becomes like a dance between two parties, acting on and with the material.

In this demonstration, we present the initial results of an ongoing body of exploratory work, investigating the potential for creative machines to communicate and collaborate with people through movement as a form of implicit interaction (Ju, 2015). Through this demonstration, we aim to gather perspectives from the computational fabrication community regarding how practitioners of fabrication with machines experience interacting with a mixed-initiative collaborative machine.

We present a Wizard-of-Oz demo, where a hidden wizard controls a modified AxiDraw drawing robot while a participant collaborates with it to draw a custom postcard. An AxiDraw pen plotter (Scientist, 2022), which is augmented with two motors adding two additional degrees of freedom, is placed on the table with two cameras aimed at the work surface and at the user. Figure 1 shows four examples of the resulting postcards on the robot’s right side.

The Wizard will improvise using a set of pre-developed possible movements—however, the interaction consists of two clear stages that are described in Table 1, categorized as “Welcoming”, and Table 2, categorized as “Collaborative Drawing”. Afterwards, participants can take their postcard home.

The demo will be set up to run on a table. We require electricity and WiFi access, permission to use live cameras, and a chair for demo participants. The wizard will connect to the robot remotely by controlling the robot’s computer via SSH. The wizard’s “eyes” will be two cameras – one of them will show the participant, and the second one will show the workspace of the robot.

This demonstration is intended to highlight the collaborative design method used in the design of the robot’s actions. One key element of our design approach is forming our interdisciplinary research team. The authors of this demo consist of two interaction design researchers, one mechanical engineer, and one dancer. The design process occurred during in-person meetings over the course of four months.

The authors would like to thank Cooper Murr, Tobias Weinberg, Avital Dell’Ariccia, Evil Mad Scientist, Antti Oulasvirta and François Guimbretière for their earlier suggestions, which fed into this work, and the Jacobs Technion-Cornell Institute for funding this work.