The Beatbots: A Musician-Informed Multi-Robot Percussion Quartet

Percussion music originated from instinct, with early percussionists relying on innate musicality. Over time, techniques were codified and passed down over generations [22, 23], with rhythmic repetition and rule-based phrasing forming a foundation for intricate patterns [24]. This enabled the creation of new, complex patterns derived from traditional phrases, much like certain procedural approaches to music composition [25], which range from medieval to contemporary works [26]. Additionally, contemporary percussion composers sometimes move away from strict rules, introducing elements of randomness to add further musical complexity [27, 28].

Steve Reich is known for repetitive rhythms and phasing [29], where phrases are played at different tempos to create desynchronized textures, as seen in Drumming and Clapping Music [18, 30]. Similarly, Philip Glass uses overlapping rhythms to build complex layers from simple patterns, as in his String Quartet 6 [31, 19]. John Cage, in works like his Composed Improvisations, pioneered rule-driven compositions [32] that use randomness, allowing performers to improvise within structured frameworks—such as having musicians roll dice to decide their next pattern [33].

Our work takes inspiration from contemporary percussion compositions by incorporating phasing, rhythmic layering, and randomness, implemented by our robotic music system. These techniques, pioneered by composers like Reich, Glass, and Cage, are particularly well-suited for our approach emphasizing unique robot strengths, as robots can easily execute random or complex patterns requiring difficult coordination, like phasing and overlapping rhythms, which would require extensive training for human musicians to perform accurately.

As robots become increasingly prevalent [34, 35], their applications in creative fields extend beyond automation [2, 3, 4, 8]. Musical robots have been developed for various instruments [36, 37], ranging from string [11, 38], to wind [39, 40], to percussion [41, 42].

Recent developments in robotic music tend to emphasize anthropomorphism, with systems mimicking human appearance or trained on human movements [8, 41, 10]. Examples include the Waseda Flutist Robot [40], Toyota’s violin-playing robot [11], and others [43, 44, 45]. Robot systems also leverage physical presence and visual cues—important elements to live performance [46]—which digital musicians cannot [6, 47].

While robotic music systems are often viewed as novel instruments [48], musicians emphasize that human control—either through programming or real-time interaction—is essential for classification as a new instrument [6]. Systems like Shimon [41, 9] or GuitarBot, in performance with violinist Mari Kimura [49], demonstrate real-time interaction through their ability to synchronize and adapt to human musicians.

Robots also possess capabilities that transcend human limitations, such as increased accuracy [50], improved ability to follow instructions while introducing unpredictable variations [6, 14], and the ability to perform feats of speed or scale that would require complex coordination of multiple humans [5, 36]. For example, Haile, a robot that plays a Native American drum, exceeds human speed [5], while other systems utilize three-dimensional space to achieve orchestral effects [36, 51]. However, these systems typically employ instrument actuation methods that mimic human movements, still aiming to produce music that sounds as human-like as possible.

Our work diverges from prior approaches by explicitly leveraging robots’ unique capabilities rather than emulating humans. Through the Beatbots, we explore new artistic possibilities enabled by our algorithmic percussion music and novel actuation method. While this approach may produce unconventional musical output, we believe it will lead to new insights toward distinctly robotic music systems and musical culture of the future [16]. Given this divergence, we prioritized input from human musicians [52] throughout our design process to ensure our system maintains artistic integrity while pushing the boundaries of traditional musicianship.

When developing the Beatbots, our primary objective was to center the values of its ideal users: musicians [53, 54]. Unlike technology-focused research that typically prioritizes system accuracy and efficiency [55], designing novel musical systems that prioritize user needs requires a deep understanding of what matters most to users [56], often diverging significantly from designers’ initial goals [55]. As robotic systems become increasingly sophisticated, incorporating user values into the design process becomes even more critically important [57].

We therefore involved musicians as key consultants in defining the robotic system through informed prototyping [21]. This approach enabled a nuanced integration of human creativity and musician values with machine potential, ensuring the system was grounded in users’ artistic and ethical values [58].

We sought feedback from three musicians during the design process: one classically trained percussionist (M1), one guitarist, composer, and instrument designer (M2), and one pianist formally trained in both classical and jazz piano (M3). All three have some level of experience with percussion playing, are practicing live performers, and compose music. Additionally, M2 earned a doctoral degree in a music domain and M3 earned a Bachelor’s degree from a music conservatory.

We gathered feedback from our musician consultants at various stages of the design process, with detailed results presented in the following sub-sections. Specifically, we sought feedback for robot choice, programmatic music composition, instrument selection, arena design, autonomous robot behaviors, and methods of human control over the robots. We collected this feedback by presenting the current prototype to the musicians in-person as a live performance, allowing them to share their thoughts verbally without any guiding questions. This approach helped identify what aspects were most important to the musicians without external influence. After hearing initial insights, researchers asked follow-up questions to clarify and ensure a thorough understanding, enabling us to effectively and accurately integrate their feedback into the system.

An early design choice by the research team was to make the robots identical in appearance and sensors, creating a cohesive look and enabling future scalability. Additionally, rather than using conventional stationary actuators, like a robot arm, our musicians advocated for leveraging the robots’ ability for dynamic, whole-body movement, and instead use kinetic motion to strike instruments—a distinctly non-anthropomorphic approach. They specifically favored Sphero¹¹1https://sphero.com robots over more traditional vehicle-like designs, citing their novel, futuristic aesthetic. Using rolling balls to hit drums along the arena’s sides was also likely to be more visually engaging, especially for non-technical viewers. We chose the Sphero BOLTs²²2https://sphero.com/collections/all/products/sphero-bolt for their built-in sensors, including a gyroscope and motor encoder, and programmable display lights, which added visual interest and interactivity. We hypothesized that these features would enhance the experience for non-technical audiences.

Based on feedback from our musician consultants, we prioritized the robots’ capability to play authentic percussion rhythms. Similar to Cage’s Composed Improvisations [33] and other works illustrating the divisible nature of music [59], the system’s music was built from pre-written, rule-based percussive patterns selected randomly. These patterns were informed by interviews with our musician consultants, particularly M1, through which we gathered twenty-five percussive rhythms which the Beatbots could combine to create a rich and complex musical performance (example rhythms in Fig. 1).

With the help of our musician consultants, particularly M1, we categorized these patterns. Uneven groupings, such as long, short, short, shortest, long, shortest, short, were categorized as “uneven patterns”, while even groupings like long, long were “even patterns”. We also differentiated between quicker and slower patterns—“slower” featured mostly longer note values, while “quicker” contained more shorter note values.

Another key concept gleaned from our musicians was the distinction between single and double strokes. A single stroke occurs when one downward motion creates a single sound, while a double stroke uses the momentum of the first hit to bounce and produce a second sound. A single stroke is heard as a longer sound, whereas a double stroke is heard more as two shorter sounds. The percussion patterns our musicians suggested included both single and double strokes, requiring additional considerations in robot behavior.

Instrument choice was another area where we sought input from musicians, piloting several instruments with them before making a final decision. We tested pitched (chimes, kalimba) and unpitched instruments (frame drums, shakers, cymbals). While pitched instruments added melody, their shape often caused the robots to veer off-course after impact, and incorrect pitches were far more sonically disruptive than mistimed unpitched sounds. Ultimately, we chose unpitched instruments, finding frame drums were the best-sounding option and could also easily function as walls for the four-sided robot arena.

We pilot-tested several drum sizes and found the ten-inch drums produced the best sound when struck by the robots. Frame drums also offered the advantage of producing two distinct tones: a deeper “bass” tone when hit in the center and a sharper “slap” tone when struck near the edge. By positioning the drums accordingly, both sounds could be played despite the robots’ small size. The original plan was to make all four arena walls out of ten-inch frame drums. However, the robots often accidentally hit drums that were not their direct target while moving toward their “goal” drum, making it difficult for audiences to distinguish between what we dubbed “purposeful” and “accidental” sounds.

Our musicians noted that percussionists in traditional quartets often play multiple instruments, inspiring our solution: we introduced ten-inch tambourines to the arena, positioning identical instruments across from each other. The tambourine’s metal jingles added a distinct sound, differentiating “accidental” hits on tambourines from “purposeful” ones on frame drums. This also enabled robots to switch instruments mid-performance, adding a dynamic element to the composition.

The robots’ enclosure, or “arena,” played a key role in shaping the rhythms the robots could produce with kinematic motion and whether single or double strokes could be played.

From the start, instruments were positioned along arena walls to optimize acoustics, as drums placed perpendicular to the robots produced the loudest sound. Each robot started the performance in the center of its own arena, with instrument placement and arena size refined through pilot testing. After testing sizes up to two feet by two feet, we settled on ten inch by ten inch arenas, with each wall made near-entirely of a frame drum or tambourine, supported by small wooden bars.

This compact design allowed for flexible arrangements, with easily movable arenas to accommodate different spaces. For instance, arenas could be arranged as shown in Fig. 2 for a clear view of all robots at once or positioned to surround the audience for an immersive experience. Our musicians suggested using distinct colors to help audiences identify individual “performers”. We therefore assigned unique colors to each robot-arena pair, helping viewers recognize each distinct unit in the quartet despite a uniform physical appearance.

We implemented two distinct roles within our quartet, designating one robot as “leader” and the others as “followers.” This structure is common in group music settings, such as with lead singers, conductors, and concertmasters [23], and it guided the selection of rhythmic patterns. Each robot, at every step of the performance, had a list of potential patterns that it randomly selected its next pattern from. This list was based on each robots’ previous instruction and—if it was a follower—the leader’s current instruction. This approach mirrors contemporary techniques, like in Cage’s Composed Improvisations [33], where the score is actually a set of instructions. These leader-follower rules were developed in collaboration with musician consultants. Fig. 3 contains a diagram depicting how the robots chose their next pattern.

To ensure the robots played basic rhythms as accurately as possible, we implemented a global “metronome-clock” synchronized to real-world time. This clock maintained a tempo of 60 beats per minute (BPM), guiding the robots when playing their selected pattern. We found that, despite the slow 60 BPM tempo, the robots struggled to play rhythms accurately due to their imprecise movement control. Even with periodic re-synchronization using the global “metronome-clock,” the robots quickly fell out of sync. For example, even small deviations in timing when aiming to spend 0.25 seconds on a note became quite noticeable within one or two patterns.

While these instabilities initially sounded jarring, they created interesting patterns when played in a group—reminiscent of the phasing seen in Reich’s Clapping Music [29, 18] and the sound layering of Glass’ String Quartet 6 [19]. When presented with the robot music, our musicians encouraged us to embrace these sonic complexities as the timing inconsistencies introduced a unique desynchronous element—one that would be difficult for humans to consistently replicate, which was our design goal. Unlike humans, robots could add or subtract truly random intervals between notes, even doing so while playing in a group, without being influenced by other performers.

Through experimentation, we also found that the robots’ spherical shape helped produce both single and double strokes. A single stroke occurred when the robot maintained forward motion after hitting the drum, whereas for a double stroke, cutting motor power immediately after impact caused the robot to bounce back then forward again for a second, softer hit. While unintended double strokes sometimes occurred (for instance, if power was cut slightly too early), this unexpected behavior further added sonic complexity, which our musician consultants saw as a positive contribution to the composition.

Finally, to address situations where the robots failed to detect collisions after hitting the walls, we implemented a fail-safe. If no collision was detected after four seconds—the length of one pattern—the robot would automatically turn around and begin their next pattern. This sometimes led to unintended silences, but our musicians felt that having moments of rest added a natural musical element.

Our musician consultants emphasized the need for incorporating human interaction into the robotic system’s behavior. This would not only enhance audience engagement but also help musicians feel actively involved in making the music. They suggested that interacting with the system felt like programming a synthesizer or creating a DJ set—while robots physically made the music, musicians saw the system as a new method of music creation under their artistic control.

The initial design allowed users to press buttons at different points in the performance to control the robots’ movements, preserving the generative composition while offering some user control. Pilot testing revealed that using a digital piano keyboard for this interaction made participants feel more musically engaged, transforming the interaction from a distant performance to a more personal musical experience.

Implemented human interactions fell into two categories: changing the robots’ lights and their movements. By default, the robots’ display lights shifted to a new hue of their assigned color every second. However, when different piano keys were pressed, the robots synchronously changed their lights to various new colors, allowing users to control an important visual element of the performance. Pilot testing showed visual control was particularly influential for non-musician users.

Pressing different piano keys also allowed users to control various aspects of robot movement. Users could make the robots spin in place, move in circles around their arena, switch primary instruments—changing between frame drum and tambourine by turning 90 degrees—or return to the center of the arena. Users could also stop and restart the robots’ default behavior at any time. A sequence of stopping, re-centering, and restarting helped manage desynchronization and any other minor differences in the global “metronome-clock,” allowing users to adjust the performance’s musical complexity.

$40$ community members attended with $28$ completing our survey. The performance was advertised via university mailing lists and bulletin boards. All $28$ survey respondents were students of the university where the performance was located. Each participant was asked to rate their familiarity with percussion music and robots on a five-point Likert scale, with average familiarity with percussion music being $2.14$ ($SD=1.18$) and familiarity with robots being $1.89$ ($SD=1.13$).

Participants were invited to first watch the Beatbots perform live, with the option to control parts of the performance afterwards. Almost all participants ($N=24$) chose to try controlling parts of the robot musical performance.

Researchers at the performance took detailed observational notes, and survey responses were collected at the scene.

In our survey analysis, we conducted inductive analysis [60] of participants’ free responses to generate a list of relevant themes. Two researchers independently reviewed responses, inductively identified, and finalized themes. Then, two independent researchers coded survey responses with the themes, and we calculated Cohen’s Kappa to measure inter-rater reliability, achieving a “substantial” agreement score of $\kappa=0.76$ (“substantial” agreement referring to $0.61<\kappa<0.80$) [61].

Both the performance and interactive aspects of the Beatbots resulted in an engaging experience characterized by what several participants described as “playfulness” in robot interaction, control, and autonomous movement. P23 remarked, “I found it to be childlike… I enjoyed the aspect of play.”

The performance evoked strong positive responses, many of which cited robots’ rolling movement as a reason for enjoyment. P6 wrote “I enjoyed seeing them roll around,” while P16 appreciated “that [the robots] roll around in little tambourine cages.” The coordinated movement between robots especially impressed viewers, with P11 noting they “liked that each of the robots worked together to create a cohesive piece.”

Most participants did not immediately approach the interactive opportunities. However, after researchers mentioned the possibility for interaction, nearly all respondents $(N=24)$ tested the human-robot interactions. Of those, the vast majority reported positive experiences, with only two expressing reservations, stating that control made the experience feel “less magical” (P24) or “more predictable” (P23).

The piano keyboard interface for controlling the robots proved particularly engaging. P2 wrote “I loved the interactions I had with robot by using the keyboard,” while P3 remarked that “it was interesting seeing how the piano was able to control the pattern.” During the interactions, several participants also remarked that using the keyboard made them feel like fellow musicians within the ensemble. P21 expressed the playful nature of controlling the ensemble, writing “It made me feel like the director of my own symphony!”

Physical, non-controlling interactions also emerged as a key element of engagement. Many participants valued being able to pick up and move the robots, which seemed to help them connect with the robots as entities rather than just machines. P13 captured this sentiment, saying it was “fun to interact with the little robots, really brought joy.” P1 echoed this: “It was fun to play around with how the robots acted.”

The visual design elements of the Beatbots helped craft a distinct aesthetic experience that participants found appealing. The combination of lights, movement, and physical design contributed to both the robots’ perceived personality and the overall atmosphere of the performance.

The robots’ lights were particularly compelling, featured in several responses. P1 noted how the colored lights contributed to giving the robots personality, observing that they “seemed to have some personality when they spinned around and changed color.” P21 also expressed that they “enjoyed the colors and rhythms. It gives rhythm and blues a whole new meaning!”

Participants also responded well to the overall charming aesthetic character of the spherical robots. Multiple participants used endearing descriptors, with P4 calling them “so cute” and P8 finding them “so fun.” This aesthetic encouraged engagement, evidenced by several participant comments, such as P5 noting they “like the funny noises and robots.”

The combined aesthetic effects of the robot system created a calming atmosphere. P14 characterized the experience as “very zen,” while P25 appreciated it as “an engaging sensory experience.” The careful integration of visuals and other aspects of physical form with the generated music seemed to transform what could have been a purely rhythmic performance into a comprehensive multi-sensory performance.

Participants’ engagement with the musical aspects centered around two distinct elements: enjoyment of the percussion music itself and the complexity of the generated rhythms.

Those who praised the music output often described it as unconventional but still appealing. P14 observed that “though the sounds sound a bit off tempo, it sounds very calming,” suggesting an appreciation for the atypical rhythms. P27 expressed surprise at the musicality of the system, noting that “it was cool that it actually produced a tune.” These responses indicate that participants found value in the robots’ musical capabilities, even when—and sometimes because—the output deviated from traditional percussion performance.

The rhythmic complexity also surfaced as an engaging element separate from music enjoyment. When asked whether humans could replicate the performance, participants’ responses revealed a fascination with its technical and rhythmic complexity. P26 noted this, suggesting “All rhythms are unique and there’s probably some 127-th tuplet that would be almost impossible to recreate,” referring to an exceptionally precise musical note value. P22 also pointed to the irregular timing, stating that “the rhythms seem a little too erratic/unstable/stop-and-go for a normal human.” Yet rather than viewing these characteristics negatively, participants appreciated them as unique features of robotic musicianship, comparing the music to “rave” or “experimental” music that moves in unexpected but still engaging ways.

A particularly striking finding was the contrast between expert and non-expert responses to the robotic performance. While nearly all survey respondents ($N=27$) reported a favorable opinion—with one respondent reporting a neutral opinion, who had a previous aversion to percussion music—their reasoning varied based on expertise level.

The robot music sharply divided participants along the lines of musical expertise—experts tended to not enjoy the music while non-experts consistently expressed positive reactions. Among our participants, non-experts ($18$ participants who rated themselves $1$ or $2$ on a five-point Likert scale) provided all positive reviews that specifically mentioned music enjoyment in the free responses, indicating a strong enjoyment of the music despite not having the technical background to understand the rhythmic patterns behind the music.

In contrast, percussion experts ($4$ participants who self-rated as a $4$ or $5$, and whom researchers identified as music students) followed a different trajectory. Researchers who spoke with expert musicians before the performance began noted that expert musicians were far more negative about their hopes for the experience than non-experts. However, by the end of the performance, all four expert musicians who completed the survey indicated positive reactions to the complete audio-visual experience. Though no music experts reported enjoyment of the music, they developed appreciation through non-musical aspects. Their music expertise also led them to recognize technical complexities that non-experts missed—for instance, all experts thought the performance would be impossible for humans to replicate exactly, while all five participants who strongly believed it could be reproduced were non-experts.

Robotics expertise also influenced engagement with the system, albeit differently compared to musical expertise. Among our participants, we had 4 robotics experts (rated $1$ or $2$ on the Likert scale) and 20 non-experts (rated $4$ or $5$ on the Likert scale) based on self-reported expertise.

Robotics experts focused more on the control and coordination aspects, with all experts highlighting their experience controlling the robots in the survey—usually expressing a desire for more granular control. This reflects a deeper technical understanding of robots’ movement capabilities. Additionally, researchers at the performance who spoke with expert and non-expert roboticists noted that roboticists actually had higher expectations for what the robots would be able to do, compared to non-experts who were initially more skeptical.

Despite this initial skepticism, non-expert roboticists showed more eventual enthusiasm for the robots’ movements and physical presence in their responses. Notably, all participants who explicitly enjoyed the robot movement were non-experts, indicating a strong mindset shift regarding robot movements. They also frequently compared the robotic sounds to familiar musical genres, potentially indicating a focus on the overall experience rather than its technical aspects. This focus may also explain why the keyboard interface proved particularly effective at engaging non-experts, as by allowing them to directly control and interact with the robots, the interface provided a hands-on way for non-experts to connect with the performance and appreciate the robots’ capabilities, without needing to understand underlying technical complexities.

Regarding R2, the differences in how music and robotics experts versus non-experts experienced the robotic performance suggest that expertise significantly shapes user perceptions.

These differences highlight the complex role of expertise in shaping audience perceptions of robot music. Initial skepticism of musical experts, stemming from attunement to precision and conventional rhythms, contrasts with the openness of non-experts to unconventional music. Similarly, the expectation of robotics experts regarding technical aspects, like granularity of control, differed from non-experts’ initial skepticism and eventual captivation with the “magical” robot movements.

Our findings reveal how different aspects of the performance shifted different expert mindsets: hands-on interaction transformed robotics non-experts from skeptical to enthusiastic, while multi-sensory elements won over music experts despite their continued skepticism about the musical output. This demonstrates that expertise does not determine audience responses in a uniform way, with various performance elements creating unique pathways to appreciation. Ultimately, the fact that both experts and non-experts found value in the Beatbots performance, even if through different routes, highlights the inclusive potential of robotic musicianship. This suggests that by centering the values of both experts and non-experts, designers can create experiences that push artistic boundaries while still fostering broad engagement and appreciation.

During our design process, our musicians provided valuable insight into what types of capabilities are most important to musicians for inclusion in future robot music systems. To address R1, we provide a list of the musical robot affordances in our system inspired by our musicians’ feedback.

Addressing R3, we offer five design principles for future design work of robotic percussion musical systems based on participant values in all (non-interactive and interactive) elements of the robotic performance.

Because we recruited via open call, most participants were likely already interested in robotic music. Additionally, most participants were laypeople in both music and robotics, and even “experts” were university students studying those fields. Future research should aim to recruit participants with more diverse opinions and more expertise in these two fields.

Additionally, our one-day study could not account for the potential novelty effect, and some survey questions were phrased in a way that may have incurred positively biased responses. Future work should investigate the longer-term impact of robotic music and use more neutral question framing.

To improve the system, future research should focus on experimenting with different robots or different control methods for more granular control, as suggested by participants. Future investigations could also modify the arena and incorporate a wider range of instruments, including unpitched (e.g., shakers, cymbals) and pitched percussion instruments (e.g., xylophones) to introduce new melodic and rhythmic elements.

Lastly, we propose continued discussion and collaboration with musicians for iterative refinement. This extended partnership could lead to more sophisticated performances that better align with musician values and expectations.