Expressive Timing in Hindustani Vocal Music

Bandish are notated lyrical compositions in the Hindustani music genre that serve as a reference for the associated raga in terms of the overall pitch movement and characteristic phrases. A bandish comprises two verses, typically with 2-4 lines each. Vocal concerts include the singing of a bandish in the chosen raga. Performers bring in different variations in the rendition of a given bandish and we also see variations in the same bandish lines across repetitions within the performance. The variations are evident to listeners, who are usually familiar with the prototypical form of the bandish. The associated auditory experience is greatly appreciated especially when executed by expert musicians.

In a work that addresses ‘musical expressiveness’ across cultures, Fabian et al. [1] refer to the effects caused by the variation of auditory parameters such as loudness, intensity, phrasing and tempo away from a prototypical performance but strictly within stylistic constraints. They make it clear that expressiveness does not refer to any features of the composition itself or the emotion that is expressed. The vocal rendition of a bandish in a Hindustani music concert can therefore be considered suitable for the investigation of the kind of variation that constitutes musical expressiveness.

We refer to the widely regarded book of V. N. Bhatkhande [2] who collected and notated a large number of traditional compositions from across the country in the early $20^{th}$ century. The Bhatkhande notated compositions serve as a convenient reference for a discussion of the measured variations of a given bandish line across performers. With schematic notation that depicts the melodic outline using the syllables of the lyrics, it is possible to relate the sung performance to the canonical version via the corresponding lyrics syllables. In this work, we use computational techniques to compare the performances of a given bandish by different artists. We present our methodology that includes automatic techniques for some of the audio feature extraction. We focus solely on timing expressiveness, leaving the study of pitch inflections and other dynamics that can also define expressiveness, to future work. In particular, we are interested in developing computational methods that capture the similarities and differences between different performances of the same bandish. We illustrate our approach with a small study of audio recordings by two expert vocalists of a well-known bandish in raga Yaman to draw insights about individual differences as well as the adherance to any genre-specific contraints.

The motivation for our work is to model the variations in acoustic parameters that can lead to the deeper understanding of music performance and the strategies used by accomplished musicians. This can potentially contribute to quantitative models for music generation from notation that are consistent with the intricacies of the specific genre and style.

Table  I describes our dataset comprising audio and metadata for several performances of bandish in raga Yaman, a popular raga which is taught early in music training and widely performed on the concert stage.

In terms of choice of bandish, we restrict ourselves to madhyalay and drut (medium and fast tempo) due to the relatively high complexity and freedom in vilambit (slow tempo) bandish. We select bandish where we have the canonical notation from Kramik Pustak Malika [2]. Next, for each bandish, we look for good quality audios recordings ranging from maestro concerts to simpler teaching websites to capture performance diversity, including also recordings from Samarpan by Pt. I. Nirody [3], following the methodology of Madhumitha [4].

The audio performance comprises the singing voice accompanied by the tabla supplying the relatively steady beat and defining the rhythmic grid for the chosen tala. A complete acoustic characterisation in terms of musically relevant parameters would need the onset instants of each of the tabla strokes and the sung syllables, and the notes (including pitch inflections or melodic ornamentation). Given the correspondence between the lyric syllables and the notes in the schematic notation of the composition, we expect the timing expressiveness to be manifested in the deviation of the sung syllables from the specified beat locations. The reliable detection of onsets of sung syllables apart from tabla stroke onsets is a component of the automated audio processing. It is necessary also to establish a temporal alignment of the note events between the performances to be compared. This is achieved using the matching of the lyrics or syllable sequences. In this section, we present our manually labeled dataset that facilitates the development of the needed text processing as well as audio processing for onsets.

Given that computational methods can help musicological analyses achieve scale, we investigate audio processing algorithms for the annotation discussed in the previous section. Automatic speech recognition models are of limited use on singing voice due to the significant differences in acoustic parameters, coupled with the compromised quality of vocals obtained with automatic source separation networks. Forced alignment of a singing voice with lyrics continues to be a topic of research and is especially challenging in non-English language settings such as ours [8]. We restrict ourselves to the automatic detection of syllable onsets for the present, followed by text based alignment of the canonical lyrics with the manually labeled audio syllables.

We now present a comparative study of two performances of the same bandish by two different singers in our dataset. One performance, as self-reported by the artist, Pt. Nirody (IN), is a close reproduction of the canonical notation. We assume that this influences the rhythmic aspect and we expect the note events to largely coincide with the beat positions indicated in the canonical notation. The second performance by a prominent artist, Ashwini Bhide-Deshpande (ABD), is typical of the concert setting and therefore strongly expected to display variations including expressive timing. We present a comparative analysis of the recordings via the Figures  3 and  4 where the temporal dynamics are visualized in different ways.

The downward trend of the points on the plot from the start of the cycle towards the cycle boundary indicates that the singer starts a bandish line a bit late (lagging) in a rather time-free manner, and compensates for this delay by compressing the following notes ensuring that a lyric segment - whether spanning a full cycle, half or even quarter, is not taken across cycle boundaries. This is essential for keeping up with the rhythm as well as providing a sense of resolution. Consistent with this strategy, the syllables corresponding to the later beats of a cycle, and the sam of the new cycle, tend to show far less temporal deviation. A cluster of points in the band near fractional deviation equal to one full matra, and very few instances within the 0 deviation band indicates a recurrent structural modification that the artist employs in rendering the composition. Generally, this kind of structural deviation is limited to one matra, and in fewer cases 2 matras.

A strong trend across performances in our dataset is for the singers to be mostly lagging as observed in Figures  2,  4, and still complete the line within the particular cycle or half-cycle by compressing the syllables together, i.e. shortening the gap between consecutive syllables, as well as using up the empty beats or note extensions (’s’ markings) according to the canonical notation. It may be remarked that this bears a striking similarity to tempo rubato in jazz [11].

Figure  5 collapses multiples rhythm cycles to achieve something like a ’fingerprint’ of artist’s expressive timing where the extent of timing flexibility at the beat level clearly depends on its location in the cycle. As expected from the previous discussion, a significant drop in the timing deviation is observed on the matra just preceding the sam and the khali. Further, the contrast between the two performances is clearly manifested in the timing deviation pattern, validating our choice of representation for timing expressiveness.

This study investigates how Hindustani vocalists strategically navigate the tension between adhering to the metric structure and injecting expressive timing variations by analyzing the alignment of syllable onsets with the rhythmic cycle. The results demonstrate how this approach can reveal distinctive patterns in the temporal phrasing of different artists, providing insights into the role of expressive timing in the aesthetics of this musical tradition. Through the audio analysis of performances of a popular composition, we have demonstrated how expert musicians take liberties with timing, particularly in regions that fall away from primary structural markers, such as the downbeat and the regions with low syllable density. In these sections, the performer may deviate from the strict tempo, introducing variations that contribute to the individuality and expressiveness of their rendition. We conclude that temporal dynamics contribute significantly towards the expressiveness of a rendition, and there is a manner to it.

Furthermore, our study introduces a systematic pipeline designed to measure certain temporal dynamics. With future improvements in the automatic audio processing pipeline including lyric alignment, large-scale analyses of such distinctive stylistic elements can become possible. It also offers valuable inputs for generative modelling. Through this framework, the model can learn to emulate the nuanced temporal flexibility found in traditional performances, thus enabling the generation of music that retains both the structure and expressive timing characteristic of a particular artist or genre. This research could pave the way for more authentic, style-specific generative AI models for music, capable of reproducing complex temporal dynamics that showcase the expressiveness and stylistic identity of traditional music renditions.