An overview of techniques for biomarker discovery in voice signal

Based on their effects on human voice, medical conditions that are known to affect humans can be divided into four clear categories. Of these, one category of diseases comprises those that have absolutely no effect on voice, such as certain dermatological conditions, hair-related conditions etc. In contrast to this, the category of conditions that is expected to have the most obvious effects on voice includes diseases that directly affect the structures of the vocal tract – vocal folds, larynx, glottis, respiratory tract, articulators etc. Examples of such diseases are otolayngological diseases of various etiology. A third category is that of diseases that indirectly affect the processes that drive voice production – including cognitive, neuromuscular, biomechanical and auditory feedback processes. These conditions cause varied effects on voice, ranging from fairly intense and obvious effects, to very subtle or almost imperceptile ones. This category includes diseases such as those listed in Table 1, and syndromes caused by secondary effects of drugs, intoxicants and other harmful substances. The fourth category – and the focus of the mechanisms presented in this paper – comprises diseases for which the existence of voice changes is hypothesized, but these may not be evident through standard analytical examinations of the voice signal. These include disease subcategories that affect intellectual abilities, visual function, temperament, personality, behavior etc. For these diseases, biomarkers in voice may be hypothesized to be present, but are elusive and must be searched for – in effect designed or created – for use in data-driven applications that attempt to detect the presence of these diseases from voice.

The changes in voice that are alluded to above are in fact biomarker patterns, or biomarkers. The term biomarker in this context refers to specific patterns of change(s) in the voice signal in the voice signal, that carry information about the health conditions that cause them. Such changes may be thought of as perturbations or deviations from a hypothetical “normal” voice signal, within its frequency, duration, intensity, amplitude and other entities that characterize it. The changes may be wide-raging and coarse in nature, or temporally transient, occurring within micro-durations of the signal. In other words, biomarker patterns may range from being highly perceptible, to completely imperceptible – to the extent that they may be undetectable even within standard analytical representations of the speech signal.

The rest of this paper introduces three categories of techniques that can potentially uncover such elusive biomarkers. Such techniques can be thought of as biomarker discovery or feature engineering techniques focused on the design or re-design of biomarker features in voice. The approaches described below include proxy techniques, model-based analytical techniques and data-driven AI techniques.

  “© 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.”

Proxy techniques may be used for the changes in voice that are human observable, but not easily measurable. For example, many of the correlations established between various diseases and voice changes in the medical literature refer to changes in voice quality. The changes, although subjective, comprise biomarkers that could potentially be used in machine learning systems for prediction of the corresponding diseases from voice. However, the problem is that for the most part, the entities that constitute the set of voice qualities are subjectively specified. For many, methodologies for objective measurement do not exist.

For example, the voice sub-qualities of nasality (or nasalence), roughness, breathiness, asthenia etc. have no objective measures associated with them. They are in fact rated by human experts on standardized clinical rating scales, such as Voice Rating scale (VRS) [32], Voice Disability Coping Questionnaire (VDCQ) [33], Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) [34] etc. Examples of such subjective correlations of various diseases to voice changes are listed in Table 1.

With the advancements in the voice profiling techniques, it has become evident that for a large set of diseases for which correlations to voice were not known to exist, the presence of such correlations can in fact be hypothesized and scientifically supported. Since for these, we clearly know that biomarkers are not perceptible or evident within standard analytical representations of the voice signal, we must devise techniques to discover them or create them in appropriate mathematical spaces within which they can be shaped and measured.

One example of such a biomarker discovery system/framework is illustrated in Fig. 2. This represents a generic setup which we call the ABCDE framework (Autoencoder based Biomarker Creation and Discovery Engine). Its exact formulation can vary from application to application. In this framework, a speech signal is first converted into a numerical representation that is hypothesized to contain the biomarker related information that we seek to uncover, or extract. The representation is then “projected” into a neural kernel space, where we can impose objective criteria on it that are tailored to the specific biomarker.

Viewing Fig.2 from the left, from a given voice signal S, several types of signal representations are obtained using digital signal processing algorithms (DSP). Thereafter, one of those (SPi) is chosen as the substrate for the target feature to be created (carrying the relevant biomarker). The other representations (SP1, SP2,…) serve as “control” representations, and are placed within the DSP stack shown in the figure 2.

The cuboid for SPi represents a stack of the same “type” of DSP features, e.g. a stack of spectrograms obtained at different time-frequency resolutions, or a correlogram. These are input into a neural network E (e.g. a convolutional neural network), which can be viewed as an “encoder” that transforms them into a kernel space, yielding a latent feature representation Z. Within this space, different constraints (including those based on prior knowledge) can be imposed on Z, so that it has the desired properties of the biomarker for the targeted health condition. One obvious property it must possess is that it must be discriminative for the underlying medical condition it is expected to encode.

To ensure that the information present in the input representation preserved in the process of transformation into the kernel space, a decoder D is trained to reconstruct the original representation from Z, while minimizing the loss between the reconstructed and original representation. All loss functions are represented by red double-lines in Fig. 2. In the training process, which is that of parameter optimization of the aggregate neural framework through gradient descent, such a loss function would be minimized.

To ensure that only the information in the original input representation is engineered for the feature at hand, the same latent representation Z is input into a generator G that can recreate the original speech signal from it (as G(Z)). Alternatively, the G can act on the output of the decoder, SPo (as G(SPo)) to generate the voice signal. Time-domain signals with same information content can be different in voice acoustics and thus, more robust losses are defined based on comparisons of DSP features from the original signal, to those predicted from the voice signal generated by G. Alternatively, the DSP features can be “simulated” by a neural stack, as shown in the figure. An additional level of detail that allows voice quality features to play a role in this process of discovery is introduced by specifying rough functional relationships between DSP features and various voice qualities V. Losses based on direct comparisons between these can also play a part in the learning process.

The entire framework is simultaneously optimized, but it is conceivable to express this process of discovery within more sophisticated AI learning frameworks that prioritize interpretability and can be trained in parts. Here, Z represents the “discovered” feature carrying the biomarker whose existence was hypothesized.

The biomarker discovery mechanisms given above are generic ones, and may have varied formulations in different settings. In contrast to traditional features derived from audio signals for use with machine learning algorithms, these features are likely to perform better with lesser training data, since they are designed to be relatively more discriminating and less ambiguous for the specific disease for which they are designed. Such features are useful in many ways. For example, they can be used in applications that serve as diagnostic aids in clinical settings, to build tools for early-detection of certain diseases, for self-monitoring of health by disabled, elderly and under-resourced people, etc.

This material is based upon work supported by the Defence Science and Technology Agency, Singapore under contract number A025959. Its content does not reflect the position or policy of DSTA and no official endorsement should be inferred.