Whale Karaoke
AI MUSIC GENERATION CHALLENGE
Team & Collaborators: Jesse Allison, Carlos G. Román, Treya Nash, Andrew Farrar, Morgan Weeks, Ka Hei Cheng
EcoSystemic Sound Lab – EMDM Program
School of Music & Center for Computation & Technology
Louisiana State University
Whale Karaoke – A Thought Experiment
Imagine a future where humans have developed advanced technology that allows them to communicate with whales. Through a breakthrough in understanding whale language and the development of sophisticated translation devices, humans can now engage in meaningful interactions with these intelligent marine creatures. Not only can humans communicate with whales, but they have also discovered that whales have a complex system of communication that involves not only vocalizations but also intricate patterns of sound signals. Scientists and musicians collaborate to create a universal language that combines human and whale communication elements.
As a result, humans and whales start engaging in collaborative music-making. Whales use their powerful vocalizations and click patterns, while humans contribute with vocalizations of their own. The resulting compositions become a unique fusion of terrestrial and marine elements, creating an entirely new genre of music that captivates audiences worldwide. This newfound communication and artistic collaboration lead to a deeper understanding between humans and whales. Cultural exchanges occur, and humans learn more about the rich history and wisdom of these oceanic beings. Conservation efforts gain momentum as humans develop a greater appreciation for the importance of preserving the oceans and the incredible diversity of life within them.
In this future scenario where humans communicate and make music with whales, a unique form of entertainment emerges: whale karaoke. Whale karaoke involves humans and whales taking turns producing sounds and vocalizations in a collaborative and improvisational manner. Humans might use specially designed underwater microphones and instruments, while whales contribute with their powerful and melodic vocalizations. The result is a harmonious exchange where both species showcase their musical talents.
Imagine a scenario where a group of humans sets up an underwater karaoke stage in a location frequented by whales. Participants, equipped with underwater headsets and communication devices, take turns singing alongside the mesmerizing sounds of whale songs. Whales respond with their own variations and embellishments, creating a unique and unforgettable musical experience.
The songs performed during whale karaoke could range from traditional human tunes to compositions specifically crafted to complement the natural rhythms and tones of whale vocalizations. The interaction becomes a form of cross-species jam session, where humans and whales learn to synchronize their musical expressions.
CONCEPTUAL APPROACH AND INSIGHTS
We want to break free from anthropocentric approaches to music, embracing the potential for a shared musical language that emerges from the interplay between terrestrial and marine life.
Beyond Human Auditory Experience
Nikolić[1] challenges the notion that music is an exclusively human auditory experience. In this human-whale communication setting, we must open our minds to the possibility that music, in its broader sense, could extend beyond the human species. Whales, with their intricate vocalizations, may possess a musical language that transcends traditional human understanding. Sperm whales [5], for instance, with their highly developed neuroanatomical features, cognitive abilities, intricate social structures, and unique click-based encoding communication, offer a compelling canvas for an extraordinary exploration in exo-musicology. The advanced neuroanatomy suggests a heightened capacity for sonic/musical perception, inviting the creation of compositions that resonate with the whales’ cognitive skills.
Ludwig Wittgenstein[2]‘s idea that language is tied to a form of life becomes particularly relevant. In imagining a shared musical language between humans and whales, we delve into a new form of life that includes both terrestrial and marine elements. The challenge is not just decoding whale sounds but understanding the cultural/environmental context in which these sounds exist for the whales, as they exhibit a remarkable capacity for cultural transmission as they learn to sing through cultural heritage. This unique aspect of their communication is exemplified by distinct groups of whales developing their own set of songs. Across the vast expanse of the ocean, these cetaceans carry their cultural heritage. It is suggested that the diversity of songs within whale populations may provide a mating advantage, as innovative and creative vocalizations could attract potential mates. The discrete click-based communication of sperm whales not only serves as a means of encoding information but introduces a novel sonic palette, offering a rich foundation for the development of a musical language that transcends traditional boundaries.
Following Chua & Rehding[3]‘s call for speculation rooted in scientific principles, we can speculate on a plausible launching point for human-whale communication. This involves not only understanding whale vocalizations but also contemplating the development of a musical language that incorporates both human and whale elements. The complex social structures of both species imply a potential for collaborative creativity, challenging the traditional human-centric approach to music-making and fostering a harmonious intersection of two distinct realms.
Human Listening Before Machine Listening
We live in the human age of recording, where every single sound is susceptible to be captured, documented, archived, reproduced, stored, and transformed. Whales remain oblivious to this, immune to the cultural shock that comes with this ability to capture sound, transform it and reproduce it indefinitely. Therefore, to embark on this musical exploration, we need a previous phase of Human Listening. This involves segmenting and annotating audio recordings of whale songs. Humans, equipped with technology and an understanding of musical structures, must identify patterns in time-frequency within whale vocalizations.
The examination of whale songs should not be limited to scientific analysis but must also involve a search for human/musical elements within them. Are there rhythmic patterns, melodic structures, or tonal variations in whale songs that resonate with human musical traditions? This bridge between two different musical worlds can form the basis for a shared musical language. The cultural memory embedded in whale songs intertwines with human interpretations, leading to a reciprocal exchange where humans comprehend and perform “whale music.” However, as Chua and Rehding [4] wisely warn, the inexorable differences between cetaceans and humans—whether physiological, cognitive, or environmental—make it impossible for one species to directly access the sonic/musical realm of the other. This is where our thought experiment comes into fruition.
Emergent Features and Interactions through Karaoke
As humans and whales engage in musical collaboration, new emergent features are likely to arise. These could include novel rhythmic patterns, harmonies, or even collaborative improvisations that result from the interaction between two ontologically distinct forms of life. This exploration goes beyond the habitual places of traditional musicology. Karaoke is introduced in the equation as a human phenomenon that encompasses a myriad of elements, reflecting a variety of cultural expression and identity through sound and performance. Central to this experience is the selection of songs, where individuals choose tunes that resonate with their personal preferences, cultural background, and emotional connection. Language becomes a crucial aspect, as participants may opt for songs in their native language or explore those from different cultures. The performance style further contributes to this cultural engagement, with each participant infusing their unique flair into an unique rendition. Beyond individual expression, karaoke serves as a powerful social bonding tool. The behavior and interaction observed during karaoke sessions emphasize the communal nature of this activity. Moreover, technology plays a pivotal role in mediating these musical moments, as karaoke systems serve as a transcultural activity, facilitating cultural diffusion and hybridization as participants explore and embrace diverse musical traditions. In this way, it becomes a reservoir of cultural memory, preserving and celebrating the global mosaic of music and performance.
[1] Nikolić, S. (2021). Five Claims for Global Musicology. Acta Musicologica 93(2), 219-235. https://www.muse.jhu.edu/article/839245
[2] Wittgenstein, L. (1953). Philosophical Investigations.
[3] Chua, D. & Rehding, A. (2021). Alien Listening. New York: Zone Books.
[4] id: p 177-180.
[5] Andreas et al. (2022). Toward understanding the communication in sperm whales, iScience,Volume 25, Issue 6. https://doi.org/10.1016/j.isci.2022.104393
TECHNICAL APPROACH AND EXPERIMENTAL DEVELOPMENT
Inspired by machine learning autoencoders, we propose a technical process aimed at bridging the gap between the radically different temporal and auditory dimensions of sound communication in each species.
The project entails a technical exploration of whale sounds to render them manageable and comprehensible for human performance. This involves a series of experiments on the temporal and timbral aspects of the whale sounds[7]. The first phase incorporates the segmentation of sounds, stretching and compressing of sound durations, elimination of silences, pitch shifting, and an in-depth timbral transfer exploration. The application of NMF-Based Concatenative synthesis, facilitated by the Flucoma toolbox for Max/MSP, further refines the temporal and timbral qualities of the whale sounds.
In the second phase, the project focuses on creating a participatory scenario for regular individuals to engage with whale sounds through a karaoke format. This involves a structured process, starting with the loading of sound files containing curated whale sounds. The files are then segmented and looped for optimal engagement. Encoding processes such as transposition, pitch compensation, and stretch factor adjustment are applied to enhance the adaptability of whale sounds for human imitation and performance. Participants engage in imitation, impression, and performance of the modified whale sounds. The reconstructed and decoded (whalified) performances are then played back or recorded, allowing for a unique and interactive experience with the transformed whale sounds within the karaoke framework.
Immersed in the enchanting ambiance of The Deep, a Halloween event held at the Atelier de la Nature[6] in Arnaudville, Louisiana, on October 28th, 2023, participants embarked on a journey that seamlessly blended experimental music, live electronic performance/improvisation, and interactive installations. Among the myriad of innovative experiences, the Whale Karaoke emerged as a unique highlight, transcending traditional boundaries of musical expression. Also, interactive installations invited attendees to actively engage with the immersive whale and underwater soundscape, further enhancing the participatory nature of the event. Whale Karaoke added a layer of interspecies collaboration, allowing humans to harmonize with the deep, melodic songs of whales.
[6] Atelier de la Nature Website
[7] Many of the sound samples employed were humpback whale songs recorded in the Colombian Pacific Coast, and were kindly provided by Andrea Bonilla, from the Center for Conservation Bioacoustics at Cornell University.
Disclaimer: This is an ongoing experimental project and a work in progress with no ultimate goal other than creating a platform for artistic creation and generating discussion and debate. Certain texts within this thought experiment were generated using ChatGPT 3.5, and fragments of these generated texts were utilized as prompts for the Image Creator by Designer, contributing to the creation of accompanying images illustrating the concept of whale karaoke. These AI tools served as catalysts for new insights, exploration, and discussions. It is important to note that while imaginative scenarios such as a submerged city of New Orleans, whales in the bayou, or laptops emitting bubbles might not be scientifically accurate depictions, these AI creations played a crucial role in hypothesizing different scenarios for our invented music tradition. Additionally, the sound databases employed, sourced from whale scientists or open access on the web, did not certainly have the acknowledgment or approval of the whales involved.