1. Introduction
There has been rapid development in the fields of audio technology, information technology and social media since the turn of this century. The proliferation in information technology has become a catalyst for the cross-pollination of cultures and enabled the exchange and influence of ideas remotely across the globe (Cornelis, Moelants and Leman Reference Cornelis, Moelants and Leman2009). Especially affected by this cross-pollination is the field of music technology, or, more specifically, the usage of technology in conjunction with ethnic instruments. While some traditionalists in ethnic fields argue that integrating technology (or anything else that diverts from their original aesthetics) with their respective metiers dilutes their art forms, many more specialists in said fields continue to argue that conjunctive research encourages the preservation of their practices while embracing the emerging hybridised cultures (Kapur, Davidson, Cook, Schloss and Driessen Reference Kapur, Davidson, Cook, Schloss and Driessen2005).
This article aims to give an overview of current practices used to extend traditional Asian instruments that involve technology, focusing on recent examples. Section 2 briefly discusses the history and background of integrating technology with traditional Asian music. The following two sections are based upon a classification system of whether works are based on performance or music information retrieval (MIR)-based applications of technology. Performance-based refers to the use of technology that extends performances and compositions of traditional Asian instruments beyond their traditional repertoire, while MIR-based research creates tools and systems to understand traditional Asian music. Section 3 looks at the performance-based applications of technology and their contributions, which are organised into three subsections: electroacoustic, hyperinstruments and musical robotics. Section 4 introduces the applications of MIR and discusses musical classification, musical analysis and pedagogical applications. Finally, we conclude with discussion of obstacles faced by current researchers and speculate the future of this hybridised field.
2. Background
The confluence of technology and traditional Asian music is not a recent conjuncture in the field of music technology. Early developments in Asia began during the 1960s, while the concept had been adumbrated in the electronic works of twentieth-century Western music composers. Stockhausen's Hymnen (http://www.stockhausen.org) and Xenakis's Bohor (Xenakis Reference Xenakis1970; http://www.iannis-xenakis.com/xen) are notable examples of this. Audio technology was used to sample and manipulate recordings, which then developed into processing acoustic instruments and playing them with audio effect processors (Young and Fujinaga Reference Young and Fujinaga2004). Significant progress was not made again until the 1990s.
Despite the large combined population of the subject countries compared to the population of the countries using Western art music as a base, the vast majority of computational music and music technology research uses Western instruments as their medium (Oramas and Cornelis Reference Oramas and Cornelis2012). A representative survey of the major publications of journals and conference proceedings such as the International Computer Music Conference (ICMC), the International Society for Music Information Retrieval (ISMIR), Organised Sound, the Journal of New Music Research, Computer Music Journal and New Interfaces for Musical Expression (NIME) from the past ten years indicates that the percentage of research published on traditional Asian instruments and music is only a meagre portion of the aggregate seen in Figure 1 (UN 2012). The vast Asian population and incongruity of research in this hybridised field indicate that there are many unexplored territories to be investigated. Furthermore, these rich musical cultures are often difficult to preserve due to the nature of their oral tradition.
Figure 1 Comparison of research in technology and Asian instruments, and the world's Asian population.
Researching and utilising technology with traditional instruments can potentially contribute to cultivating interests in their origins. These research findings can also provide insights and alternative perspectives that will benefit the research of many fields.
3. Performance-based
One major classification of research pertaining to Asian instruments using technology is performance-based research. This is similar to the electroacoustic and computer-music fields that emerged in Western music in the mid-twentieth century. Its utilisation includes, but is not limited to, performance of traditional instruments with electronic sounds, audio signal processing of traditional instruments, building new controllers to extend traditional performance, and creating autonomous systems to play the traditional instruments. The following subsections will present the developments of traditional Asian instruments in the fields of electroacoustic music, hyperinstruments and musical robotics.
3.1. Electroacoustic
The initial efforts of electroacoustic music often comprised sampling Asian musical instruments, manipulating the recorded samples and using them with the instruments, theory or aesthetics of either Western or Asian music. Figure 2 shows works prior to the twenty-first century in the field of electroacoustic music that hybridises Asian and Western aesthetics and instruments in the Asia region by their native composers (B. Gluck Reference Gluck2009a, Reference Gluck2009b, Reference Gluck2009c; R. Gluck Reference Gluck2005, Reference Gluck2008). There have also been examples of non-local composers working in the metier, such as Randy Raine-Reusch. Raine-Reusch has been specialising in creating new and experimental works for ethnic instruments of many cultures since the 1980s, with primary focus on the Japanese ichigenkin and the Chinese zheng. While academic institutions played a crucial role in the seed and growth of this field of research in the general Asian regions, India became an outlier and began its integration through colloquial mediums (B Reference B2010).
Figure 2 Electroacoustic works prior to the 21st century by their native composers.
Other composers who have been working in the realm of electroacoustic composition include: Jin Hi Kim, Yoichi Nagashima (Nagashima Reference Nagashima1999), Alesandro Cipriani (Cipriani Reference Cipriani2002), Christopher J. Keyes (The Li Jiang Etudes 2004) (Keyes Reference Keyes2005), Luca Bonvini (Bonvini Reference Bonvini2009), David Rosenboom (Rosenboom Reference Rosenboom2010), Larry Polansky, Leah Barclay, Sandeep Bhagwati, Mei Han and Miya Masaoka.
Jin Hi Kim is the first kumongo performer to use the instrument in a modern context. She performed both solo and collaborative works utilising an electric komungo built by Joseph Yanuziello in conjunction with a live interactive Max/MSP system created by her and Alex Noyes (http://www.jinhikim.com). Her works preserve and augment the essence and aesthetics of the tradition.
Byungjun Kwon and Ensemble Sinawi's performance at the final concert of the International Conference of New Interface for Musical Expression (NIME) 2013, brought electronic sounds together with traditional Korean improvisational music that accompanies the rites of Korean shamanism, also known as Sinawi. The Future Sounds of Folk, a project developed by STEIM and the Unifiedfield (Marta Moreno Muñoz and Marc Chia), features two Indonesian folk musicians, Iman Rohman (Jimbot) and Wukir Suryadi (Bambu Wukir) (Chia Reference Chia2010). In their performance series with OneManNation, Jimbot uses traditional Sundanese instruments and explored contemporary techniques for musical expression (http://thefuturesoundsoffolk.com). Bambu Wukir uses his self-built bamboo plucked-string instrument and a loop pedal (Suryadi Reference Suryadi2009). Works in this field have also extended towards live performance with audio and visual interactions, as well as with other art forms such as calligraphy, dance, painting, poetry and ceremonies. The Tea Rockers Quintet is one such example, combining music with traditional Chinese tea ceremony. The quintet features Guqin master Wu Na, noise artist Yan Jun and tea master Lao Gu (http://www.tearockers.com).
These examples show the aesthetics and theory of traditional Asian music can co-exist with audio technology in complement with each other.
3.2. Hyperinstruments
Todd Machover coined the term ‘hyperinstrument’ when he used technology to expand the capabilities of acoustic instruments (Machover and Chung Reference Machover and Chung1989). Yoichi Yagashima is a pioneer in augmenting traditional Asian instruments with sensors. His works SHO (traditional Japanese bamboo mouth organ) (Nagashima and Ito Reference Nagashima and Ito1999) and Hyper Pipa (Chinese plucked string instrument, Figure 3) (Nagashima Reference Nagashima1999) utilises wireless technology, sensors and biosensors to control audio effects.
Figure 3 Hyper Pipa by Yoichi Nagashima.
Ajay Kapur is another pioneer who also augments traditional Asian instruments. The eTabla in Figure 4 was his first creation. The eTabla is a tabla, an Indian percussion instrument, fitted with sensors to capture performance gestures for human–computer interaction (Kapur, Essl, Davidson and Cook Reference Kapur, Essel, Davidson and Cook2002). In 2004, he presented the eSitar (Indian plucked string instrument) and eDholak (another type of Indian percussion instrument) (Kapur, Wang, Davidson, and Cook Reference Kapur, Wang, Davidson and Cook2005).
Figure 4 Kapur's eTabla.
Other instruments of different cultures that have been augmented include the Cyber Kendang (Sundanese percussion instrument) (Nagashima Reference Nagashima2007), e-Suling (Indonesian bamboo flute) (Erskine and Kapur Reference Erskine and Kapur2011), Hyper-shaku (Japanese flute) (Beilharz, Jakovich and Ferguson Reference Beilharz, Jakovich and Ferguson2006), eJanggu (Korean percussion instrument), and eHaegum (Korean bowed string instrument) (Kapur, Kim, Kapur and Eom Reference Kapur, Kim, Kapur and Eom2013).
In most cases of instrument augmentation, the actual instrument itself is modified when fitted with sensors. Some Asian instruments are regarded as prized objects, and their status as a representation of nature and spirituality often discourages the modification of the physical entity. In order to avoid modification of the instrument, another approach is to use sensors on the performer or their relative performance apparatuses (such as their bows and mallets). The AoBachi is a hyper-beater that is inspired by Japanese ‘Bachi’ drumsticks, where the sensors are fitted in the beaters instead of the Taiko drums themselves (Young and Fujinaga Reference Young and Fujinaga2004). There are other such entities that follow the same approach, such as the Hyperpuja, an augmented Tibetan Singing Bowl (Young and Essl Reference Young and Essl2003). The method these instruments follow allows the interface to not be limited to the instrument.
Similarly, wearable sensor systems are also explored to capture performance gesture without modification of the instruments (Benning, Kapur, Till, Tzanetakis and Driessen Reference Benning, Kapur, Till, Tzanetakis and Driessen2007; Tanaka and Knapp Reference Tanaka and Knapp2002). Miya Masaoka (http://www.miyamasaoka.com) has worked with Tom Zimmerman (1991) and STEIM (1994, 1996, 1998) on interfacing traditional koto performance gestures. Kontrol also falls into this category as sensors are fitted onto the hands of the performer to extract the performance gesture of a Guqin player, saxophonist and a dancer performing South Indian Dance (Christopher, He, Kapur and Kapur Reference Christopher, He, Kapur and Kapur2013).
Yet another approach to avoid modifying the instrument is to build a new instrument that is influenced by the original instrument. One such example is Echo Ho's Slow Qin, a reconstruction of the Chinese fretless plucked seven-stringed instrument (Ho 2007).
As seen in Figure 5, it is constructed like a Chinese Guqin, but uses transparent acrylic material instead of wood for its body and is fitted with sensors such as buttons, switches and knobs.
Figure 5 Echo Ho with Slow Qin.
The above examples show that traditional Asian instruments can be extended for human–computer interaction by using appropriate methods to extract performance gestures. The aesthetics and philosophies of cultural music traditions can be observed when the implementation of technology is negotiated strategically. Through integrating sensors, these instruments acquire additional capabilities that can be used in other applications such as controlling audio effects parameters for live electroacoustic performances.
3.3. Musical Robotics
Robotic musical instruments are automatic musical instruments that use motors, solenoids and other mechanisms to actuate levers to physically play musical instruments (Kapur Reference Kapur2005). Automatic musical instruments have been in existence since as early as 875 CE and have been confined to Western culture until recently (Murphy, Kapur and Carnegie Reference Murphy, Kapur and Carnegie2012): researchers only began incorporating musical robotics into traditional Asian music during the past ten years. However, even institutions in Asia that develop and research musical robotics (such as Waseda University) focus on Western instruments instead of traditional Asian instruments.
The first Asian robotic orchestra was the Gamelatron, created in 2008 by Aaron ‘Taylor’ Kuffner and the League of Electronic Musical Urban Robots (LEMUR) (http://gamelatron.com). The Gamelatron project features traditional Balinese and Javanese gamelan instruments augmented with custom robotic counterparts, as seen in Figure 6, that is modelled after the traditional Balinese and Javanese gamelan orchestra. It is fully automatic and can be controlled using the Musical Instrument Digital Interface (MIDI), with a setup that enables a person to play or conduct the entire gamelan orchestra. This allows for the performance of traditional instruments by a reduced number of players, making up for the lack of traditional performers available. Other musical robotic instruments of similar influence include Gamelan Elektrika (Pardue, Boch and Boch Reference Pardue, Boch and Boch2011) and Robotic Reyong (Kapur and Darling Reference Kapur and Darling2010).
Figure 6 Padma Bhuwana by Gamelatron Project.
Kapur combined the use of augmented traditional Asian instruments with musical robotics, and presented the idea of a one-man ensemble playing North Indian music (Kapur, Tzanetakis, Schloss, Driessen and Singer Reference Kapur, Tzanetakis, Schloss, Driessen and Singer2006). Later, he furthered this implementation into the Machine Orchestra (Figure 7), where he built a cog of robotic musical instruments using traditional practices to guide aesthetics and design, and had his cog engage with musicians using augmented traditional North Indian instruments through performances based on North Indian classical music (Kapur, Darling, Diakopoulos, Murphy, Hochenbaum, Vallis and Bahn Reference Kapur, Darling, Diakopoulos, Murphy, Hochenbaum, Vallis and Bahn2011). The latest addition to the Machine Orchestra is the Kritaanjli (Figure 8), a robotic Indian Harmonium (Kapur, Murphy and Carnegie Reference Kapur, Murphy and Carnegie2012).
Figure 7 Machine Orchestra.
Figure 8 Kritaanjli, robotic Indian harmonium by James Murphy.
As mentioned earlier in the section, research in musical robotics is highly active in the Asian region as well. While Korea-based art and technology company KoIAN created several Western-influenced musical robots, they also created robots that perform traditional Korean dances, ceremonies and other cultural events based on historical research (http://koian.org). This is the first sighting of the use of robotic technology in a cultural context outside the musical field.
The use of musical robotics in traditional Asian instruments makes up for the lack of accessible traditional instrumentalists, with the added benefit of new interactions in these performances.
4. Music Information Retrieval (MIR)-based
Another research area pertaining to Asian art music using technology is the research in Music Information Retrieval (MIR). MIR-based research is driven by data analysis, with the data being retrieved from scores, performances, data sets, audio collections and other similar musical sources. Data processing is usually done offline because the datasets are relatively large. The application of MIR-based research results in the creation of tools used for classification, performance-gesture recognition systems, music recommendation systems and automatic music creation.
4.1. Analysis/classification tools
MIR research started in the late 1990s (Bryd and Fingerhut Reference Bryd and Fingerhut2002) and has been biased towards Western music (X. Serra, Reference Serra2012b). Recently, research specific to Asian music began to appear in major conferences and journals such as the International Computer Music Conference (ICMC) (Chordia Reference Chordia2004), and the International Conference for Music Information Retrieval (ISMR) (Lee, Downie and Renear Reference Lee, Downie and Renear2002). Since then, there has been approximately one publication yearly on the analysis techniques of non-Western music in major music technology conferences.
In 2011 Xavier Serra started the CompMusic project to develop automatic analysis techniques that are specifically for non-Western music (X. Serra Reference Serra2011). The team currently focuses on five non-Western musical cultures: Hindustani (North India), Carnatic (South India), Turkish-makan (Turkey), Andalusian (North Africa) and Han (China). The computational models that were available had been developed for Western music, and were neither accurate nor suitable for non-Western music. This realisation resulted in the team proposing culture-specific approaches to MIR for non-Western music (X. Serra Reference Serra2011). Serra identified the sources from which information could be mined, and approached the data-gathering process according to the characteristics of each culture (X. Serra Reference Serra2012a). With prospects of contributing to the communities involved, Serra and his team have hosted two annual CompMusic workshops sharing their MIR research. Research outcomes from CompMusic include various methodologies for identification and classification (Gopala Krishna Koduri, Gulati, Rao and Serra Reference Koduri, Gulati, Rao and Serra2012; Ross, Vinutha and Rao Reference Ross, Vinutha and Rao2012; Salamon, Gulati and Serra Reference Salamon, Gulati and Serra2012; Vidwans, Ganguli and Rao Reference Vidwans, Ganguli and Rao2012), tools for musical analysis (Bellur, Ishwar and Murthy Reference Bellur, Ishwar and Murthy2012; Koduri, Miron, Serra and Serra Reference Koduri, Miron, Serra and Serra2011; J. Serra, Koduri, Miron and Serra Reference Serra, Koduri, Miron and Serra2011) and approaches to gathering data from the different non-Western musical cultures (Sordo, Serrà, Koduri and Serra Reference Sordo, Serrà, Koduri and Serra2012). Serra's Music Technology Group (MTG) at Universitat Pompeu Fabra also maintains an online database of royalty-free sound clips for researchers and sound artists, available at Freesound.
Other outcomes of MIR research in traditional Asian music include music and score recognition (Chordia Reference Chordia2004), performance gesture extraction and recognition (Chordia, Reference Chordia2005, Reference Chordia2006; Henbing and Leman Reference Henbing and Leman2007; Kapur, Percival, Lagrange and Tzanetakis Reference Kapur, Percival, Lagrange and Tzanetakis2007), music recommendation systems (Chordia, Godfrey and Rae Reference Chordia, Godfrey and Rae2008) and automatic music generation (Chordia and Rae Reference Chordia and Rae2010; Kapur et al. Reference Kapur, Tzanetakis, Schloss, Driessen and Singer2006).
Compared to the MIR research focusing on Indian classical music, the amount of research done on other types of Asian music is minute.
4.2. Pedagogical
As most traditional Asian music is heavily dependent on oral tradition, physical transcriptions of notated music can be difficult to find. Furthermore, the available transcriptions are in the musicians’ native languages and utilise non-Western notation systems. Notated transcriptions are usually transcribed by performers instead of the original performer or composer, which results in the same piece being transcribed significantly differently because of varying interpretations. This makes it difficult to understand and analyse music through notation, thus making it more challenging to pass on and learn if a new performer does not understand the native language.
In conjunction with the new interfaces developed through performance-based research, pedagogical tools have emerged for traditional Asian instruments. The first significant publication is Kapur's ‘Pedagogical Transcription for Multimodal Sitar Performance’ (Kapur et al. Reference Kapur, Percival, Lagrange and Tzanetakis2007). Utilising the eSitar along with music information retrieval techniques of audio feature extraction and processed sensor data, and machine learning, it transcribes the performance to a unique notation system that notates both fret position and audible notes.
More recently, systems focusing on other instruments have emerged. Kikukawa and his colleagues at Wakayama University developed a learning system using sensors to assist novice players of the Erhu (a Chinese bowed-string instrument) to assume correct finger positioning (Kikukawa, Ishihara, Soga and Taki Reference Kikukawa, Ishihara, Soga and Taki2013). The same year, Hochenbaum introduced a more general system aimed towards practising instruments using hyperinstruments (Hochenbaum and Kapur Reference Hochenbaum and Kapur2013). This system presents a similar use of multimodality to monitor practice, and utilises customised software for capturing sensor data and audio recordings of performances synchronously.
Currently, pedagogical systems are additional tools to assist students in effective practice, and are not intended to replace the teacher. As the system allows the capturing of almost every aspect of a performance, it has the potential to archive teachings and performances. It also allows more accurate capturing of interpretation, which could be used by traditional performers in addition to transcriptions. This contributes to the preservation of traditional Asian music by providing playing techniques of instruments in a globally understood language.
5. Conclusion
There are still many issues, concerns and obstacles facing current researchers working in the hybrid realm of technology in combination with traditional Asian music. There is a large language barrier for many of the researchers as only a few of them natively speak the respective languages of the instruments they research. Furthermore, notation may be in a modified or archaic form of the original language, forcing even the researchers who are native speakers of the languages to face a steep learning curve. Recordings of traditional repertoire also are unreliable, as very few recordings are made. The few recordings that are available are also thought to be far from what the original composer intended, as they are usually interpretations realised by the individual performer's aesthetics. Even the accessibility to these professional performers can be difficult, since there are so few of these specialised performers available in the world.
Yet, even with these setbacks, the extensive works and collaboration of ethnic instruments with modern technology, as summarised in Figure 9, further reemphasise the relevance, capability and compatibility of it in today's world. This new attitude provides a new outlook for the younger generations towards traditional Asian instruments. This is achieved by entering the less explored territories of traditional Asian practices, such as instrument augmentation and musical robotics, yet still following the aesthetics of traditional music heritage. Augmented traditional Asian instruments have opened up new interactions between instruments and performers, while sensor systems extract performance gesture for additional intuitive control of other media such as audio and visual effects and robotic instruments. Asian musical robotics provides an alternative solution to gathering enough players for traditional ensembles requiring a large number of players.
Figure 9 Outline of extensive works of Asian traditional instruments with modern technology since 2000.
MIR-based research develops tools that are culture specific, allowing better understanding of the culture's compositions and acoustics without prior knowledge of non-standardised musical notation. In addition, the new interfaces created through performance-based research can be used in conjunction to capture and analyse performance gesture data. Having multimodality in data also allows for more accurate representation of the work, which proves helpful because detailed data is frequently unavailable due to the nature of oral tradition and lack of documentation.
Pedagogical applications provide an alternative to accessing cultural traditions, which are most often of oral tradition and are intangible in nature. These applications could further increase the number of people who could perform these traditional instruments.
All these mentioned examples demonstrate the potential to be tools or methods for preservation and cultivation, as well as to sustain the developments of traditional Asian music. The future of hybridisation between the traditional Asian arts and technology is bright. With the rapid growth of development in information technology infrastructure in Asia coupled with the broadening of computational musical research that focuses on its cultures, it seems that there will only be more research and researchers making an appearance on international grounds in the near future.
