1. IntroductionFootnote 1
The majority of the works created at the ZKM | Institute for Music and Acoustics (IMA) are intended for performance in the concert hall. However, a number of the works can be categorised as ‘installations’. Whereas approximately 400 of the works produced at the IMA have been compositions, an additional 20 have been installations. While the large majority of the compositions have been created by guest artists, about half of the installations were developed by staff members of the ZKM. The continual creation of new installations since the opening of the ZKM in 1997 – a new installation has been produced virtually every year – reflects the importance that these works hold among the works produced at the IMA. These installations are the focus of this article.
The IMA generally specifies no guidelines for the design of installations, other than perhaps the broadly formulated condition that they somehow incorporate new media or computers. The IMA installations therefore do not constitute a uniform group of works. Instead, they display a great range of different approaches, styles and content. And yet they still have a great deal in common. As we will see, one of the most salient features of the majority of the IMA installations is their interactivity. The interactivity in turn gives rise to further typical characteristics of the IMA installations, such as their ability to take the form of a kind of game, a didactic tool, or a musical instrument. In the following text I will describe these and other qualities, providing a portrayal of the installations produced at the IMA. Following this depiction I will then present the individual works in more detail in the second part of the article.
1.1. Interactivity
Interactivity constitutes a central component of the majority of the installations created at the IMA. The installations are intended to be truly ‘controlled’ by the user, and some installations even remain silent if there is no active user-input (such as the installation Music Pipeline). Most of the installations have a kind of ‘automatic mode’, so that they play independently even without the input of an active user (such as the installation KlangWeltKarte). However, this automatic mode is often not the central feature of the installation. Instead, it has more of the function of attracting curious visitors, or of offering a kind of ‘demo’.
If users are expected to actively participate in operating an installation, they must be shown the path to this interaction. This can produce a conflict: on the one hand, the visitor should be able to intuitively interact with the installation without having to first read an instruction manual. On the other hand, installations are sometimes intended to address complex topics, which can possibly lead to the necessity for a complicated user-interface. Some installations solve this problem by distinguishing between inexperienced and advanced users. This distinction is not necessarily seen in the form of completely separate processes. Rather, advanced users may have more options for interaction available to them than inexperienced users. It is important, however, that even the inexperienced user can engage in a meaningful interaction with the installation. The steepness of the learning curve for the users of an installation is obviously also connected with their level of experience. A visitor who has experience with computer games will typically have no trouble finding their way in engaging with an installation. For example, children can often be seen showing their parents how to operate the exhibits in the ZKM’s Media Museum.
1.2. The installation as a (computer) game
Interactivity often gives the installation a game-like quality. The user’s influence is not limited to the mere selection of options. Rather, interaction with an installation often consists of a cycle of mutually dependent actions and reactions on the part of the user and the installation. Some installations establish a distinct reference to existing games, such as a pinball or slot machine (for example, the installations Small Fish and Mozart-Würfel). Other installations show a more general connection to the aesthetic or functionality of computer games (such as the installations Oracle or SonoMorphis).
Another relationship to computer games can be seen in the learning curve mentioned in section 1.1. In order for inexperienced users of an installation to become advanced users, they can often take advantage of written assistance (which either accompanies the installation in paper form or is integrated into the user interface). Alternatively, the user can also learn the skills for interacting with the installation in a playful manner, without any instruction, and independently make intentional or unanticipated discoveries. This manner of learning is typical for many computer games, and sometimes even the object of the game (for instance in the case of adventure games).
The topic of ‘playing’ in general has a tradition in sound art, as seen for example in Michael Jüllich’s Spiel- und Klangstraße (Play and Sound Street) or in Bill and Mary Buchen’s Sound Parks and Sound Playgrounds (Föllmer Reference Föllmer1999: 217), even if the meaning of ‘playing’ in this context has more to do with ‘playing a musical instrument’.
1.3. The installation as a musical instrument
Many of the installations can be used as musical instruments. There are many precedents for this double function of exhibit and musical instrument found in the field of sound art and its precursors. The work of creators such as Harry Partch, Harry Bertoia or the Baschet brothers includes musical instruments that have also been displayed as sculptures in exhibitions (Gertich Reference Gertich1999: 138ff, 174, 177f). In the museum setting, some of these instruments are indeed only ‘exhibited’. In other words, they can only be viewed but not heard. However, others have been specifically designed to be interacted with by the museum visitors.
It is particularly true of the IMA installations that they have been designed for active use in the museum. Spontaneous little ‘concerts’ can even arise among the visitors, for example when a group gathers around an installation and one person operates it while the others listen and comment. The installation KlangWeltKarte has several user stations, so that the visitors can even improvise together. The social component of such a ‘concert situation’ should certainly not to be underestimated. This is especially true with regard to the manner in which groups of school children take in exhibitions, but it also applies to visitor groups in general. It particularly promotes discussion and mutual thought about the installation, or about the content that the installation takes as its central topic.
Some of the IMA installations have also been used in the concert setting (such as the installations KlangWeltKarte (inter-art project n.d.) and Audio Fraktal (ZKM n.d.-b)), and some have special concert versions (for example, the installation Small Fish). The primary environment for their use, however, still remains the exhibition.
1.4. The installation as a tool for exploration
For a large number of the IMA installations, one of the primary goals is to stimulate thought about their content. The content of these installations does not only consist of topics specific to sound art. Most commonly they address more general topics relating to music. For example, they demonstrate compositional techniques from the realms of loop-based music, minimalism, or musique concrète (such as the installations Pattern Machine, Random Machine and Sonoloop). The visitors can gather their own experiences with these compositional techniques by actively employing them themselves. In doing so they may stumble upon questions, such as why they like one combination of sounds better than another. They can make assumptions about these questions, and the installation provides them with a means of testing these assumptions directly using new combinations of sounds. The installations thus become empirical tools, with which the visitors are able not only to research the compositional techniques that form the subject matter of the installations, but also, ultimately, to research their own perception.
1.5. The installation and sonification
Some of the IMA installations do not serve the purpose of examining a musical topic, but rather concern themselves with the acoustic representation of data; that is, with sonification. These IMA installations not only sonify data; they also present the data in a visual form. In this way, the visitor has the opportunity to relate the two forms of representation to each other and compare them. In some cases, it is not important which data are concerned. Instead, the focus is on the artistic implementation (such as in the installation SonoMorphis). In other cases, however, there is a concrete interest in the data themselves (for instance in the installation Algorithmic Echolocation). In these cases the installation once again becomes a form of tool, this time for revealing the underlying data structures. Even so, the visitor’s own perception of course ultimately becomes the object of observation here as well, particularly through the comparison of visual and acoustic presentations of the same source data.
1.6. The installation and didactics
Several of the topics discussed in the previous passages show that many of the IMA installations have an important didactic component. Some of the IMA installations have the explicit goal of introducing the visitors to musical topics that they wouldn’t normally think about. Many of the installations could also easily be shown in the context of a science museum (for instance, the installations Music Pipeline and Architektur Musik Labor; the latter can be seen in the Phaeno Museum in Wolfsburg, which is a well-known science museum in Germany). Other installations are particularly oriented towards children – even if not exclusively (for example, the installations Small Fish and Bubbles). To put it in the words of the Baschet brothers: ‘Reservation and timidness, which may arise when faced with the keys of a piano, for instance, disappear in this setting. When everything is completely new, there is no teacher who can give advice and call attention to mistakes’ (Gertich Reference Gertich1999: 141). This citation, which reports on the positive experiences of the Baschet brothers when visitors making music with their sound sculptures, could apply equally to the IMA installations.
One example of a didactic element that has been incorporated into several installations presented in exhibitions of the ZKM | Media Museum is the function for making sound recordings. The installations that make use of this function provide a USB port for the users, to which they can connect their private USB memory sticks or MP3 players. The users can thereby record all the sounds that they or the installation generate. Thus, they also have the opportunity to use this recording outside of the museum for further study and to heighten their insights.
1.7. Video projection
One final point should still be addressed before I go into the details of the individual installations: most of the installations discussed here incorporate a computer screen, usually in the form of a projection on the wall but in some instances also in the form of a monitor (exceptions to this include the installations KlangWeltKarte and Musikschrank Rheingold). The interaction often takes place using a mouse, though it sometimes consists of special buttons. The sound is played through loudspeakers.
These installations can therefore be described as audiovisual installations. In ZKM exhibitions they are often classified as sound installations. This can be interpreted as an indication that the acoustic component is considered to be more important than the visual element.
A further feature of many installations consists of the flexibility in the form of presentation. Many installations have both a version for wall projection and loudspeakers as well as one for monitor and headphones. Furthermore, the same installation can have both sculptural manifestations as well ones consisting of just image projections. Finally, parallel versions exist for the museum and for at home, on CD-ROM; it would also be conceivable to create versions offered over the Internet.
2. The installations
The following passages will present the individual installations in chronological order. Unfortunately, the limited scope of this article will not allow me to go into the same level of detail for each of the installations.
2.1. KlangWeltKarte/Acoustic World Atlas (Thomas Gerwin, 1997)
The KlangWeltKarte (Movie example 1) is a collection of short sound portraits of many different locations around the world. These sound portraits can be played back by the visitors. Each of the sounding locations are displayed through light effects on a large world map (approximately 10 metres long and 2.5 metres high), which hangs on the wall (figure 1). In front of the map there are three user stations, each of which controls a separate third of the world map. A relief map of the world is depicted in small format on the consoles of each of the stations. Buttons are embedded at the geographic locations for which compositions exist, so that the users can play back the specific sound portraits. In addition to buttons for certain selected locations, there are also buttons for regions and continents, and finally for the entire world. The various types of buttons are distinguished on the user station through both their colour and their shape. This makes it possible for the installation to also be operated by the blind or the visually impaired.
Figure 1 Thomas Gerwin: KlangWeltKarte. Photo: Götz Dipper.
Pressing a button will trigger the playback of one of the previously compiled sound portraits. There are a total of 210 such sound portraits, each of which lasts between 5 and 40 seconds (DEGEM n.d.). The material that Gerwin compiled for the sound portraits stems from original recordings that were each made on location. Some of the recordings that can be heard consist of sounds of nature, machines, traffic, human voices and folk music. The ‘world’ button mixes all of the sounds together, metaphorically speaking. The button triggers a unified noise that contains all frequencies, implying a merging of all the sounds of the world, in which it is impossible to hear any of the separate sounds individually.
The source recordings were made available to Gerwin for the KlangWeltKarte by people from all over the world (SWO n.d.). To this end, a press release was published by the city of Karlsruhe prior to the project, stating: ‘Acoustic snapshots sought from around the world – The ZKM/Center for Art and Media in Karlsruhe is looking for sounds and noises from all continents. They are to serve as the raw materials for the ‘Acoustic World Atlas’, which is currently being produced at the ZKM for the future Media Museum’ (Kulturamt Karlsruhe 1997).
Gerwin defines the KlangWeltKarte as a musical instrument, ‘with which one can play live ‘musique concrète’ ’ (inter-art project n.d.). As mentioned in section 1.3, Gerwin also used the KlangWeltKarte in the concert setting. In the museum, visitors can either improvise three at a time and hear the sounds through loudspeakers, or they can improvise independently of each other and listen to the results on headphones. Any number of sounds can be layered on top of each other during playback. The user stations also have further keys available that allow the user to fade the sound in or out either gently or rapidly. The KlangWeltKarte was the first installation at the ZKM that allowed the visitors to make recordings of their improvisations and take them home with them – originally this was still done with cassette tapes. Other installations have followed this model today, by saving recordings to a USB memory stick.
Overall, the KlangWeltKarte possesses a distinct acoustic-environmental component. The individual sound portraits are situated within the tradition of ‘soundscape compositions’. The KlangWeltKarte can be seen as ‘a kind of documentation of the audial state of the world at the end of the last century’ (inter-art project n.d.), while calling for ‘a responsible interaction with the fragile world of vibration and sound’ (ZKM n.d.-e).
The KlangWeltKarte was first exhibited publicly at the opening of the ZKM in 1997 and has been a permanent feature of the ZKM | Media Museum ever since. The KlangWeltKarte was produced in cooperation with the Department of Mechatronics at the Karlsruhe University of Applied Sciences, the Institute for Computer Music and Electronic Media (ICEM) of the Folkwang University in Essen, and the ZKM | Media Museum.
2.2. AML – Architektur Musik Labor/Architecture and Music Laboratory (Pierre Dutilleux and Christian Müller-Tomfelde, 1997)
The Architektur Musik Labor (AML) installation (Movie example 2) deals with the topic of acoustic spaces (figure 2). The visitors can explore the changes in a source sound that is played back in different spaces; in other words, when the space adds the element of reverberation to the sound.
Figure 2 Pierre Dutilleux and Christian Müller-Tomfelde: Architektur Musik Labor (version 2007; exhibition design: Matthias Ossmann). Photo: Sónia Alves.
The user can choose from 10 real spaces – such as a concert hall, a church or a living room – and 6 artificial spaces, such as a sphere or a cube. The visitor has a further choice of around 30 source sounds, from a piano sonata by Beethoven, to a Bruckner symphony, to jazz pieces, speech recordings, and sounds of nature or machines. The users additionally have at their disposal a controller with which they can modify, on the one hand, the distance between the sound source and the listener in the real rooms or, on the other hand, the size of the artificial rooms.
The source sounds were recorded in spaces with very little reverberation, so that virtually no aspects of those spaces are contained in the recording. The reverberation of the real rooms is calculated using impulse response in real time.Footnote 2 The reverberation of the artificial rooms is calculated with the help of less complex algorithms. A detailed technical description can be found in Dutilleux and Müller-Tomfelde (Reference Dutilleux and Müller-Tomfelde1999).
AML is a very good example of an installation that fits into the category of ‘tools for exploration’ described in section 1.4. It takes the personal experience of the visitor as a point of departure, since everyone has a distinct impression of how his or her voice sounds in a tunnel, in a church, or in the open. But only with a tool such as AML can the visitor directly compare the acoustics of various rooms. The available rooms and parameters were consciously designed so that even an inexperienced listener can easily discern the differences between the rooms (Dutilleux and Müller-Tomfelde Reference Dutilleux and Müller-Tomfelde1999: 9). On the other hand, subtle differences can also be produced by modifying the distance setting very slightly, allowing the visitor to gradually become sensitised to finer nuances. AML also allows ‘nonsensical’ combinations of sound and space, such as a symphony orchestra in a bathroom, or airplanes in a church. And when the artificial spaces are used, the reverb suddenly becomes a ‘shaping tool’ and AML becomes a station for experimenting with processes of creating electroacoustic music.
AML was produced within a cooperation between the IMA and the ZKM | Media Museum. Its current form is already its third version. The original version possesses a sculpture-like operating console (designed by Frank den Oudsten). The second and third versions, however, make use of video projections and mouse-based control or a computer touchscreen (developed by Jordi Janer and Paulo Ferreira-Lopes, respectively).
2.3. SonoMorphis (Bernd Lintermann and Torsten Belschner, 1998)
Unlike the KlangWeltKarte and AML, which distinctly place their focus on the acoustic component while their visual elements primarily make up the user interface, SonoMorphis places equal focus on both aspects (Movie example 3). The basic concept of SonoMorphis is to use the visual and the acoustic components as two different, independent means for representing the same (abstract) source data. The sound doesn’t follow the graphics, nor do the graphics follow the sound. Rather, both elements trace back to a third, common origin. Its creators refer to ‘visualisation’ and ‘sonification’ of the same ‘data structure’ (Lintermann and Belschner Reference Lintermann and Belschner2007). SonoMorphis successfully manages the balancing act by which ‘the sonic and visual representations of the installation’s structure [are put] into as close a relationship as possible (Lintermann n.d.-b), while avoiding a ‘mere doubling of that which is being represented’ (Belschner Reference Belschner2000: 341).
The visual component consists of a stereoscopic projection. The visitor sees an organic-looking shape that is composed of various segments (figure 3). It arouses recollections of leaves or blossoms, insects, arms, tentacles or the like. It floats, weightless, on a black background, in constant motion and gradual change. The observers feel as though they have been transported to a strange ‘genetic laboratory’.
Figure 3 Bernd Lintermann and Torsten Belschner: Sono reMorphed. Photo: Christina Zartmann.
The acoustic part seems abstract in comparison, and hardly any concrete associations come to mind. The sounds are produced using the technique of ‘physical modeling’. In order to avoid a simple doubling of the visual element, the aural component places a preference on the use of metallic sounds, in conscious contrast to the organic forms of the visual layer. This contrast further intensifies the impression of strangeness brought about by the visual forms.
SonoMorphis uses either four or six loudspeakers that are placed in a circle around the visitor. This spacial presentation assists the visitor in grasping the parallel nature of the graphics and sound. For example, if one of the graphical objects – in other words, one of the ‘body parts’ of the organic form – moves forward and to the left, the corresponding sound also moves in the same direction. The visitor has many diverse options for influencing both the spatial position of the graphical and tonal shapes and the kind of changes they undergo by manipulating the controls of a user station. Spatial movement can be controlled by sliders. Changing the shape is done by selecting an item from a list of suggested ‘mutations’. The shape then changes accordingly; for example, it may begin to grow tentacles. ‘The interaction follows the evolutionary principle of the mutation of the graphical object and the selection of variants’ (Belschner Reference Belschner2000: 340). Oliver Grau has classified SonoMorphis as ‘genetic art’ (Grau Reference Grau2004: 312). The list of mutations is generated randomly. The selection from this list of mutations, however, is performed by the user. Through the opportunity for the user to influence the process, SonoMorphis becomes an ‘audiovisual instrument’ that can be played (Belschner Reference Belschner2000: 339; Lintermann and Belschner Reference Lintermann and Belschner2007).
A new version of SonoMorphis was presented to the public in 2007: Sono reMorphed. Sono reMorphed is an adaptation of SonoMorphis to more recent technologies developed by the ZKM: the PanoramaScreen (Institute for Visual Media) and the Sound Dome (IMA). SonoMorphis itself was produced in a cooperation between the Institute for Visual Media and the IMA.
2.4. Small Fish (Kiyoshi Furukawa, Masaki Fujihata and Wolfgang Münch, Reference Münch1999)
Small Fish (Movie example 4) consists of a collection of 15 movable, interactive images or ‘scenes’. The visitor selects one of these scenes from a welcome page. The individual scenes contain graphical objects, some of which move automatically on the projection surface and some of which can be moved by the user (figure 4). Their movements result in sounds, which typically arise when the objects touch each other. The exact principles and rules that govern the occurrence of sounds and movements of the objects vary from scene to scene. The creators refer to Small Fish as a game (Staff Reference Staff1999: 33). Most of the scenes are remotely modelled after traditional games, such as billiards or pinball. The sound production is oriented around reality, in that the objects make a sound when they collide. The user is thus usually able to understand the functionality of a scene intuitively and easily operate the game. Nevertheless, the result is neither trivial nor completely predictable. Instead, it continues to surprise: even though the behaviour of an individual object is easily comprehensible, the continuous involvement of several objects can quickly result in a high degree of complexity (see also Münch Reference Münch1999: 30).
Figure 4 Kiyoshi Furukawa, Masaki Fujihata and Wolfgang Münch: Small Fish. Screenshot.
Small Fish was created at nearly the same time as SonoMorphis, and both works address the interconnection of image and sound. Even if both installations succeed in integrating image and sound to an equally high degree, the initial point of departure for SonoMorphis still lies in its images, and for Small Fish in its sound. SonoMorphis can thus be traced back to its predecessor Morphogenesis (Lintermann n.d.-a), which operates without any sound. The basic concept behind Small Fish, on the other hand, appears to be more musically founded (see also Goebel Reference Goebel1999: 26). The scenes can be read as a kind of moving graphical score. The visual impression of a score is aroused, for example, by the fact that the pitch level of the objects usually corresponds to their vertical position on the screen.
Just like SonoMorphis, Small Fish can also be seen as an instrument. It has already been used as such in the concert setting, even if in a modified version. In the concert version, which was produced for the Ars Electronica Center and the Warsaw Autumn, Small Fish is controlled in real-time by the computer operator as well as by a trumpet player. This is done using a pitch tracker, so that certain pitches played by the trumpet trigger specific actions within Small Fish. The actions are orchestrated in such a way that the audience can easily attribute them to the trumpet (ZKM n.d.-d).
Small Fish is a very good example of the flexibility in the form of presentation addressed in section 1.7. I will therefore briefly outline some of the other existing forms of presentation. In addition to the aforementioned concert version and the ‘simple’ museum version (with wall projection), Small Fish also exists in the form of a sculpture. In this form, the scenes are projected onto a three-dimensional, opened book. The book is made of wood and is approximately 1 metre wide. The interaction is performed using a computer mouse. A further version of Small Fish consists of a table. In this version, the scenes are projected vertically onto the tabletop from above. Instead of a computer mouse, a few real objects are located on the table. These objects can be moved by the users and are integrated into the Small Fish scene as virtual objects (the position of the objects are tracked by the computer using a camera). Yet another version, the floor version, functions in a similar manner to the table version, with the exception that the projection encompasses several metres, and instead of the small objects, pillows resembling stuffed animals are used. The floor version could also be classified as an ‘environment’. With its floor version as an ‘environment’ and its book version as a ‘sculpture’, Small Fish unifies two forms of presentation that are actually described as opposite positions in the discussion of sound art (Helga de la Motte-Haber Reference de la Motte-Haber1999).
Finally, Small Fish was also produced in a CD-ROM version that could be purchased for Mac and PC. Anyone could install Small Fish on their own computer and ‘play’ with it at home. Unfortunately, this CD-ROM is no longer in stock. There is currently a new edition in progress, again with a new version, titled Let’s make Small Fish. This version allows users to design their own scene for Small Fish. They can integrate self-made graphical objects and sounds into the Small Fish environment. In this case, both the interactivity and the involvement of the user in the creative process begins one stage earlier. A similar approach can incidentally also be seen in the work of the Baschet brothers. In the 1960s, they also began involving the audience in playing on their experimental musical instruments, as mentioned in section 1.6. They later expanded this involvement to the construction of these instruments within their children’s workshops (Gertich Reference Gertich1999: 141). Small Fish has also been successfully used in many workshops for children. The Let’s make Small Fish version is especially oriented towards children (ZKM n.d.-d).
Like SonoMorphis, Small Fish was also produced in a cooperation between the IMA and the Institute for Visual Media.
2.5. Musikschrank Rheingold (Torsten Belschner, Johannes Goebel and Bernhard Sturm, 1999)
As with the KlangWeltKarte, the installation Musikschrank Rheingold (Rheingold Radio Cabinet) also forgoes the use of a computer projection. The installation consists of an old radio set (the ‘Rheingold’ model), integrated into a living room set in a 1950s style (figure 5). The visitors can use the radio. The radio set possesses the customary buttons for shortwave, medium wave, and so on, and a control dial for selecting the broadcasting station. What makes Musikschrank Rheingold unique is that the radio’s innards have been rebuilt. Instead of traditional radio, the visitor receives Internet radio.
Figure 5 Torsten Belschner, Johannes Goebel and Bernhard Sturm: Musikschrank Rheingold. Photo: Bernhard Sturm.
Musikschrank Rheingold holds a special position among the installations discussed here. It is a very conceptual piece and principally focuses on non-musical topics, such as the ‘Internet’ or the issue of the ‘interface’. The radio set in this context thus constitutes an interface to Internet radio. The visitor is intended to take a critical look at the question of what the difference is between traditional radio and Internet radio, and whether the old familiar radio set presents an appropriate interface for both. The explanatory notes that accompany the installation provide a response to these questions: ‘The new thing [about Internet radio] is: Anybody can transmit a broadcast, can send it into the Web, with very little effort. This new form of distribution could also lead to a change in both production and content. However, the user has to fish around in the infinite tributaries of the Web in order to find that specific broadcast; there is no linear dial like there is on a [traditional] Radio’ (ZKM n.d.-f).
Musikschrank Rheingold was produced with great attention to detail. The user is given feedback on an LCD display and by a voice over the loudspeakers during their search for a station. The feedback informs the user in several languages that the connection is currently being established, until the station itself is finally audible and indicated on the LCD display. The so-called ‘magic eye’ of the original radio set is even used, giving users a bit of the feeling that they are using a ‘real’ radio. The sound is played back using the Rheingold’s internal speakers and tube amplifier.
2.6. Bubbles (Wolfgang Münch and Kiyoshi Furukawa, 2000)
Bubbles (Movie example 5) uses the computer projection in a very special way. The projector and the projection are arranged in such a manner that the visitors can step into the light beam cast by the projector and become visible as silhouettes (figure 6). Virtual soap bubbles float among these shadow figures. The soap bubbles move in a very natural way. They obey the laws of gravity and gradually sink towards the floor. Sometimes they even pop, and – most importantly – they interact with the silhouettes. When one of the soap bubbles converges with a shadow, the soap bubble is repelled; it ‘ricochets’ off the shadow (ZKM n.d.-a).
Figure 6 Wolfgang Münch and Kiyoshi Furukawa: Bubbles. Photo: ZKM.
The nature of the soap bubbles’ motion and interaction is reminiscent of some of the scenes from Small Fish. Even the camera-tracking technique used here was previously implemented in the tabletop version of Small Fish. However, Bubbles incorporates something very new. The visitors themselves become a principal component of the installation through their representation as shadows. The soap bubbles appear to have the primary purpose of animating the users to playful gestures. In Small Fish, the visitor was motivated to musically and visually arrange given material. Bubbles encourages the users to personally perform as an actor. The installation’s focus is thereby shifted to the visual element, while the acoustic component receives a predominantly supportive function. Yet the sound is still very important for the success of the installation. It creates the atmosphere that animates the visitors to unreserved playfulness. The sound is not limited to acoustically highlighting the collisions of the soap bubbles. It also forms a continuous background, against which the active events can take place. Unlike with Small Fish, visitors do not directly associate this acoustic background with individual visual events.
Bubbles was produced within a cooperation between the IMA and the Institute for Visual Media.
2.7. Algorithmic Echolocation (Ramon Guardans, Adolf Mathias, Matthias Gommel, Martin Schüttler, Götz Dipper and Renata Sas, 2003)
Algorithmic Echolocation (Movie example 6) is an installation fully dedicated to the topic of sonification. It concerns climate data derived from ice core samples from the Antarctic. The data describe the periodic fluctuation in atmospheric concentrations of, for example, carbon dioxide, oxygen, or dust over the course of approximately 400,000 years. The source material thus consists of several rows of data that are depicted both visually and acoustically (figure 7).
Figure 7 Ramon Guardans et al.: Algorithmic Echolocation. Photo: Matthias Gommel.
Users have access to a control panel with which they can influence the presentation of the data. Using the controls, they can decide whether they would like to track the data for carbon dioxide or oxygen, for example. They can also determine the time frame to be presented, and can vary the playback speed – that is, the rate at which the underlying data are transferred and depicted. Algorithmic Echolocation is thus a tool with which data can be visualised and examined. Whether the information consists of climate data or other content is ultimately of secondary importance.
Algorithmic Echolocation is the result of a cooperation between the IMA and the Institute for Visual Media, the Institute for Basic Research, and Soundplots and Medialab Madrid.
2.8. Audio Fraktal (Joachim Goßmann, 2004)
A further installation from the area of sonification is Audio Fraktal (Movie example 7). As opposed to Algorithmic Echolocation, the data depicted by this installation are not the product of any given measurements. Instead, they are the result of a mathematic formula, namely, the famous Mandelbrot Set. The visual element consists of a tabletop, onto which a graphical, black-and-white presentation of the Mandelbrot set is projected (figure 8). The projection simultaneously constitutes the user interface. Visitors can move a computer mouse on the tabletop and trigger sounds by mouse-click. The sounds are generated from the mathematic formula in a similar manner to the graphics (Goßmann Reference Goßmann2005). The graphical representation of the Mandelbrot set is typically calculated by setting the colour value of a given point to correspond to the number of iteration cycles required to calculate that point. The acoustic representation in Audio Fraktal is not only based on the number of iteration cycles, but also on the values produced by each cycle. This is done by mapping their (geometric) intervals to the frequencies of sine tones and their (geometric) directions to the spatialisation of these sine tones. The user can thus use the acoustic element to examine properties of the Mandelbrot set that are not accessible through mere graphical depiction.
Figure 8 Joachim Goßmann: Audio Fraktal. Photo: Franz Wamhof.
2.9. Pattern Machine, Random Machine (Ludger Brümmer, Chandrasekhar Ramakrishnan and Götz Dipper, 2004)
Pattern Machine and Random Machine (Movie examples 8 and 9) are two installations that are closely related and are usually presented together in exhibitions. They function like little step sequencers that can be controlled by a graphical user interface (figure 9). The input field encompasses two octaves of pitch and ten steps along the time axis. The user can enter short melodies and have them played back in a loop by various instruments. The special thing about this installation is that these melodies can then be transformed. Pattern Machine and Random Machine differ in the manner by which they transform the melodies.
Figure 9 Ludger Brümmer, Chandrasekhar Ramakrishnan and Götz Dipper: Random Machine (version 2004). Photo: Anatole Serexhe.
Using Pattern Machine, the visitor can choose from the transformation techniques of retrograde, inversion, palindrome, accumulation, rotation and phasing. Random Machine offers random transformations, such as one randomly selected note being changed per cycle, or all of the notes being reordered in a randomly determined sequence. The user can also assign specified probabilities to the pitches, or define pitch ranges within which the tones are randomly selected.
Pattern Machine and Random Machine are good examples of the ‘didactics’ category described in section 1.6. The user is presented with strategies for creatively dealing with musical patterns and random processes. Of great importance is that the the manner by which the installations are controlled is extremely easy and intuitive, and that the user essentially cannot do anything ‘wrong’. Thus even inexperienced users can achieve an immediate feeling of success. This encourages them to further experimentation, during which they continue to arrive at an increasingly differentiated manner of handling the material. The installations’ reduction and concentration of the material to a few fundamental points is decisive in this context. The installations isolate their key issues, the transformation of patterns and the principle of random process. They function as a kind of research lab, in which the user can study the effects of these musical tools.
2.10. Music Pipeline (Joachim Goßmann, 2006)
Music Pipeline is another installation with a didactic emphasis. It consists of several listening stations that the users visit in a predetermined order, and which are connected through a ‘musical pipeline’. At the first station, the users load into the pipeline music that they have brought with them, such as from an audio CD or an MP3 player. Alternatively they can use a microphone located at the station. This recording, having now been entered into the system, slowly wanders along the pipeline. The recording is not continuously audible on its way through the pipeline. Instead, it can only be heard at the listening stations that are placed along the pipeline, at which the recordings arrive in sequence (figure 10). These recordings are listened to through loudspeakers and headphones. The visitors linger at each station for approximately two minutes, and then have 20 seconds to proceed to the next station.
Figure 10 Joachim Goßmann: Music Pipeline. Diagram: Joachim Goßmann.
Various methods of sound processing are presented at the individual stations. These methods are used to modify the original recording. The majority of this processing is performed by the visitors using computer terminals located at the stations. One of the headphone stations, for example, allows the user to work with a delay between the left and right channels. The duration of the delay can be varied by the visitor. A different station processes the recording using granular synthesis. Here, the visitor can modify various parameters such as pitch or the density and overlap of the grains. Other stations involve reverberation and echo effects, loudspeaker comparisons, wavefield synthesis, or a demonstration of the Sound Dome that is located in the ZKM Cube performance hall.
The Music Pipeline installation is thus a kind of ‘learning path’ for sound processing. The exact configuration of the stations is flexible. An important point in the concept of the installation is that the visitors load their own music into the pipeline. Their familiarity with the music allows them to perceive subtle differences in the sound more easily, while still being able to discern the original signal even when it has been significantly modified. This helps them to achieve a clear understanding of each sound-processing technique’s functionality and effect. Music Pipeline is typically presented at special occasions, such as open house events. At such occasions, the daily press invites visitors to bring their own music with them.
2.11. Epigenesis (David Psenicka, 2007)
Interactivity is not only a central component of the audiovisual installation Epigenesis (Movie example 10); it is also a conceptual topic of the work. Epigenesis implements a video projection and four loudspeakers (figure 11), located in the four corners of the projection screen. The images that appear on the screen and the sounds that are audible through the loudspeakers are indeterminate and transform over the course of the installation’s duration. The source material comes from either current or previous visitors that have been recorded using a video camera and microphones. The computer constantly redetermines which actions it carries out (i.e. which material it selects and how it presents and transforms this material) on the basis of artificial intelligence (reinforcement learning). The computer has been programmed to strive for interaction with the visitors. If a visitor interacts with the installation, the computer perceives this as a reward and is ‘satisfied’. If a visitor leaves the installation area, the computer classifies this as ‘punishment’. The computer keeps a record of which of its actions were successful and which were not. By this means, it gathers a pool of experience over time. This pool of information provides the basis for its future actions, which it again rates as successes or failures. The installation is thus in a constant state of flux, and it is impossible to predict the direction in which it will continue to develop.
Figure 11 David Psenicka: Epigenesis. Photo: David Psenicka.
More detailed information on the functionality of Epigenesis can be found in Psenicka Reference Psenicka2008.
2.12. Mozart-Würfel (Götz Dipper, 2007)
The Mozart-Würfel (Movie example 11) traces back to a popular ‘musical dice game’ that was invented in the eighteenth century. The game was very widespread at the time and existed in different versions. The historical dice game consists of a sheet of music and a table of numbers. Using these two items, each roll of the dice yields a new measure in a piece of music, typically encompassing 16 measures that are always based on the same harmonic progression (figure 12). The Mozart-Würfel installation is a reference to a version of this game that was published in Mozart’s name two years after his death. It implements the game in the form of a modern slot machine, with musical notes on virtual ‘reels’. Each of the 16 measures has one reel that is set into motion and played back by a random generator. The visitors can also specifically operate each of the reels individually, allowing them to compose and compare their own versions. In this sense, the Mozart-Würfel is also a tool that allows users to explore the functionality of the historical musical dice game.
Figure 12 Götz Dipper: Mozart-Würfel. Screenshot.
2.13. Oracle (Robert Darroll and Sean Reed, 2007)
Oracle (Movie example 12) is a Gesamtkunstwerk consisting of text, video and music. The visitor is surrounded by a video panorama and a circle of loudspeakers. A user station stands at the centre of the space. By entering text into the interface, the visitors can pose questions to the Oracle that are subsequently answered. The reply takes place in the style of traditional rituals, with a structured sequence of scenes. The user is presented with alternating scenes, some with no text, in which surreal landscapes spread out around the visitor, and some with text, in which the Oracle speaks from giant heads that appear in a circle around the visitor (figure 13).
Figure 13 Robert Darroll and Sean Reed: Oracle. Detail of video still-image.
An oracular statement encompasses ten scenes and lasts approximately eight minutes. Each of the individual, computer-animated scenes was produced in advance. Nevertheless, practically every oracular statement differs from the previous one since there are several variants for each of the ten scenes. Which of the variants is shown in each particular case is determined randomly. For the creators, the application of a random process follows the models of traditional forms of oracular consultations, such as the I Ching. Given the fact that Oracle implements random process, it follows that the installation is, technically speaking, not interactive. The questions that the visitors pose to the Oracle have no influence whatsoever on the answers that are given. Yet visitors instinctively attempt to establish a relationship between their questions and the ‘answers’ offered by the Oracle. Interactivity is thus intentionally simulated here. This simulation of interactivity is part of the basic concept of the work.
However, the question of interactivity, or its simulation, is not the only topic at the centre of Oracle’s concept. It also addresses the issue of ‘seeking and finding truth’. In this context, the irrationality that forms the basis of traditional oracles is compared with modern faith in technology (ZKM n.d.-c).
Oracle is a production of the Institute for Visual Media and was created with the support of the IMA. It is a work for the PanoramaScreen developed by the Institute for Visual Media.
2.14. Sonoloop (Götz Dipper, 2008)
Sonoloop (Movie example 13) is even more directly based on the concept of a sequencer than Pattern Machine and Random Machine. Unlike Pattern Machine and Random Machine, Sonoloop does not transform the melodies entered by the user; they remain unchanged. Instead, the user has more freedom in shaping the melodies and resulting sounds. The visual component functions as a graphical score (figure 14). A cursor continuously travels from left to right across the surface of the score, causing the objects found on the surface – circles, triangles etc. – to sound. Users can move these objects freely with the mouse, thus producing their own sound sequence.
Figure 14 Götz Dipper: Sonoloop. Screenshot.
Sonoloop is an example of an installation with differentiated learning curves, as was addressed in section 1.1. Its basic functions can be grasped intuitively, allowing users to implement them right away. After further interaction with the installation, the user quickly encounters an expanded set of functions. For example, objects can be created and removed, and their form and size can be modified; or they can be assigned different sounds.
2.15. Other works
The installations presented in this article implement the concept of interactivity in diverse ways. The various approaches span from installations that are intended to be played as an instrument, to environments into which the visitor is integrated by means of camera tracking, to conceptual works that motivate the visitor to reflect upon interactivity itself. With their focus on interactivity, these works fit in excellently with the ZKM’s overall concept and can be seen on both a regular and permanent basis in the exhibitions of the Media Museum.
A number of other installations produced with the support of the IMA have not been discussed here because they do not possess the feature of interactivity emphasised in this article. These include the following installations:
• en face (Gerhard Eckel, 1993)
• Stele (Gerhard Eckel, 1998)
• Bedlam (Robert Darroll and Sean Reed, 2005)
• Wsystko Juz Bylo/Alles gab’s schon mal (Lidia Zielinska, 2006)
• Klang-Stücke (Götz Dipper, 2008)
We should mention one final installation that is currently in production and will be shown for the first time in August of 2009. Zachary Seldess is currently working on the installation A Head of View, which constitutes another example of the category ‘(computer) game’. The visitors are surrounded by four projection screens and a circle of loudspeakers. They move around as avatars in a virtual 3D landscape, where they encounter a broad variety of sound objects and sound landscapes.
