The most interesting impact of digital media on sound artistry is surely the way it has changed our compositional mind. I believe it has pushed composers to manipulate musical material in innovative ways, especially in response to video editing. In particular, the impact of video techniques that relate to ‘compositing’ and what may be their musical equivalent. Compositing is the process by which two or more images from different sources are combined so that they appear to belong in the same visual frame and sequence. So, for instance, the compositing technique of ‘chroma-key’ allows a film director to mask out portions of the image and substitute for them entirely different ones such that an actor can say their lines against a green background in which later the audience will see the Grand Canyon, or entire objects in a room can be selectively masked in green and their texture substituted in digital post-production. Transpose this idea onto music creation and the ease with which we can drag, drop and merge audio onto the blank canvas of our digital audio workstations (DAWs) is as much compositing as it is composing. In terms of inspiration, I also believe that our continuous exposure to visual work that freely appropriates and re-presents borrowed visual sources has permeated our compositional imagination – and by visual work, think beyond ‘art’ film to commercials, the MTV of the 1980s and 1990s, and now YouTube and other music video online networks.

Thanks to this omnipresence of visual media, a non-linear way of thinking about musical material has become second nature to us, even allowing composers to think of music as ‘visual sound scenes’ that are edited together. The latter makes even more sense in the case of composition for audio-visual media, as compositing sound to visual sequences is often a task of finding meaning in musical building blocks that can then be freely mixed to picture. This is something I learnt from my work in music for commercials and narrative film in the 1990s. Film and video directors taught me through their briefing how to respond to the language of the camera beyond traditional musical syntax. They also encouraged me to cross the boundaries between music and noise or music and the sound effect. This knowledge equipped me to deal with the fluid nature of visual narrative, where the estimation of the length of a shot is largely subjective and not bound by quantised musical beats. Where the composer often thinks of time in proportion to the metre, the director thinks in more subjective and durational terms. When composers do the same, scenes can become flowing and intuitive, and long enough to convey the intended meaning and emotion. The moving image becomes a terrain in which composers are architects designing houses, villages and cities of sound. As composers we then work with the emotional topography of the image; we carefully build, either for contrast or for similarity. The result must make sense for the terrain, be proportional, allow space for the sound, and be modular and adaptable like a new media object. Where this once was solved by symphonic textures on a rhythmic scaffolding, it is now approached through the remix mentality that pervades our culture. The building blocks of sound are arranged on the computer screen with irreverence towards their content: the soundbite cohabits with the romantic orchestral ‘wild’, and sounds of machinery with dubstep synth bass wobble. Not to mention the syntactic use of location sound: whether anticipating scene changes, lingering long after the cut, or wholly unrelated to the visuals, it often assumes a quasi-musical role. Composited, these blocks of sound form a semantic network alongside the visuals they shadow.

Enabled by digital technology, this compositing approach has emerged in my work as well as that of my contemporaries, and it helps solve two practical challenges that have remained constant for me in the twenty-seven-odd years of my creative practice. One is refitting an existing and arguably successful score to a new image edit from the director, usually due to small changes in the story. The other is understanding enough about video/film editing to make sense of its compositional language: what does a fade-in/out mean for the narrative? A close-up? The angle of a shot? If we rethink music for picture in terms of compositing, both these challenges are approached in a new way. As implied earlier, a compositing approach frees us from the constraints of purely musical metre as regards the visual hit-point. Creating music for moving images becomes a question of remixing and timing, as we shift, blend, and re-sync sound objects along the timeline of our DAWs in ways we didn’t tend to treat our music scores. This allows us to create richer sound responses to changes in the video sequence, outside a traditional rhythmic model. On the other hand, our understanding of the compositional language of film can now be more freely echoed by the implied meanings of the sound building blocks that populate our DAWs, as they also fade in/out, pan across, establish a sound-shot, focus on a particular sound event or support a certain action on-screen.

If our creative approach has changed, our craftsmanship has also changed. In addition to traditional counterpoint and harmony, digital technology has forced us to go outside our disciplines and learn a variety of new skills. A musician or sound artist working for visual media needs to know about frame rates, transcoding, and video editing and production; which compressor and wrapper to use so that our DAW will efficiently play back video; how to program synths and samplers; how to audio edit to picture and orchestrate via MIDI. On top of this, we must produce realistic mock-ups that could be perceived as real, all at the whim of non-musical directions from film directors and producers. I believe we are the better for this changing practice enabled by new digital technologies, one step closer to whatever an audio-visual composer is meant to be and certainly closer to how musical creativity needs to work in an age of increasing digital collaboration.