Note: this is an edited rendition of a reflective work for my MSc in Sound Design.
Background
The majority of realistic sounds that we hear in media from video games to radio theatre are commonly created by capturing the source e.g., the closing of a door used as Foley in a film or theatre is the recorded sound of a real door. This is either captured by interacting with the door as a performance to the media or stored as a sample and added at a later time in an editing session (Academy Originals, 2016). In contrast, procedural synthesis is referred to as a method that creates the sound without first capturing the content using recording equipment.The term procedural audio is often used interchangeably to mean either adaptive sound or the process of designing realistic sound using synthesis (What Is Procedural Audio?, 2018). While the latter is perhaps better categorised as procedural synthesis it can also be referred to as procedural content generation or simply procedural sound.
This method of sound generation can be done manually or using an algorithmic program to produce the content as described by Andy Farnell in his book Designing Sound (Farnell, 2010). Different synthesis methods can be used to produce sound content such as subtractive, wavetable and granular where the generated output is often further randomised for variety and to allow for adaptation to events or interactions (Farnell, 2010, pp.318, 321).
In computer games, procedural content generation is an increasingly common tool for game objects and art (Shaker et al as cited in Game Dynamics, 2017, p.3), where it can be used to create highly detailed and realistic real-world simulations (Project Titan | SideFX, 2022).
However, for sound, this method of creation has not had the same uptake possibly, at least in part, because of the difficulties of producing assets of consistent quality. In a 2022 talk organised by AES Prof.Josh Reiss discussed the range of likeness to real-world recordings and concluded that the procedural model can come very close to the real-world example and almost indistinguishable for some sound types but for others such as thunder it is very difficult to model to a convincing level. (aesuksection, 2023, 34:15).
Related to this is the possibility that we as humans may react negatively to sound simulacrums something that was suggested by Francis Rumsey in an AES talk titled “Spatial Audio-Reconstructing Reality or Creating Illusion?” (AES Section Meeting Reports » Chicago – May 21, 2014, 2014.) where he discussed how artificially produced sound that is “close but not quite” could generate similar outcomes as the “uncanny valley” effect that exists in fields such as robotics and 3D (Robert Schulein – Binaural AV Recordings, 2017, 56:22)
However, creating sound using a computer algorithm can be very fast and also automatable and could therefore be a tool used in world-building concepts such as the “Metaverse’ where the sheer magnitude of sounds potentially needed may not be possible to provide using conventional methods such as recording the source (Vincent, 2022).
Process
In his book, Farnell discusses procedural sound creation using a range of methods for the process such as pure wave oscillators and granular synthesis as the fundamental origin of the final sound. The approach to this project derives from these methods but differs in implementation for some sound (see Table 1).
| World Sound | Method used for this project | Method used in Designing Sound |
| Fire | Transients from fire | Noise |
| Water | Sound of water drop | Pure Wave / Noise |
| Wind | Noise | Noise |
Farnell uses Pure Data as the creation tool, this visual scripting software similar to MaxMSP enables users to create sound, sound effects, or audio effects by combining graphical objects instead of writing code (47 • Miller Puckette • Max/MSP & Pure Data, 2020). As Unreal Engine 5’s MetaSounds environment also employs a comparable visual scripting method with objects referred to as “nodes,” I can apply similar techniques (see Table 2; List of Objects | Pure Data, 2024).
| Examples of MetaSounds components and their application | ||
| Node | Pure Data similar | Application |
| Trigger | Bang | Executes other nodes e.g., play/stop |
| Audio | tabplay~ | Plays an audio file e.g. Wav |
| bool/int/float | expr, int, float | Represent numbers or conditions e.g., value for a volume, or play condition. |
Farnell proposes a method of “Sound Hybridisation” (Farnell, 2010, p.309) utilising granular synthesis as a foundation, a sort of kernels of smaller sound to create a larger entity e.g., a sampled water drop for the sound of streaming water. Granular synthesis is often an experimental and iterative process, but using such an approach still allows for predictable results. This form of a curated process is something that is both discussed and heard in work by Curtis Roads and in an article analysing his composition “Half-Life” he traces his approaches to developing an organised compositional process using granular synthesis where much of the work is based on iteration but results in a method allowing for controlled results using sequencing and layering of individual sound objects or a bottom-up strategy as he calls it (The Path to Half Life, 2004; Curtis Roads – Topic, 2020).
For this project, MetaSounds granular delay (see Fig.1) was used as the algorithmic representation as it is an abstraction in the environment providing parameters and settings for granular outcomes but does not require a custom modular build. The reference manual states that it “performs delayed audio granulation on a given audio buffer by sampling it into “grains” and playing them after a set delay.” (MetaSounds Reference Guide, 2024). The use of this object was almost completely iterative, however, the articles on granular synthesis by Roads helped consider strategies for how to achieve desired outcomes (Articles, 2024).
Fig 1. MetaSounds Granular Delay with parameters to algorithmically adjust the outcome.
The sequential part of the system is made up of a trigger repeat object, essentially a metronome, using a noise node to control the timing of the pulses. These modules in turn drive a sample wave playback node i.e., playing back samples one at a time (see Fig 2).
Fig 2. Node trigger sample playback using random time derived from noise.
As the aim is to emulate a live sound source such as fire and water the Perlin noise node modulates the timing of the trigger and continuously randomises the flow of the pulses. This helps to better emulate the natural random behaviour of real sound and randomisation is also key in adaptive sound and a solution to avoid repetitions noticeable to the listener (Sound Design for Video Games | Nicolas Titeux, 2021).
In visual scripting applications such as Pure Data, the graphs can become very large and complex and it’s common to save a modular patch as an abstraction that can be inherited in other patches. (Pure Data, 2024). This abstraction type of sub-patch compartmentalises the process and makes it easier to manage and overview larger visual scripts. In MetaSounds this can be applied using child and parent graphs (See Figure 3)
Fig 3. The Fire graph child inherited in the Fire_Controller parent graph.
Reflection:
Concluding what has been learned from an iterative and experimental process can be difficult as often outcomes feed into the next step of a longer chain of events that ultimately results in the desired aim, or most cases, a satisfactory resemblance of that aim. However, by adopting a systematic method such as those discussed by Farnell, Roads and others I’m able to better reflect on the experience and how I can apply similar methods for other work. Some of the stages or steps in preparing for this type of project are possibly clearer than others.
| Decide on the working tool and environment: – does it fit the initial approach and the overall outcome both technically and aesthetically? – how well does it align with the field in general |
Choosing a familiar tool is often easier in the beginning but will stall the process if it needs to be altered later on. Pure Data/MaxMSP are excellent tools for generative sound creation but do not fit directly with a game engine. Investing time in learning MetaSounds is more productive in the long run as it’s already part of the Unreal engine. The game audio industry is not necessarily centred around any particular software, however, MetaSounds is the official audio system in Unreal Engine and is likely to persist long term (MetaSounds, 2024.)
| Create predictable and measurable steps for the iterative process. – create a starting point/s that can serve as a foundation for all the experimental processes. – separate functions that can be shared and inherited for other processes. |
By using short samples for the granular process I can simply swap the sample content for different aims e.g., water drops instead of single fire cracks. In addition to a more streamlined process, I can also analyse the effectiveness of this process against the outcome and determine its suitability e.g., how suitable is this method for fire vs. wind.
Separating functions such as the sound catalyst (trigger-noise-time) from the sound manipulator (granular delay) means I can similarly analyse the effectiveness better as the functions are decoupled and also be more effective as the functions can be shared as standalone graphs/objects. Decoupling is a common approach in programming as it simplifies changes and debugging (Decoupling Patterns · Game Programming Patterns, 2011).
| Use a template method to iterate on – use an established workflow/s as the foundation for a project. – gather the relevant thoughts and concepts for that method before starting the project but also to use as a reference during the iterative process. |
Working from a template like is set out in Designing Sound it becomes more about filling in the blanks rather than figuring out what the best approach could be. This also means that I can prepare both technically and conceptually before starting any work. This approach can also be used to test multiple creative solutions to a project, for instance, a sound design project for linear media could evaluate approaches like field recording vs. library based design to quickly prototype what solution best fits the aesthetic and overall narrative before deciding on what direction to take.
Key Learnings:
The malleability of procedural synthesis makes it a useful approach when creating audio for interactive purposes and for certain sound types such as fire the constant evolvement in the sound lends to the realism of such game or interactive objects.
However, the process of synthesising the patches is quite time-consuming and unlike sample-based sound design, there are currently no pre-made libraries to rely on in the process which increases production time. While this can be mitigated by using abstractions as exemplified in this project and in time it’s likely to presume that commercial libraries aimed at systems like MetaSounds will be developed this increase in design time currently makes it a less obvious choice for a commercial project.
As a personal project, it has many benefits especially the process of breaking down the anatomy of a sound to understand how to emulate its components I find it an excellent method for increasing my skills as a sound designer and as an audio professional in general. In the creation of this project, I’ve found aspects where my abilities can be strengthened and I’m relying too much on iteration. Setting aside time for these types of practise projects will certainly help in developing my own practise more broadly.
Resources/References
47 • Miller Puckette • Max/MSP & Pure Data. (2020). Future of Coding. https://futureofcoding.org/episodes/047.html
Academy Originals (Director). (2016, June 13). Foley Artists: How Movie Sound Effects Are Made. https://www.youtube.com/watch?v=U_tqB4IZvMk
AES Section Meeting Reports » Chicago—May 21, 2014. (2014). https://www.aes.org/sections/reports/?ID=1634
aesuksection (Director). (2023, January 20). Procedural audio worlds—Josh Reiss. https://www.youtube.com/watch?v=OJ5HJMRvmmc
ASPress—Designing Sound—Code examples for “Designing Sound” textbook. (2010). http://aspress.co.uk/sd/
Curtis Roads – Topic (Director). (2020, May 7). Half-Life, Pt. 1: Sonal atoms (1999). https://www.youtube.com/watch?v=8D5ObNnHgck
Decoupling Patterns · Game Programming Patterns. (2011). https://gameprogrammingpatterns.com/decoupling-patterns.html
Farnell, A. (2010). Designing sound. The MIT Press.
Game dynamics: Best practices in procedural and dynamic game content generation – Edinburgh Napier University. (2017). https://napier.primo.exlibrisgroup.com/discovery/fulldisplay/alma9923486314302111/44NAP_INST:44NAP_ALMA_VU1
List of objects | Pure Data. (2024). https://pd.iem.sh/objects/
MetaSounds Reference Guide. (2024). https://docs.unrealengine.com/5.1/en-US/metasounds-reference-guide-in-unreal-engine/
MetaSounds: The Next Generation Sound Sources. (2023). https://docs.unrealengine.com/5.0/en-US/metasounds-the-next-generation-sound-sources-in-unreal-engine/
MetaSounds: The Next Generation Sound Sources. (2024). https://docs.unrealengine.com/5.0/en-US/metasounds-the-next-generation-sound-sources-in-unreal-engine/
Project Titan | SideFX. (2022). https://www.sidefx.com/titan/
Pure Data. (2024). https://archive.flossmanuals.net/pure-data/dataflow-tutorials/abstractions.html
Roads, C. (2004). The path to Half Life.
Roads, C. (2024). Articles. Curtis Roads. https://www.curtisroads.net/articles
Robert Schulein – Binaural AV Recordings (Director). (2017, May 28). Francis Rumsey-Spatial Audio-Reconstructing Reality or Creating Illusion? https://www.youtube.com/watch?v=y82nth2Pnwk
Sound design for video games | Nicolas Titeux. (2021). https://www.nicolastiteux.com/en/blog/sound-design-for-video-games/
Vincent, A. (2022, January 31). What will the Metaverse sound like? Sound Object. https://www.soundobject.io/sound-of-the-metaverse/
What is Procedural Audio? (2018, February 25). Dara Crawford. https://daracrawford.com/audio-blog/what-is-procedural-audio
Video resources
Fire
Leave a Reply