What the future sounds like

Arguably the world’s first ‘omnidirectional sound environment’, 4DSound offers the chance to experience individual recorded sounds as independent physical entities. Its inventor, Paul Oomen, is only just beginning to understand its implications…

WORDS WILLIAM RALSTON

Situated in the east of Berlin, on the banks of the Spree, is Funkhaus, an enormous multi-purpose campus with a long and cultured history. Built in 1950 as the broadcasting base for GDR’s state radio, it’s a peculiarly chilling and rather desolate spot to visit nowadays. Head out of the main hall – itself home to a plethora of musical studios, performance rooms and offices – and you’ll find an assortment of weathered warehouse spaces, the majority of which are unoccupied. But behind one of those large industrial iron doors is the world’s first ever 4DSOUND system, the most advanced spatial sound instrument of its kind (the only other one of this scale is at Budapest’s Spatial Sound Institute). The system itself is an elegant and sophisticated construction, the product of more than a decade of dedicated research and development by Paul Oomen and his team of engineers. Each system is tailored to its space, but this particular one consists of 16 industrial metal cylinders, each about a foot in diameter and around 12 feet tall, distributed in a four-by-four grid on an acoustically transparent platform no bigger than a regular dancefloor. The room itself is just a little larger. The floor is grated for the purposes of the nine sub-woofers below. Inside each of these cylinders are bundles of speakers positioned at three heights: ‘below feet’, ‘head height’, and ‘overhead’. Off to one side is a large workstation, like a futuristic DJ booth, with a desk, and several monitors and amplifiers.

Intent on creating an integrated software and hardware system that provides an omnidirectional sound environment, Oomen began his research in 2007. In more simple terms, this means conceiving a fully immersive space where sounds appear as independent physical entities. This allows listeners to actually locate sounds and walk over to them. It’s something that must be experienced to be properly understood. And despite having spent a large portion of his adult life developing the system, Oomen says that understanding the full capabilities of his creation remains a work in progress.

Born and raised in Amsterdam, Oomen spent much of his youth in theatre performance. He began acting aged just 14 and enjoyed moderate success, performing in various local productions. In parallel with this was a burgeoning interest in music. He recalls ‘scribbling his own compositions’ aged seven or eight, albeit with an invented form of notation. At 18, he entered Amsterdam’s Music Conservatory, later studying at Berlin’s University of the Arts.

Though compelling, his studies of classical music felt like ‘a suit that never quite fitted’. His main focus was writing music for theatre, and his tutors encouraged him to produce his own experimental sound-based pieces and accept any commissions that came his way. It soon became clear that his most striking works were all composed with a particular spatial environment in mind. ‘It was always about more than just music,’ he says. ‘I would never just imagine a melody; I would always imagine the space where the piece would be performed.’

One natural progression was site-specific theatre, where the setting is integral to the production. Oomen had long wished to break through the idea of the passive spectator, and his growing fascination for the dynamics of spatial sound inspired him to take action. ‘I tried hiding speakers in corners and also using different sound systems,’ he says, ‘but from a composition perspective, I felt extremely limited.’ Only a more advanced sound system would allow him to properly sculpt the space for both audience and performers. ‘I just didn’t have the tools available,’ he says. And so he started to develop them.

An important catalyst in this decision was his fascination with Serbian-American physicist Nikola Tesla. Oomen wished to compose a work about Tesla’s discoveries in movement and energy – an ‘extremely ambitious idea,’ he admits. The essence of the piece was to use sound to show how energy moves through space. To do so, he needed a sonic instrument far more advanced than anything already available. ‘I wanted people to be able to locate sounds around them and physically feel them moving within the space,’ Oomen says. Drawing on his (and others’) knowledge, he sketched out an initial draft system.

But it wasn’t going to be easy. The obstacles could be sorted into two broader categories: finding a capable speaker system (and configuring a layout); and coding a sufficiently advanced software interface to allow him sufficient control over the sounds.

Oomen’s first step was to consult some talented friends in the Music Technology department of Utrecht’s School of Arts. He detailed his intentions before hiring a team of programmers: Poul Holleman, Luc van Weelden and Salvador Breed (all of whom remain involved as partners). After just 12 months in development, the team had settled upon the software and conducted their first tests using a grid of 27 regular PA speakers. It worked ‘surprisingly well’ Oomen recalls, but there was much more work to be done. For instance: traditional PA speakers point at the listener, and so the angle of projection is narrow. But for the sound to feel like it was coming from everywhere, they needed speakers that projected much more widely.

There weren’t too many options available. The most advanced developments back then were in academic contexts and not suited for creative application. In the light of this, Oomen started to design his own speaker that would disperse sound equally over 180 degrees. His work was cut short when he connected with Leo de Klerk, founder of Bloomline Acoustics and another leading spatial sound enthusiast, who had designed the world’s first fully omnidirectional speaker and was preparing to patent his invention. It was more advanced than anything Oomen had developed, and so he adopted it as one his system’s main components. ‘It was the missing piece of our jigsaw,’ he explains. After saving up for more than a year, in 2010 Oomen placed an order for 48 omnidirectional speakers.

With speakers in hand, in 2011 Oomen and the team rented a large Amsterdam warehouse to begin configuring the system. The plan was to distribute the speakers equally through the space in the same rectangular grid as before. Desite warnings that the balance of sound would never be quite right, Oomen proceeded and was ‘pleasantly surprised’ by the result. The ensuing period was then spent refining the technologies and learning how to use them. ‘It was purely intuitive – and involved a lot of trial and error.’ After a year of research and development, in September 2012 Oomen presented the Tesla production – titled NIKOLA – which he describes as the most nerve-wracking thing he’s ever done.

By this point, however, Oomen’s eyes had moved onto bigger things. NIKOLA’s success highlighted a wealth of new creative opportunities, few of which even Oomen could have anticipated. Rather than just a tool for composing sounds and adding new textures, he had created an entirely new instrument in itself – one that allows artists to explore new performative disciplines and express their ideas with a new dimensionality. ‘Only then did it become clear just how much potential the system has,’ he explains, ‘and I wanted to begin exploring what was really possible.’ Interestingly, his explorations have long been privately funded; Oomen accepted some support for the Tesla production but cut all ties thereafter in favour of financial independence and operational flexibility. ‘At this point, I was completely obsessed with the idea,’ he explains. ‘I had some capital myself that I had made from earlier productions – and I invested all of that in the further development.’ Any additional outlays have been covered through bank loans.

The system entered the more conventional electronic music world in October 2012 at Amsterdam Dance Event, where it featured music composed by Peter Van Hoesen. There have since been numerous other artist showcases, but this has given rise to its own obstacles. The system requires artists to learn an entirely different medium of communication –a whole new language, if you will. ‘Artists have to begin by exploring, sound by sound, what you can do with the system,’ Oomen explains. ‘It’s an instrument and it requires you to learn it as such.’ For these reasons, Oomen is particularly excited about working with younger artists. ‘If you want to make a meaningful production you have to start from scratch,’ he says. ‘Established artists can deliver great content but they already have an ingrained approach for sound.’

Questions must now be asked about where 4DSOUND can – and will – go. The technology has advanced well beyond our understanding of it; there isn’t anyone in the world proficient enough to exploit even a modicum of its capacity. ‘When people learn to play the piano there are centuries of tradition on how to cultivate skills on such an instrument,’ Oomen explains. ‘There is interesting research into the exploration of space, but we are really dealing with an entirely new instrument. It’ll take decades for us to properly comprehend the possibilities.’

Nonetheless, Oomen is convinced that those will never take shape in a club context. Not only does 4DSOUND require an entirely different way of performing – much more than just playing records – it also relies heavily on the listener. Spatial sound demands near silence so as to allow sounds to drift to the edge of perception and return; the experience will never work in situations where people are used to talking. It’s similar to cinema: attendees must abide by certain conditions to maximise the experience. ‘This doesn’t exist with spatial sound yet,’ Oomen explains. ‘Instead of bringing the system into contexts where there is a regular way of listening, we need to create a new context, or a new form of presentation, to make this really come to life.’

Important in this endeavour is Monom, the Berlinbased collective headed up by Australian William Russell. With the help of an angel investor, Zak Khutoretsky (aka DVS1), Russell purchased the original system and installed it at Funkhaus in July 2017. Much of the focus since then has been on electronic music production and performance, whereby established producers spend some weeks learning the system before performing in front of a live audience – but, Russell explains, ‘this is for any artist to explore’. Classical musicians have already utilised the system to great effect. ‘It’s not just about music; it’s about emotions and harnessing the way we have evolved to perceive sound and space, harnessing emotional responses inscribed in our ancient memories, to deliver us to new states,’ Russell adds. There are plans in place to explore classical and transcendental applications such as healing and meditation. Meanwhile, Oomen and his team at Budapest’s Spatial Sound Institute are focusing on the broader applications of spatial sound outside of music, including the scientific, the architectural, and in media design.

These are extremely exciting times, Oomen insists. A growing global interest in spatial sound has facilitated developments and stabilised the company from a financial perspective. The team will also soon present some more mobile and flexible systems that will, of course, allow them to demonstrate the system much more readily. 4DSOUND has already started supplying various installations, not all of which are omnidirectional, using the software and the approach as tools to provide auditory experiences.

For now, the journey continues as interest grows. ‘We have built an extensive instrument that people are just beginning to explore,’ Oomen says. ‘It’s a huge development in the tools with which artists can express themselves.’

And it’s quite possible that our understanding of it will only become clear in the many decades to come. Watch this (4D) space… 4dsound.net