The Future of AI Music & Instrument Making

NNA POLSZAJSKI

Hello old friend. Should I do a few little toots? Like little warm-up toots?

[uplifting trumpet sounds]

I love playing my trumpet. I love the sensation of creating sounds that can fill a huge space. And as an engineer who's obsessed with making, I love being totally at one with a delicately crafted metal tool that rests on centuries of human creativity and innovation. If you took this instrument away from me and instead handed me a computer, I would be pretty disappointed.

[sad trumpet sounds]

Ha ha ha, is that too silly?

I'm Anna Polszajski and in this episode of Create the Future, I'm going to interrogate my reservations and consider how digital technology and AI is transforming the world of music. How has the music we love already been influenced by digital tech?

TOM FOX

If you think about how many genres of music there are now, you can't count them. Technology has allowed people to access new ways to make sounds and new musics.

ANNA POLSZAJSKI

What do we risk as new software takes us away from beautifully crafted physical instruments and into the nebulous world of AI?

LEX DROMGOOLE

I think we’re going to have to be very disciplined about it otherwise it’s just going to veer off and we can potentially lose all of the things that enrich us as a culture.

ANNA POLSZAJSKI

And is it really good for us to be able to click a mouse and compose an infinite number of tracks in mere minutes and in total isolation?

LEX DROMGOOLE

If you don’t see it as a tool to be creative, if you start fearing it straight away you’re not going to be able to use it.

ANNA POLSZAJSKI

I'll be joined by two music experts who are already grappling with these very questions. Lex Dromgoole, a recording engineer who spent decades in the studio with artists like Madonna and Kanye and is now building a new music software called Bronze. And Tom Fox, a craftsman, maker and real life design and technology teacher, who builds physical instruments that incorporate digital technology. Tom, let's start with you, tell us a bit about your work.

TOM FOX

So my name is Tom Fox, I'm the Head Of Design Technology at Beechwood Park School. I also run a group called Hackoustic. We've been running music instrument maker events in London for the past 10 years. I also work for Music Tech Fest, who are like a European wide community of instrument builders at the forefront of music tech innovation. And I'm also a maker and artist myself. So I've got my fingers in a few different pies.

ANNA POLSZAJSKI

Amazing. So is it true to say that you're our kind of hardware, digital hardware expert for this conversation?

TOM FOX

I'm both, I build acoustic stuff and digital stuff and I code and I try and learn as many different things as I can just to squeeze as many different skills into new projects.

ANNA POLSZAJSKI

Amazing. I love it. Thank you. And how about you Lex?

LEX DROMGOOLE

So yeah, I'm founder of Bronze, which is a new technology that allows artists to experiment with animative music. And I'm also a record producer and my background is in composition and music production.

ANNA POLSZAJSKI

Okay, so you've been in the music game for a long time.

LEX DROMGOOLE

Yes, I mean, originally starting more in the kind of classical world and yeah, electroacoustic composition world, but then kind of veering more into popular music and then ultimately trying to combine the two in hopefully quite interesting ways.

ANNA POLSZAJSKI

So let's start by setting the scene. We're going to be talking about potentially some futuristic topics today, but of course, digital technologies have been with us for a long time and they haven't escaped the notice of music and musicians already. So how has digital technology already influenced instrument making throughout history?

TOM FOX

This is quite a fun one because it's, when I talk about this kind of stuff I do and I explain, I make sort of crazy instruments and they go, what, like a Theremin? I'm like, well, no, because that, that was already over a hundred years ago and technology has moved on a bit since 1920.

ANNA POLSZAJSKI

Right.

TOM FOX

I've got some amazing old books where they talk, they've got like music, like instruments categorizations and there's a tiny snippet about electronic music. And back then they were like, it's not going to take up. Don't worry about this. This is just noise. They were afraid of the new instruments back in the twenties and thirties when this stuff was starting to creep in. In the thirties was like the Hammond organ, which was still very early on, that's still used nowadays. The sound of a Hammond organ is just, it's incredible and hard to replicate.

ANNA POLSZAJSKI

What does it sound like?

TOM FOX

Think Stevie Wonder, a lot of his stuff.

ANNA POLSZAJSKI

Ok. Nice.

TOM FOX

I think he used the Hammond quite a lot and a lot of like prog rock stuff used a Hammond organ because it's not like a church organ, it's an electric organ and it uses like magnetic tone wheels instead of like synthesis. But that was back in the thirties. These instruments as they've been built, they've spawned new genres around them. So a lot of music you hear nowadays are because of instruments that were built 50 years ago, 60 years ago. There's an amazing article I read recently about tons of reggae and roots music, which was based around one specific drum beat on a Casio keyboard, which was developed by a Japanese woman in Tokyo working for Casio. And so she programmed this drum beat for their keyboard back in the eighties and she had no idea it would go on to spawn this entire genre of like incredible music. It was an amazing thing. Yeah. I mean, if you think about how many genres of music there are now, you can't count them because there are so many. And a lot of that is because the technology has allowed people to access new ways to make sounds and new musics. It's quite fun that even from the beginning, the musicians were quite scared of what was going to come.

ANNA POLSZAJSKI

Yeah, so this idea of reluctance to adopt new technologies as one, but then on the other side, kind of embracing new technologies as well and seeing where the technology leads you creatively as well, as you say, spawning new genres. Lex, can you tell us a bit more about Bronze? What does it aim to do, and how does it relate to AI?

LEX DROMGOOLE

AI is really only one kind of aspect of Bronze really. So what we're attempting to do with Bronze is actually change the medium through which recorded music is experienced predominantly and to do that through streaming, through gaming, and through all the other areas in which we experience music. So it's more about a kind of new experience of music that's kind of rooted in, I would say, generative music concepts, but kind of moves beyond those. And AI is just one small aspect that facilitates that or helps us facilitate that. So it's actually not really a kind of AI product. It's a product that uses AI to facilitate a particular creative goal. Obviously everyone's talking about AI now and you know, there's lots of current sort of discourse about all of the first emerging kind of mass market tools. In music, maybe somewhat controversially, I would say that most of the things that we've seen in music so far are actually maybe not the best examples of what it can do. There are many other applications of this technology in which it could really be quite beneficial if we guide it in the right way. One of my favourite books is a book by Richard Sennett called The Craftsman, which is actually about the craft - and actually, you know, a lot of my practice in making records is about the craft of record making, and obviously Tom's practices around the craft of instrument building. We actually need to approach the applications of AI in a way that honours that. So we need to think about building interfaces, particularly, creating a sort of human computer interaction framework around these systems that actually makes them feel more like instruments.

ANNA POLSZAJSKI

Tom, can you give us a bit of a picture of the sorts of creations that you make? Because it's not purely digital as it is, it's interfacing with physical stuff as well. Yeah, paint us a picture.

TOM FOX

So I started off with purely physical things with instruments out of like recycled materials and using, like, electronics to be pickups. Like, often I would build an instrument out of just what I could find. Sometimes I had no idea what sound it was going to make. So I would put it together and pluck it and go, oh that's kind of fun. Then learn how to play it. But what I found is when I started doing events and I would bring these creations to events, I’d say “come and play on this thing” and they would be, “oh no, I'm not a musician”. And so they would back off and say, they don't want to play it. And so that's when I got more into the technology side of it and finding new ways for people to interact with the instruments. So I got more interested in using microcontrollers and sensors and finding new ways to have the audience or the people at our events to play these instruments without any knowledge of being a musician. So they could either just touch some things and it would automatically play the right notes, or they would change the shadows on a sensor to play different MIDI notes. I use lots of gesture sensors. So I've got a fun one. It's behind me somewhere. It's just a, it looks like a log, but inside the log, there's an accelerometer. So if you pick up the log, you start playing music and it's a lot of fun ways to interact with it. That's, yeah, that's what I focus on now is more the fun ways to interact with interfaces and music and combining the physical and the digital.

ANNA POLSZAJSKI

I think that's actually a really lovely illustration of how music is so much more than just a thing that we listen to. It's something to participate in as well. And digital technology can actually open up new routes for people to get involved in that. Does that ring true for you, Lex?

LEX DROMGOOLE

Yeah, I think, I think obviously all of like, all of our kind of early experiences of music as a culture were participatory. So we talked a little bit earlier about instruments, but maybe we didn't talk about the transition from acoustic and kind of physical instruments to the kind of digital realm, because it's quite an interesting trajectory. Obviously we had early kind of digital synthesis and then we obviously had digital recording. But actually the digital technology that we now use to create music, a lot of it's just simulating acoustic instruments, which to me isn't particularly interesting. There's a huge opportunity to completely rethink the kinds of interfaces that we use to interact with digital systems. So particularly within computers, and I think obviously Tom's exploring this a lot with interfaces, there's a lot of skeuomorphism in the way people have thought about instruments throughout this kind of physical to digital transition. And a lot of the time we're still rooted in the kind of old world model of it when actually we need to completely rethink how these interfaces work and how we interact with them.

TOM FOX

And there is a lot of that stuff going on. There's a piece that is a microcontroller called Baylor and the team behind that run by the incredible Andrew MacPherson, who's a professor. They used to be at Queen Mary's University, but now they're at Imperial. And this microcontroller has been designed specifically with audio in mind. So there's basically no delay or latency in any kind of audio processing. And you can build just insane things with this stuff. Any sensor you can even possibly imagine can be used with this board to create synthesis or control MIDI or whatever you want to do with it on the other end. So there is some really amazing like high-end research still going on with instrument design.

LEX DROMGOOLE

I mean, Andrew's magnetic resonator piano is still one of my favourite kind of augmented instruments. It takes a completely acoustic instrument like a piano and then gives you an extended set of gestures that manifest acoustically through the piano, which I think is like a really, really amazing and kind of magical way of doing that because it's not trying to sort of combine digital sounds with acoustic sounds. It's actually using gestures as a way of creating an extended language of acoustic sounds. More people should be experimenting with that. And also the frustration for me is that many of these things that really incredible developments which are being created predominantly in academia aren't really filtering through into instruments that musicians actually use.

TOM FOX

On the point that Lex was just making, what Hackoustic, the group I run in London, has been doing for the past 10 years is showcasing the instrument builders at events. We'll have people either bringing the stuff they've made and they can either just perform with it or talk about it or demonstrate it. We've had like well over a hundred different artists and makers in the past 10 years, showing off what they've made. And some of them are like high level people doing it for a living. Then we just had like people who built something in their bedroom and they don't quite know what to do with it. So we like, we get them down and we get them to show off in front of an audience.

ANNA POLSZAJSKI

That sounds like a very cool and very nerdy show and tell. I'd be well into that.

TOM FOX

I mean, it is, but it's also really fun. Because of our events, people have collaborated on projects because they found out they've got similar interests. A couple of people come through our events and they signed up to be students at the Augmented Instruments Lab and now they're doing PhDs.

ANNA POLSZAJSKI

Oh wow.

TOM FOX

Because they didn't realise that that was out there.

ANNA POLSZAJSKI

Amazing.

TOM FOX

So yeah, it's been a really fun sort of networking thing as well as just you get to meet cool people doing cool stuff.

LEX DROMGOOLE

I would actually say as well Tom, I think there's kind of a sort of deeper, deeper kind of aspect to that as well, which is that almost all instruments that endure, have, they kind of emerge as part of this kind of reciprocal relationship between musicians and makers. Often they're the same people, but often, in other times they're actually, you know, it's actually a dialogue, it's an ongoing dialogue and the reason that these instruments emerge in a particular way is because the process in which they're built, it relies on this feedback between musicians using them in practice and then the people who making them, refining them. When I started at Olympic Studios a long time ago, the tech department there had four full-time techs. At the time, there weren't really enough things for them to be fixing. I would argue that most of those techs spent as much time creating new recording equipment for us to try out in the studios as they did repairing the things that we needed repairing. And what was wonderful about that is that they would come up with an idea or maybe there was an idea that was just kind of very loosely defined and they would bring it to us and we would use it and we would give them feedback and often amazing things would come from that. And actually if you look at the history of recording technology, that's how almost all of the kind of amazing things that we use today were established and I think it's the same with instruments.

ANNA POLSZAJSKI

Yeah, that definitely rings really true to me. I'm a trumpet player and I've spent time with trumpet makers, but the trumpet maker that I've spent time with is called Andy Taylor and he always works one-on-one with the musician. They might come to him not with an instruction about the instrument, but more an instruction about their sound. I want to sound bigger or I want to sound, I don't know, livelier or whatever. They'll kind of use these sorts of feelings-based terminologies, or they want to feel more powerful when they're playing, and Andy has to render that in brass and solder. And it was really fascinating to hear him talk about the decisions, the material decisions that he has to make based on those requests, you know, what alloy do you choose? What extra weighting do you add? What shape do you make the bell to be able to create that sound for them?

LEX DROMGOOLE

That's kind of an amazing example of it, this idea of the luthier or the craftsman having this dialogue with musicians. But interestingly, it also kind of strangely extends into architecture sometimes where the particular room or the particular space in which music emerges can have a very profound effect on the way that we play and the way that those rooms are designed. It's quite interesting, you hear records like some of the records that Bill Withers made, and you can hear that like the way James Gadson's playing drums, the kind of rhythmical feel that they have can only kind of emerge from a particular acoustic environment. And interestingly, when we moved into recording studios that were very controlled and very tightly defined, a whole new sort of like culture of playing came out of that as well.

ANNA POLSZAJSKI

Super interesting. So as an instrument maker and as therefore a music maker, Tom, your work seeks to shed new light, I suppose, on elements of feeling, elements of being human. Can you give us an example of a piece of work that forms, I guess, a kind of collaboration between these technologies and the players?

TOM FOX

I've played a lot with data in the past and live data. One thing I did is I wrote some code that looked on Twitter for any instances of the words ‘hope’ or the words ‘fear’ and every five seconds, it would count how many times hope was mentioned and how many times fear was mentioned. And if the world was more hopeful, be ascending. And if the world was more fearful, it would be descending. So you could constantly just hear if the world was feeling better or worse. And it was a really nice project until Elon Musk removed all the API access and then I had to scrap 10 years of like developing it. And, yeah, that was fun.

ANNA POLSZAJSKI

So what was the dominant feeling then on Twitter?

TOM FOX

It was always hope, but because hope can sometimes have bad connotations. You can hope harm on people. And there was quite a lot of that happening as well.

ANNA POLSZAJSKI

That's so impressive, but also kind of sinister as well. I love the idea that music can respond to our digital lives and it completely flips the dynamic actually of how music is connected to new technology. One question that I really wanted to ask you both was one that I was thinking about in terms of instrument design and sort of interface between making and new possibilities with AI is this: The Stradivarius violin is often seen as the pinnacle of violins. It's seen as the perfect shape. It hasn't changed for hundreds of years. It's sort of the one that now almost every violin is a replica of. Would AI be able to design us an even more perfect violin?

TOM FOX

I don't think, I’m going to chime in on this one. I don't think so. In the history of violins, if you look like really far back, the designs of them, some of them are just bonkers. The shape of the violin tended to settle on how it was partly due to fashion and trends and stuff like that. So the shape, it is a very elegant shape and they've done lots of studies on it being like a really perfect shape, but they weren't always that shape. They've done lots of double blind tests on it as well. So, unless you've got like an incredible ear, then it's going to be really hard to tell the difference between a carbon fibre violin or 3D printed violin and a proper Stradivarius. They've done these tests. And the other thing with the Stradivarius ones is the wood from those violins was from a region of Europe. It got hit by a particularly cold winter. And so when the trees grew, the rings on the tree were closer together. So that changed for actual sort of sonic resonances of the woods. And that gave it a much richer tone in theory.

ANNA POLSZAJSKI

Interesting. I heard it was also some of the glues that they use or the lacquers as well, but yeah, very small subtleties of the materials.

TOM FOX

I don't think AI would be able to just because, well, AI can't pick up a hammer.

ANNA POLSZAJSKI

Yeah, okay. Yeah, that's true. Not yet.

LEX DROMGOOLE

It's a difficult question because it depends what you're aiming for and ultimately I don't think there's any better, any better gauge of whether you've, you've achieved that than actually experiencing the sound of it yourself. And ultimately that's a quality, that's not quantity. If you can't really kind of define what it is other than maybe someone with supposedly golden ears telling you whether it's the greatest violin ever made, it's difficult to say. Obviously, there's a big trend in AI for finding new materials, being able to create like structures that we haven't thought of before. So it may be that it produces a material that might be incredible, just because it's faster at going through all the permutations of the relationships between those different particles. So my answer is it might create a material that allows us to make an even better violin, maybe, but maybe not.

ANNA POLSZAJSKI

Love it. Thank you.

TOM FOX

Also depends if the player is any good.

ANNA POLSZAJSKI

That’s true. That is a huge component.

LEX DROMGOOLE

Of course. It mostly depends on if …

TOM FOX

The incredible violinist could make the cheapest violin in the world sound like a Stradivarius.

ANNA POLSZAJSKI

You're absolutely right. And that's the joy of it, right? Is we can never divorce it fully from the humans. I want to move on now to the fears that surround digital technology and AI in relation to music. Is it something that we should be super fearful of?

TOM FOX

I can see both sides of the fence. I really wish I could remember who I heard this quote from. I heard it on the radio, somebody talking about AI and saying AI is like the weather. It's coming, you just got to dress appropriately. There's nothing you can do about it at this point. Wear the right shorts or an umbrella, whatever. If you don't see it as a tool to be creative, if you start fearing it straight away, you're, you're not going to be able to use it, you're going to get stuck straight away trying to fight it because it's going to be really hard to actually stop the oncoming rush of it just being everywhere. It already is everywhere.

LEX DROMGOOLE

I would take it a step further and say that we not only need to dress appropriately, but we also have an obligation as a culture to try and guide this in the right way. Because I think if we don't, if the right people don't step up and try and guide it in the right directions, we just end up with the prevailing winds of capitalism. It always baffles me that people think that we want shortcuts for creating music when every one of the artists that I work with wants to spend 18 hours a day in a recording studio. We actually love the process of making music because we find it enriching. We love the process of learning to play instruments. We love the processes of engaging with these things. I think we're going to have to be very disciplined about it, otherwise it's just going to veer off and we can potentially lose all of the things that enrich us as a culture. I mean I wrote an article recently in which I compared the outputs of these kind of very big commercial AI models that are creating music to ultra-processed food. They may well cause the same kind of ailments to us as a culture intellectually and culturally that ultra-processed foods have done for us physically. We really can't just go out there and think that sticking a whole load of existing music into a machine and then experience the output of it is going to be, we can't say for sure, or we shouldn't just assume that that's a good thing to do.

ANNA POLSZAJSKI

I think this aspect of the humanity or the kind of the human endeavour, the human meaning of music is really pertinent here. I think that's maybe the point of it. What is it that we value from music?

TOM FOX

Yeah, I mean, yeah, it should be used as a tool. It's not a shortcut to just generate bland music that no one's going to listen to. Music is about communicating and expressing yourself. You're not a computer. A computer is not going to give you a human expression. My hope at the moment is that we're in a sort of a bubble where it's all exciting and lots of stuff is happening and look how cool this stuff is. And then people are going to get bored of it and keep the bits that are useful and just, it just becomes fairly monotonous. And yeah, we just get rid of all the shiny glittery things that are catching people's attention, making money for no particular reason. It can still be used as a tool, but it's not the end goal. It's not the whole thing, it should be part of the process.

ANNA POLSZAJSKI

I think that leads us really nicely into the final part of this conversation really, which is to continue your analogy about the weather. How do we prepare? How do we know what shorts to wear and what umbrellas to take with us? What should listeners, what should we all be doing to prepare ourselves for what's ahead?

TOM FOX

That's a fairly tricky one for me to answer, just because a lot of the stuff I do is very physical-based, it's all about hardware interaction and sensors and building instruments that make physical sounds. Like I said, AI, it can't yet pick up a hammer. So I'm not that concerned with my particular work. One thing that did sort of ruin a big project of mine was Elon Musk. He sort of, he took away access to data that I was using to compose things and that was really annoying. So less of him would be nice.

ANNA POLSZAJSKI

Or maybe greater access to data for all might be another one.

TOM FOX

Yeah, definitely.

ANNA POLSZAJSKI

Democratisation of data.

TOM FOX

Or both. Yeah.

LEX DROMGOOLE

That'd be good.

TOM FOX

Lots more open source. I'm a big advocate of open source data and open source knowledge. And so more of that would be wonderful.

ANNA POLSZAJSKI

And what about you Lex? How should we learn from you what umbrellas to take with us?

LEX DROMGOOLE

I think the first thing is really choosing not to use the most pernicious aspects of it. Just say, well, I'm just not going to use that. If you feel that automatic music creation isn't really what we should be doing, then just don't use those tools. The second thing is maybe a little bit more of a tangent, but I think we need to look back and like, honour the kind of history and culture that all of these great things that we now, that we already have, have emerged from. And I think, I think instruments are a great example. I think recording technology, analogue recording technology, is a great example. We need to actually have more reverence towards them. Many of the things that we could design in the future could be made in the image of many of the kind of triumphs of the past, if we pay attention in the right way, not by just mimicking them, but actually by thinking in the same way about what it is that we want those things to produce and how we want them to become part of a process of imagination.

ANNA POLSZAJSKI

That's one of the key things that I've taken away from this conversation is, it's an evolutionary step that I think we're witnessing right now. And it's really interesting to see how that step is just a small one at the end of a very, very long history of the development of music and how music has shaped the people that play it, the people that listen to it, the meaning that it has in our society. We've been here before, obviously, you know, new technology is not new. That's what we do as humans. I really like how you framed it as a tool in the same way that a handle on the end of a rock was once a tool. And that led us on to new and interesting things. So yeah, you've left me feeling very hopeful. Thank you.

I really hope we don't forget the importance of classic instruments and the experience of playing a handcrafted trumpet or a perfectly designed Stradivarius violin. Tom's idea of preparing for an AI future like you do the weather makes so much sense to me. And Lex is spot on as well saying that any new software should be about more than just replacing what we've already got. It should allow for new types of imagination and creativity that enhance our experience of one of life's greatest joys. So I definitely won't be putting my trumpet down in favour of a screen anytime soon, but I think I am going to try to be more open-minded when it comes to the future of music, tech, and AI. I hope this conversation has made you feel the same.

You've been listening to Create the Future, a podcast from the Queen Elizabeth Prize for Engineering and Peanut & Crumb. This episode was presented by me, Anna Polszajski and was produced by Jude Shapiro. It featured Lex Dromgoole and Tom Fox. To find out more, follow QEPrize on Twitter, Instagram, and Facebook. Thanks for listening and see you next time.