Interview with CJ Carr and Zack Zukowksi

Hi! Please tell us: who are CJ and Zack?.

CJ Carr

Zack, you go first.

Zack Zukowski

I'm Zack Zukowski. I am a music technologist and machine learning researcher, and we make art with Dadabots. CJ, go for it!

CJ Carr

CJ from Dadabots. I'm a machine learning researcher, allround security hacker, blockchain developer, web developer, signal processing specialist. I do work with natural language processing, EEG, deep learning,... Lots of skills all coming together. But primarily I'm a musician, a metal musician!

FAST45

How would you describe your relationship to higher arts institutions?

CJ Carr

Well, I guess there's music schools and there's art schools. Zack and I went to Northeastern University. I was studying computer science and Zack was studying music technology. Across the street from the university was the Massachusetts College of Art and Design. I ended up hanging out there way more often than my own college. The people there were way more interesting. They had a lot more ideas that were way out there. And they were really good at creating communities around music events, around art events, things like Burning Man for example. That was really significant for me culturally, and it was something that I wasn't getting out of my own community of programmers. Zack and I met at the Berkeley College of Music, as interns, and we were developing the… What were we doing at Berkeley? [Offering the word to Zack]

Zack Zukowski

We were making games for kids to learn music theory. These were online fun Flash games. It was that time when Adobe Flash was going away and everyone was moving to HTML5. We were translating these games into something that was accessible on the web.

We also started looking into what we were calling ‘music intelligence’, using really basic algorithms to just understand more about music theory through the music itself. At that time we were also going to hackathons at MIT and other colleges nearby that were really nurturing this hybrid crossover of arts - technology - music - hacking into concise weekend projects. There you would just quickly come up with something outrageous and present that in a five minute pitch. We thought that this was a really cool medium. This five minute pitch was almost a little stage performance. It was a really cool place that was open to people who weren't even from the university. I feel like that was something that really nurtured our aesthetic and got us working together outside of the job.

CJ Carr

Oh, totally! Hackathon culture is what we're born out of. You know, computer science school didn't have enough art in it and art school didn't have enough tech in it. Both of these things were satiated at those hackathons, where art and tech were at the same level and people were just programming things just for the hell of it. Not to solve any big world problem, but to just tinker and to see what comes out. That kind of creative thinking was addicting. I've been at over 65 of these events internationally. And together, Zach and I have won over a dozen of these, including some notable ones like Music Tech Fest, Music Hack Day, …

Zack Zukowski

South by Southwest Music Hack…

CJ Carr

Yeah, the list goes on.

FAST45

Super cool! From your perspectives, with regard to both today and the future, what are the most impactful roles of artists in society?

CJ Carr

Hmm. Ooh, big questions…

FAST45

Yes!

CJ Carr

I guess the artist's role is more than just designing any product or designing any system. It is more about designing the human experience. Like: what do we want the human to be? What should life be? This is not a question in the realm of science. We can't set up an experiment to find an answer to the question ‘what is the best life one can live?’ This is something for philosophers to think about and for artists to actually enact.

Zack Zukowski

Yeah, and I was also thinking that, in our case, we've been noticing that you become a curator sometimes, In our case, we're making some kind of process that then makes the art. And so by being removed a little bit, you get to kind of hit it with a wider view. And, rather than being stuck inside one genre or one movement of art, you can actually play with the different movements against each other and take a more global look at what's being created. That's something that I really like moving towards. So for example, rather than composing a song by writing the notes, you can think of just the inspiration that gets you there and the blends of the outputs that you would like to see stylistically.

FAST45

What could this mean for authorship as a concept?

Zack Zukowski

I think it means that we're going to have to have a discussion about what authorship means, and what ownership means also. What are you allowed to mess with? What does it mean if in reality you’re just kind of pointing to some data somewhere? These questions are going to come in…

CJ Carr

Yeah, there's some controversy around ‘what is fair use in the context of sampling music for a remix?’ But if an AI model has been trained on entire discographies of music, can a music producer use content that is generated from that AI? And can they own it, or is it derivative work? It's a grey area.

Zack Zukowski

The way we're technically referring to it is that it's data. Data is not something that is cultural and alive. But of course, the art represented by the data is cultural and alive…

FAST45

What is the impact of art and creative work on your field of expertise, like machine learning, music technology, et cetera.

CJ Carr

One of our big objectives is developing these technologies to give to musicians. Other machine learning groups are mostly research focused, and they'll do a little bit of artist outreach. We on the other hand are working with musicians 80% of the time and spend only 20% in research and development. We saw there was a huge gaping hole right there that no one was really bridging. And, yeah, it was really, really fun for us to just jump in and be that bridge. You know, everyday we were reading the latest research papers, downloading the latest github repositories, tinkering with deep learning and neural audio synthesis and reaching out to musicians that we looked up to. Sometimes we would just cold email these bands, like, ‘Hey, do you want to collaborate? We have this new tech, do you want to make something that never existed before?’ And almost everybody says yes!

FAST45

In your opinion, what are the most influential trends and developments in society, economy and in the arts?

Zack Zukowski

Many people probably mention NFTs when you mix society and economy. That's the one big trend that people are talking about a lot. And I think that there's going to be both good and bad that comes out of NFTs, and we'll have to sort out where the value actually is to society. It could be a burden to us in some ways, or a distraction. I think CJ has a lot of thoughts on how crypto and art are kind of merging together these days, right?

CJ Carr

Yeah. NFT is a really interesting early experiment in this fusion. It's kind of like a new way of doing patronage. It's like Patrion meets Pokemon meets chess meets Dungeons & Dragons. It's like chess because each move can't be taken back. It's like D&D because you can create worlds and ways for people to interact and build the platform. It's another way for money to flow to artists, to be supported, which musicians have been struggling for for a long time. So hopefully that is not just a fad.

FAST45

How do you imagine your work to change and evolve in the next few years and what would be the main challenges and drivers of this change?

CJ Carr

We're on the verge of a few different technical challenges. Once we achieve them, it will open the floodgates for a lot of things. One such challenge is making these models cheap enough that anybody can run them on their computer. Right now, training models requires really expensive hardware that, we have to rent it to train and run these models. But once it becomes very accessible, the average bedroom producer will be making music this way, which is really exciting. And I want to see what five-year-olds are going to come up with, you know, like producing music, not by sequencing, but by specifying a list of influences that you can add or even subtract. It becomes like a bird's eye point of view sort of creation. What do you think, Zack?

Zack Zukowski

Yeah, I definitely want to see that too. and yet the technical is the first thing to overcome. There will be some discussion of copyright and ownership to be done again. People are going to want to mix their favourite artists. And if that artist is in on being part of that model, that's great. But if they aren't, where and how are we going to decide what is right or wrong? So yeah, there's both the question of who owns art and who can do things with it, and then there's what we can technically do, whether we can make it efficient and cheap enough so that it becomes accessible to people?

FAST45

What roles can artists, art schools, and the creative sector as a whole potentially play within these trends you just described?

CJ Carr

Maybe I have a bias here. but I think that, as the world becomes increasingly online and technical literacy becomes more and more important, that learning to code will be as essential as learning to speak and write. And probably artists should be part of the wheelhouse. Everyone has to take an illustration class. Why not have everyone take a creative coding class? That makes a lot of sense to me. And most of the world's being built on the internet nowadays and you know, during the pandemic, everyone's lives were on the internet. And those that were building stuff in that medium, they were connecting people, they were influencing people's lives. That was where a lot of the creativity was occurring.

Zack Zukowski

And the classic storytelling is never going to go away. Whether it's a computer generating the story or the human steering the computer, art schools are still going to have peer criticism. Even with this new digital medium, a lot of those traditional things you learned in art school will still be relevant. Just the tools and technology might be different. Instead of a pencil or a paint brush, it just might be a graphics processor.

FAST45

Okay. Maybe we can dive a bit deeper into this. Good artists are said to have great control over the medium they work with. What skills would an artist have to possess in order to use the full potential of artificial intelligence as a medium to create works of art?

CJ Carr

Ooh, that's a good question. I think about that a lot. So, I guess a list of hard skills to have is operating Linux or knowing your way through a batch terminal, knowing Python, understanding how digital audio works, statistics, machine learning. There’s no need to learn all these things from the ground up. No, no, no, you start with tinkering with a CoLab notebook that already trains and runs a deep learning model and then spits out creative content. Then you start tinkering from there and you learn in the opposite direction: not from the fundamentals up, but rather from using the tool and learning how it works as you're tinkering. And along the way you're discovering new things and slowly building up the low level understanding.

Currently it’s hot on Twitter to play with OpenAI’s Clip to do automatic captioning. You give it an image, it could be a photo, it could be art, and it comes up with a caption. Then people were able to make a model that inverts that. So you give it a word, a phrase, a sentence, a caption, and then it generates an image. Starting from noise, it just trains and optimises and you arrive at an image iteratively. This style of creation is great.

Recently they figured out how to ‘subtract’ words. You can imagine an avocado chair, which is the addition of ‘avocado’ and ‘chair’. So you can also subtract the words. And you can do other things with it, like telling it you want your avocado chair to be shaped like an egg and placed in the middle of the picture. And if you don’t give it a prompt, the result could be a little bit more surreal, which is cool. And you can also prompt it with an image of just an oval, and then it would generate the avocado chair based on that oval shape. It doesn't require hardcore knowledge of statistics to be tinkering with these things. And yet artistic minds are just inventing new ways of working with this kind of tech and breaking it.

FAST45

How should we understand Moore's law in relation to artificial intelligence? Or: how should we imagine the impact of this technology on our world and on the arts, five years from now, ten years from now and twenty years from now?

Zack Zukowski

That sounds like a question for the computer scientist!

CJ Carr

Well, I guess part of the singularity is that technology influences the ability of people to accelerate the technology. Obviously, the hardware is getting faster and cheaper. The $10,000 cards that Zach and I had to rent on the cloud a couple of years ago, you can now get $1,500 cards that are marketed to gamers. You can now run and train the same models on those. And now Apple's new laptop comes with a neural chip in it, the M1. so that's all just going to get faster and cheaper up to the point where, you know, everyone can train models. And then there’s Google Colab, they just give you a free GPU. That has made it accessible now. You don't need a huge budget anymore to train average size models.

What are other parts of the singularity? So, the amount of data being produced every year is huge, like petabytes or exabytes. So that's huge. No human can ever go through that. You need AI to learn from that. We now have giant models like GPT-3. You could actually train its predecessor GPT-2 yourself in Colab without any budget. But GPT-3 is just huge. You can't train it on one machine. The budget for that was like, millions of dollars. GTP-3 has probably read every single book that has ever been digitised, it has probably read every Reddit post. It just contains knowledge from all these domains. And we’ll need more and more hardware and parallelism, just to digest everything that humans are producing. Zach, I think you have a good idea of how much video content is going up on the internet every year?

Zack Zukowski

Yeah. It definitely grows exponentially every year. Billions of videos are going online every year. So eventually CJ, do you think we get to a point where people like you and I never train models anymore? We just fine tune large scale pre-trains?

CJ Carr

Yeah, self supervised learning, pre-trained models and prompt engineering, that’s the direction that we're moving in for artists. Which is nice, because it makes it more accessible.

FAST45

Can you explain what you mean with prompt engineering?

CJ Carr

So for example, for GPT-3, if you want it to write you a song, you could just say, “Hey, GPT-3, write me a song.” Or more specifically, you mimic it in the format of what a song would look like. So for example you could input artist name, a song title, and for lyrics you could write the first line and then let GTP-3 just continue to keep writing. So it's a smarter and more natural user interface, whereas with the earlier iteration of GPT-2 interacting with it was much more complicated.

FAST45

Okay, now please imagine your job 25 years in the future. What does it look like?

CJ Carr

Wow. I'm hoping by then, that we'll have really advanced brain computer interfaces where we can download the whole sensory and mental experience of what it means to be human. We’ll be able to understand how it can be digitized and accessed, and that means input and output. Let's assume that this exists, and that you can also do this with dreams. We'll have artists developing dreams, composing experiences. And we would also have AI models that will generate experiences, just like they're doing media synthesis right now with music and images. And we’ll have psychonauts and oneironauts that are exploring the latent space of these models to find out “what else is there in the human experience?”. So that's kind of interesting.

And I also hope that the human brain becomes hackable enough that we could find ways to rewire it. That might make people have superhuman abilities like higher levels of cognition, higher levels of compassion, high levels of conscious awareness. That would be cool.

FAST45

Is there anything you would like to add to this Zack?

Zack Zukowski

Yeah, I don't think that jobs are going to be something where we're just exchanging our time for money. I think that we shouldn't even be valuing that in our society. We should probably, as CJ said, be encouraging people to think they're successful if they're compassionate. If they care about each other, if they can express something beautiful to each other, we'll probably have to shift away from the idea that your value is what you can provide to the economy. And maybe the economy will be a little bit more autonomous, and we're going to have to start thinking of new ways to create value for ourselves as humans.

FAST45

Now that's a future to look forward to! Before we end this interview: is there anything that we haven't covered yet and that you consider super relevant to the topics we’ve discussed?

Zack Zukowski

Well, there’s one thing I thought about when you were talking about Moore's law… It's not just a hardware thing that we're facing. I think the algorithms will make significant steps that brings us quite a bit far forward, compared to the current deep learning that we're doing.

We don't know whether deep learning is going to solve general artificial intelligence, we might need different approaches. Maybe we even need different ways to structure parts of the world as well. And nobody knows what that's going to be. Researchers are looking into the future, writing theoretical papers on how we can do this. But I think that there could also be a software improvement as well, in the whole way that we're approaching these algorithms in the next 10, 20 years.

CJ Carr

Right! And I think what's converging with AI and media synthesis is a leap that will be as significant as the invention of the search engine. Just think of how the search engine interfaces with creative problem solving today. If someone asks me a question, I feel confident that I could type it into Google and after a little bit of searching come up with a decent answer. And the latency of that keeps getting shorter and shorter and shorter. Like now I can even ask Siri a question, and then it gives me an answer. So technological advances also seem to be about reducing all kinds of latencies. And of course, this relates not only to answering questions, but to creating as well.

You know, it can take me months to compose a song that I heard in my head. But, with AI's ability to just understand what it is that you want and where you're trying to go - or even with brain computer interfaces that are able to extract that intention and immediately express it - that just makes the human, the creator, so much more powerful. It’s like one step up in the singularity.

FAST45

This brings up the question of craftsmanship. The arts and craftsmanship are very much related. But listening to your ideas, should we maybe rethink our notions of craftsmanship? Or do you see that differently?

CJ Carr

Yeah. We're moving away from brick laying and more towards hunting and gathering. That's the kind of UX change that machine learning brings. It's driven by your aesthetic and a sense of what you want rather than by your ability to put it together. So creating is more like flying a spaceship through the cosmos of possibilities. Like last summer, Zack and I were working with this model ‘OpenAI Jukebox’ which has been trained on over 800 sub genres of music. You can build latent codes that are combinations of styles. We generated 6000 songs worth. For us, the work mainly went into building this spaceship that allowed us to input a style combination, after which it generates the song for us. It would take a whole full day just to generate it. But imagine that that's getting faster and faster, until we can do this real time. So you just think about a combination, and then you could instantly hear it.

FAST45

Okay. Wow! Thank you for the inspiring interview. I hope we can meet each other again 25 years from now, and together look back on this interview...

CJ Carr

For sure!

Image: Dadabots CJ Carr and Zack Zukowski working on a new death metal album - © Dadabots