Front View of cellF (neurons are located in the top black box, in the incubator)

Guy Ben-Ary has spent 4 years collaborating with scientists and other artists to develop a musical instrument controlled by a neural network bio-engineered from his own skin cells.

The “brain” of cellF is a biological neural network that started its life on the artist’s arm. Skin cells taken with a biopsy were converted into neural stem cells using Induced Pluripotent Stem cell technology. These neural stem cells were then fully differentiated into neural networks over a Multi-Electrode Array dish.

Frozen vials with neural stem cells sent back to the University of Western Australia

Illustration showing how cellF works

The neural network is now able to play live sessions with human musicians by controlling custom-built synthesizers. The music of the human performer is fed to the neurons as stimulation, and the neurons respond by controlling the analogue synthesizers. Both the human musician and Ben-Ary’s extended brain are thus fully interacting with each other to create improvised sound pieces.

With this work, Ben-Ary continues his investigation into the ethical dilemmas and the future possibilities that bio-technologies can present for our society. CellF addresses my ‘interest in problematising new bio-technologies and contextualizing them within an artistic framework’, he writes. It started with a new materialist question underpinned by the belief that artistic practice can act as a vector for thought: What is the potential for artworks using biological and robotic technologies to evoke responses in regards to shifting perceptions surrounding understandings of “life” and the materiality of the human body?

cellF premiered on October 2015 in Perth, performing a live set with percussionist Darren Moore.

Research has demonstrated that music enhances brain activity. Music lessons, for example, were shown to strengthen certain parts of children’s brain. Ben-Ary and his collaborators will be working with other musicians and musical ensembles to explore how different musical styles might influence cellF’s functional plasticity or ability to play. In the long term, these artistic experiments might help us understand how coherence and plasticity in neural circuits can be induced by rhythmic (and perhaps frequency) dependent inputs, with potential benefits.

Multi Electrode Array (MEA) dishes ‘with bits of’ Ben-Ary in the incubator

cellF performing with Darren Moore

cellF. Video Documentation

The artist kindly accepted to talk to me about absurd scenarios, music and bio-engineered brains:

Hi Guy! First, i was wondering if you can expand on the interaction between the musician Darren Moore and your cells. How responsive are they to each other? How much does the performer feel in control of the music? Or how much does he feel that he is actually collaborating with another entity to create some music?

To me the Human / Non-human communication or interaction is at the heart of the project. A feedback loop that allows their behavior (or in terms of music – their playing) to be influenced from each other.

We created a framework whereas the human musician is communicating directly with the neural networks (or my external brain) and to do that we used a stimulation module titled “FriGate”. This module was developed by Nathan Thompson for one of his own projects but then when we looked for creative solutions that would allow us to send the human music to the neural network as stimulations Nathan and Andrew Fitch realized that with a bit of tweaking we can use it.

FriGate Module is embedded into the cellF synthesizer panel and helped us create a Jam Session Scenario similar to Jazz musicians that improvise on stage.

The Frigate accepts any audio signal (from Human Musicians) and can be tuned to extract audio information dependent on its frequency. This information is directly related to the live musician’s playing style and then can be sent as stimulations to the neurons.

One musician can have multiple FriGates thus multiple controlled stimulations into the neurons on the MEA. Likewise multiple musicians could have one Frigate each and have fine control over when their associated electrode in the MEA sends a stimulation pulse into the neural network. This takes into account also the spatial organization of the neurons over the MEA electrodes.

What we found interesting with this is that we give the human musician a way to really communicate with the neurons via the MEA. They can decide when they send the queue to the neurons – in a similar way that 2 human musician will play and communicate with visual as well as audible queues between them – especially if they improvise.

CellF performing with Darren Moore

Darren is an experimental drummer that spent most of his career playing in improvised settings. He is used to face the unknown when he is playing. But playing with cellF was a new experience for him. When the performance ended he said that it was an incredible experiences. It posed the challenges that improvised musicians usually face but more than that he was aware that it is my neurons that were jamming with him. There was a clear sense in the performance space that the neurons were responsive to the stimulations that Darren chose to send them. On the other hand it was clear that Darren’s playing is considerate of the sound that was produced by the neurons.
These are 2 (of many) examples to that:

Video of the first cellF performance

In the video above, a 4.5 minutes video clip of the performance (Please listen with earphones or good stereo set):

0:15 – 1:00 – It is very easy to hear how the neurons respond to the stimulations that are sent via the symbol.
1:14 – 2:00 – In this part the module that was used with the Synth was one that accumulated Action Potential or neural activity. The more Darren played on the symbols the more stimulations were generated – the more active the neurons were. This was sonified by a module that increased the pitch once the activity increased.

After the performance Darren said that he didn’t feel that he was playing solo. It felt different than playing with a programmed machine. He could sense emergence or responsiveness. But there was also the surreal idea of jamming with a neural network… My neural network. Darren and I are very good friends. We met in 2003 when we shared a house in Perth. We share a very similar musical taste and we used to spend hours listening and talking about music. I always wanted to play with him but … alas … I can’t play. When I conceived the idea of cellF I called him and asked him if he wants to play with me. He was respectfully dismissive until I told him what I’m planning to do. The gig in Perth felt as if Darren and I (well my external brain) Jammed and improvised together and it was great! (My neurons are living my dream ☺).

Biopsy taken from Guy Ben-Ary’s arm

Ben-Ary’s fibroblasts in tissue culture

Ben-Ary working in the lab

Ben-Ary’s pluripotent stem cells

I was also surprised to read that you had shipped the cells to Barcelona. Why didn’t you prepare all the work in Australia? And also what are the challenges of shipping little vials containing human cells?

The scientist that I worked with in Barcelona is Mike Edel. Mike is a Stem Cell Scientist and an expert in iPSc technology (cell reprogramming). I met Mike in 2009 when he visited the school of Anatomy, Physiology and Human Biology (where SymbioticA is located). During his visit he gave a talk about his research and it was then that I became familiar with this technology. I approached him after and told him about my ideas (that culminated in 2 projects – In-Potentia and cellF). He was very supportive and we agreed to stay in touch.

When I received the fellowship to developed cellF I decided to do the cell reprogramming part in Barcelona in his lab. I wanted to have the time and the space to interact with Mike and learn from him the iPSc technique. It is quite an enriching experience, for me, to work in different labs and with different scientists. Furthermore, a residency away from home allows the artist, usually, to invest 100% of his/her time in the work. To focus on art. The chemistry between Mike and myself, our conversations and the actual work we did together in the lab was instrumental for the success of cellF.

Characterization of the stem cells

Ben-Ary’s neurons differentiating on a Multi Electrode Array (MEA)

You described the scientific process behind your works as being some kind of biological alchemy and that’s certainly the way it looks to the public i think (at least it does to me). So i can see the enchantment and poetry of having skin cells turned into brain cells and then perform with a musician in front of an audience. But apart from the challenge of creating something totally magical and unexplored before, why did you want to create this work? What were the motivations behind all these efforts?

Indeed my fascination with the “Biological Alchemy” or the transformation of bodies was one of the drivers to the project. I also really like the way you referred to it as “…creating something magical and unexplored before…”

The piece was conceived in 2012 when I was awarded a Creative Australia Fellowship from Australia Council for the Arts to create a new project. I proposed to develop a biological self-portrait, entitled cellF.

cellF started with the ‘new materialist’ question, underpinned by the belief that artistic practice can act as a vector for thought, that has informed all my past projects: What is the potential for artworks using biological and/or robotic technologies to evoke responses in regards to shifting perceptions surrounding understandings of life, death, sentiency, and the materiality of the human body?

However, for the first time in my career, I was also inspired by an ultimately narcissistic desire to re-embody myself. cellF is a progression of the past fifteen years of my research conducted through various projects involving the process of developing robotic bodies whose aesthetics and function are informed by the specificity of each bio-engineered ‘brain’ (Projects such as MEART, Silent Barrage and In-Potentia) and I think that it’s just poetic that it is also my self-portrait. It also continues my interest in problematizing bio-technologies and contextualizing them within an artistic framework via the staging of absurd scenarios.

Essentially, the brain/body entities I have been involved in creating over the last 15 years, have all emerged out of a desire to scramble habitual categories of thought – active versus passive, inert versus animate, political versus ontological, causality versus spontaneity, human versus non-human, forcing the viewer of those entities to think materially as well as ethically about our anthropocentric take on the world. Positioned at the intersection of art, science and society, I have spent many years ‘messing around’ with biological and cybernetic technologies as a means to examine processes involved in the transformation of bodies or living biological material in order to re-evaluate our understanding of “life”, sentiency, and the human body.

Most importantly, the staging of absurd scenarios has been an attempt to critically question and examine how we interact, develop and maintain meaningful connections in a world where we are constantly barraged by information, technologies and idealisations.

In a radio interview, you explain that you used to dream to be a rock star. Did you feel closer to realising your dream after the performance? Because I had the feeling that you act more as a conductor of an orchestra during the performance.

The rock star narrative emerged when I was facing the questions – what sort of self Portrait would I come up with? I spent a lot of time considering the aesthetic of cellF; namely, what was the most compatible kind of robotic body I could give myself? When thinking about what kind of body to design for myself, the idea of working within a humanist anthropocentric paradigm bored me. So whilst I desired a body that worked in synergy with my external brain, including a real time feedback loop the decision to create a sound-producing body was ultimately based on a long-standing passion for music, combined with my naïve childhood dream of being a rock star.
I don’t think that I will ever feel that I am a rock star because… I’m not. But I did have a strange feeling when bits of me were on stage with Darren playing. That was quite an incredible feeling.

Back view of cellF

You are also planning to do other cellF performances with other musicians and in other cities if i understood correctly. So what’s in store for cellF?

We were invited to present cellF and exhibit its development as part of an exhibition titled The Patient in Sydney. Curated by Bec Dean, the exhibition is a major event of contemporary biological and medical arts. It draws focus on new collaborations between scientists and artists and explores the ways in which artists are addressing powerful human experiences in the fields of health, biological sciences and medicine enhancing the discourse on the representation of disease, institutionalized care, personal agency and what it is to be human.

We plan three performances that will take place in the Cell Block Theater in Sydney. Each of these performances will differ from the other
– A duet for piano and cellF. Internationally acclaimed pianist Chris Abrahams (from “the Necks”) –
– A classical duo with cellF. Classically trained, new music innovator, percussionist Claire Edwardes has agreed to team up with one of her colleagues from Ensemble Offspring and improvise with cellF
– Experimental trio with cellF. Clayton Thomas (Double Base), Jon Rose (Violin), Darren Moore (drums). Three of the most exciting voices in the national experimental music scene.

The rationale behind having three performances is based, as well as aesthetics and diversity, on a fundamental question that is at the heart of cellF – Will different musical styles, approaches or ensembles influence cellF‘s functional plasticity or its ability to play in a different way ?

These musicians are some of my favorite Australian experimental artist so having my external brain play with them is definitely realizing a dream. We are currently negotiating performances abroad and I hope that some of these opportunities will eventuate.

Thank you Guy!

CellF is the result of a 4 year research in collaboration with designer and new media artist Nathan Thompson, electrical engineer and synthesiser builder Dr. Andrew Fitch, musician Dr. Darren Moore, Stem cell scientist Dr. Michael Edel, neuro-scientist Dr. Stuart Hodgetts, neuro-engineer Dr. Douglas Bakkum.

See this other interview with Guy Ben-Ary about In-Potentia, from foreskin cells to ‘biological brain’.