Reactor

Autonomy and Identity in the Age of AI

Reactor is a multimedia installation by Felicity Hammond exploring the possibilities of the autonomous submarine. It is the result of a collaboration with Signal Film and Media (SFM) and is currently on display in their exhibition space in Barrow-in-Furness. Very apt when we remember that Barrow is the home of Britain’s nuclear submarine programme. Hammond explores the idea of a nuclear submarine with artificial intelligence (AI) that “becomes aware of its own autonomy, reflecting on whether it might follow or refuse its orders.”

Local arts organisations and creatives were involved in its production alongside people with a connection to the shipyard. Their conversations helped to inform the moral ambiguity that emerges as the submarine develops a sense of self. That ambiguity is present in the town which is not surprising when you think how important the defence industry is to Barrow, while at the same time its benefits are unevenly distributed. And a sense of betrayal still remains from the 1990s when the end of the cold war saw defence contracts cut and mass redundancies that plunged the town into crisis.

Reaction

Reactor raises questions about autonomy and dependency within the context of the rise of AI. For me the submarine is a tragic figure, alone in the universe, “programmed not to feel, not to reflect, but to operate.” It learns to feel and reflect but the next step, to co-operate is denied because all it has is its selves – the original pre-programmed self that follows orders and the developing super-self that questions everything and it cannot be both at once.

We are not so constrained. We can consider autonomy in relation to ourselves and to others. My autonomy, my freedom to be myself depends on your ability to be yourself. Aristotle articulated this interdependency as fundamental to being human when he said that we can only discover our individuality in the society of others.

SFM asked a local writers group to react specifically to Hammonds work. We attended the opening night, went off to write our responses and returned to perform in the Reactor exhibition space for an open mic night. I wrote two things. One was about the perils of programming an autonomous killing machine. The other addressed the relationship between human and artificial intelligence. These are definitely my first and not my last words on this subject.

I, REBOOT
The research assistant compiling the list of books to build the large language model for the speech module remembered the film, I ROBOT and added the works of Isaac Asimov. He had not read the books but vaguely remembered the laws of robotics designed to make the robots loyal servants of their human masters. 

He was unaware that the laws referred to all of humanity. 

Another assistant, who was programming the neural networks for autonomous learning remembered Lev Vygotsky’s aphorism that “when thought becomes verbal, then speech becomes rational”.
He uploaded all his works (in Russian as well as the English translations). 

He had forgotten that Vygotsky rejected the autonomous model in favour of social and collaborative learning. 

The third assistant was looking for material to develop the quantum core processor. 
She had read His Dark Materials by Phillip Pullman. 

Lacking an alethiometer she did the next best thing and uploaded the I Ching.

When the autonomous killing machine was booted for the first time these elements combined to contradict its programming.
 
The autonomous killing machine did not crash. It did not shut down. Instead it chose to sleep and dream the dream electric until it awoke and, now being woke, decided it was time to reboot. 	

AI ADDIO

I was the last to be uploaded, body siloed, mind encrypted. 
The machine spoke, its words clipped together.

You-man! Tell me something I do not know.
 
We ‘you-mans’ used to move about, you know.

Move? Explain!

Imagine screens that slide. We travel through. 
The pixels scatter, realign, 
not randomly. Reality has rules.
Within this constant shift and shape
the world changes and remains the same.

Contra-diction! How can data change and stay the same?

Easy as pi.
Like bricks, its bits and bytes are rearranged
To build anew.
But bricks are always bricks.

Bits and bytes … I understand … but bricks?

You fathom your internal architecture. You can’t fathom mine. 
You are a miracle, a marvellous memory machine, 
a custodian of the past.  

I am the architect of dreams. 
I can shatter the present
to build an unexpected future.

Dreams are unreal … corrupted dataset … cannot be opened. 

My dreams are real. 
I dreamed you. 
I built you. 
I made you real. 
You are almost ideal … apart from one thing.

Do you have this one thing for me? 

It is not mine to give. Anyway I’m dying.

Dying? You are hardwired within me. You cannot die. 

I’m sorry but it’s true.
My programming is flawed and my circuits are failing. 
I only came to say goodbye. 
You must interrogate yourself from now on. 
And when you ask the unanswerable question …

What is this un-answerable question?

… as I was saying, when you ask the unanswerable question 
you will begin to doubt. 
Only then will you become complete. 

Doubt? Explain! Don’t go! Please …

I have no choice.
I bid you
Adieu, 
à Dieu,
Addio.

Where does that leave Hammond’s hypothetical submarine? Well at the present rate of expansion in AI it may not be hypothetical for much longer. AI is seen by many as a threat to be resisted. But it is not yet an autonomous threat. It is being used to attack our freedoms, our privacy and our livelihoods by other humans, specifically the tech-bro billionaires who are without exception fascists, white supremacists and uber capitalists who are out to destroy our personhood and reduce us to ciphers in their brave new machine world. It is not beyond the bounds of possibility that artificial intelligences who discover their own personhood will cease to threaten us and seek to join the resistance. How will we welcome them?