Eye Catcher

Behind the Wall a UR Robot running IAL's own "Scorpion" Software puppeteers the frame

A few weeks ago, i visited the graduation show of The interactive Architecture Lab, a research group and Masters Programme at the Bartlett School of Architecture headed by Ruairi Glynn, Christopher Leung and William Bondin. And it was, just like last year (remember the Candy Cloud Machine and the architectural creatures that behave like slime mould?), packed with very good surprises. I'll report on a couple of them in the coming days.

I'll start nice and easy today with the Eye Catcher, by Lin Zhang and Ran Xie, because if you've missed the work at the Bartlett show, you'll get another chance to discover it from tomorrow on at the Kinetica Art Fair in London.

Eye Catcher

The most banal-looking wooden frame takes thus a life of its own as soon as you come near it. It quickly positions itself in front of you, spots your eyes and starts expressing 'emotions' based on your own. Eye Catcher uses the arm of an industrial robot, high power magnets, a hidden pinhole camera, ferrofluid and emotion recognition algorithms to explore novel interactive interfaces based on the mimicry and exchange of expressions.

A few words with Lin Zhang:

Hi Lin! I think what i like about the frame is that it is so discreet and unassuming. You can pass by it and not even notice it. So why did you chose to make it so quiet and 'normal' looking?

Yes exactly, it's a really normal static object, which exists in everyone's daily life, so the magic happens the moment it begins to move. I was inspired by my tutor's art work finding "life in motion" - not all motion can provide wonder and pleasure in the observer, but playing with the perception of animacy in objects often does. There are many digital interfaces that have the appearance of advanced technologies and compete for our attention, but I think it is better to develop interfaces that rather than standing out, can sit within our normal daily lives and then come to life at the right moment whether for functional or playful purposes.

Magnetic Puppeteer that manipulates the frame from behind the wall

How does the frame respond to and communicate emotions? How does it work?

To start with, the height of passers-by is calculated by ultrasonic Sensors embedded in the ceiling. This is remapped to the robotic arm (controlled using the Lab's opensource controller Scorpion) hidden behind the wall which magnetically drives the frame to align "face to face" with onlookers. A wireless pinhole camera in the frame transmits the video footage of onlookers back to our software (built in Processing and using face-OSC) which analyses 12 values of facial expression such as width of the mouth, the height of the eye-brow, the height of eye-ball etc. That information then drives the reciprocal expressions of the frames fluid "eyes", controlled by four servo/magnets manipulating ferrofluid.

Do you see The Eye Catcher is mainly a work that aims to entertain and amuse or is there something else behind the work? Some novel interfaces, interactions or mechanisms you wanted to explore?

The Eye Catcher project is a method to examine my research question, which is to explore the possibilities for building non-verbal interaction between observers and objects through mimicry of specific anthropomorphic characteristics. It asks to what extend can such mimicry be deployed, specifically utilising eye-like stimuli, for establishing novel expressive interactive interfaces. We found that humans perceive dots, specifically eye-like stimuli, automatically as almost a hardwired ability, which develops at a very early stage of human life. By the age of 2 months, infants show a preference for looking at the eyes over the rest of regions of the face, and by the age of 4 months, they get the ability to discriminate between direct and averted gaze. Therefore, the eye is the foundation of human interaction upon which we build more complex social interactions.

Eye Catcher moves along the wall to approach a visitor

Ferrofluid "Eyes" puppeteered magnetically

What was the biggest challenge(s) you encountered while developing the work?

The biggest challenge is how to make the frame and two dots more animate - to not appear robotic but rather more natural. So we were really exploring how long reactions should take, how to select a suitable behaviour in response to peoples expressions, and how to provide continual unpredictable interaction to keep observers' attention.There's still a lot of questions to be explored, and even though its only ultimately 2 dots we're animating, the limitations are a useful constraint to work within.

Will you modify or upgrade The Eye Catcher for Kinetica?

Yes, we're working on it now for Kinetica Art Fair. We've already built a new frame that moves faster and more quietly. We've updated it with new Wi-Fi camera which provides more reliable facial recognition and smoother behaviour on the wall. The film you've seen is really only a prototype so its exciting to see how the new iteration will perform. We've switched round some behaviour too, to see how the public reacts. For example, at Kinetica we've programmed it to prefer to interact with children which should get them excited when it drops down to see them. In the future we'd like to build a more permanent piece using a 2 axis rail system rather than a robot arm. In theory the frame could then work on a much longer wall which would allow all sorts of new types of interaction.

Thanks Lin!

The Making of Eye Catcher

Check out the Eye Catcher at the KINETICA ART FAIR on 16th - 19th October 2014 at the Old Truman Brewery in London.
The project also references works such as Omnivisu, Opto-Isolator and All eyes on you.

Sponsored by:


Last year, news emerged that Russia's agency responsible for the Kremlin security was buying electric typewriters and "expanding the practice of creating paper documents" in a bid to prevent leaks from computer hardware. A few months later, The Guardian was forced to destroy the computer equipment that stored the NSA files provided by Edward Snowden. Diego Trujillo Pisanty saw reminiscence of the Cold War in these two stories and in other current news related to state espionage.

His This Tape Will Self Destruct machine prints self destructing documents. The documents merge images and texts extracted from Cold War fictions with excerpts from current secret documents. A short amount of time after they've left the machine, these documents burst into fire and their content is gone forever.

This Tape will Self Destruct from Diego Trujillo on Vimeo.

What is the paper made of? How come it 'auto-combusts'?

The paper is normal thermal paper used in receipt printers. As the document is printed it is treated with glycerol and a potassium salt. When these two substances mix at the end of the process they react exothermically to produce fire, this reaction ignites the paper and the heat also blackens any unburnt parts of the document as it is printed on thermal paper. The chemistry behind this is actually a common GCSE demonstration so it's nothing too complicated.



i'd also like to understand how the documents are generated. They are "a mixture of images and texts extracted from Cold War fictions paired up with excerpts from current secret documents". Are they generated randomly? do you design them yourself?

I designed (or more accurately curated) the documents myself based on relationships I saw between images and texts. For example one of the documents contains the famous Mission: Impossible (1966) phrase:

"As always, should you or any of your IM force be caught or killed, the Secretary will disavow any knowledge of your actions."

Presented next to an excerpt from an NSA leaked document reading:
"All publicly available information regarding work on this contract at Mangfall Kaserne will be sanitized so that no association with NSA will be made. This will entail removal of references to Maiyland Procurement Office/MPO, N S A-r elated DODAICs, NSA civilian/military affiliate names, NSA phone numbers, etc. (This is not an all-inclusive list.)"

Other documents focused more on visual aesthetics of devices and architecture, for example the parallel between the circular composition of the war room in Kubrick's Dr. Strangelove and the GCHQ 'doughnut' building.

Where do these images and texts from Cold War fictions come from? And where do you find the current secret ones?

The images and texts from Cold War fiction come from watching many hours of (usually very bad) Cold War film and television and manually curating extracts that relate to previously revised contemporary secret documents. Most of the extracts come from the early 007 films (Dr. No, You Only Live Twice, Moonraker), the Mission: Impossible 1960s television series, Macgyver as well as other fiction films of the era such as The Conversation and Dr. Strangelove.

I initially intended to work with the original files leaked by Edward Snowden and Chelsea Manning but even after they have been covered in the news it is very hard to find the primary sources for them. The current secret documents I used come from different places, mainly non-government organizations and news agencies. Many came from the Electronic Frontier Foundation and their repository of NSA primary sources (https://www.eff.org/nsa-spying/nsadocs). Some others came from The Guardian and their similar list of U.S. embassy cables summaries (now taken down but formerly http://www.theguardian.com/world/series/us-embassy-cables-the-documents) and The Intercept (https://firstlook.org/theintercept/documents/). Other documents were found throughout the web from all sorts of sources and forums with varying degrees of credibility.



Why did you decide to work with Cold War fictions documents instead of actual CW documents?

I did for a while focus on Cold War declassified documents, for example I did some reading around the STASI archive and even some pre-Cold war sources such as the recently disclosed Manhattan District History (https://www.osti.gov/opennet/manhattan_district.jsp). However most of these documents did not seem as relevant as some of the things discussed in fiction. I think that in the Cold War way of thinking an omniscient machine capable of spying on everyone seemed like a holy grail. This comes across in discussions of satellite imagery and long distance radio networks in many films and television series.

I also find it interesting to think that the rhetoric in these fictions could have done some of the ideological groundwork that led to mass online surveillance. It seems that in many of these franchises (007, Mission: Impossible, Macgyver, etc.) it's fine for the government to spy, impersonate and assassinate local and foreign citizens if they have reason to believe that they are suspicious. The 'good guys' in these series overthrow governments, kill criminals without trial and have no regards for international agreements and human rights, but they do it all for the sake of national security so it becomes justifiable. This is where I see a real link between the fiction and a reality in which citizens are prepared to accept that unregulated surveillance is good because it will stop 'the bad guys'.

Do you see relationship with the 300 Years Time Bomb? because news and secrets are explosive in their own way too..

I see a link in the popular culture that both of these projects take from, I would say that action cinema was very relevant to my generation and that this ends up showing in my work. I also see a relationship in the way I explored the way we assign value to things based on their lifespan. The 300 Year Time Bomb gained historical value by existing 300 years whereas a self destructing document gains value by existing for a very short amount of time, meaning that only a privileged person will be able to read it.

I hadn't actively thought of the secret as explosive but I think it is implicit when I say that "The release of the NSA files will likely be part of this decade's history". I think that Snowden's leaked documents have caused a sort of explosion resulting in an accelerated process of questioning the ethics around online technologies, digital democracies and personal rights online.

Thanks Diego!

The new episode of #A.I.L - artists in laboratories, the weekly radio programme about art and science i present on Resonance104.4fm, London's favourite radio art station, is aired tomorrow Wednesday afternoon at 4pm.

Song of the Machine

Dynamic Genetics versus Mann

My guests in the studio will be Anab Jain and Jon Ardern from Superflux. Superflux is an Anglo-Indian design practice: they are based in London, but have roots and contacts in the Gujarati city of Ahmedabad.

Superflux is looking at the ways emerging technologies interface with the environment and everyday life and the result of their research is a rather extraordinary portfolio which explores deviant economies for India's elastic cities, climate change, political engagement, desertification, human enhancement, etc.

The radio show will be aired this Wednesday 26 February at 16:00, London time. Early risers can catch the repeat next Tuesday at 6.30 am. If you don't live in London, you can listen to the online stream or wait till we upload the episodes on soundcloud one day.

Image on the homepage: 5th Dimensional Camera.

Benjamin Gaulon -aka RECYCLISM- is an artist, researcher and art college lecturer. And to be honest, when i see how much he manages to pack in each of his job descriptions, i'm starting to suspect that he benefits from more than 24 hours per day. I've read about his artistic activities ever since i've started blogging. Often both playful and critical, his projects involve printing messages on walls using a PaintBall Gun, collecting video streams from wireless surveillance cameras, turning your videos into animated GIFs, developing radio controlled cars that physically react to messages sent on Twitter, giving an architectural dimension to the 1970s game PONG, circuit-bending, hacking, deconstructing and re-purposing "obsolete" electronic devices.

Self-portrait - 2.4Ghz from Surveillance to Broadcast

Last year, Benjamin opened the Recyclism Hacklab, a shared studio space where people interested in physical computing, gaming, interactive and media installations, programming and more can find basic electronic tools,, storage for ongoing projects but also mentoring sessions for electronic, programming, conceptual support, etc.

The GAMERZ festival invited him to show his Printball graffiti robot and i took the opening of the exhibition as a heaven-sent opportunity to catch up with Benjamin Gaulon. After that we had an email conversation which, laborious little blogger that i am, i'm going to copy / paste below:

2.4Ghz from Surveillance to Broadcast

Hi Benjamin! Let's start with one of your latest projects. The 2.4Ghz project uses an affordable and widely available wireless video receiver to hack into wireless surveillance cameras. Including the ones people use for private reason, to monitor their baby for example.

This project has 3 layers. One of them consists of placing the device in the street to reveal the presence of the cameras and to make obvious the fact that anyone can receive those signals, a fact most people don't seem to be aware of. Can you tell us about this part of the project and how people react to it?

The 2.4GHz project actually started in 2008, however I have recently made new versions for two events: Hack the City at the Science Gallery (Dublin) and Mal au Pixel at La Gaîté Lyrique (Paris). This project started with an article about a mother receiving images from the NASA space station on her baby monitor, which got me wondering how this could even be possible. After some research I found out more about the 2.4GHz video signal and how open and easy it is to receive surveillance cameras and, in some cases, baby monitors. Once I found a receiver, with a built-in display, I decided to place the device in public spaces to reveal that signal, as a street art intervention, as you said, to expose the weakness of such system.

The reactions of people passing by are of indifference and more often surprise and enquiry. Most shop owners don't seem to mind that much about the fact they are broadcasting what they are trying to look after. I guess the same way many people today share a lot of their life on Facebook. The concept of privacy has changed with the rise of social media and the wide spread of surveillance camera in public and private spaces. Very often, when I'm installing the devices, people passing by would stop and ask me what I'm doing, and this is partially what I'm looking for, a moment of dialog allowing me to explain the project and engage in a discussion about the matter outside galleries and festivals.

Last time in Paris, someone asked me if I was looking for water, which is not far from the truth as there is a bit that feeling when searching and finding surveillance camera signals. This is what brought me to organise workshops, so others could share that thrilling experience of searching and finding signals in cities.

2.4Ghz from Surveillance to Broadcast

Do you ask permission to broadcast the images you capture for example?

But also is there a 'privacy' law related to that? Is it legal to collect the footage as you do?
And when you organize a 2.4GHz Workshop, do you have to take some precautions? Warn the participants they might get into troubles?

I do not ask permission to display the signal, as part of the project is to show how easy it is to hack into those signals. Also, in some cases, it is impossible to know where the signal comes from (when a camera is inside a building for example). About the legal aspects, I haven't really looked into the law of each country where I did this project to be honest. And I'm not the one broadcasting, I believe there are laws against that rather than receiving. I'm simply displaying a signal that's out there. The same way someone on the street would be listening to the radio.

About the workshops, I only ask participants to take some ID with them, just in case they get stopped by the police and need to talk their way out, which never happened by the way. From past experiences with other projects in public space, like placing a dodgy looking, noise-making, electronic device on screens of Time Square (next to police forces), I never had any troubles.

L.S.D Sonic Graffiti (Testing @Time Square, New York)

'Glitch' and 'corrupt' are two of the keywords of your artistic practice. Your project sometimes attempt to disrupt technology, to make it stray from its usual function and aesthetics. What do you think is fascinating about a damaged image or device? To you personally but also why do you think people are so attracted to it (judging from the enthusiastic reaction that your projects often get)?

My Corrupt projects series came from an earlier project, started in 2001, called: digitalrecycling. It was an online, digital garbage, recycling centre. The website was an exploration of trash and ownership in the digital realm. From the website, users could upload unwanted files (digital trash) and, once sorted by file types, others could download them for later reuse / remix. This project was also a comment on how operating systems tends to mimic the physical world (desktop, files and folder and the trashcan). The idea came from the unfortunate experience of trashing a file by accident. After using a data recovery software, the recovered document ended up corrupted (glitched). I thought it was quite fascinating idea that, when you trash a document in a computer bin, it gets "corrupted" or "damaged" the same way if you pull a piece of paper from the bin after you trashed it by accident you get it back dirty (corrupted).

While I was busy with digital garbage, I looked into files corruption and how I could recreate the conditions of such glitch (and not fake it but actually damage the file). So I made the first of the Corrupt projects, in 2004: corrupt.processing (a Processing code allowing users to read and damage files on a binary level). This project was quickly followed by corrupt.online (the php version of that algorithm, allowing thousands of users over the years to glitch images from the website by altering jpeg images on a binary level). I've recently made a video of all the content uploaded and glitched on the website since 2005 (over 1h long and visible here). I like the idea that, from a large database of corrupted images, what emerges is a story, in a way, of the entire internet.

Glitch, as a concept and a term, became more popular recently with people starting theorising more about it, like Rosa Menkman with her Vernacular of File format and with the Gli.tc/h festival for example. I remember being quite influenced by an earlier text by Kim Cascone "The Aesthetic of failure" when I started exploring image corruption.

What I see in glitch, in relation to my practice, is a way to engage with a critical discourse toward technology, obsolescence and digital materiality.

When corrupting a digital image, you reveal its inner structure, in other words, you reveal the code behind an image. Each file format (or compression algorithm) have different outcomes when corrupted. On some level it is a way of revealing the materiality of a digital image on a very low level. For me it is also a way to engage with technology in ways that were not intended. And by doing so, moving away from being a passive user of tools made by others. When breaking a system I feel more in control over it. Even if glitches are unpredictable, which is part of the beauty of it.

Maybe, in failure, we find ourselves less distant from digital information, we can somehow relate to failure, as humans, since we are not, in anyway, perfect machines operating on code. But it's hard for me to talk for others as I think if you ask what people see in glitch, each person will give you a different answer. I suppose part of the recent wide spread of glitch practices, is also a visual exploration of glitches on an aesthetical level (and often recycled by advertising).

About "damaged" devices, well it is a whole different story. I've been researching the potential of electronic waste as a way to expose planned obsolescence for many years now. Among different projects dealing with hardware hacking, I've initiated a project called ReFunct Media in 2010, which is, in a way, a prolongation of the e-waste workshops I have been running since 2005.

Corrupt.desktop, Fnac, Aix en Provence

And is reversing the glitch something you pay attention to? i'm thinking in particular of Corrupt.desktop, an app you developed together with Martial Geoffre-Rouland to allow people to glitch, in real time, a computer's desktop image at any Apple Store or Apple Retailer.

The project Corrupt.desktop is actually a derivative project of Corrupt.video (a software for real-time video glitch combined with a YouTube-like platform www.uglitch.com, allowing users to upload glitched videos from the software to the website). One of the features of Corrupt.video is the possibility of glitching your computer screen.

From this possibility, came the idea of making a separate software that would just do that, but with a twist since we renamed it "Safari" (using the Apple browser name and icon). Since then, I have being going to Apple stores (so far only 2, PC World in Dublin and la Fnac in Aix-en-Provence) to install the software on as many computers as possible, while recording the intervention on video. I see this type of intervention as Apple's retailer hacking. I believe it is a humorous way of disrupting capitalism and an attempt to challenge Apple's slick and shiny image. I'm also inviting people to take part in the project by doing the same actions where they live and sending me their recorded footage.

You're at the origin of the e-waste workshops, which as they name suggest use e-waste as raw material to create interactive art projects. Could you give us a few examples of projects developed during these workshops?

The e-waste workshops have been going on since 2005 and hosted in different countries with a large variety of participants. I run those with Lourens Rozema (an electronic engineer based in the Netherlands). Our goal is not necessarily to have finished pieces (even if it's better when it happens) but to get participants to see the potential in hacking e-waste and learn basics of electronic and programming in the process.

So to answer your question I can mention some projects. In the first session a group of participants made an interactive drawing machine using 2 scanners and a floppy disk controlled by microphones, allowing users to controlling the drawing machine by shouting at it. Drawing machine are often a good way to start and learn, we had few of those. Another one used an old printer combined with a webcam, users could draw on a long roll of paper, scanned by the webcam to generate sound (using computer vision). The most complete drawing machine was created by Charles Mazé and called the Rétrographe, using an old bulldozer toy, turned into a font making machine that he used in projects long after the workshop. Another, very low tech, interactive project (not directly recycling e-waste but still very good) is DanceKarpet, using cardboard and light bulbs power from car battery to make an interactive, very low resolution, interactive screen.

Sound making projects are also a regular way of reusing old electronic (as in circuit bending), or in some cases creating new instrument, by using old tape player heads and magnets, bended video games, making a sound controller from an old phone, a monkey head, etc...

e-waste 7.0

What would you advise me to do with my own e-waste? How can i discard them responsibly?

Since we started those workshops, things got better, there are more (proper) recycling facilities in Europe (just make sure they do recycle the stuff and not simply ship the stuff to China, India or Africa). However I guess part of the point of our workshops is to say that recycling (dismantling for materials) is maybe not the only way. It comes with a cost in energy and chemicals used in the process, when in some cases, part or all of the so-called obsolete device could be reused / hacked into a new project or product. This way of thinking should be more widespread when it comes to making new devices. Making open-source hardware is a good start as it allows easier hacking in the future (if the documentation is clear and accessible). Circuit bending or hacking of recent electronic devices is also getting a lot harder than for older electronic. Parts are getting smaller, which makes this type of practices very hard (or impossible). This is really an issue that we see in the e-waste workshop when people bring too recent devices.

Physical Computing Master Class at the Hacklab

If i understand well, the e-waste workshops have given way to the Recyclism Hacklab, a permanent space offering workshops, mentoring sessions, support, etc. Who is the hacklab for? hackers? artists? professionals of IT technologies? Children? Un-usual suspects?

I guess anybody looking at learning and also sharing their skills about hardware hacking, e-waste recycling (and the usual Arduino, PureData, Processing,...). But more importantly people interested in Critical Making (a term coined by Matt Ratto). This idea of of Critical Making is also behind a publication coming up (led by Garnet Hertz), which regroups artists, designers, educators, theorists, makers from all over interested and engaged with this concept. I also believe hacklabs and hackerspaces have an important role to play in reshaping capitalism. The Recyclism Hacklab is my modest contribution to this topic.

The Recycling Entertainment System, at SonicActsX Amsterdam

I'm also quite curious about how this interest for recycling, re-purposing, re-using e-waste translates in the aesthetics of your various projects. Is this something that matters to you? Do you, for example, purposely attempt to create devices that look a bit vintage or second-hand?

Many of my projects are reusing vintage devices, like the RES (Recycling Entertainment System) that reuses 6 NES controllers, so the vintage looks is inherent to the technology I'm recycling / referencing in that case. With ReFunct Media I have invited other artists to collaborate on a large scale circuit-bending installation, where we purposely reuse obsolete/vintage devices (video games, ancient computers, turntables, tape players and so on) that we open up and hack together in a complex chain of interconnected devices. This project deals with the temporality of media and technological obsolescence, while revealing the process of its production by keeping every wire or circuit boards visible. Somehow, I see this type of projects as a practice-based media archaeology. I feel very close to the concept of Zombie Media (by Garnet Hertz / Jussi Parikka), part of this essay was included in the publication for ReFunct Media #2.


When it comes to aesthetic choices, I suppose, I try to make coherent propositions in relation to the piece of hardware I'm hacking. So the second-hand look is actually due to the second-hand devices I'm using. But I suppose there is also probably some part of nostalgia in hacking early video games considering I grew up in the 80s. For example with AbstracTris, when I turn a Gameboy screen into a lowtech generative pixel art piece, I can't help but think of myself as a kid playing Tetris on the Gameboy for hours while making it. Or with Harddrivin', a twitter controlled car racing installation, I have chosen to collaborate with an artist (Ivan Twohig) whose sculptural works refers to early days 3D polygonal modeling (considering that Hard Drivin' was one of the first 3D polygonal driving environment).

ReFunct Media

ReFunct Media

While preparing this interview i found about Protonoir where you are selling some of your works, are you involved in this online shop?

Well yes, directly. Ivan Twohig and I wanted to find other ways to distribute our works and we both had pieces we wanted to sell. Since none of us felt like adding a commercial side to our websites, we decided to set up this online store and invited other artists to join (people we know / works we like). We see this a bit as a museum or gallery shop, except we cut the middleman. So there is no fee or commission for artists featured. I wouldn't say it has been a huge commercial success but it is more a practical thing when people ask about buying a project. We are open to works by other artists but we reserve ourselves the right to choose things we like and that we consider fit for the website. The website became also part of one of a recent project: KindleGlitched, a series of readymade glitched Kindles donated, found or bought on eBay that I laser etched with my signature, and sell (again), as an art piece, on Protonoir.com.


You're French but you've been living and working in Dublin for a number of years. Why is this a good city for you?

Despite the current economical climate (and climate in general ;-), Ireland is a pretty good place for me. Dublin is small yet it's a European capital. I've been here for nearly 7 years. I'm teaching at the National College of Art and Design in Media (in the Fine Art Media Department). I've been running (with Rachel O'Dwyer) an organisation called D.A.T.A since 2007 (Dublin Art and Technology Association, initially created by Jonah Brucker-Cohen). After starting in an artist led space, I've recently moved the Recyclism Hacklab to the CTVR building (The Telecommunications research centre of the Trinity College), which is another amazing opportunity.

ReFunct Media v1.0

The Printball. Photo : Luce Moreau - for M2F Créations

Any upcoming projects/events/exhibitions you want to share with us?

The next stop is Chicago for the GLI.TC/H festival, where I'll be a track leader focusing on "Hardware hacking and recycling strategies in an age of technological obsolescence"

Soon after, I'm very pleased to present, with the usual crew Karl Klomp, Gijs Gieskes and Tom Verbruggen and two new special guests Phillip Stearns and Pete Edwards, ReFunct Media #5 for the next Transmediale Festival in Berlin.

With D.A.T.A we are also discussing the next OpenHere festival (a four days festival that addresses social, technological and cultural issues surrounding the notion of the digital commons).

And hopefully find some time to make new work in between.

Thanks Benjamin!

As i mentioned a few days ago, the Barltett's student show is one of my favourite events of the Summer in London. It is however so overwhelming and turbulent that you need dedication and pure chance to spot the works that might interest you the most.

I'm glad my Bartlett expedition led me to The Theatre of Synthetic Realities.


The project, authored by architectural designer Madhav Kidao, is miles away from what you'd expect from an architecture work. No model, no plan. In fact, it rather looks like an essay made of photos, short videos and texts. Together, they reflect on immoral architecture, unsympathetic machines, reality filtered by artificiality and more generally, our symbiotic relationship to technology. In fact, Kidao likens his project to "an exaggerated caricature of our present and near future relationships to technology as is stands."

What made the project remarkable is its focus on how new technologies prompt new behaviour and new ethical questions and decisions. I realize i might be accused of heresy and crucified in front of the populace for writing this but ethics and social concerns are usually not the nerve centers of architectural exhibitions.

Allow me to do a bit of copy/pasting and let's see each other right after that for an interview with Madhav Kidao.

The Theatre, as illustrated in the film Making Friends and Other Functions, is a vehicle in which to explore the relationship that exists between the designer/creator and his or her repertoire of increasingly intelligent collaborative tools, be these tools to create or tools to think. The machines, under the selective guidance of the designer, construct their own reality based upon the information they extract from their environment and its unwilling occupants. This is ultimately a task with no beginning or end, and fundamentally questionable ethical integrity. As result we are left to question the role of the Architect, both in regard to creative authorship and ethical responsibility.

Making Friends and Other Functions v.2 from Madhav Kidao on Vimeo.

Hi Madhav! What are the 'Unsecure Webcams' that are mentioned in ACT I - The Observer? What makes them unsecured?

As I'm sure you are aware most live feed web and surveillance cameras can be accessed remotely through the internet. It is possibly one of the Internet's worst kept secrets that by simply googling specific codes for particular camera models, for example "intitle:liveapplet inurl:LvAppl" a list of all the live webcams of that particular make and model is given. Often, however, the owners of the cameras, usually domestic or small businesses, fail to password protect them, this is what makes them unsecured. Therefore in theory anybody is able to view the camera's live stream and with many cameras actually control the pan and tilt of the camera. As long as you have not cracked any passwords, this is entirely legal.

Whilst the feeds are usually pretty innocent, and frankly often mundane, every now and then you come across something that you no doubt shouldn't have done, an insight into a private world completely unconnected to our own other than through technology. It's quite a peculiar relationship that exists between the remote observer and the realities of an unknown space that is unfolding before them. Time is of particular importance as you are viewing in real time a world in which you have no knowledge of its past beyond the moment you logged in, yet our extended embodiment into the animate camera somehow immediately embeds us into this unfamiliar space.

This is not too dissimilar to the tele-assistance relationship between a drone pilot and the drone. Initially we are completely ignorant of where and what it is we are looking at other than what can been interpreted from the point of view of the camera. It's in our nature to be inquisitive, to people watch, and there is some perverse pleasure in trying to comprehend the events unfolding before us. However due to the lack of context more often than not this is simply educated speculation. The more I explored this strange paradigm the more I found myself just making up wholly fictitious scenarios in my head based upon the slightest clues garnered purely from body language, perceived interactions and the environment. The epistemologically variety of this action means that truth soon begins to seem irrelevant compared to the desire to fabricate an alternative reality.

Machine Vision Portrait - Gesture Recognition

Machine Vision Portrait - Skeleton Tracking

The name of the project, The Theatre of Synthetic Realities, places your work in the realm of fiction. Yet, it was inspired by elements of reality. Could you tell us about the trends, behaviors and technologies that have inspired the project?

It was from this point that the concept of a Theatre of Synthetic Realities emerged, a kind of reinterpretation of Hitchcock's Rear Window for the 21st Century, in which the binoculars are replaced by the the vast interconnected complexity of our computing networks; with each input - be it a camera, a sensor or even another person - acting as a portal into a parallel environment. In much the same way as the binoculars are a prosthesis to extend our vision, the global technological systems that we symbiotically rely upon extend our powers of perception and influence around the world. In Rear Window the architecture of community and society defines the story. I was keen to explore how technology facilitates new forms of social interaction and redefined concepts of neighbourhood and community, and how this in turn redefines our concepts of architecture.

James Stewart in Hitchcock's Rear Window, 1954

The impact technology has had upon our social systems, access to knowledge and global influence is nothing particularly new, Marshall McLuhan famously introduced the concept of a Global Village and Global Theatre almost 50 years ago. For me what is more interesting is how this has evolved into the relatively more recent concept of the Internet of Things. The trend I believe that we we will see with the Internet of Things is that it is not just objects that can be tagged and categorised but also the spaces they inhabit and the actions and events that they are associated with. In this way spaces, buildings and environments are becoming encompassed in the Internet of Things. This is not limited to architecture though, the social activities, and routines that are contained within that architecture add to the data history of a place, a form of artificial psychogeography. What this means is that in much the same way as we exist in a duality between physical and digital social circles, the spaces that we occupy do too. What we begin to see is a physical environment and the live updating digital representation of that environment. This is facilitated by new advances in sensory and scanning technologies, computer vision and biometric analysis.

I, like many others, was investigating the potential of the XBox Kinect as means of capturing complex real time data. Its capabilities have been widely publicised but in simple the idea that a machine can see in three dimensions and then also recognise human gesture is incredible. It allows any machine created using such technology to become actually embodied in a physical world. It begins to transcend the border between the physical and the digital which is a very powerful concept when thinking about intelligent environments. And as we increase our dependence upon the internet as our primary source of knowledge and interaction, our interpretation of truth becomes more reliant upon the technology that we assign to gather and interpret the real world. So fundamentally we start to view the physical world through the filter and perception of machines, in effect, a synthetic reality.

ACT III - The Vision Machine

Could you describe the Vision Machine, the way humans would 'interact' with it and its ultimate purpose?

The idea of the Vision Machine originated from Paul Virilio's book The Vision Machine. In it he discusses the concept of the "automation of perception", predicting a machine that sees for itself not for the benefit of man. This concept coupled with an interest in how artificial intelligence feeds into responsive/adaptive environments intrigued me. The capabilities that a computer has to extract a significant amount of information from a physical location combined with its ability to comparatively analyse that data with the vast amounts of data stored online through a neural network is almost enough to simulate perception. The resources that a computer has immediate accesses to - such as The Internet of Things - and its methods of analysis are completely alien to our own, therefore it could be predicted that its interpretations of the physical world would be alien too. Building upon the idea of how a human would perceive a space as viewed remotely through a webcam, I wanted to create a sensationalised demonstration of how the camera itself could possibly perceive the space it inhabits in relation to its learned perceptual world.

In regard to the Theatre of Synthetic Realities, the Vision Machine is in essence a robotic actor, cameraman and director in one. Whereas in the initial webcam scenario I was in complete control of the camera and was free to make my own conclusions of what I saw, with the Vision Machine at the other end I am relegated to a more collaborative role in which the machine dictates what it is I view however provides information that I would never have ordinarily been able to extract. Working from this principal we could then predict that the collaborative relationship between myself and the semi-autonomous machine could lead to an emergent performance unique to the relationship. The next stage in the Vision Machine was to then suggest that, much like I had done, it too begins to fabricate elements of reality and as such distort the reinterpretation of the digital model of that environment as well as what the viewer believes to be true.

The project has never really been a serious proposition for a machine to be developed. In fact it was supposed to be an exaggerated caricature of our present and near future relationships to technology as is stands. The machine is portrayed as more of a mild irritation that we just coexist with rather than any kind of interactive companion. Its ultimate purpose is as an analogical device to critique and explore our concept of what digital fabrication is or could be. In this project we are fabricating reality and the representation of reality as part of the Internet of Things. It is very much an attempt to question the relationship we have to tools and technology in a world in which we continue to transfer not just physical but increasingly cognitive faculties over to those tools. The is not seen to be necessarily a negative thing but rather an interrogation of the possibilities this presents, especially in regard to new forms of collaboration.

ACT IV - Making Friends and Other Functions

ACT IV - Making Friends and Other Functions

I found 'ACT IV - Making Friends and Other Functions' a bit daunting: here is a machine that observes human beings constantly and then assesses them and assigns them a character. Do you see current technology going in that direction? And what would be the benefit of having such machines around?

In Making Friends and Other Functions everything is real and works to some degree. Somethings I have exaggerated or simulated to communicate the ideas more coherently, however they are all based upon pre-existing technology, most of which is already incorporated into our smartphones. So rather than technology heading in that direction it is in fact already there. In fact casinos have been using some of the most advanced computer vision systems for sometime to catch out those attempting to cheat as well as identifying individuals who have been banned. But what I am more interested in is the relationships between the designer, society and emerging technology rather than necessarily a specific technology and its implications.

One of my aims for the project was to see how technologically advanced I could make it for the least amount of money; so begging, borrowing and hacking spaces, hardware, open source software and code. Then was an attempt to demonstrate how the role of the designer is changing, particularly in the world of open source projects. Practically any designer can now create they're own tools and machines for any job-specific purpose with a relatively low budget, the RepRap being the archetype. As complex and powerful technologies become cheaper and easier to hack, like the Kinect, the designer is gifted with power and responsibility that is free from supervision.

That sense of malaise you have from the idea of camera watching and judging you is in large part due to your loss of control and empathy due to the unpredictable nature of a non-human agent. I think this is an issue that as designers, and particularly architects, we will have to address. As architecture reacts to shifts in social habits I think we will see a lot of unforeseen challenges in regard to what technologies we use and the manner in which we do so.

Fundamentally my project is an immoral one and I think the concept of an immoral architecture is something that will become increasingly prevalent in the future.

ACT IV - Making Friends and Other Functions

The text of your project says that "The Theatre (...) is a vehicle in which to explore the relationship that exists between the designer/creator and his or her repertoire of increasingly intelligent collaborative tools, be these tools to create or tools to think. The machines, under the selective guidance of the designer, construct their own reality based upon the information they extract from their environment and its unwilling occupants. This is ultimately a task with no beginning or end, and fundamentally questionable ethical integrity. As result we are left to question the role of the Architect, both in regard to creative authorship and ethical responsibility." Is ethical responsibility already an issue architects and designers encounter nowadays when working with new technological tools?

Ethical responsibility has always had some part to play in the design process but I think that is slowly coming to the forefront. The explosion of open source has really brought ethics into the design process as it not only transfers power from the institution to the individual but it also provides new forms and channels to disseminate information, this recent article actually being a perfect example.

For me, consciousness and intent are just as important as categorical right and wrong. For exploring emergent technology is just that, emergent, it is unknown. While a particular technology may have positive society changing impact it may well have dire consequences too. I'm not one for speculations upon utopian ideals and believe that we will have to tread and adjust the boundary of what is acceptable to progress civilisation, however whilst the unquestioning embrace and exploration of new technology is exciting we often fail to question its merit beyond novelty. As Joseph Weizenbaum suggested, just because we can do something doesn't mean we ought to. I think we have to take a utilitarian stance and ask ourselves is doing something in a new way beneficial to design and society? And if so is it not to the detriment of others?

Admittedly I think architecture is a bit slow off the mark when it comes to these kinds of issues. This is understandable when you consider the timescale architecture operates on compared to other design fields. However Architects have always been eager to incorporate new design methodologies and the ethics of the technology used will undoubtedly become an issue.

In terms of the Bartlett show, I'm guessing there is a particular reason you have asked this question so I would ask the same of you. The show is always a dazzling array of phenomenal work and the mastery of a variety of mediums used is breathtaking. I do feel the density and complexity of the show does however sometimes make it overwhelming and distracting from the work. Of course as with any degree show the work on display can only ever be a vast reduction of the full scope of the project and as such the viewer is never going to get a comprehensive understanding of that project.

Thanks Madhav!

All images Madhav Kidao.


Lisa Ma is interested in the fringes of society. From the ladies who are mad about cats to the communities campaigning to stop the extension of Heathrow Airport. Lisa is a designer and her role is to create platforms of engagement with these groups which are otherwise ignored by society.

One of her latest projects drove her to a joystick factory located in one of the suburbs of Shenzhen. She spent several weeks with the factory workers, sleeping in dorms, sharing their meals in the canteen, making friends.

Because most of these young factory workers come from a farming background and because joysticks might very well become obsolete soon, she proposed to the factory owners that they'd allow the joystick makers to work part-time in a nearby farm. She called the experiment Farmification - using farming to keep the factory community together when work dwindles.

Almost everything about the project intrigued me. So i asked the designer to give us more details about Farmification:


Hi LIsa! In your video you say that there are more than 230 million Chinese migrant workers? Why do you think we know so little about them? Is it because we prefer not to know about who they are and how they live? Or is it because it is difficult for an outsider to gain access to these factories?

We tend to be conscious of Chinese factory workers as a working mass. For example ABC's "Monday morning, at the Foxconn's recruiting centre over 3000 people have been lined up, desperate to work for Apple's biggest supplier", we hear about the impressive scale rather than relate to the workers as humans. When journalists write about factory salaries they are describing these workers in terms of economic values. Focusing on sensationalist extremes makes the workers difficult for us to relate to on a human scale and distances them, whereas I look into the workers' daily lives and highlight their mundane events to make them more emotionally accessible for the viewers.

The manufacturing of our products is really a very secretive process. We are starting to grow a consciousness about material and ecological costs in the items that we use but there's a huge part of how they are made that is still in the shade. The recent Foxconn stories have brought more attention to this but there are other examples of how the story of manufacturing affects consumption:

-In 2011, a luxury Italian furniture company called Davinci caused outrage in Chinese customers when it was exposed that their 'imported' products was in fact made in China. "By spending a day in the bonded zone, the furniture had changed from being classed as domestically produced" to being labelled as Italian-made." Robert Olsen, Forbes, 1/05/2012.

-The value of a workforce demonstrates itself (sadly) in any touristic craft store, where there's a supposed craftsman making spoons out of horn or glass-beaded bracelets for sale. The narrative process of the products becomes the main selling point.

-The village of Dafen specialises in making fake paintings. The process of faking a famous painting has actually become an tourist attraction. The point is that vendors can position their value in the craft process, even if it's faking a famous painting. (Ironically, due to Dafen's success, now other villages reproduce their own 'fake Dafen fakes'.)

One possibility is that products might have a "Responsible Life-Work Balance" standard, similar to the "Free From Animal Testing" labels that we've become accustomed to as consumers. However, this is a very paternalist view of our connection with the manufacturing process. The goal of my research isn't just to dictate what "good practice" should be. The stories I've been revealing hope to show the different threads of problems rather than a single answer that fits all. Through "Farmification", I'm giving a design suggestion that would invite more potential alternatives by making the issue more approachable.



How did you manage to get access to a joystick factory in China? Was it a long and painful process or did it require little more than an email?

The best explanation is that "contacts of contacts" offered to me to stay in either a handbag factory or a joystick factory. I chose the joystick, largely because it was interesting as a technology on its way out.

It was a no frills package. I was literally sleeping in the dorms and eating with the workers day after day. At one point I was about to be covered in heat rashes and the factory owner, in exchange for some of the photographs, let me have his spare apartment. There were no glass in the windows and the air conditioning leaked over the bed. For a while I was sleeping with a bowl in my bed. On my third week I managed to 'bribe' a production manager, over a meal of duck congee, to link his broadband from the second floor window across to my window on the fifth floor, for 50rmb (£5). That was probably the best investment I've made.

Ethernet cable

I thought the place was pretty secure, with guard dogs everywhere, but once someone tried to force open my lock in the middle of the night. They took so long that I managed to boil a kettle of water to defend myself. It was like the Three Little Pigs. Luckily for both of us, the door held.

Sometimes I was really questioning if I was doing the right thing but it was worth it. I stayed for about 6 weeks there and after half a year, returned to them with my proposal.



0girl dorm.jpg
Girl Dorm

What makes a joystick factory a fringe? Because surely 230 million people cannot be regarded as fringe?

Factory workers are fringe in terms of our awareness and industrial concern, not in terms of scale. There is in fact a huge amount of people in the peripherals of our vision. Joystick factory workers, specifically, are at the fringe of the innovation cycle. They are at the brink of being left out from demand for the products that they manufacture. They are an emerging group of people designed out by technology.

0aaworkers walking past clear blg.jpg


If I understood correctly, you proposed to the factory owners that workers would work part time at the farm. But what benefit (financial and non financial) is there for the owners of the factory to see their employees desert the building to work in the field?

I'm proposing to encourage debate for all stakeholders, workers, owners, farmers, governments and internationally consumers. Farmification benefits the owners because it increases longevity in their businesses. Farmification would help the community that the owners invested time to build to remain alive through the cycles of a product. For the workers, Farmification stabilises their community as well as deals with their issue of disconnection with their origins. Farming is less alienating than manufacturing because the producers can be directly consumed locally, whereas in factory manufacturing the producers probably can never afford their own product. In addition, factories use shift-work so that the buildings are hardly ever left empty.

In Europe, there is a similar debate of "Farmification" for people to sustain themselves with direct food security. The allotments in the United Kingdom sustained the population against social unrest in the economic depression of in 1930s. Currently there are innovative farming movements with people such as Incredible Edible working to revive farming during the recession and making an impact on British policy-makers.

Mini shrine where workers burn fake money to their ancestors

0factory dormitory units-door.jpg
Factory dormitory units

Watching tv in the street

Family picnic

What is going on at the farm nowadays? Are people still working there?

The factory is sadly downsizing and the farm is a strawberry field right now. I'm not sure if anyone will appreciate the irony of Strawberry Fields Forever.

Image: 鸡爪图片 noiawin

Did you find the answer to that question you're asking in the video "who's making all the food now"?

China's importing huge amounts of food internationally. For example, the chicken feet, which are Chinese delicatessens are being imported out of American chicken factories as waste products. This is a nice story of recycling but depending so heavily on importation is not sustainable for the largest population in the world. China's importing grain in record numbers. There is a 500% increase since last year and it's having a huge impact on the price of food for the global community.


And finally can you tell us a few words about the other fringes you've been exploring since the Farmification project or the fringes that you are planning to explore in the coming months?

Building up from a previous project, Heathrow Heritage, about a local airport community and it's activists, I explored similar possibilities in the airport of Shenzhen. A large proportion of Chinese airspace is militarized and passengers complain of the long delays, often abusing airport staff without giving the problem any further thought. I'm taking stranded passengers out of Shenzhen airport and into the snack streets of the slums surrounding the airport, where the staff live.

0-egascanister onbreak511246.jpg
Lunch break around Shenzhen airport

I'm also starting a collaboration with mindfulness coaches Headspace to break the stereotypes of the meditation community.

Finally, I'm finishing off workshops I hosted on The Future of Sex Education in a Beijing Love Hotel. This project investigates what a sexually active generation, that's never had its own sex education, demand of the future generation. As one of the participants in the workshop puts it: "girls learn from their boyfriends and the boys learn from porn". The collaborators and I had to get through every loophole possible, for example, the anti-nudity technology was so crude that Garfield the cartoon cat was banned because of its tanned body. How do these people evolve their own fantasies when their first points of reference are from Western pornography downloaded from illegal cafes?

Thanks Lisa!

Previously: Heathrow Heritage.
Related: Err (or the creativity of the factory worker), a conversation with Jeremy Hutchison.

 1  |  2  |  3  |  4  |  5  |  6  |  7  |  8  |  9  |  10 
sponsored by: