Image courtesy LABoral Centro de Arte y Creación Industrial (Art and Industrial Creation Centre)/Paula Andrés
Image courtesy the artists
Sofian Audry, Stephen Kelly and Samuel St-Aubin started working on Vessels in 2010. The aquatic installation is a fleet of 50 autonomous robots that gradually build up their own micro system by interacting with each other and by collecting and interpreting data related to water and air quality, temperature, ambient light, sound, etc.
However, the robots do not simply process scientific readings, they also communicate through behaviours and interactions. For example, an increase in temperature sensed by one agent may cause it to act more aggressively, with erratic or irrational (random) movements. This change in behaviour will influence its neighbouring agents, who may respond with relative changes to their own behaviour. These agents will in turn influence their neighbours, thus creating a ripple effect of actions.
The ecosystem is thus generated over time by the robots themselves and by their particular environment.
Photo Beatriz Orviz, LABoral
Image courtesy LABoral Centro de Arte y Creación Industrial (Art and Industrial Creation Centre)/Paula Andrés
Samuel, Sofian and Stephen have just spent the Summer improving and researching Vessels as part of their artistic residence in Platform 0 at LABoral Centro de Arte.
Since i was curious about those luminous little robots in a white swimming pool, i asked the artists to talk to us about the work:
Hi Samuel, Sofian and Stephen! I read on Laboral’s blog that your artistic residency is based on the Vessels project. So what will the residency consist of exactly? Are you going to make Vessels more sophisticated? Or build on it to make an entirely different project? Or investigate another aspect of the installation?
We started the Vessels project in 2010 at the Center for Art Tapes, where the concept and technical structure was initiated within two fairly brief 2-week residencies. Since then, we’ve spent time fine tuning various aspects of the work to better withstand real-world environments. For example, we had to abandon our original design with air propulsion because it was too energy-consuming and would easily catch in the wind. The version we are currently working on is the third prototype and works with water propulsion. We validated the final design of the electronic boards last winter during a short residency at the Perte de Signal art center.
The goal of the LABoral residency was to assemble the first large group of robots with our new technical improvements, to finalize the material/aesthetic design and to make a first working version of the software. Because we ran into all kinds of technical problems, we decided to put less effort on the material design and more on the software. Thus we spent a large portion of our time at LABoral developing the behaviour of the robot collective.
Vessels at Nocturne 2010 (Public Garden location) in Halifax, Canada. Photo credit: Mat Dunlap
Vessels is a fleet of 50 aquatic vehicles. That is a lot of robots. So first of all, why did you need to build so many robots? And how big are they exactly? i suspect that they will also need a large area to float around…
When we did our presentation in Halifax, we had about a dozen of robots and we felt it was hard for them to occupy an outdoor space, given their relatively small size (about 20-25 cm in diameter depending on the version). They looked kind of lost. By scaling up their population, we believe we can give a real presence to the installation in large natural environments such as lakes and ponds.
Also, we are interested in the kind of behaviors that can emerge from the interaction between a massive group of autonomous robots, which is something that has not been fully explored in the art world. A lot of work in robotic art has been done on singular robots or small assemblies of big robots but not so much with large groups of small, autonomous robotic agents. In the past decade, a lot of research in the scientific world has been carried out involving swarm robotics and multi-agent collaboration, with encouraging results. Behavioural diversity is something we’re interested in exploring with Vessels, and more robots means more potential diversity within the ‘population’.
Finally, we felt like 50 robots would give us more flexibility. For instance, we could show two groups of 25 robots at the same time in two different spots in a city, or even in different cities. Because the robots will react to their immediate environment, they will behave differently in the different contexts they are put in.
Image courtesy the artists
Image courtesy LABoral Centro de Arte y Creación Industrial (Art and Industrial Creation Centre)/Paula Andrés
Is Vessels a group of identical robots? Do they all start with the same set of sensors?
Almost. All the robots have more or less the same “body”. They each have a pair of distance sensors, a compass, a directional IR communication system, a pair of underwater pumps for propulsion, a set of LEDs, an onboard real-time clock and some external flash memory for data logging. They will also be using the exact same software.
Their only difference lies in the fact that each bot will eventually be equipped with a unique “environmental” sensor. Each robot has an external, pluggable “card” that we designed to take care of sound production and accommodate this unique sensor. For instance, one robot can be able to measure air temperature, while another one will know about the air pressure, another one about the pH of water, and so on. This sensor will give the robot its “personality”, so to speak. They will react to their own sensor in a specific way and their reaction will influence the actions of other robots. The idea is that by putting the same group of robots in different settings (i.e. with different environmental conditions) they will produce a distinctive collective behavior.
But we’re still a long way from that! In the version that we produced at LABoral, we don’t yet have these environmental sensors. We focused more on establishing the software framework that will enable individual personalities AND group behaviors.
Samuel St-Aubin and Sofian Audry at work in LABoral
Image courtesy LABoral Centro de Arte y Creación Industrial (Art and Industrial Creation Centre)
Image courtesy LABoral Centro de Arte y Creación Industrial (Art and Industrial Creation Centre)
Photo courtesy of the artists
Each aquatic vehicle learns and develops a behavior through Reinforcement Learning and “Over an extended process of trial and error, RL makes it possible for computers to do things that they were not explicitly programmed to do.”
If i understood correctly you do not have complete control over what the robots learn and how they evolve. So have they surprised you in the way they learn, interact, behave?
We’ve just begun to implement learning for individual robots in very simple tasks. We did some small experiments with Reinforcement Learning in which we were able to get a robot to learn how to go straight, which is not an easy task for these round-shaped robots (they tend to spin easily!) At this point we’re taking baby steps with learning, so we have yet to see the implications of the entire population of robots with learned behaviours.
Since we wanted to build a first version that “worked” somehow, in terms of the robots going around the surface of water, being able to move straight, approaching one another, etc. we had to work at a much higher level. We thus let the learning stuff aside for a start and decided to work using an Artificial Intelligence approach that is currently very popular in the video game industry for the design of intelligent behaviors. This method, known as Behavior Trees, allows the design of complex, hierarchical behaviors. It makes it easy to design priorities for the robot and to allow it to try different strategies to achieve its goals (or fulfill its desires if you prefer). For example, in our current implementation, the robots move around freely, but when they hit an obstacle they interrupt their moving and try to avoid getting stuck. They also interrupt their behavior when they receive a message from another robot, which might change what they are doing at that moment.
Machine learning methods such as Reinforcement Learning and Genetic Programming are tricky, especially in the context of creating an artwork. They’re optimization techniques, so they work well when one tries to solve a specific problem like ‘how to navigate in a straight line’. But in an artistic context, the problems are blurry, so we have to invent new methodologies. For example, you can achieve interesting results by playing with the reward functions of the agents, such as what Sofian did last year at LABoral as part of the installation/performance n-Polytope by Chris Salter and collaborators. Also, the process of learning itself has sometimes a very interesting aesthetic value. So, the current focus of our research is how to use machine learning as a critical tool, helping the robots learn behaviours with respect to their environment that might eventually surprise us.
Image courtesy LABoral Centro de Arte y Creación Industrial (Art and Industrial Creation Centre)
The spectator plays a role in the work. Can you explain it how the public might be involved in and maybe even influence the installation?
Although the environment sensor of some of the robots might be influenced directly by the audience (e.g. if some of them have microphones or light detectors), the installation is not meant to be interactive per se. We see it more as a piece you experience through indirect, slow interaction, where the bots are simply added to our existing ecosystem, responding to it. We hope the audience will respond to it by projecting their own cultural references on the robots, that they will recognize themselves in them.
On another level, we’d like the audience to begin to ask themselves questions. Why are the robots grouping together? Why is this robot making that sound? Why are they so hectic now while they were calm a minute ago? How does that relate to the site they’re currently swimming on?
Any upcoming project, exhibition, field of research you’d like to share with us?
Samuel will be attending the Bozar Electronic Art Festival (BEAF) in Brussels from September 25 to 29. Stephen just finished a major work titled Patch at Dalhousie University (Halifax, NS/CA) with robotic agents that react to the presence of students in classrooms. Sofian’s underwater artificial life installation Plasmosis is still running at the marina of Carleton-sur-Mer until September 7th (QC/CA). We are also trying to organize another research residency next year for Vessels but we have no definite plan yet.