Today I carried out a simple experiment to demonstrate behaviours that emerge in a social environment when a simple set of rules are followed. The tech. platform I chose to demonstrate this is mobile phones, as they are a readily saturated technology, and is the focus throughout my project for applying the swarm paradigm to a social environment (as the technology already exists).
Each person had to follow these basic rules;
- Walk a straight line, in any direction you choose.
- When somebody crosses your path in front of you, take a photo.
- Act naturally as you would in a normal social environment (probability of greeting others, collision avoidance, etc).
Here is a condensed video of the experiment:
Connections made within the swarm are represented by the camera flash on each persons’ phone. We can see that from just 8 people walking randomly over a period of a few minutes, many person to person connections are established. With current mobile technology and software, these potential connections taking place are in no way exploited in any degree. Mobile phones currently require user interaction to initiate communication via centralised control systems, but what if we could exploit local area user connections on mobile phones, without requiring constant and intimate attention to your device…?
Learning Python for S60 is probably the most awkward, frustrating and undocumented one out of every new language I have learnt in the past 4 years. On many occasions I have been convinced that a function such as connecting to another bluetooth device or making the phone vibrate would be a walk in the park, only to find that the documentation / example code is out of date or doesn’t work on my device. I will briefly cover a couple of gotchas that I have encountered.
Signed Extensions / applications
Many extensions that allow functionality when coding in python need to be signed to the phone itself, due to (overly) strict installation restrictions on all new Symbian S6o devices. This is a major frustration to my project, as it means that I will have to sign my finished app to each phone that it needs to be installed on (plus any supporting files). Signing apps is as easy as uploading the file to this website and waiting for them to email you back a link to the updated file. Then you can proceed to install the app to your device as normal via Bluetooth or USB cable.
https://www.symbiansigned.com/app/page/public/openSignedOnline.do
Discovering Bluetooth devices on Symbian S60 3rd gen OS
Many Python examples on the internet (most on Nokia Forum) of Bluetooth device discovery use an outdated library/extension to be used, that is not compatible with many of the latest Symbian S60 devices (aosocketnew). It doesn’t help the problem that the inbuilt function for this included in Python crashes the phone when called.
To overcome this use PYAOSOCKET, which is the old extension ported to the latest Symbian platform. However suprisingly enough it comes with absolutely NO DOCUMENTATION!! And only 2 examples that I can find on the net using it!
http://www.hiit.fi/files/fi/da/pdis/pyaosocket/web/
Vibration on Symbian S60 3rd gen
Again, many examples of vibration functionality available where written for old versions of Python for S60, yielding them useless for me to use in my project on a new(ish) phone. I managed to find an updated version of an extension called MISTY. Download misty193_unsigned.SIS
* Don’t forget - Make sure vibration is set to ON in your current profile - even if you are running the phone without a SIM card in offline mode - there is a profile for that too!
Non-Blocking Bluetooth Socket Listening
So you want to listen for incoming connections on a Bluetooth socket without it freezing up the UI of your script? This took a while for me to figure out, and it would be easier for me to paste my code to demonstrate how it was achieved. Make sure you use socket.setblocking(Flase)
Considering the open nature of this swarming-experiment platform, I decided to host the code and development of Swarm Sandbox on Google Code. It allows me to use a SVN repositry (version control) for the development of this app. It also allows other members of the development community to become involved in the project if they wish, and allows the general public to browse the source code.
I decided to develop Swarm Sandbox in Python for S60 because it has a vast array of easily accessible API’s for the features of most modern Nokia phones running on the Symbian S60 operating system, also there are some great resources and examples to help developers.
However there are many many bugs and unexpected problems that I have encountered in the short time I have spent developing for mobile Python. My main gripe is that my script crashes whenever I try and scan neighboring bluetooth devices. I have trawled the internet for days looking for a fix to this bug, even tried installing the latest beta releases of Python, however nothing cures it and it is a known bug in the community. I find this absolutely appalling, as it is such a basic inbuilt feature that should work, and they do not seem concerned to fix 
Another issue i’ve come across is the phone not vibrating when told to because I was running it in Offline mode (without a SIM in it)! Bizare!
Because I cannot scan neighboring Bluetooth devices (for the time being), I am using a hard-coded list of phones in the swarm, and also will allow users to manually add the address of other “swarm” phones to the application. A far from ideal solution!
Anyway heres a video of my current prototype and some basic functionality for the app.
Having considered the opportunity to expand on my Pirate Jacket idea from the sound module, I have deemed it not suitable and decided to pursue a slightly different take on my original idea of swarming using mobile phones. I present..
Swarm Sandbox
Having spent many months researching Swarm Intelligence, I have come up with a number of findings and conclusions that have contributed to my dissertation on the subject (download here). It is these findings and knowledge that I would like to explore further in this project.
Keywords
Emergent Behavior, De-centralized Organization, Swarm Intelligence, Mobile Experiences, Social Behavior, Swarm Art
Background
The key principle of any Swarm Intelligence based system is that a number of independent “agents” follow simple rules that dictate their actions. The interactions between these agents and their environment may individually be un-intelligent, however when a swarm of many “agents” follow these given rules the behavior that emerges from this is very interesting to study, and can provide a number of elegant solutions to current problems in computer science and other areas.
Bonabeau, Dorigo and Theraulazn (1999) describe Swarm Intelligence as “any attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societies”. Meaning that everything we artificially create that is inspired by natural swarms is Swarm Intelligence. (Reynhart, 2009)
I have identified that a significant part of these systems that has vastly unknown levels of impact on the emergent properties of the entire swarm is the local ruleset that each “agent” follows. For example small changes in the local ruleset of each “agent” can greatly differ the emergent behaviour of the swarm. Many of these complex processes at work in a swarm system are still not fully understood, and my intention is to produce a platform that enables anyone to explore these processes without any specific hardware or training. In fact it is the goal of this project to situate this experiment of local processes within a swarm to an everyday social environment. The underlying technology that enables me to create this platform is bluetooth-enabled mobile phones, which I will cover in more technical detail at a later date.
Innovation & Differenciating this project
Technical Implementation
Predicted Uses and Findings
I have been recently looking into the possibility of building upon my Sampler Jacket (as documented here) for my final project, replacing my previous idea of a mobile-phone based orchestra. A couple of thoughts and improvements have come to mind, namely the ability to broadcast over local FM frequencies (guerrilla radio style) which adds some new depth to the idea, broadening the potential reach of my 1-man-music machine 
Here is a quick concept drawing.
I really like the idea of hijacking established radio setups, e.g. bars/clubs, supermarket tannoy, etc. Embracing and abusing existing infrastructure to aid spreading an individual message. A system such as this bares similarities to other anti-establishment gadgets and tools developed such as the TV-B-Gone which allows you to turn off TV’s in shop windows and bars. An interesting practical joke carried out by some American students was modifying a CB radio to hijack the drive-through ordering system of a fast-food restaurant, allowing them to talk directly with the customers. The video and howto guide can be found here.
While researching FM transmitters I stumbled across this tiny FM transmitter that runs from a 9V battery and can transmit your voice upto 500M on any user-defined frequency between 80 - 120Mhz on FM band. Very cool!
For my final project, I have been researching some existing installations and projects that sit in similar fields to where my project lies. My dissertation research is heavily based on swarm intelligence, and there is significant crossover between the two modules. Swarm intelligence research is the basis for much of my context in my project, so I will start by taking a look at these first.
Notable swarm intelligence based installations
AtomSwarm by Daniel Jones
This is an interesting software experiment that explores the interaction between a flock of simple agents that follow a set of rules that allows them to create audio textures. Interesting and pleasant audio and visual aesthetic created. Performance mode allows a “conductor” to affect the swarm’s behaviour by modifying variables such as population size and interactions. - I like that each agent can have different genomes and hormones that effects behaviour (inc audio).
The Fragmented Orchestra by Jane Grant, John Matthias and Nick Ryan
This project takes the form of a more permanent installation, and is “a huge distributed musical structure modeled on the firing of the human brain’s neurons.. The Fragmented Orchestra connects 24 public sites across the UK to form a tiny networked cortex, which will adapt, evolve and trigger site-specific sounds via FACT in Liverpool.”
I find this project interesting firstly because of the scale and distribution of the project, literally spanning the entire length of the United Kingdom and using the internet as a network. The simulated “sensors” for the project are microphones, listening to the local happenings at each node and then reacting as a distributed brain, influencing the sounds that are output at the other locations. I also like the feedback loop that the project constructs, allowing participants to interact with the piece, and then listen to their contribution in the global track.
In an effort to keep my final project turning over, I decided to begin work on a prototype that provides some basic functionality of the finished product. Hopefully doing this and experimenting with settings and scenario’s I shall be able to refine my idea and let it evolve into something better!
I was originally going to create a Flash application in Flash Lite and run that from the phone, however I have since learned that for the mobile operating system S60 (which my development handset runs) the simplest, most integrated and supported language to write scripts for is Python.
The Nokia forums & Wiki provide an excellent resource for python development; http://wiki.forum.nokia.com/index.php/Category:Python
I have managed to set up a development environment on my phone and Macbook that allows me to very easily write and debug scripts on the phone very quickly, via Bluetooth. This Bluetooth connection is also what allows me to send the accelerometer data from my phone directly to Max/MSP on my Macbook.
Instructable on making the Bluetooth link between phone and OS X; http://www.instructables.com/id/SRSP943F6B7SYBG/
Instructable on setting up PUTools (allows file synchronizing across OS X & phone); http://www.instructables.com/id/SPSD55IF6B7T24U/
MobileLenin - An impressive achieve of fantastic beginner mobile python scripts; http://www.mobilenin.com/pys60/menu.htm
Over the Christmas break I had time to think more about my project and bounce some ideas off family and friends getting some “fresh” feedback.
I have managed to refine my idea slightly, and my prototype can be summarized in the following;
I intend to create a platform which promotes an ad-hoc network of compatible mobile devices to be formed, which can occur anywhere, provided they are within a short distance of each other. The software on each user’s mobile device allows them to “conduct” or play a unique instrument, which when played with others creates a “mobile orchestra”.
Each musician can control their instrument by way of movement and gestures, much the way that orchestra conductors behave. This is achieved by using tilt sensors and accelerometers on the mobile device, the data is then transmitted via Bluetooth to the others.
My initial prototype will only achieve some basic functionality, and will not resemble the finished project. I plan to write a basic Flash Lite app for the phone, which retrieves values from the accelerometer sensor and transmits them to my Macbook via Bluetooth. I will then write a patch in Max/MSP that can pickup the values recieved via Bluetooth and create a basic instrument synth based on them.
I have spent today creating a London Underground-esq map to try and visually represent my final project in a clear and easy to understand way (Click for an enlarged version).
More to come soon…
I have recently been researching the use of swarm intelligence in generative music software and installations, and have stumbled across some lovely examples. The flocking orchestra is a lovely example of using the classic boids flocking model in 3D space and having the agents triggering basic midi instruments.
Another example of this sort of swarm-driven performance piece is AtomSwarm by Daniel Jones. This example is beutiful both in visual complexity and takes a original and more developed approach to the sound synthesis. I have included of it here.
Another interesting piece that relates to my area of research is the Bacteria Orchestra, this is an installation containing a number of different nodes or “cells” which contain both speakers and a microphone each. Each cell is “born” with a different set of DNA which dictates how it reacts to it’s surroundings, together the cells act together in a harmonious synchronised orchestra, constantly evolving and changing due to each cells environment.
While doing some web research for my final year project, I stumbled upon this fantastic installation that epitomizes all of the areas of interest that I am currently pursuing for my project. You could say that this is the inspiration for it, however I hope that my idea evolves from this and doesn’t end up too similar!
Phlock is an interactive installation that allows viewers to sequence sounds and create music using the emergent behavior of digital creatures.
Cameron Browning’s Phlock