Creating a battery from a lemon is apparently a common science text book project.... so for Christmas we decided to put our lab coats on and try harnessing the power of the Lemon to light up our Xmas tree. For a battery to work, two different metals must be suspended in an acidic solution. Copper and zinc work particularly well as metals (we used a galvanised nail and a penny) and the citric acid of a lemon provides the acidic solution. Connect a few up (as one lemon produces less than one volt) and tadah!...a Christmas tree twinkles (with LED's)
Here's a simple example using the same colour tracking as the Sticky Striker post-it tracker. Keep in mind that this is not a finished technology and can be optimised and fine tuned.
If you don't have a webcam, this is what it looks like:
We've created a colour tracking application that uses post-it notes as a marker. Here's what it looks like:
And if you have a webcam (and a post-it!) and want to try it out click here to play!
Background
The original idea was to create a simple game that made some use of Augmented Reality (AR) to further explore its uses and applications. And since the World Cup was just around the corner we felt it would appropriate to do a football game. Our main objectives were to create something that would be lightweight, accessible without any installation and, mainly, fun. This is why we went for a Flash application.
So we started to experiment with how we could use AR, specifically it’s tracking engine, to help the player interact with the game. We came up with the idea of having a corner kick situation where the player would be represented by a character in the goal area waiting to head the ball. The player would control the character’s movements by moving it’s own head.
Doing that would require having an AR marker somewhere on the user’s head. We made some headbands with AR markers on the front that, when worn would allow us to track the marker’s movement and thus making it possible to move the character itself.
Augmented Reality
AR has a really good tracking system where the application looks for a specific black and white marker and returns all the coordinates for it. That includes x, y and z positions and rotation on all three dimensions. This is really useful when you want to apply these values to a 3D model for example.
The main problem we had with this approach was that AR tracking on Flash is not really responsive. Since the application is looking for a specific type of marker, if the user moves this said marker too fast, it will become blurry and therefore impossible to track. It also depends on the user’s webcam quality. Another problem we had was CPU usage, which was pretty high with just the tracking system. So we decided to look for alternatives.
Colour Tracking
Colour tracking (CT) offered a different view on this situation. CT means looking for a group of colours and determining their positions. Using CT meant the user could move faster and didn’t have to use a specific marker, which would allow the player to use any common colourful object, say for example, a post-it. Thus was born…
Post-it-Reality™ (not really trademark)
Tracking a post-it that could be stuck on the user’s forehead: simple, funny and weird!
To make this work we had to come with solutions for the problems that CT brings: background interference and ambient light changes and, of course, whilst keeping CPU usage to a minimum.
So how does it actually work?
It’s not too complicated and can be divided into the following processes:
1. Capturing the image and simplifying it. Instead of using colour ranges, we went for a more “post-it centric” approach. The original image from the webcam is “pixelated” and then has the number of colours drastically reduced. This allows us to select small number of colours for the tracking.
2. Tracking the colours. This is basically going through every pixel on the image and deciding whether it is or not part of the group of colours we want to track. This will give us an image with only the desired colours on it.
3. Cluster tracking. The result we get from the above can be almost perfect in some cases e.g. the user is on a room with the right lighting and the background doesn’t have any colours that we are tracking. But this is most commonly not the case. So the next step is to detect groups, or clusters, of colours. This is done by checking the distance between the pixels and separating them into clusters (some times based on their colour).
4. Selecting the appropriate cluster. After we tracked the colours and separated them into clusters we’re still left with the problem of selecting which one of those clusters is the one that represents the post-it. Now, this is the tricky part. In most cases it’s possible to assume that the biggest cluster is the right one. But if the user’s background is, for example, yellow, that might lead to a huge cluster that’s not the post-it. With this scenario in mind we decided to use motion detection to create exclusion zones.
5. Motion detection. By simply checking the current colour of a pixel against its last colour we can, in a way, determine if there’s movement on that area. Doing that we’ll end up with a map of movement areas. Using the same cluster detection as for the colours we can tell where there is motion. Finally we check the distance between the colour clusters and the motion clusters to define if they’re background or not.
Conclusion
We had really good results in most cases. It’s very responsive and allows the user to interact with the game without any sort of calibration. Just stick a post-it on your forehead and play!
It most certainly can be improved for different uses. Specifically if used for installations where it’s possible to know what machine is going to run it and, in some cases, the lighting can be controlled. In our game we didn’t have these possibilities so we had to keep everything quite open.
Even though it might not work 100% in all scenarios it’s still pretty responsive and fun and, after all, it’s just a post-it!
