Da Fish in Sea

These are the voyages of Captain Observant

AR You Serious?

| Comments

Using Augmented Reality with native Flash 3D (ie. no 3D library)

Augmented Reality (AR) was one of the big things to appear on the flash scene last year. AR has been around for a while, but it new in flash… The FLARToolkit, is an ActionScript port (by Saqoosha, whom I got to see at FITC last year) of a Java AR library (NyARToolkit) IIRC, which was itself derived from the original C version (ARToolkit).

As an interesting aside: ARToolkit isn’t the only AR library implemented in C, though it was the first… there is also ARTag which “uses more complex image processing and digital symbol processing to achieve a higher reliability and immunity to lighting”. For the limited processing power of Flash, ARToolkit was probably a good start, but perhaps some smart people may be able to enhance its recognition by improving the algorithms.

When I tried to use FLARToolkit, I’ll admit I had some trouble, as it seemed rather complex and not very well documented (at least not in English). But fortunately I found FLARManager which makes setting up an AR project in Flash a lot simpler. It also provides some optimizations to the recognition process.

However I feel it could be made still be simpler, and I’m working on that.. although the code still needs some refactoring, I have it basically working with a simple animated butterfly, which is simply a couple of bitmaps with Y rotations applied to them. The trick is to get the matrix that FLARManager gives you back when a marker is recognized, and apply that to your Sprite with 3D content. To use the app, you need to:

  1. make sure you have a webcam attached to your computer and turned on. Most macs have them built-in by default. Don’t use any other application which uses the camera at the same time or it won’t work… thus you can also only run one AR enabled website at a time.

  2. print out a AR Marker ( see here ) – right click and download any of the .png files or the .pdf, if you want a whole bunch at once, and print on white paper. At a pinch, you can use a black rectangle drawn on paper, as I did in the video below.

  3. Go here for the AR Butterfly (but finish reading instructions first ;)

  4. When the flash app starts, it will prompt you to allow access to the camera… click the ‘allow’ button. Now I have found this settings thingy to be very glitchy (it can also be accessed by right clicking on a flash movie and selecting ‘Settings…’). On my MacBook, I found it took a long time for the dialog to go away after clicking the ‘allow button’. I’ve also noticed similar problems on Linux. If you’re having trouble with it you can enable the site permission to access the camera by going here: flash player global settings Then you’ll need to find this site in the long list, select it, and click ‘always allow access’ for the camera. That way you won’t be bothered by the buggy dialog anymore (at least on this site).

  5. Once you can see the output of the camera in the app, make sure you have decent lighting, hold up the marker to the camera, and adjust position so that there is enough contrast, but also no glare, as FLARToolkit does not handle that very well. Also make sure nothing is in front of the marker, breaking the outlines, or it won’t detect (this is something that ARTag can apparently handle). You should also not move too fast, or have shaky hands… If you’re lucky, you’ll see something like this:

flash 10 required

I noticed that the z-sorting on the butterly wings is inverted in this video… will have to sort that out. I’ll post a link to the code later, once it is a bit more polished. I have several ideas for AR applications, which I’ll hopefully get a chance to work on soon…