Last year Seb Lee-Delisle posted about how it would be nice to be able to draw in 3D with the same ease as you can draw in 2D, using the flash drawing API. It inspired me to try something similar, but with a different approach. I was looking into WebGL and OpenGL late last year, and though I got a bit bogged down with my lack of C knowledge, I found I liked the procedural API. The entire scene is actually redrawn every frame. This year I’ve also been thinking about the Turtle graphics system I once used in school, and how it works in a similar way (albeit in 2D). So I took a stab at creating a simple procedural 3D drawing API, along the lines of OpenGL and Logo. First the demo..

flash 10 required

It is currently implemented as a base class which exposes the following methods:

    rotate(angleX, angleY, angleZ)
    draw(x,y,z)
    move(x,y,z)

    pushMatrix()
    popMatrix()

The x, y, z values here are relative to the current position. The current position (and rotation) is maintained by the modelViewMatrix. By calling pushMatrix() you can save the current state and return to it later with popMatrix(). If you have subroutines, it is generally a good idea if they return the modelViewMatrix to the same state. For example in this example the drawSquare() method does this:

private function drawSquare()
{
    draw(100,0,0);
    rotate(0,0,90);
    draw(100,0,0);
    rotate(0,0,90);
    draw(100,0,0);
    rotate(0,0,90);
    draw(100,0,0);
    rotate(0,0,90);//return to starting orientation
}

So this simply draws 4 lines of length 100, rotating 90 degrees around the Z axis. The final rotation isn’t necessary for the drawing, but it ensures that the state is not changed after the routine is called. To draw a cube, we call the drawSquare routine while rotating and moving in 3D.

private function drawCube()
{
    //draw cube
    //back
    drawSquare();

    //left
    pushMatrix();
    rotate(0,90,0);
    drawSquare();
    popMatrix();

    //top
    pushMatrix();
    rotate(-90,0,0);
    drawSquare();
    popMatrix();

    //right
    pushMatrix();
    move(100,0,0);
    rotate(0,90,0);
    drawSquare();
    popMatrix();

    //bottom
    pushMatrix();
    move(0,100,0);
    rotate(-90,0,0);
    drawSquare();
    popMatrix();

    //front 
    pushMatrix();
    move(0,0,-100);
    drawSquare();
    popMatrix();

}

finally, to draw the cubes, we call the drawCube() routine inbetween calls to push/popMatrix() and any changes in position we wish to make.

    pushMatrix();
    drawCube();
    popMatrix();

    pushMatrix();
    move(150,0,0);
    drawCube();
    popMatrix();

    pushMatrix();
    move(-150,0,0);
    drawCube();
    popMatrix();

Once you get the hang of this approach, it is quite intuitive, and reduces the need for calculating a lot of 3D coordinates. This is done behind the scenes in the draw() method, which adds the real 3D coordinates to a list of vertices, which ultimately get projected to 2D coordinates. It’s easy to rotate the objects in the scene by calling rotate() before drawing the objects in the scene.

I’ve used Haxe here because of the faster compile time. There is also an earlier ActionScript version which I’ll link to below.

Da Code

DrawCubes.hx

Draw3D.hx

MatrixStack.as (the AS3 prototype)

GitHub

For Valentine’s Day I’ve dusted off an idea I initially submitted last year as part of the now defunct 25 Lines ActionScript contest. By tweaking the formula which generates the coordinates of the sphere, I was able to get something that more or less looked like a heart. I was able to get the heart modeled and lit within the 25 lines, but could not quite squeeze in the code to animate it. Now, without the limitation of 25 lines, I’ve done that, as well as optimize the performance using the normal map + PixelBender normal map shader that I used in my normal map tutorial. Here is the result - use the sliders to change the direction of the lighting.

flash 10 required

How it Was Done

The mechanics of this are essentially the same as the lighting of a sphere technique described in the normal map tutorial. However, since the shape of the heart is not the same as a sphere, I could not use the sphere formula to generate the normal map. So I just used the same formula to generate the normal map as I did to generate the geometry. For each point on the texture map, I determined the lat/lon of that point, and plugged it in to the formula to get its 3D position. I did the same for the pixel above and to the right. Since we now have three points, we have a triangle, and we can get the normal of the triangle by the cross-product of two of the sides (considered as vectors). This normal gets encoded as a color in the resulting normal map. This is a fairly intensive operation, but since it is only required once at the start, it doesn’t affect the running performance. I tried to optimize this process by using a PixelBender kernel to generate the normal map, but got some odd results, so returned to more familiar ActionScript territory. Using a shader to generate the normal map will become necessary if the geometry changes a lot.

I also refactored the code a lot so that it’s not all in one class. I’ve separated the model, shader and renderer into separate classes, and used a Scene to coordinate them. This is the beginnings of a primitive 3D engine, which I’ve dubbed DeepSee, so if that goes anywhere I may break it out as a separate project on GitHub.

Da Code

Heart2
DeepScene
Heart.as
IModel.as
NormalMapShader
NormalMapShader.pbk

If you are not a Vim user this post will probably not be of much interest or value… anyways, I just wanted to document this, for myself as well :)

Ctags, specifically exuberant ctags is a code-indexing system that creates files which summarize all the methods and properties in your classes. Its also the name of the command line utility that performs the indexing on source code and generates the tags files.

Ctags and Vim go hand in hand.. by specifying tags files to use in your .vimrc file, vim will be able to jump to the file where a class, method, or property is defined. Tags are also used for code-completion, and syntax highlighting.

Unfortunately ActionScript is not natively supported by ctags. Fortunately it is not hard to add support for it. I found out here how to do this for JEdit, but the process is much the same for Vim.

1. Download Ctags source. I used ctags-5.8.tar.gz which worked on my Ubuntu and Mac. I just put it in $HOME/ctags/ and extracted it with ‘tar zxvf ctags-5.8.tar.gz’

2. Ok , go into the source directory (ctags-5.8 or whatever your version is)

3. Add this actionscript.c file into the source directory (you’ll need to Save As… actionscript.c, as github doesn’t seem to have a per-file download link). I found that the version linked the JEdit example did not work in Vim, so I modified the RegEx expressions in it so that they produced more standard tags.

4. In source.mak, add a line that says “actionscript.c " under SOURCES and “actionscript.$(OBJEXT) " under OBJECTS.

5. In parsers.h, we need to add the following line under #define PARSER_LIST:

   ActionScriptParser, \ 

   (an easy way to make these changes is just copy the line and edit the language name)

6. Build it. Do the usual

   ./configure make

You may need to do a ‘make clean’ first to remove any previous builds.

1. Try ‘./ctags –list-languages’, and actionscript one should show up. Then if looks good, go ahead with the final ‘sudo make install’. Thats it.

Now for usage … to build a tags file for a AS3 project in a ‘src’ folder, just do the following:

ctags -R -f tags path/to/src

Eg. I made a tags file for the Flex SDK:

ctags -R -f ${FLEX_HOME}/frameworks/projects/framework/src/tags ${FLEX_HOME}/frameworks/projects/framework/src

ctags may not be happy about generating a tags file over an existing file, so you may need to remove a previous one first. It may also warn about some null references in source code.. this doesn’t seem to matter.

Then I added the line

set tags+=$FLEX_HOME/frameworks/projects/framework/src/tags

to my .vimrc, and I can jump to any Flex SDK file by ctrl-] on the name of a class, method, or property. (ctrl-t jumps back). I can also recommend the taglist plugin. This instantly makes tags for your current file and shows the symbols (organized by type) in a window split.

I also found that it was easy to add support for Haxe, by renaming a few things in the actionscript.c file and renaming it haxe.c (and making the other additions to sources.mak and parsers.h). You can probably figure it our yourself, or you can download the added and patched files here:

ctags_as3_haxe_patched_files.tar.gz

Using Augmented Reality with native Flash 3D (ie. no 3D library)

Augmented Reality (AR) was one of the big things to appear on the flash scene last year. AR has been around for a while, but it new in flash… The FLARToolkit, is an ActionScript port (by Saqoosha, whom I got to see at FITC last year) of a Java AR library (NyARToolkit) IIRC, which was itself derived from the original C version (ARToolkit).

As an interesting aside: ARToolkit isn’t the only AR library implemented in C, though it was the first… there is also ARTag which “uses more complex image processing and digital symbol processing to achieve a higher reliability and immunity to lighting”. For the limited processing power of Flash, ARToolkit was probably a good start, but perhaps some smart people may be able to enhance its recognition by improving the algorithms.

When I tried to use FLARToolkit, I’ll admit I had some trouble, as it seemed rather complex and not very well documented (at least not in English). But fortunately I found FLARManager which makes setting up an AR project in Flash a lot simpler. It also provides some optimizations to the recognition process.

However I feel it could be made still be simpler, and I’m working on that.. although the code still needs some refactoring, I have it basically working with a simple animated butterfly, which is simply a couple of bitmaps with Y rotations applied to them. The trick is to get the matrix that FLARManager gives you back when a marker is recognized, and apply that to your Sprite with 3D content. To use the app, you need to:

1. make sure you have a webcam attached to your computer and turned on. Most macs have them built-in by default. Don’t use any other application which uses the camera at the same time or it won’t work… thus you can also only run one AR enabled website at a time.

2. print out a AR Marker ( see here ) – right click and download any of the .png files or the .pdf, if you want a whole bunch at once, and print on white paper. At a pinch, you can use a black rectangle drawn on paper, as I did in the video below.

3. Go here for the AR Butterfly (but finish reading instructions first ;)

4. When the flash app starts, it will prompt you to allow access to the camera… click the ‘allow’ button. Now I have found this settings thingy to be very glitchy (it can also be accessed by right clicking on a flash movie and selecting ‘Settings…’). On my MacBook, I found it took a long time for the dialog to go away after clicking the ‘allow button’. I’ve also noticed similar problems on Linux. If you’re having trouble with it you can enable the site permission to access the camera by going here: flash player global settings Then you’ll need to find this site in the long list, select it, and click ‘always allow access’ for the camera. That way you won’t be bothered by the buggy dialog anymore (at least on this site).

5. Once you can see the output of the camera in the app, make sure you have decent lighting, hold up the marker to the camera, and adjust position so that there is enough contrast, but also no glare, as FLARToolkit does not handle that very well. Also make sure nothing is in front of the marker, breaking the outlines, or it won’t detect (this is something that ARTag can apparently handle). You should also not move too fast, or have shaky hands… If you’re lucky, you’ll see something like this:

flash 10 required

I noticed that the z-sorting on the butterly wings is inverted in this video… will have to sort that out. I’ll post a link to the code later, once it is a bit more polished. I have several ideas for AR applications, which I’ll hopefully get a chance to work on soon…

So last time we ended up with a chunky looking ball. Great. Lets spice things up a little by turning it into a planet, and a big one at that: Jupiter. Here is what the end result looks like: (note - slider controls direction of lighting, in the x,y,z planes.. I apologize for the crappy GUI)

flash 10 required

If the planet is not centered, but instead you only see the bottom right part of it in the top left, refresh your browser.

There’s two main changes since last time. Firstly, in the pixelbender kernel I’ve applied the shading value to the color value of a second input image (called texture). Since all color values in pixel bender are between 0 and 1, simply multiplying each channel value by a constant in the same range (0-1) has the effect of lightening or darkening it, without changing the hue. First I’ll just dump out the kernel since it is not that long:

 


kernel NormalMap2TextureMap
<   namespace : "com.dafishinsea";
    vendor : "David Wilhelm";
    version : 2;
    description : "kernel to apply shading to a texture map using a normal map";
>
{
    input image4 src;
    input image4 texture;
    output pixel4 dst;
    parameter float xr
    <
        minValue: -1.0;
        maxValue: 1.0;
        defaultValue: 0.1;
    >;
    parameter float yr
    <
        minValue: -1.0;
        maxValue: 1.0;
        defaultValue: 0.0;
    >;
    parameter float zr
    <
        minValue: -1.0;
        maxValue: 1.0;
        defaultValue: 0.0;
    >;

    void
    evaluatePixel()
    {
        //get currrent pixel color
        pixel4 p = sampleNearest(src,outCoord());
        pixel4 tp = sampleNearest(texture,outCoord());
        //image4 is a 4 element vector of RGBA channels
        //we want a vector of the first three as the normal
        float xd = 2.0*p.r - 1.0;
        float yd = 2.0*p.g - 1.0;
        float zd = 2.0*p.b - 1.0;
        float3 normal = float3(xd, yd, zd);
        normal = normalize(normal);
        float3 light = float3(xr, yr, zr) ;
        light = normalize(light);
        float dotp = dot(light, normal);
        //angle between vectors = acos (a dot b)  
        float angle = acos(dotp);
        float shade = angle/3.14159265358;
        //add contrast
        shade = shade*shade;
        dst = pixel4(shade*tp.r, shade*tp.g, shade*tp.b, 1.0);

    }
}

The xr,yr,zr params are the direction of the light, which we passed in from flash. We have now added a second input image called ‘texture’ - this is a spherical map of jupiter. I scaled it down a bit to 640 x 640. We grab the pixel color on line 37, just as we got the color of the normal map. Then we don’t do anything else until the end, after we have calculated the shading value. Then, instead of giving the result as the shading value, we multiply each channel of the texture by the shading value. I also fixed the glitch in the lighting by normalizing the light and normal values.

Now the other change is that I refactored the createNormalMap() method so that the normal map is generated on a per-pixel basis, by getting the normal of the surface of the sphere at each point (pixel) in the texture map. This results in a much smoother normal map.

/**
 * create normal map --evaluate normal for each pixel based lat/lon at that position in the sphere
 */
private function createNormalMap():void
{
    var radius:Number = 1;
    var lon:Number = 0;
    var lat:Number = 0;
    
    for(var v:int = 0; v < mapHeight; ++v)
    {
        lat = (v/mapHeight)*PI;
        var y3d:Number = radius*Math.cos(lat);

        for(var u:int = 0; u < mapWidth; ++u)
        {
            lon = (u/mapWidth)*TWOPI;
            var x3d:Number = radius*Math.cos(lon)*Math.sin(lat);
            var z3d:Number = radius*Math.sin(lon)*Math.sin(lat);
            //normal is vector from center of sphere (consider = origin) to this point
            var norm:Vector3D = new Vector3D(x3d,y3d,z3d);
            norm.normalize();
            var r:uint = 255*(norm.x + 1)/2;
            var g:uint = 255*(norm.y + 1)/2;
            var b:uint = 255*(norm.z + 1)/2;
            var normalColor:uint = r << 16 | g << 8 | b;
            normalMap.setPixel(u,v,normalColor);
        }
    }
}

And here is the result:

As you can see it is really smooth, which results in smooth lighting, even though our sphere is not a very high-polygon mesh. And I’m getting a reasonable frame rate, though using PixelBender always gets my old MacBook’s fan whirring, since it is does not have a supported graphics card, and does everything on the CPU. Even so, the performance is a good deal better than it would have been without using PixelBender. A further optimization would be to save out the normal map to a file and compile it in to the app – this would certainly be worthwile if you had a lot of 3d objects on stage. Another technique to improve performance of this sort of thing is to dynamically reduce the resolution of the texture map when the size of the projected 3d object changes. No need to process a 640 x 640 texture and normal map, if the object itself is only 100 pixels high. This is known as mipmapping, and I may well explore that later.

Also want to point out that this is very primitive lighting. The first time round I did this, the lighting seemed flat, so I squared the shade value to make it a bit more contrasty. A lot more can be achieved here, as lighting techniques are a whole science..some really cool things can be done. I also wanted to explore generating a rough surface texture on the sphere, but that proved more complex than I initially thought so I’ve decided to try that on a plane first. Coming up next… height maps!

Da Code

NormalMap4.as (GitHub)

NormalMapShader.pbk (GitHub)