flash 10 required

It is currently implemented as a base class which exposes the following methods:

1
2
3
4
5
6
    rotate(angleX, angleY, angleZ)

    draw(x,y,z)

    move(x,y,z)



    pushMatrix()

    popMatrix()

The x, y, z values here are relative to the current position. The current position (and rotation) is maintained by the modelViewMatrix. By calling pushMatrix() you can save the current state and return to it later with popMatrix(). If you have subroutines, it is generally a good idea if they return the modelViewMatrix to the same state. For example in this example the drawSquare() method does this:

1
2
3
4
5
6
7
8
9
10
11
private function drawSquare()

{

    draw(100,0,0);

    rotate(0,0,90);

    draw(100,0,0);

    rotate(0,0,90);

    draw(100,0,0);

    rotate(0,0,90);

    draw(100,0,0);

    rotate(0,0,90);//return to starting orientation

}

So this simply draws 4 lines of length 100, rotating 90 degrees around the Z axis. The final rotation isn’t necessary for the drawing, but it ensures that the state is not changed after the routine is called. To draw a cube, we call the drawSquare routine while rotating and moving in 3D.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
private function drawCube()

{

    //draw cube

    //back

    drawSquare();



    //left

    pushMatrix();

    rotate(0,90,0);

    drawSquare();

    popMatrix();



    //top

    pushMatrix();

    rotate(-90,0,0);

    drawSquare();

    popMatrix();



    //right

    pushMatrix();

    move(100,0,0);

    rotate(0,90,0);

    drawSquare();

    popMatrix();



    //bottom

    pushMatrix();

    move(0,100,0);

    rotate(-90,0,0);

    drawSquare();

    popMatrix();



    //front 

    pushMatrix();

    move(0,0,-100);

    drawSquare();

    popMatrix();



}

finally, to draw the cubes, we call the drawCube() routine inbetween calls to push/popMatrix() and any changes in position we wish to make.

1
2
3
4
5
6
7
8
9
10
11
12
13
    pushMatrix();

    drawCube();

    popMatrix();



    pushMatrix();

    move(150,0,0);

    drawCube();

    popMatrix();



    pushMatrix();

    move(-150,0,0);

    drawCube();

    popMatrix();

Once you get the hang of this approach, it is quite intuitive, and reduces the need for calculating a lot of 3D coordinates. This is done behind the scenes in the draw() method, which adds the real 3D coordinates to a list of vertices, which ultimately get projected to 2D coordinates. It’s easy to rotate the objects in the scene by calling rotate() before drawing the objects in the scene.

I’ve used Haxe here because of the faster compile time. There is also an earlier ActionScript version which I’ll link to below.

Da Code

DrawCubes.hx

Draw3D.hx

MatrixStack.as (the AS3 prototype)

GitHub

flash 10 required

How it Was Done

The mechanics of this are essentially the same as the lighting of a sphere technique described in the normal map tutorial. However, since the shape of the heart is not the same as a sphere, I could not use the sphere formula to generate the normal map. So I just used the same formula to generate the normal map as I did to generate the geometry. For each point on the texture map, I determined the lat/lon of that point, and plugged it in to the formula to get its 3D position. I did the same for the pixel above and to the right. Since we now have three points, we have a triangle, and we can get the normal of the triangle by the cross-product of two of the sides (considered as vectors). This normal gets encoded as a color in the resulting normal map. This is a fairly intensive operation, but since it is only required once at the start, it doesn’t affect the running performance. I tried to optimize this process by using a PixelBender kernel to generate the normal map, but got some odd results, so returned to more familiar ActionScript territory. Using a shader to generate the normal map will become necessary if the geometry changes a lot.

I also refactored the code a lot so that it’s not all in one class. I’ve separated the model, shader and renderer into separate classes, and used a Scene to coordinate them. This is the beginnings of a primitive 3D engine, which I’ve dubbed DeepSee, so if that goes anywhere I may break it out as a separate project on GitHub.

Da Code

Heart2

DeepScene

Heart.as

IModel.as

NormalMapShader

NormalMapShader.pbk

Augmented Reality (AR) was one of the big things to appear on the flash scene last year. AR has been around for a while, but it new in flash… The FLARToolkit, is an ActionScript port (by Saqoosha, whom I got to see at FITC last year) of a Java AR library (NyARToolkit) IIRC, which was itself derived from the original C version (ARToolkit).

As an interesting aside: ARToolkit isn’t the only AR library implemented in C, though it was the first… there is also ARTag which “uses more complex image processing and digital symbol processing to achieve a higher reliability and immunity to lighting”. For the limited processing power of Flash, ARToolkit was probably a good start, but perhaps some smart people may be able to enhance its recognition by improving the algorithms.

When I tried to use FLARToolkit, I’ll admit I had some trouble, as it seemed rather complex and not very well documented (at least not in English). But fortunately I found FLARManager which makes setting up an AR project in Flash a lot simpler. It also provides some optimizations to the recognition process.

However I feel it could be made still be simpler, and I’m working on that.. although the code still needs some refactoring, I have it basically working with a simple animated butterfly, which is simply a couple of bitmaps with Y rotations applied to them. The trick is to get the matrix that FLARManager gives you back when a marker is recognized, and apply that to your Sprite with 3D content. To use the app, you need to:

1. make sure you have a webcam attached to your computer and turned on. Most macs have them built-in by default. Don’t use any other application which uses the camera at the same time or it won’t work… thus you can also only run one AR enabled website at a time.

2. print out a AR Marker ( see here ) — right click and download any of the .png files or the .pdf, if you want a whole bunch at once, and print on white paper. At a pinch, you can use a black rectangle drawn on paper, as I did in the video below.

3. Go here for the AR Butterfly (but finish reading instructions first

4. When the flash app starts, it will prompt you to allow access to the camera… click the ‘allow’ button. Now I have found this settings thingy to be very glitchy (it can also be accessed by right clicking on a flash movie and selecting ‘Settings…’). On my MacBook, I found it took a long time for the dialog to go away after clicking the ‘allow button’. I’ve also noticed similar problems on Linux. If you’re having trouble with it you can enable the site permission to access the camera by going here:
   flash player global settings
   Then you’ll need to find this site in the long list, select it, and click ‘always allow access’ for the camera. That way you won’t be bothered by the buggy dialog anymore (at least on this site).

5. Once you can see the output of the camera in the app, make sure you have decent lighting, hold up the marker to the camera, and adjust position so that there is enough contrast, but also no glare, as FLARToolkit does not handle that very well. Also make sure nothing is in front of the marker, breaking the outlines, or it won’t detect (this is something that ARTag can apparently handle). You should also not move too fast, or have shaky hands… If you’re lucky, you’ll see something like this:

flash 10 required

I noticed that the z-sorting on the butterly wings is inverted in this video… will have to sort that out. I’ll post a link to the code later, once it is a bit more polished. I have several ideas for AR applications, which I’ll hopefully get a chance to work on soon…

flash 10 required

If the planet is not centered, but instead you only see the bottom right part of it in the top left, refresh your browser.

There’s two main changes since last time. Firstly, in the pixelbender kernel I’ve applied the shading value to the color value of a second input image (called texture). Since all color values in pixel bender are between 0 and 1, simply multiplying each channel value by a constant in the same range (0-1) has the effect of lightening or darkening it, without changing the hue. First I’ll just dump out the kernel since it is not that long:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
<languageVersion : 1.0;>



kernel NormalMap2TextureMap

<   namespace : "com.dafishinsea";

    vendor : "David Wilhelm";

    version : 2;

    description : "kernel to apply shading to a texture map using a normal map";

>

{

    input image4 src;

    input image4 texture;

    output pixel4 dst;

    parameter float xr

    <

        minValue: -1.0;

        maxValue: 1.0;

        defaultValue: 0.1;

    >;

    parameter float yr

    <

        minValue: -1.0;

        maxValue: 1.0;

        defaultValue: 0.0;

    >;

    parameter float zr

    <

        minValue: -1.0;

        maxValue: 1.0;

        defaultValue: 0.0;

    >;



    void

    evaluatePixel()

    {

        //get currrent pixel color

        pixel4 p = sampleNearest(src,outCoord());

        pixel4 tp = sampleNearest(texture,outCoord());

        //image4 is a 4 element vector of RGBA channels

        //we want a vector of the first three as the normal

        float xd = 2.0*p.r - 1.0;

        float yd = 2.0*p.g - 1.0;

        float zd = 2.0*p.b - 1.0;

        float3 normal = float3(xd, yd, zd);

        normal = normalize(normal);

        float3 light = float3(xr, yr, zr) ;

        light = normalize(light);

        float dotp = dot(light, normal);

        //angle between vectors = acos (a dot b)  

        float angle = acos(dotp);

        float shade = angle/3.14159265358;

        //add contrast

        shade = shade*shade;

        dst = pixel4(shade*tp.r, shade*tp.g, shade*tp.b, 1.0);



    }

}

The xr,yr,zr params are the direction of the light, which we passed in from flash. We have now added a second input image called ‘texture’ – this is a spherical map of jupiter. I scaled it down a bit to 640 x 640. We grab the pixel color on line 37, just as we got the color of the normal map. Then we don’t do anything else until the end, after we have calculated the shading value. Then, instead of giving the result as the shading value, we multiply each channel of the texture by the shading value. I also fixed the glitch in the lighting by normalizing the light and normal values.

Now the other change is that I refactored the createNormalMap() method so that the normal map is generated on a per-pixel basis, by getting the normal of the surface of the sphere at each point (pixel) in the texture map. This results in a much smoother normal map.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
/**

 * create normal map --evaluate normal for each pixel based lat/lon at that position in the sphere

 */

private function createNormalMap():void

{

    var radius:Number = 1;

    var lon:Number = 0;

    var lat:Number = 0;

    

    for(var v:int = 0; v < mapHeight; ++v)

    {

        lat = (v/mapHeight)*PI;

        var y3d:Number = radius*Math.cos(lat);



        for(var u:int = 0; u < mapWidth; ++u)

        {

            lon = (u/mapWidth)*TWOPI;

            var x3d:Number = radius*Math.cos(lon)*Math.sin(lat);

            var z3d:Number = radius*Math.sin(lon)*Math.sin(lat);

            //normal is vector from center of sphere (consider = origin) to this point

            var norm:Vector3D = new Vector3D(x3d,y3d,z3d);

            norm.normalize();

            var r:uint = 255*(norm.x + 1)/2;

            var g:uint = 255*(norm.y + 1)/2;

            var b:uint = 255*(norm.z + 1)/2;

            var normalColor:uint = r << 16 | g << 8 | b;

            normalMap.setPixel(u,v,normalColor);

        }

    }

}

And here is the result:

As you can see it is really smooth, which results in smooth lighting, even though our sphere is not a very high-polygon mesh. And I’m getting a reasonable frame rate, though using PixelBender always gets my old MacBook’s fan whirring, since it is does not have a supported graphics card, and does everything on the CPU. Even so, the performance is a good deal better than it would have been without using PixelBender. A further optimization would be to save out the normal map to a file and compile it in to the app — this would certainly be worthwile if you had a lot of 3d objects on stage. Another technique to improve performance of this sort of thing is to dynamically reduce the resolution of the texture map when the size of the projected 3d object changes. No need to process a 640 x 640 texture and normal map, if the object itself is only 100 pixels high. This is known as mipmapping, and I may well explore that later.

Also want to point out that this is very primitive lighting. The first time round I did this, the lighting seemed flat, so I squared the shade value to make it a bit more contrasty. A lot more can be achieved here, as lighting techniques are a whole science..some really cool things can be done. I also wanted to explore generating a rough surface texture on the sphere, but that proved more complex than I initially thought so I’ve decided to try that on a plane first. Coming up next… height maps!

Da Code

NormalMap4.as (GitHub)

NormalMapShader.pbk (GitHub)

But it is not necessary to recalculate the normals every frame, since they don’t change, at least in the object coordinates. Thus it makes sense to cache the normals somehow – this is where the normal map comes in. So we can refactor our current example by converting the normals of each surface to a color, where the Red, Green, and Blue channels represent the x, y, & z components of the vector. How do you transform a vector into a color? I did it like this:

1
2
3
4
            var r:uint = 255*(normal.x + 1)/2;

            var g:uint = 255*(normal.y + 1)/2;

            var b:uint = 255*(normal.z + 1)/2;

            var normalColor:uint = r << 16 | g << 8 | b;

Since the normal is in the range of -1 to 1, we add one to it to put it in the range of 0 – 2, then divide by 2 to get it in the 0 -1 range. Multiplying by 255 gives a 8-bit value. The we shift these over to the right channels for a RGB color using binary operators. So that gives a normal map.. it actually looks quite pretty… click the buttons in the example below to see the normal map and the texture map (which currently just has the shading in it).

flash 10 required

Another advantage of having the normal map in the form of an image, is that we can use PixelBender to process it to produce the lighting/shading from the combintation of the normals and the light direction. All we have to do is the same calculations we were doing before (explained in previous lession) inside the Pixelbender kernel. As it turns out, PixelBender had lots of builtin functions to help with this kind of calculation, and it pretty straightforward. The kernel is really tiny, so here it is in full:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
    <languageVersion : 1.0;>



    kernel NormalMap2TextureMap

    <   namespace : "com.dafishinsea";

        vendor : "David Wilhelm";

        version : 1;

        description : "kernel to generate a texture map (shading only) from a normal map";

    >

    {

        input image4 src;

        output pixel4 dst;

        parameter float xr

        <

            minValue: -1.0;

            maxValue: 1.0;

            defaultValue: 0.1;

        >;

        parameter float yr

        <

            minValue: -1.0;

            maxValue: 1.0;

            defaultValue: 0.0;

        >;

        parameter float zr

        <

            minValue: -1.0;

            maxValue: 1.0;

            defaultValue: 0.0;

        >;



        void

        evaluatePixel()

        {

            //get currrent pixel color

            pixel4 p = sampleNearest(src,outCoord());

            //image4 is a 4 element vector of RGBA channels

            //we want a vector of the first three as the normal

            float xd = 2.0*p.r - 1.0;

            float yd = 2.0*p.g - 1.0;

            float zd = 2.0*p.b - 1.0;

            float3 normal = float3(xd, yd, zd);

            float3 light = float3(xr, yr, zr) ;

            float dotp = dot(light, normal);

            //angle between vectors = acos (a dot b)  

            float angle = acos(dotp);

            float clr = angle/3.14159265358;

            dst = pixel4(clr, clr, clr, 1.0);



        }

    }

All this does is convert the normal (color at the source pixel) to a shading color (greyscale) depending on its variance with the lighting direction (which is passed in to the kernel as a parameter — actually 3, for x,y,z components). The huge advantage here is that this process gets called by Actionscript as a ShaderJob, and it runs ansynchronously. The result gets used as the texture for the sphere. I won’t go over the mechanics of using a ShaderJob — check out this article for more info. Or you can just read the code below, for all the gory details.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
        package com.dafishinsea.tutorials.normalmap {



            import com.bit101.components.*;

            import flash.display.Bitmap;

            import flash.display.BitmapData;

            import flash.display.Shader;

            import flash.display.ShaderJob;

            import flash.display.Shape;

            import flash.display.Sprite;

            import flash.display.StageAlign;

            import flash.display.StageScaleMode;

            import flash.display.TriangleCulling;

            import flash.events.Event;

            import flash.filters.ShaderFilter;

            import flash.geom.Matrix3D;

            import flash.geom.PerspectiveProjection;

            import flash.geom.Point;

            import flash.geom.Rectangle;

            import flash.geom.Utils3D;

            import flash.geom.Vector3D;

            import flash.utils.ByteArray;

            import net.hires.debug.Stats;



            [SWF(backgroundColor="0xffffff", width="800", height="600", frameRate="50")]    

            public class NormalMap3 extends Sprite {



                private var texture:BitmapData = new BitmapData(500,500,false,0xFFFFFF);

                private var textureBmp:Bitmap = new Bitmap(texture);

                private var normalMap:BitmapData = new BitmapData(500,500,false,0xFFFFFF);

                private var normalBmp:Bitmap = new Bitmap(normalMap);

                private var normalMapShp:Shape = new Shape();

                private var vertices:Vector.<Number>;

                private var indices:Vector.<int>;

                private var uvtData:Vector.<Number>;

                private var perspective: PerspectiveProjection;

                private var projectionMatrix : Matrix3D;

                private var rotationMatrix : Matrix3D;

                private var projectedVerts:Vector.<Number>;

                private var focalLength:Number = 50;

                private var container:Sprite;

                private const PI:Number = Math.PI;//half revolution in radians

                private const HALFPI:Number = Math.PI/2;//1/4 revolution in radians

                private const TWOPI:Number = 2*Math.PI;//full revolution in radians

                //x,yz rotation

                private var rx:Number = 60;

                private var ry:Number = 40;

                private var rz:Number = 30;

                private var testBmp:Bitmap;

                private var r:Number = 0;//yrotation of sphere

                private var normalMapShader:Shader;

                private var shaderJob:ShaderJob;

                private var input:ByteArray;

                private var output:ByteArray;

                private var normalMapRdo:PushButton;

                private var textureMapRdo:PushButton;

                private var noMapsRdo:PushButton;



                [Embed(source="NormalMapShader.pbj", mimeType="application/octet-stream")] 

                private var NormalMapShader:Class; 



                public function NormalMap3() {

                    init();

                    trace("init");

                }



                private function init():void {

                    stage.align = StageAlign.TOP_LEFT;

                    stage.scaleMode = StageScaleMode.NO_SCALE;

                    vertices = new Vector.<Number>();

                    indices = new Vector.<int>();

                    uvtData = new Vector.<Number>();

                    //set up perspective

                    perspective = new PerspectiveProjection();

                    perspective.fieldOfView = 50;

                    //3D transformation matrix - used to rotate object

                    projectionMatrix = perspective.toMatrix3D();

                    rotationMatrix = new Matrix3D();//used to keep track of rotation of the sphere

                    projectedVerts   = new Vector.<Number>();

                    //container  to hold scene

                    container = new Sprite();

                    container.x = stage.stageWidth/2;

                    container.y = stage.stageHeight/2;

                    addChild(container);

                    createSphere(100, 40, 80);

                    createNormalMap();

                    createShaderJob();

                    addEventListener(Event.ENTER_FRAME, onEnterFrame);

                    addEventListener(Event.RENDER, render, false, 0, true);

                    addChild(new Stats());

                    //texture map

                    //controls

                    normalMapRdo = new PushButton(this, 500, 30, "Show normal map", onNomalMapBtnClick);

                    textureMapRdo = new PushButton(this, 500, 50, "Show texture map", onTextureRdoClick);

                    noMapsRdo = new PushButton(this, 500, 10, "Show only sphere", onNoMapsRdoClick);

                }

                protected function onNomalMapBtnClick(event:Event):void

                {

                    if(textureBmp.parent == this)

                        removeChild(textureBmp);

                    addChild(normalBmp);

                }



                protected function onTextureRdoClick(event:Event):void

                {

                    if(normalBmp.parent == this)

                        removeChild(normalBmp);

                    addChild(textureBmp);

                }



                protected function onNoMapsRdoClick(event:Event):void

                {

                    if(normalBmp.parent == this)

                        removeChild(normalBmp);

                    if(textureBmp.parent == this)

                        removeChild(textureBmp);

                }



                private function createSphere(radius:Number, rows:int, cols:int):void {



                    var lon_incr:Number = TWOPI/cols;

                    var lat_incr = PI/rows;

                    var lon:Number = 0;//angle of rotation around the y axis, *in radians*

                    var lat:Number = 0;//angle of rotation around the x axis

                    var x:Number, y:Number, z:Number;

                    var vnum:int = 0;

                    var ind:int = 0;

                    //a full rotation is PI radians

                    for(var h:int = 0; h <= rows; ++h)

                    {

                        lon = 0;

                        y = radius*Math.cos(lat);//need to shift angle downwards by 1/4 rev

                        for(var v:int = 0; v <= cols; ++v)

                        {

                            x = radius*Math.cos(lon)*Math.sin(lat);

                            z = radius*Math.sin(lon)*Math.sin(lat);//seen from above, z = y



                            //add vertex triplet

                            vertices[vnum] = x;

                            vertices[vnum+1] = y;

                            vertices[vnum+2] = z;

                            //uvts

                            uvtData[vnum] = v/cols;

                            uvtData[vnum+1]  = h/rows;

                            uvtData[vnum+2] = 1;



                            vnum+=3;

                            //add indices

                            if(h < rows && v < cols){

                                indices.push(ind, ind+1, ind + cols+1);

                                indices.push(ind + cols+1, ind+1, ind + cols + 2);

                            }

                            ind+=1;

                            lon += lon_incr;

                        }

                        lat += lat_incr;

                    }

                }

                /**

                 * create ShaderJob for use later

                 */

                private function createShaderJob():void

                {

                    normalMapShader = new Shader(new NormalMapShader());

                    normalMapShader.data.src.width = normalMap.width;

                    normalMapShader.data.src.height = normalMap.height;

                    normalMapShader.data.src.input = normalMap;                

                    

                    shaderJob = new ShaderJob(normalMapShader, texture, width, height);

                    shaderJob.addEventListener(Event.COMPLETE, shaderJobCompleteHandler);

                    shaderJob.start();                

                }



                /**

                 * shade job complete

                 */

                private function shaderJobCompleteHandler(event:Event):void

                {

                    //update params and restart job

                    var lightDir:Vector3D = rotationMatrix.transformVector(new Vector3D(1,1,1));

                    lightDir.normalize();

                    normalMapShader.data.xr.value = [lightDir.x];

                    normalMapShader.data.yr.value = [lightDir.y];

                    normalMapShader.data.zr.value = [lightDir.z];

                    shaderJob = new ShaderJob(normalMapShader, texture, width, height);

                    shaderJob.addEventListener(Event.COMPLETE, shaderJobCompleteHandler);

                    shaderJob.start();

                }



                /**

                 * create normal map from geometry to reuse for lighting

                 */

                private function createNormalMap():void

                {

                    for(var i:int = 0; i < indices.length; i+=3){

                        //get the 3 points of the triangle as Vector3D objects

                        var pt1:Vector3D = new Vector3D(vertices[indices[i]*3],

                                                    vertices[indices[i]*3+1],

                                                    vertices[indices[i]*3+2]);

                        var pt2:Vector3D = new Vector3D(vertices[indices[i+1]*3],

                                                        vertices[indices[i+1]*3+1],

                                                        vertices[indices[i+1]*3+2]);

                        var pt3:Vector3D = new Vector3D(vertices[indices[i+2]*3],

                                                        vertices[indices[i+2]*3+1],

                                                        vertices[indices[i+2]*3+2]);

                        //subtract the first two from the third

                        var d1:Vector3D = pt3.subtract(pt1);

                        var d2:Vector3D = pt3.subtract(pt2);

                        //get the cross-product of the results to get the normal

                        var normal:Vector3D = d1.crossProduct(d2);

                        normal.normalize();

                        var r:uint = 255*(normal.x + 1)/2;

                        var g:uint = 255*(normal.y + 1)/2;

                        var b:uint = 255*(normal.z + 1)/2;

                        var normalColor:uint = r << 16 | g << 8 | b;

                        normalMapShp.graphics.beginFill(normalColor);

                        var p1:Point = new Point(uvtData[indices[i]*3]*normalMap.width, uvtData[indices[i]*3+1]*normalMap.height);

                        var p2:Point = new Point(uvtData[indices[i+1]*3]*normalMap.width, uvtData[indices[i+1]*3+1]*normalMap.height);

                        var p3:Point = new Point(uvtData[indices[i+2]*3]*normalMap.width, uvtData[indices[i+2]*3+1]*normalMap.height);

                        normalMapShp.graphics.moveTo(p1.x, p1.y);

                        normalMapShp.graphics.lineTo(p2.x, p2.y);

                        normalMapShp.graphics.lineTo(p3.x, p3.y);

                        normalMapShp.graphics.lineTo(p1.x, p1.y);

                        normalMapShp.graphics.endFill();

                    }

                    normalMap.draw(normalMapShp);



                }

                /**

                 * save the normal map -- so I can play with it in PixelBender Toolkit ;)

                 */

                private function onSaveBtnClick(click:Event):void

                {

                    var savingBmp:SavingBitmap = new SavingBitmap(normalMap);

                    savingBmp.encoderType = "png";

                    savingBmp.save("normalMap.png");

                    

                }



                private function onEnterFrame(event:Event):void {

                    //createSphere()

                    update();

                }



                private function update():void {

                    r+=0.5;

                    rotationMatrix = new Matrix3D();

                    rotationMatrix.prependRotation(-r, new Vector3D(0,1,0))

                    projectionMatrix = perspective.toMatrix3D();

                    projectionMatrix.prependTranslation(0.0,0.0,250);

                    projectionMatrix.prependRotation(r, new Vector3D(0,1,0))

                    stage.invalidate();//trigger render event



                }

                private function render(e:Event):void {

                    

                    container.graphics.clear();

                    Utils3D.projectVectors(projectionMatrix, vertices, projectedVerts, uvtData);

                    container.graphics.beginBitmapFill(texture,null, false, false);

                    container.graphics.drawTriangles(projectedVerts, indices, uvtData, TriangleCulling.POSITIVE);

                    container.graphics.endFill();

                }



                

            }

        }

You may have noticed that there is a white blotch in the darkest part of the shadow… this is a bug in the shader, which I have yet to figure out… let me know if you know what it is

The next step is to liberate the normal map from the actual geometry, in order to make the lighting more detailed than the geometry… the dramatic conclusion to this series is coming soon!

As usual, code is on GitHub…

NormalMap3.as

A while back I came up with a solution for lighting a mesh object using the Flash 10 3D APIs. This was my own idea based on the knowledge that the normal of a triangle can be computed by getting a vector which is perpendicular to two of its sides. The following crude diagram may help.. the triangle is ABC, the normal is AN. The normal is considered to be facing away from the surface. You’ll need to imagine that ABC is not flat on the picture plane, with B pointing away from you, and with NA perpendicular to AC and AB, which it doesn’t really look like it is in the picture, but theres only so much you can do with ASCII art :p

   N
    \        B
     \      /|
      \    / |
       \  /  |
        \/___|
         A    C

So the plan then is to iterate over every triangle in the sphere, and determine its normal, and then the luminosity of that surface by getting the difference between it and the light. The math for this involves vectors, and was new stuff for me. I recommend the excellent book ‘3D Math Primer for Graphics and Game Development’, but any introductory text to 3D graphics should be helpful to grok these concepts. Fortunately the Vector3D class has some handy methods to do such calculations, and you just need to know which method to call.

To get the normal of a triangle you need to convert the points to vectors, and then get the cross-product between two of these vectors.I just created 3 Vector3D objects for each point and subtracted them from each other to get the vectors representing the sides of the triangle. Eg. subtracting point A from B in the above triangle, gives you the vector representing AB. Note that you need to use the Vector3D subtract() method, not regular ‘-’ since the vector subtract() subtracts each component of the vector for you. At this point the code may help clarify (I will give complete code later) :

Then the final step to get the normal of the triangle surface is to get the cross-product of the two vectors we just derived.

The normalize() method just makes sure the length of the vector is between -1 & 1. Now to get the difference between the surface normal and the light vector, we can use the angleBetween() method.

Then I converted the this difference to a color, and draw it onto the bitmapData being used as the texture of the object. I just used the drawing API to draw each triangle on a Sprite, then drew the sprite onto the bitmapData being used in the beginFill() method just before the call to drawTriangles() in the render method.

When I first did this I found that when I rotated the sphere, the lighting was rotating along with it. So I had to transform the light vector using the Matrix3D which I was using to rotate the sphere. This did not work as expected. Then I realized I had to transform the light vector using a Matrix3D which was the inverse of the rotation of the sphere (the projection matrix). So I thought I could just call the invert() method on the projection matrix and use that to transform the light. But it turns out that the projection matrix, which is derived from a perspective projection, is not invertible. SO… I had to make another Matrix3D, and rotate it in the opposite direction, in order to use it for correcting the light direction.

Here’s the working example, followed by the code. There’s a lot of it, but that will improve soon… see comments after code.

flash 10 required

So although I was happy that this worked at all, it has some problems… performance has taken a hit, as we are re-creating the shadow/texture map every time. And the ‘texture map’ a this point only has lighting in it , so we’d have to merge the lighting onto a real texture map (with other colors). And the sphere looks more like a disco ball, not smooth at all, so if we wanted a smooth appearance, we’d have to increase the number of points, which would further degrade performance. I did get some improvement by blurring the shadow-map but this would only help with smooth curved surfaces. So this is not a scalable solution at all. I almost didn’t want to show this version at all, but it does introduce the concept of normals, and how to get the lighting on a surface.

After a while it occurred to me that maybe I should really look into normal-maps as a way to improve my lighting code. I was also hoping that I could optimize performance by using PixelBender to do the lighting calculations, based on the normal map. It turned out to be correct, so stay tuned for the next exciting installment…

First of all, we need some object to light.. for this I revisited the Sphere I did earlier, and improved the method of its construction, using radians instead of degrees, and also generating the indices in the same loop as the vertices. First, here is what it looks like with a flat, outlined fill…

flash 10 required

Since I just installed the snazzy CodeColorer plugin, here’s the code…

And if you actually want to download the code, here’s the link:

Da Code

NormalMap1

(@GitHub)

flash 10 required

You may be wondering, ‘why bother with haXe?’ so here are some reasons I can think of …

• faster compiler (pretty much instantaneous in my experience so far)
• richer language constructs in haXe (enums, more flavours of types)
• access to superfast ByteArray currently only available to Alchemy
• multiple platform targets…Javascript, PHP, even iPhone

Da Code

Test3D.hx

Test3D.hxml

the .hxml is just a way of setting compiler arguments. In future I’m just going to set them inline , but its useful in this case to see the argument for targeting Flash Player 10. I have some ideas for using my homegrown Vim Actionscript autocompletion script in haXe.

Apparently Japan has a thriving flash community. Here are some of the things that stood out for me:

Perhaps best-known is the wonderful site: http://wonderfl.kayac.com/ which allows you to edit actionscript code and see the results almost immediately rendered alongside it. I love the simplicity of this , and how it encourages you to experiment. You can also browse code by others, and if you want to edit it, you just ‘fork’ it and away you go. In the presentation this was described as ‘collaborate by forking’ which got a few chuckles from the crowd I must say that this site performs really well, considering the massive usage it must be getting, so the engineers – eg. mash – behind it clearly know what they are doing. I was really happy to discover that I can use ItsAllText firefox plugin to edit the code with Vim, so I can use my familiar environment for coding. I think the big win of this is the collaborative aspect – I think I’m going to learn a lot by reading and tinkering with other people’s code.

Of course, there’s the awesome FLARToolkit by saqoosha, which lets you do Augmented Reality stuff in flash (with a webcam). Check out http://livingsasquatch.com/. But there are also many other interesting projects in the SPARK project, as explained by ‘yossy’, a 20 year old who’s been doing actionscript development for 10 years! There’s a full listing of the Spark projects in English: http://www.libspark.org/wiki/WikiStart/en where you will also find links to the wikis by each developers about their projects.

sazameki
this features a virtual instrument called ano gakki (?) There’s also a very simple sequencer and mixer, allowing you to control generated sound. It’s not hobnox but you can roll your own UI to do something different (I’d like to see a virtual tenori on)

betweenAS3 yet another tweening engine… I know, but this has a functional programming style (you can nest tweens inside each other) which I like. Also, it can handle tweening massive numbers of particles.. it outperformed other tween engines by far in the demo particle system yossy showed.

Frocessing a graphics library with a surprising range of capabilities, from HSL color, to 2D and 3D transforms, and SVG support.

swfasssist – a flash compiler and player done with actionscript – ?? Holy crap.

SwfWheel – better cross-browser and platform support for the scroll wheel, very useful for the UI developers.

OK, what else…

Joe Berkovitz’s moccasin framework for developing visual editor applications looks really good, having developed such an application before, I know it is hard, and having some of the mechanics readymade will help a lot.

Brian Lesser presented on using RTMFP , the new protocol for real time streaming between flash players. You still need a server to do the initial connection either Status or the yet-to-be-released version of Flash Media Server. Apparently Flash PLayer 10 now has UDP capabilties built into it, who knew?

Speaking of Flash Player, I checked out the Flash PLayer Internals session by Jim Corbett. This was pretty much the same as one I saw online but there were some interesting tidbits about Ichabod – the headless flash player being used by Google to index flash content. It operates in conjunction with a virtual user. They are currently working on issues to do with how it accesses network resources. There are many other possible applications for this technology, so I hope that it eventually becomes available to the rest of us, and not just Google. The new Text Layout Format is powerful, but low-level, so you probably want to use the actionscript components for it, which will be available in Flex 4 , coming soon ( I hope).

ByteArray for beginners gave me a head start on using the ByteArray class in flash, which is incredibly powerful, allowing you to implement encoders/decoders for any filetype, and also use any existing network protocol over a socket (or invent your own). The class provides methods to read and write data to a ByteArray as a String, or various other types, so you don’t have to speak binary

Ralph Hauwert was mind-blowing … starting off with his formative inspirations, he then took a deep dive (for me) into how to make normal maps, texture maps, bump maps and lighting even without doing traditional polygonal 3D. Then he explained some of the technical breakthroughs – a 3-way coordination between Alchemy, ByteArray, and PixleBender, which are going to appear in the next version of Papervision. He also explained how they are refactoring PV3D to allow alternative renderers. The demos of the new engine were very impressive, lightning fast, even rendering a version of Quake in the flash player without a single stutter. I also enjoyed the few demos of voxel-based 3D and 8 bit music, specifically a Commodore64 sound emulator, based on Tiny SID which lets you play the original music from those games.

The Commodore / Amiga era games cropped up again as inspiration in James Paterson’s presentation. James Paterson is behind presstube.com. He combines his own drawings and animations in flash, and recently has been developing some interative environments as a home for the various strange beings he loves to invent see here and click on the thumbnails to enter the enviroment. You may need to check on the presentation to understand how to use it, which I highly recommend as it gave a lot of insight into the creative process as well as being very funny. It’s great to see hand drawn art still alive and well in this over-digital era, and this is inspiring me to get back to my own drawings and see if I can bring them to life in a similar way.

I started off the conference with te presenation on SourceBinder a collaborative node-based editor for flash, built in flex. I was able to make an interactive 3D cube controlled by my WiiMote, in minutes. I found the interface mostly intuitive, though it is an alpha product, so there’s the occasional glitch. Its interesting to compare this with Wonderfl – similar goals, but different approaches: Sourcebinder avoids codingbut lets you connect other people’s code like lego. However you can also create your own nodes if you want.

To be honest, the Adobe Keynote seemed a bit flat compared to the other sessions, as they didn’t have anything major to announce, except that a new version of Flash Cs4 will be released soon, to address performance issues. There was an interesting demo of creating your own tool to do procedural drawing.

Joshua Davis was entertaining, the first time I’d seen him speak. Again, I enjoyed the insights into the creative development process of what he does : ‘computational design’. Instead of interactivity as an end goal, he ends up creating a static design by iterating on an interactive process. What I found interesting is how much time this takes, even weeks, before something worthwile is achieved.

This emphasis on iteration was a feature of Mario Klingeman’s session, called The Tinkerer’s Box. By use of some heavy computational geometry and recursive algorithms (as well as some simpler ideas) and many, many variations and experimentation, some really cool effects were achieved, like portraits made up of tiny triangles or spaghetti like lines.

Finally, I attended two sessions by or featuring Grant Skinner… the first about his work as a flash developer in the context of console game devlopment .. apparently console games are making increasing use of flash to develop in-game interfaces. To do this, they sacleform developed a custom flash player which pushes triangles up to the GPU. Some really hardcore engineering going on here. Grant has architected a component set for use in this enviroment, called CLIK. I enjoyed the insight into how to build components which are highly flexible and modular, a topic which he also touched on in his other presentation on Actionscript Development. I got some concrete tips on improving my coding practices here, such as the concept of cohesion, which is how focused a chunk of code (say, a Class) is. You should be able to express what it does in a single sentence, or at least very briefly, or it may be lacking in cohesion (ie, doing too much stuff that it shouldn’t be).

Of course this just scratches the surface… there was so much more which I’m still digesting, and I’m feeling inspired, motivated and informed to do more and better work (and play) with flash in the future.

flash 10 required

Da Code

PanoramaTest

Sphere

Note: I got the panorama image from here http://www.all-in-one.ee/~dersch/. Using this and other software you can create your own panoramas from multiple images.