JiblibFlash + Augmented Reality (Take Three)

Well here’s one more take. Now there’s two levels – you’ll bounce back and forth between them as you get through each maze. I made everything a bit smaller which improves cpu usage and also makes it easier to see. Also (hopefully) fixed it so that it will be viewable on Macs (Don’t know why Apple makes it so difficult to target the default webcam of a Mac, but man, what a pain). Once again, the marker can be found here.

[kml_flashembed fversion=”9.0.0″ movie=”http://blog.onebyonedesign.com/wp-content/uploads/2009/05/armaze2.swf” targetclass=”flashmovie” publishmethod=”static” width=”640″ height=”480″]

[/kml_flashembed]

I’m still not happy with how the movement is accomplished. It takes a whole lot of mucking around to get a good movement based on the AR transformation system, but it works. Mostly…

While not as clean as I normally like my code, it’s all posted below, if you’d like to try it out.. You’ll have to supply your own materials as well as flartoolkit, papervision3d, and jiglib libraries, but other than that, it’s all there. Enjoy

The two maze levels and their interface:

package { import org.papervision3d.materials.utils.MaterialsList; /** * Maze Level Interface * @author Devon O Wolfgang */ public interface ILevel { function get wallList():MaterialsList; function get map():Array; } }

package { import org.papervision3d.materials.BitmapMaterial; import org.papervision3d.materials.utils.MaterialsList; /** * First maze level with brick wall * @author Devon O Wolfgang */ public class Level1Maze implements ILevel { [Embed(source = "../assets/wall.jpg")] private static const WallClass:Class; private var _wallMaterial:BitmapMaterial = new BitmapMaterial(new WallClass().bitmapData); private var _wallList:MaterialsList; private var _map:Array = [ [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [0, 0, 0, 0, 0, 0, 0, 1, 1, 1], [0, 0, 0, 0, 0, 0, 0, 1, 1, 1], [1, 1, 1, 1, 1, 0, 0, 0, 0, 1], [1, 1, 0, 0, 1, 0, 0, 0, 0, 1], [1, 1, 0, 0, 1, 0, 0, 1, 1, 1], [1, 1, 0, 0, 1, 0, 0, 1, 1, 1], [1, 1, 0, 0, 0, 0, 0, 0, 0, 1], [1, 1, 0, 0, 0, 0, 0, 0, 0, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1] ]; public function Level1Maze() { buildMaterialList(); } private function buildMaterialList():void { _wallList = new MaterialsList( { all:_wallMaterial } ); } public function get wallList():MaterialsList { return _wallList; } public function get map():Array { return _map; } } }

package { import org.papervision3d.materials.BitmapMaterial; import org.papervision3d.materials.utils.MaterialsList; /** * Second level maze of hedges * @author Devon O Wolfgang */ public class Level2Maze implements ILevel { [Embed(source = "../assets/hedge.jpg")] private static const WallClass:Class; private var _wallMaterial:BitmapMaterial = new BitmapMaterial(new WallClass().bitmapData); private var _wallList:MaterialsList; private var _map:Array = [ [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 0, 0, 0, 0, 0, 0, 1, 1, 1], [1, 0, 0, 0, 0, 0, 0, 1, 1, 1], [1, 0, 0, 1, 1, 0, 0, 0, 0, 1], [1, 0, 0, 1, 1, 0, 0, 0, 0, 1], [1, 0, 0, 1, 1, 1, 1, 0, 0, 1], [1, 0, 0, 1, 1, 1, 1, 0, 0, 1], [1, 0, 0, 0, 0, 0, 1, 0, 0, 1], [1, 0, 0, 0, 0, 0, 1, 0, 0, 1], [1, 0, 0, 1, 1, 1, 1, 1, 1, 1] ]; public function Level2Maze() { buildMaterialList(); } private function buildMaterialList():void { _wallList = new MaterialsList( { all:_wallMaterial } ); } public function get wallList():MaterialsList { return _wallList; } public function get map():Array { return _map; } } }

Not perfectly decoupled, but the FlarObject handles most of the AR stuff, while the MazeScene handles the PV3d and Jiglib stuff.

package { import flash.display.BitmapData; import flash.display.DisplayObject; import flash.display.Sprite; import flash.events.Event; import flash.media.Camera; import flash.media.Video; import flash.system.Capabilities; import flash.utils.ByteArray; import org.papervision3d.core.render.IRenderEngine; import org.papervision3d.scenes.Scene3D; import org.papervision3d.view.Viewport3D; import org.libspark.flartoolkit.core.FLARCode; import org.libspark.flartoolkit.core.param.FLARParam; import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData; import org.libspark.flartoolkit.core.transmat.FLARTransMatResult; import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector; import org.libspark.flartoolkit.pv3d.FLARBaseNode; import org.libspark.flartoolkit.pv3d.FLARCamera3D; /** * Manages most FLAR goings on * @author Devon O. Wolfgang */ public class FlarObject extends Sprite { [Embed (source="pat1.pat", mimeType="application/octet-stream")] private var Pattern:Class; [Embed (source="camera_para.dat", mimeType="application/octet-stream")] private var Params:Class; private var _fparams:FLARParam; private var _fpattern:FLARCode; private var _raster:FLARRgbRaster_BitmapData; private var _detector:FLARSingleMarkerDetector; private var _transformObject:FLARTransMatResult; private var _virtualcam:FLARCamera3D; private var _vid:Video; private var _webcam:Camera; private var _bmd:BitmapData; private var _renderEngine:IRenderEngine; private var _scene:Scene3D; private var _viewport:Viewport3D; private var _flarContainer:FLARBaseNode; private var _mazescene:MazeScene; private var _doRender:Boolean = false; public function FlarObject(videowidth:int = 640, videoheight:int = 480) { initFlar(); initVideo(videowidth, videoheight); initBitmap(); initCamera(); } private function initFlar():void { _fparams = new FLARParam(); _fparams.loadARParam(new Params() as ByteArray); _fpattern = new FLARCode(16, 16); _fpattern.loadARPatt(new Pattern()); _transformObject = new FLARTransMatResult(); } private function initVideo(vw:int, vh:int):void { _vid = new Video(vw, vh); // find default camera on Mac var camIndex:int = 0; for ( var i : int = 0 ; i < Camera.names.length ; i++ ) { if ( Camera.names[ i ] == "USB Video Class Video" ) { camIndex = i; break; } } _webcam = Camera.getCamera(String(camIndex)); _webcam.setMode(vw, vh, 31); _vid.attachCamera(_webcam); addChild(_vid); } private function initBitmap():void { _bmd = new BitmapData(_vid.width, _vid.height); _bmd.draw(_vid); _raster = new FLARRgbRaster_BitmapData(_bmd); _detector = new FLARSingleMarkerDetector(_fparams, _fpattern, 80); } private function initCamera():void { _virtualcam = new FLARCamera3D(_fparams); } private function renderHandler(event:Event):void { _bmd.draw(_vid); try { if (_detector.detectMarkerLite(_raster, 80) && _detector.getConfidence() > .45) { _viewport.visible = true; _detector.getTransformMatrix(_transformObject); _flarContainer.setTransformMatrix(_transformObject); _mazescene.moveBall( _transformObject ); _renderEngine.renderScene(_scene, _virtualcam, _viewport); } else { _viewport.visible = false; } } catch (err:Error) { // do nothing } } public function startRendering():void { if (_mazescene != null && !willTrigger(Event.ENTER_FRAME)) { _viewport.visible = true; addEventListener(Event.ENTER_FRAME, renderHandler); _doRender = true; } else { throw new Error("MazeScene must be set before rendering can begin!"); } } public function stopRendering():void { _viewport.visible = false; removeEventListener(Event.ENTER_FRAME, renderHandler); _doRender = false; } public function set mazescene(value:MazeScene):void { _mazescene = value; _renderEngine = _mazescene.engine; _scene = _mazescene.scene; _viewport = _mazescene.viewport; _flarContainer = _mazescene.flarContainer; addChild(_viewport); } } }

package { import flash.display.Sprite; import flash.events.Event; import flash.events.EventDispatcher; import jiglib.cof.JConfig; import jiglib.geometry.JSphere; import jiglib.math.JNumber3D; import jiglib.physics.PhysicsSystem; import jiglib.plugin.papervision3d.Papervision3DPhysics; import jiglib.plugin.papervision3d.Pv3dMesh; import org.libspark.flartoolkit.core.transmat.FLARTransMatResult; import org.papervision3d.core.math.Matrix3D; import org.papervision3d.core.math.Number3D; import org.papervision3d.materials.BitmapMaterial; import jiglib.geometry.JBox; import org.papervision3d.cameras.Camera3D; import org.papervision3d.lights.PointLight3D; import org.papervision3d.materials.shadematerials.FlatShadeMaterial; import org.papervision3d.materials.utils.MaterialsList; import org.papervision3d.objects.DisplayObject3D; import org.papervision3d.objects.primitives.Cube; import org.papervision3d.objects.primitives.Plane; import org.papervision3d.objects.primitives.Sphere; import org.papervision3d.render.BasicRenderEngine; import org.papervision3d.scenes.Scene3D; import org.papervision3d.view.layer.util.ViewportLayerSortMode; import org.papervision3d.view.layer.ViewportLayer; import org.papervision3d.view.Viewport3D; import org.libspark.flartoolkit.pv3d.FLARBaseNode; import org.papervision3d.Papervision3D; import org.papervision3d.core.render.IRenderEngine; import org.papervision3d.render.BasicRenderEngine; import org.papervision3d.scenes.Scene3D; import org.papervision3d.view.Viewport3D; /** * Manages pv3d/jiglib shenanigans * @author Devon O. Wolfgang */ public class MazeScene extends EventDispatcher { [Embed(source = "../assets/metal1.jpg")] private static const BallClass:Class; private var _scene:Scene3D; private var _viewport:Viewport3D; private var _engine:BasicRenderEngine; private var _flarContainer:FLARBaseNode; private var _physics:Papervision3DPhysics; private var _ground:JBox; private var _world:DisplayObject3D; private var _ball:JSphere; private var _visibleBall:Sphere; private var _cellsize:int = 12; private var _level:ILevel; private var _physicsWalls:Array = []; private var _visibleWalls:Array = []; private var _vplObjects:ViewportLayer; public function MazeScene(videowidth:int = 640, videoheight:int = 480) { // no tracing Papervision3D.PAPERLOGGER.unregisterLogger(Papervision3D.PAPERLOGGER.traceLogger); initScene(videowidth, videoheight); } private function initScene(vw:int, vh:int):void { _viewport = new Viewport3D(vw, vh, false, false, true, true); _engine = new BasicRenderEngine(); _flarContainer = new FLARBaseNode(); _scene = new Scene3D(); _scene.addChild(_flarContainer); _physics = new Papervision3DPhysics(_scene, 2); _physics.engine.setSolverType("ACCUMULATED"); PhysicsSystem.getInstance().setGravity(new JNumber3D(0, 0, -5)); } public function buildMaze():void { _vplObjects = new ViewportLayer(_viewport, null); _vplObjects.layerIndex = 1; _vplObjects.sortMode = ViewportLayerSortMode.Z_SORT; _viewport.containerSprite.addLayer(_vplObjects); var light:PointLight3D = new PointLight3D(); light.x = 0; light.y = 300; light.z = 0; var groundMat:FlatShadeMaterial = new FlatShadeMaterial(light, 0x999999); var matList:MaterialsList = new MaterialsList( { all:groundMat } ); var groundPlane:Cube = new Cube(matList, 125, 2, 125); _ground = new JBox(new Pv3dMesh(groundPlane), 125, 2, 125); _ground.material.restitution = 1.5; _ground.material.friction = 2; _ground.movable = false; _physics.addBody(_ground); // create walls var w:int = _level.map[0].length; var h:int = _level.map.length; for (var i:int = 0; i < h; i++) { for (var j:int = 0; j < w; j++) { if (_level.map[i][j] == 1) { var c:Cube = new Cube(_level.wallList, _cellsize, _cellsize, _cellsize, 1, 1, 1); _flarContainer.addChild(c); _vplObjects.addDisplayObject3D(c); _visibleWalls.push(c); var wall:JBox = new JBox(new Pv3dMesh(c), _cellsize, _cellsize, _cellsize); wall.moveTo(new JNumber3D(((j * _cellsize) - 65) + (_cellsize * .5), ((i * _cellsize) - 65) + (_cellsize * .5), _cellsize * .6)); wall.movable = false; _physicsWalls.push(wall); _physics.addBody(wall); } } } var ballMat:BitmapMaterial = new BitmapMaterial(new BallClass().bitmapData); _visibleBall = new Sphere(ballMat, 6); _flarContainer.addChild(_visibleBall); _vplObjects.addDisplayObject3D(_visibleBall); _ball = new JSphere(new Pv3dMesh(_visibleBall), 6); _ball.mass = 10; resetBall(); _physics.addBody(_ball); } private function destroy():void { var len:int = _visibleWalls.length; for (var i:int = 0; i < len; i++) { _flarContainer.removeChild(_visibleWalls[i]); _vplObjects.removeDisplayObject3D(_visibleWalls[i]); _physics.removeBody(_physicsWalls[i]); } _physicsWalls = []; _visibleWalls = []; _physics.removeBody(_ball); _physics.removeBody(_ground); if (_visibleBall != null) { _vplObjects.removeDisplayObject3D(_visibleBall); _flarContainer.removeChild(_visibleBall); } } private function resetBall():void { _ball.moveTo(new JNumber3D(40, 40, 25)); } public function moveBall(trans:FLARTransMatResult):void { var mat:Matrix3D = new Matrix3D(); mat.n11 = trans.m01; mat.n12 = trans.m00; mat.n13 = trans.m02; mat.n14 = trans.m03; mat.n21 = -trans.m11; mat.n22 = -trans.m10; mat.n23 = -trans.m12; mat.n24 = -trans.m13; mat.n31 = trans.m21; mat.n32 = trans.m20; mat.n33 = trans.m22; mat.n34 = trans.m23; var objRot:Number3D = Matrix3D.matrix2euler(mat); trace(objRot.x); if(_ball.z < 10 && _ball.z > 4) // too much kludge to get this working well. Experiment with these numbers and see what's good for you _ball.addWorldForce(new JNumber3D((-objRot.z + 100) * .15, ((objRot.x - 120) * -2 ) , 0), _ball.currentState.position); _physics.step(); if (_ball.z < -30) dispatchEvent(new Event("nextLevel")); } public function get scene():Scene3D { return _scene; } public function get viewport():Viewport3D { return _viewport; } public function get engine():IRenderEngine { return _engine; } public function get flarContainer():FLARBaseNode { return _flarContainer; } public function get level():ILevel { return _level; } public function set level(value:ILevel):void { _level = value; destroy(); buildMaze(); } } }

And finally the Main class to tie it all together:

package { import flash.display.Sprite; import flash.events.Event; /** * Augmented Reality with Jiglib Maze game * @author Devon O. Wolfgang */ public class Main extends Sprite { private var _flarObject:FlarObject; private var _mazeScene:MazeScene; private var _levels:Array; private var _currentLevel:int = 0; // game always starts with ball falling. Give it 2 tries private var _firstTry:Boolean = true; public function Main():void { if (stage) init(); else addEventListener(Event.ADDED_TO_STAGE, init); } private function init(e:Event = null):void { removeEventListener(Event.ADDED_TO_STAGE, init); // entry point _levels = [ new Level1Maze(), new Level2Maze() ]; initGame(); } private function initGame():void { _flarObject = new FlarObject(); _mazeScene = new MazeScene(); _mazeScene.addEventListener("nextLevel", changeLevel); _mazeScene.level = _levels[_currentLevel]; _flarObject.mazescene = _mazeScene; addChild(_flarObject); _flarObject.startRendering(); } private function changeLevel(event:Event):void { if (_firstTry) { _firstTry = false; } else { _flarObject.stopRendering(); _currentLevel++; if (_currentLevel > _levels.length-1) _currentLevel = 0; _mazeScene.level = _levels[_currentLevel]; _flarObject.startRendering(); } } } }

Have fun...

EDIT:
I cleaned up the code a tad more uploaded the entire project. Anyone wanting to play around with this can download the whole kit and kaboodle here. Let me know if you discover anything interesting.