Rendering 3D Flash Content on an HTML5 Canvas

This week, I have had a project which uses Javascript extensively in order to create a web application interface. An important component to the application is a 3D render, which represents a product you are building through this application.

Now that version 1.0 of the specification has been released, WebGL may be the best technology for achieving this type of integration in the future. In the meantime, I will be using Flash to perform my rendering, since I do not know of a suitable Javascript engine which supports the Collada model format, reliable depth sorting and quality UV mapping.

Similar to most media applications, Flash Player uses an “overlay” layer which the operating system displays over the top of your web browser. This works well for performance, but always displays Flash in front of other content in your page. Flash has support for “window modes” that will display inside of the browser, but these run a lot slower. In the case of my web application, running Flash caused all of my Javascript animations to appear choppy and unprofessional. Since I do not need to render my 3D scene in real-time, I can push my renders to an HTML5 Canvas so that the application remains smooth and responsive.

HTML

<!DOCTYPE html>
<head>
 
	<title>Render Example</title>
 
	<link href="stylesheets/styles.css" rel="stylesheet" type="text/css">
 
	<script type="text/javascript" src="scripts/jquery-1.5.1.min.js"></script>
	<script type="text/javascript" src="scripts/jquery.class.min.js"></script>
	<script type="text/javascript" src="scripts/swfobject.js"></script>
	<script type="text/javascript" src="scripts/app.js"></script>
 
</head>
<body>
 
	<canvas id="canvas"></canvas>
 
	<div id="flash-content">
 
		<div id="renderer"></div>
 
	</div>
 
</body>

To get started, we need an HTML file to create our Canvas element, a container for our Flash renderer, and include our CSS stylesheets and Javascript code. I am using jQuery, a class plugin for jQuery, and SWFObject for embedding Flash content. You can feel free to use your own “mix-ins” to suite your taste.

CSS

#canvas { width: 800px; height: 600px; }
#flash-content { position: absolute; left: -10px; }

Because I prefer to keep my design and the structure of my web application separated, I have decided to use CSS to specify the width and height of my Canvas element, and have not put it in HTML. Normally if you only set a CSS width and height for a Canvas element, it will be stretched, but I will read these values in Javascript later so that it sizes properly. I want my Flash content to be invisible, so I have put it inside of a container which is absolutely positioned off-screen. If I use display:none or a width and height of 0 to hide my Flash content, it will not work in some browsers, so I am using this approach to keep it out of sight.

Javascript

var RenderExample = $.Class ({
 
	Canvas: null,
	DrawingContext: null,
	Renderer: null,
 
	canvasHeight: null,
	canvasWidth: null,
 
	init: function () {
 
		$(document).ready ($.proxy (this.document_onReady, this));
 
	},
 
	construct: function () {
 
		this.Canvas = document.getElementById ("canvas");
 
		this.canvasWidth = $(this.Canvas).width ();
		this.canvasHeight = $(this.Canvas).height ();
 
		if (this.Canvas && this.Canvas.getContext) {
 
			this.Canvas.setAttribute ("width", this.canvasWidth);
			this.Canvas.setAttribute ("height", this.canvasHeight);
			this.DrawingContext = this.Canvas.getContext ("2d");
 
			swfobject.embedSWF ("assets/renderer.swf", "renderer", "4", "4", "10,0,0", "", {}, { }, { id: "renderer", name: "renderer" } );
 
		} else {
 
			swfobject.embedSWF ("assets/renderer.swf", "canvas", this.canvasWidth, this.canvasHeight, "10,0,0", "", {}, { wmode: "opaque" }, { id: "renderer", name: "renderer" } );
 
		}
 
	},
 
	render: function () {
 
		if (this.DrawingContext) {
 
			var image = new Image ();
 
			image.onload = $.proxy (function () {
 
				$(this.Canvas).fadeOut (800, $.proxy (function () {
 
					this.DrawingContext.drawImage (image, 0, 0);
					$(this.Canvas).fadeIn (400);
 
				}, this) );
 
			}, this);
 
			image.src = "data:image/png;base64," + this.Renderer.render ();
 
		} else {
 
			this.Renderer.render (false);
 
		}
 
	},
 
	// Event Handlers
 
	document_onReady: function () {
 
		this.construct ();
 
	},
 
	Renderer_onReady: function () {
 
		App.Renderer = swfobject.getObjectById ("renderer");
		App.Renderer.initialize (this.canvasWidth, this.canvasHeight);
		$.proxy (App.render, App) ();
 
	}
 
});
 
App = new RenderExample ();

First I define my RenderExample class, then create an instance called App. This will make it easy to reference our application later from Flash. I am using a class structure because it is most comfortable for my coding style, but it would not be difficult to write this same functionality without it.

Once the HTML document is ready, I get a reference to the Canvas element and check if the browser supports an HTML5 Canvas. If it does, I set the width and height of the Canvas to the CSS width and height I defined earlier, then I get it ready for 2D drawing. If a Canvas is not available, I replace the Canvas element with the Flash renderer directly. As I discussed earlier, this approach runs slower, but it is a good fall-back for IE 8 and older.

When the Flash renderer is ready, it calls App.Renderer_onReady. That is when we know that we can begin interacting with the Flash renderer. First I call an initialize function on the SWF in order to define the width and height we will need for rendering, then I call render() to draw the first frame. As I continue to develop my application, I will be add more calls to the renderer so I can control which objects it loads, how it positions the camera, and other settings I’ll need to make it useful.

When I render, I check to see if I am using a Canvas or using Flash directly. If I am using Flash, I tell it to render the current frame. If I am using a Canvas, I first fade out the Canvas, then I draw the next frame, then I fade it back in. This way it will transition very smoothly. If I wanted to, I could certainly write the frame directly without fading the Canvas. It really depends on the application you are building.

The trick to getting images out of Flash and into Javascript is to A) encode them as a standard format, like JPG or PNG, then B) turn them into a Base64-encoded string. You can set the src attribute of an image to “data:image/png;base64,” plus the Base64 string, or something similar when using a different image format. When I ask Flash to render the current frame, it will return a Base64 string of the frame.

Actionscript

package com.example.renderexample {
 
	import away3d.cameras.TargetCamera3D;
	import away3d.containers.View3D;
	import away3d.core.session.BitmapSession;
	import away3d.primitives.Cube;
	import by.blooddy.crypto.Base64;
	import by.blooddy.crypto.image.PNG24Encoder;
	import flash.display.BitmapData;
	import flash.display.Sprite;
	import flash.external.ExternalInterface;
	import flash.geom.Point;
	import flash.utils.ByteArray;
 
	public class RenderExample extends Sprite {
 
		private var View:View3D;
 
		private var backgroundColor:Number;
		private var setHeight:Number;
		private var setWidth:Number;
 
		public function RenderExample () {
 
			if (ExternalInterface.available) {
 
				ExternalInterface.addCallback ("initialize", initialize);
				ExternalInterface.addCallback ("render", render);
 
				ExternalInterface.call ("App.Renderer_onReady");
 
			}
 
		}
 
		public function initialize (width:Number, height:Number, background:Number = 0xFFFFFF):void {
 
			setWidth = width;
			setHeight = height;
			backgroundColor = background;
 
			View = new View3D ();
			View.x = setWidth / 2;
			View.y = setHeight / 2;
			View.session = new BitmapSession (1);
 
			// testing
 
			var targetCamera:TargetCamera3D = new TargetCamera3D ();
 
			var test:Cube = new Cube ();
 
			targetCamera.z = -1000;
			targetCamera.y = 500;
			targetCamera.x = 500;
			targetCamera.zoom = 30;
			targetCamera.lookAt (test.position);
 
			View.camera = targetCamera;
			View.scene.addChild (test);
 
			addChild (View);
 
		}
 
		public function render (takeSnapshot:Boolean = true):String {
 
			View.render ();
 
			if (takeSnapshot) {
 
				var snapshot:BitmapData = new BitmapData (setWidth, setHeight, false, backgroundColor);
				snapshot.copyPixels (View.getBitmapData (), View.getBitmapData ().rect, new Point ());
 
				var bytes:ByteArray = PNG24Encoder.encode (snapshot);
				var base64:String = Base64.encode (bytes, false);
 
				return base64;
 
			} else {
 
				return "";
 
			}
 
		}
 
	}
 
}

To run my Flash renderer, I am using Away3D and the PNG encoder and Base64 encoder from blooddy_crypto. Because the renderer is actually small when it is embedded in my application, it is important that you set the SWF width and height when you compile to a size that is as large or larger than your Canvas. Otherwise you will experience unwanted clipping when you export frames. When the renderer begins, it first adds callbacks so that Javascript can call functions on it. Then it calls App.Renderer_onReady to say that it is ready to go. I used FDBuild under Linux in order to compile my Flash content easily, but if you are running Windows, I highly recommend FlashDevelop.

When the renderer has been initialized, it creates a new View3D instance and adds it to the display list. Because Away3D centers its content on (0, 0), we move it to the center of the stage. For testing purposes, I added a cube and configured a camera to view it at an angle, so it would look 3D.

When Javascript tells the Flash renderer to render the current frame, it prompts Away3D to render, then to improve performance, the pixels are copied into a new object without transparency. This is encoded as a PNG, then as a Base64 string, then passed back to Javascript.

Using this method, my entire web application is built with HTML, CSS and Javascript. Only the hidden renderer is built in Flash. If WebGL becomes supported by most browsers, Away3D or another engine may be available to handle the rendering. Otherwise, it would always be possible to pre-render or utilize some kind of renderer on the server, then to load Base64 images using AJAX.

  • triynko

    Demo?