Unity3D and Flash integration – Part 1

Unity3D is a excellent 3D games development system, but lacks the ability to leverage existing Flash based content.

This article describes my work so far, to enable Flash and Unity3D content to be used together.

As both a Flash and Unity3D developer, I’ve always wished that there was a way to use Flash content inside Unity, as Flash is a great tool for quickly creating an animation or 2D game.

A partial solution, pioneered by the guys at Aquiris, is to place Flash and Unity content in two different DIV’s on the same web page, and switch between the two DIV’s using javascript in the browser, and the external interface API’s of both Flash and Unity. An example of this is the SuperVolei Brazil website.

However on the SuperVolei Brazil site, Flash is only used as a menuing system before the Unity game is activated. The background of the Flash and the Unity content is the same pre-rendered image, so that when the browser switches between between the Flash and Unity DIV’s the player doesn’t see any visual change.

The big limitation of this technique is that the switch between Flash and Unity can only occur at times when the there is a pre-rendered background for both the Flash and Unity content, e.g. at the start and end of games.

While reading the Unity3D docs, I realised that there was a way to improve on the methods employed by Aquiris.

The technique employs texture.readPixels to take a snapshot of the user’s current view of the Unity scene, and using it as a background for the Flash content. Hence making it possible to switch from Unity to Flash at virtually any time.

The basis of the c# code for Unity is shown here.

int width = Screen.width;
int height = Screen.height;
 
Texture2D tex = new Texture2D( width, height, TextureFormat.RGB24, false );
 
tex.ReadPixels( new Rect(0, 0, width, height), 0, 0 );	// Read screen contents into the texture
tex.Apply();
 
byte [] imgBytes = tex.EncodeToPNG();// Encode texture into PNG
Destroy( tex );
sendImageData(imgBytes);// Send data to Flash to use as a background

Using the technique as describe above, I craeted this prototype / proof of concept. Where the user can move around a 3D scene and can activate Flash based popups by clicking on any active item.

Note. In the 3D scene, movement is controlled using the arrow keys. Objects are selected using the mouse. There is currently only one Flash item , the “Variables test” app.
It’s launched from both the computer screen, that is in the middle of the Unity scene when the prototype first loads, and again on the coffee pot, which is to the left of the computer screen. Clicking on either of these objects launches the same Flash app, however it is possible to launch different Flash apps off different hotspots.

It should be noted that the main caveat of this method is that the Unity scene needs to be “paused”, and some visual metaphor e.g. a darkening of the Unity content needs to be used, to inform the player that this has happened so that they don’t expect things to still be animating in the Unity scene. However I’ll discuss that in another posting.

In the next part of this series I’ll cover the method used to pass the image data to Flash as well as the operation Flash based swf launcher, which makes use of the background image.

To be continued…

7 Responses

  1. pinaud nicolas
    |

    Hello, sorry for my poor english, i’m french.
    Very good. I’m a beginner with unity. I’ve made some flash animation and i want to developp a unity game for a client of mine. It’s for elearning. Could i ask you some question if i’m lost ?
    thanks

  2. Roger Clark
    |

    Hi Nicolas,
    I’ve not done any Unity development for over 1 year. My clients do not want to install the Unity3D plugin, so all of my work is now just in Flash.

    You’re best bet is use the official Unity3D forums at http://forum.unity3d.com/ , I’ve found that they are very helpful.

    Cheers

    Roger

  3. pinaud nicolas
    |

    @Roger Clark
    Hi Roger,
    ok i don’t want to annoy you.
    but i don’t understand how to take the data in flash to use in a background. Could you give me the soluce ?
    I would like to have the next tutos ! It will be great.

  4. Roger Clark
    |

    Hi Nicolas,

    The development I did, sent the background image from Unity to Flash, but it sounds like you want to send an image from Flash to Unity.

    The process is basically the same.

    The method I used, was to render the camera in Unity to a PNG file, then the file data is sent to Flash, via JavaScript in the browser.
    However you can’t send binary data via JavaScript.
    So the image file data needs to be converted to text i.e UUEncode before it is sent, and the UUEncoded data needs to be decodded back to binary at the other end.

    But there appears to be a 15k limit to text sent to JavaScript from Unity, so I had to cut the String (of image data) into 15k blocks and send these separately, then re-assemble at the other end.

    But you cannot do send 15k, send 15k, send 15k, etc because this overloads the JavaScript, so you have to do

    1.Unity : send 15k
    2.Unit: Wait for message from flash
    3.Flash: recieve 15k
    4.Flash: send “OK message: to Unity
    6 Unity: receives “OK message” – go back to step 1 (until all data is sent)

    If you are sending from Flash to Unity, you will need to do the same thing, but in reverse.

    If sending from Flash to Unity, I guess you need to create a material in unity based on the PNG or JPG that you make in Flash, but I don’t know how to do that. You’ll need to look in the Unity API docs or post to the Unity forums

    Cheers

    Roger
    Unity

  5. Roger Clark
    |

    Hi Parvin,

    I’m not aware of a method to do this. My experiments were to make it look as if Flash was inside unity, but in reality the Flash movie was separate and just playing onto a snapshot of the last rendered Unity frame.

    You should try posting you question to the main Unity forums

  6. Andrea
    |

    Hi Roger!

    I found this solution to my Flash Unity dilemma, but I can’t actually understand (maybe because I am a Unity newbie) where to put your code and what is the way to call the JavaScript function from Unity.

    Do you happen to have an example? It would be perfect.

    Thanks a lot!

  7. Prash
    |

    Awesome! Thanks for sharing.

    ~Prash