Displaying In-World Video and Web Pages

From Multiverse

Jump to: navigation, search

Overview

The Multiverse Client can display videos (both steaming video and local video in the asset repository) and web pages in a virtual world. Sampleworld contains examples of both a local video and a web page displayed on floating cubes. Video or web pages displayed this way will appear on a surface of a 3D object in the world, and may be occuluded by objects that come between the player and the object.

Displaying video

There are two ways to display video:

Replace an object in the scene entirely with a video. This is the easiest approach in development and testing. See Displaying a video by replacing a world object.
Attach a video to a 3D object using a material script, following the steps below. See Displaying video in a material.

Displaying web pages

The Multiverse Client can display a web page in three different ways:

In a texture, using the same techinque as described for a video in this article. The Client will render a web page on the surface of a 3D object in perspective. You cannot interact with web pages displayed in this manner. See Displaying a web page in a texture.
In an external browser window, launched by a Client event such as a button click. See Displaying a web page in an external browser window.
In a Browser UI widget. This will be a 2D user interface element that the Client displays "on top" of the 3D world. Such an widget is not occluded by objects in the world. You can show or hide it based on client events, such as a button click. See Displaying a web page in a UI widget.

Limitations

Web pages can display rich media such as Microsoft Silverlight and Adobe Flash. However, the following are not supported:

Quicktime video
Windows Media Player content
Java applets

Displaying a video by replacing a world object

You can use a client script to replace an object with a video. For example, this script will watch for the special movie object to be loaded, and replace it.

import ClientAPI

def LoadMovieHandler(object):
  ClientAPI.Log("Added object " + object.Name)
  if object.Name == "MOVIEBOX":
    cm = ClientAPI.MovieManager.Instance.FindCodec("DirectShow")
    im = cm.LoadFile("Satisfaction", "satisfaction.mpg")
    if im != None:
        im.ReplaceWorldObject(object.Name)
        im.SetParameter("volume", "100")
        im.SetParameter ("balance", "0")
        im.SetParameter("looping", "true")

ClientAPI.RegisterEventHandler('ObjectAdded', LoadMovieHandler)

This code waits for the MOVIEBOX object to be loaded, and then replaces it with a new movie object.

The code above selects a codec to load the movie, creating a MovieCodec object with the following line:

cm = ClientAPI.MovieManager.Instance.FindCodec("DirectShow")

A codec determines how to decompress the movie and play it. The "DirectShow" codec is faster, and tends to make clearer videos, but streaming can be problematic. The "DirectShow" codec is the best choice to play local MPEG, WMV, and ASF files. The "Browser" codec will load any web page, so it will display Flash animation, but not embedded video players that use a secondary DirectDraw surface. NOTE: With the Browser codec, there is no volume control.

Then the following line uses the MovieCodec.LoadFile() method to load a movie:

im = cm.LoadFile("Satisfaction", "satisfaction.mpg")

The LoadFile() method takes two or three parameters:

The name of the movie that will be used in referring to it later,
The filename of the movie in the Movies directory,
An optional third parameter is the name of the texture on which to put the movie. If you don't specify this parameter, it will generate a unique name.

If the movie loads, the code then specifies to "take over" a world object, replacing the existing geometry with a plane that's "movie width x movie height" pixels across. The movie will play as soon as it loads.

The im object is a Movie object, and the "cm" object is an MovieCodec object.

The Movie.SetParameter() method takes string name/value pairs to support a variety of codecs that require different parameters. For example, the Browser codec can create a web browser at any size, whereas the DirectShow player expects to use the size of the movie. So, in this example, the code sets the volume of the movie to 100%, the balance to the middle (it goes -100% to 100%), and specifies to loop the movie, that is, replay it on completion.

Displaying video in a material

The basic procedure is:

Use World Editor to add a "screen object" to your world that will display the video.
Create a material script file for the object that refers to the desired video.
Add the new material script to your asset repository.
In World Editor, edit the object to refer to the material script.
If desired, for a video, add scripts to control it (play, stop, and so on).

Add screen object

The "screen object" is just the in-world object that will display the video or web page. In Sampleworld, it is the unit cube (unit_box.mesh), which is just an untextured box. But it can be any object that you want to use for this purpose. If the object does not have a flat surface at least as large as the specified size of the video or web page, then the image will be truncated.

Create material file

To make the screen object display the desired video or web page, you must create a material script that refers to the desired video or web page, and then make your screen object use it. For example, in Sampleworld, the unit cube normally uses the unit_box.material script, which is just:

material unit_box.unit_box : MVRed
{
}

However, to display a video, this is replaced with wallvideo.material:

material wallvideo.wall
{
    technique
    {
        pass
        {
            shading phong

            ambient  1.00000 1.00000 1.00000 1.00000
            diffuse  1.00000 1.00000 1.00000 1.00000
            specular 0.00000 0.00000 0.00000 1.00000 1.00000
            emissive 0.00000 0.00000 0.00000 1.00000

            texture_unit
            {
               texture_source mvMovie
               {
                       name DShow Movie
                       codec DirectShow
                       path RafAndCorey.wmv
                       looping true
               }
                tex_coord_set 0
            }
        }
    }
}

Notice the texture_source section that specifies the name of the video file in the asset repository Movies directory. This section also specifies to use the DirectShow codec. The name is used to refer to the movie in the movie API, and the path is the name of the movie file in the Movies directory. For more information on texture_source, see Texture Units.

If you omit the path, a placeholder texture will be created but not displayed, using either a "textureName" parameter in the material, or the containing material name if none is provided. The logs will indicate what it created. If you provide a path, it will start playing on startup.

Displaying a streaming video

To display a video streamed over the internet, simply put the URL in the path attribute instead of the video file name, for example:

path http://www.multiverse.net/videos/times_square.wmv

Add to asset repository

If you create the material file by copying an existing material file, make sure you put the new material in the /Materials directory in your asset repository. You may wish to use Asset Importer to import it to you asset repository, so that it will have a corresponding asset definition file.

Associate material with screen object

In World Editor, attach the new material to the object. Follow these steps in World Editor:

Select the screen object.
In the Properties View at lower right look for the SubMeshes property.
Click on the "..." button that appears to the right of this property to bring up the Submesh Editor dialog box, shown at right for the video example in Sampleworld.
Select the desired submesh.
The Material text field at the bottom of the dialog box will show the associated material file. Change this to refer to the material script you created previously.
Click OK.
Save your world file.

Scripting

If you want to be able to control a video (start it, stop it, and so on), then you have to do some scripting. Add a new file to the /Scripts directory to contain your script, for example, PlayMovies.py.

To pause it or load a new video when it's already running, you basically find the movie object, in this case named "Target Movie", and tell it what to do:

def PauseMovieHandler():
  movie = ClientAPI.MovieManager.Instance.FindMovie("Target Movie")
  if movie != None:
    movie.Pause()

To replace a running movie with a different movie, use code such as the following:

def ReplaceMovieHandler():
  movie = ClientAPI.MovieManager.Instance.FindMovie("Target Movie")
  texture = "Target Movie Texture"
  if movie != None:
    texture = movie.TextureName()
    codec = ClientAPI.MovieManager.Instance.FindCodec(movie.CodecName())
    codec.UnloadMovie(movie)
  codec = ClientAPI.MovieManager.Instance.FindCodec("DirectShow")
  movie = codec.LoadFile("Target Movie", "movie.mpg", texture)

What this script is doing is first, finding the already playing movie object (in this case, named "Target Movie.") If it finds it, it saves the name of the texture the movie is playing on. If the movie isn't found, it creates a new texture called "Target Movie Texture". It then tells the codec that owns the movie to unload the movie.

Finally, it has to load the replacement movie. It finds the "DirectShow" codec, and tells it to load a new movie, naming it "Target Movie" so that subsequent calls to replace the movie find the new movie object, using the file "movie.mpg" from the Movies directory, using the same texture name as the previous movie did.

If the movies are the same size (likely), then everything's set. If you loaded a different sized movie, you need to reset the texture coordinates. You then can call:

 movie.SetTextureCoordinates("Material Name")

This will adjust the display of the movie to match the new movie size.

Note that you can't work backwards from a movie and find all the materials that reference it, you must know the name of the materials that are used to display the texture.

Reference

Import your script

In your asset repository Scripts directory, change Startup.py to import a new script file. For example, if your script file is PlayMovies.py, then at the end of the file, just before the line ClientAPI.Log("Startup.py loaded"), add:

import PlayMovies

Other steps

Once you've created this material and put it in your world, you can add UI elements to start and stop the existing movie, unload it, then create and load a new one, and so on.

There's one more step that this touches on: when you create this new material, you should make sure it gets packaged in your asset repository. If you add a movie, you will need to add it with Asset Importer as well.

Displaying a web page in a texture

You can display a web page instead of a video in a texture using basically the same procedure as described above in Displaying video in a material. The only difference is the material script.

Material script for web pages

To display a web page instead of a video, use the value "Browser" for the codec, and specify a URL for the path instead of the name of a vide file. For example:

 texture_source mvMovie
 {
  name DShow Browser
  codec Browser
  path http://www.multiverse.net
  videoSize 1024x1024
  }

The browser codec takes two of its own parameters: "videoSize", which takes a string "WIDTHxHEIGHT" that specifies the dimensions of the browser window as the source of the texture. So, for example, "320x240" is a valid value. The texture size will default to the next highest power of two in both directions, in this example 512x256, or you can set the size with the "textureSize" parameter.

The browser codec also uses LoadStream instead of LoadFile; the parameters are the same, but the path as the second parameter is a URL, for example http://www.multiverse.net/.

Displaying a web page in an external browser window

Use the ClientAPI.LaunchBrowser() method to launch a browser displaying the URL passed as the argument to the function. The browser is your default web browser. The window will be external to the Multiverse Client window.

Displaying a web page in a UI widget

Example of a browser widget in Sampleworld

Example of a browser widget in Sampleworld

Use the Browser widget to display a web page in a two-dimensional UI element. You can click to follow links in such web pages, and scroll them within the Browser widget frame; however you cannot interact with them via the keyboard. When the Browser widget is active, it will capture all mouse clicks and keyboard events (though it won't respond to keyboard events); you won't be able to move your avatar or change viewpoint. So, you must put a "close button" in a parent frame of your Browser widget to hide the browser and restore normal Client event capturing.

Example

The following example consists of two files that create a Browser widget that appears on startup.

 <Browser name="$parentWidget" 
     url="http://www.multiverse.net/"
     scrollbars="true" errors="true" line="1" >
  <Anchors>
   <Anchor point="TOPLEFT">
    <Offset>
     <AbsDimension x="0" y="-24"></AbsDimension>
    </Offset>
   </Anchor>
   <Anchor point="BOTTOMRIGHT"> </Anchor>
  </Anchors>
 </Browser>

You can close the browser by clicking on the "X" button at the upper right of the Browser. Add these two files to your asset repository /Interface/FrameXML directory:

Example Browser widget - Copy and paste the code into Browser.xml.
Example Browser script - Copy and paste the code into Browser.py.

Retrieved from "http://update.multiverse.net/wiki/index.php/Displaying_In-World_Video_and_Web_Pages"

Category: 1.1 Documentation

Views

Personal tools

Log in / create account

General

Search

Power Search

Toolbox