Vertex and Fragment Programs

From Multiverse

Particle Scripts

Syntax  •  Attributes  •  Emitters  •  Affectors

Material Scripts

Techniques  •  Texture Units  •  Passes  •  Vertex and Fragment Programs  •  Calling Vertex and Fragment Programs  •  Copying Materials

Compositor Framework

Compositor Scripts  •  Compositor Techniques  •  Compositor Target Passes  •  Compositor Passes

Shaders

Writing Custom Shaders  •  Built-in Shaders  •  Troubleshooting

Contents 1 Overview 1.1 References 1.2 Default program parameters 2 Implementing animation in vertex programs 2.1 Implementing skeletal animation 2.2 Implementing morph animation 2.3 Implementing pose animation 3 Vertex texture fetching 3.1 Declaring the use of vertex texture fetching 3.2 Render system texture binding differences 3.3 Texture format limitations 3.4 Hardware limitations 4 Vertex programs with shadows 5 Cg programs 5.1 Low-level syntaxes 6 DirectX9 HLSL 6.1 Matrix multiplation 6.2 Advanced parameters 7 Using unified high-level programs

Overview

A material script can call a vertex or fragment program (shader). Any number of material scripts can use a single shader. The only prerequisite is that the shader must be defined before being referenced in the pass section of a material. NOTE: this documentation will use the terms vertex program and shader script (or just shader) interchangeably.

For a general introduction to shaders, see Shader Scripts.

You can define a vertex program:

• In the .material script itself (in which case it must precede any references to it in the script)
• In an external .program script, if you wish to use the same program across multiple .material files.

In both cases, you define the program in the same way. The only difference is that all .program scripts are parsed before all .material scripts, so you can guarantee that your program has been defined before any .material script that might use it. Like .material scripts, .program scripts are read from the current asset repository, and you can define many programs in a single script.

The Multiverse Client supports vertex programs written in:

• Cg, an API-independent and card-independent language from NVIDIA.
• HLSL, DirectX 9 High-Level Shader Language.

You can support both languages by including separate techniques in the material script, each referencing separate programs. However, if the programs are basically the same, with the same parameters, and the techniques are complex, this can bloat your material scripts with duplication. If the only difference is the language of the vertex and fragment program, use unified high-level programs to pick a program suitable for your rendering system while using a single technique.

Although in most cases, you will want to use a high-level language for a vertex and fragment program, The Multiverse Client also supports using assembly language. For more information see the OGRE documentation.

References

• Microsoft HLSL documentation
• NVIDIA Cg Homepage

Default program parameters

Specify the default parameters for materials that use a vertex or fragment program in the default_params section, as follows:

vertex_program the Multiverse Client/CelShadingVP cg
{
	source Example_CelShading.cg
	entry_point main_vp
	profiles vs_1_1 arbvp1

	default_params
	{
		param_named_auto lightPosition light_position_object_space 0
		param_named_auto eyePosition camera_position_object_space
		param_named_auto worldViewProj worldviewproj_matrix
		param_named shininess float 10 
	}
}

The syntax of the parameter definition is the same as when you define parameters when using programs; for more information, see Program Parameter Specification. Defining default parameters allows you to avoid rebinding common parameters repeatedly (clearly in the above example, all but 'shininess' are unlikely to change between uses of the program) which makes your material declarations shorter.

Implementing animation in vertex programs

If desired, a vertex program can implement:

• skeletal animation
• morph animation
• pose animation

Implementing skeletal animation

Implement skeletal animation in hardware by writing a vertex program that uses the per-vertex blending indices and blending weights, together with an array of world matrices (which the Multiverse Client provides if you bind the automatic parameter 'world_matrix_array_3x4').

Indicate hardware skeletal animation with the following attribute in the vertex program definition:

includes_skeletal_animation true

With this attribute, the Multiverse Client will not perform skeletal animation in software: any skeletally-animated entity using the material will forgo the usual animation blend and expect the vertex program to do it, for both vertex positions and normals.

NOTE: You must assign a material which implements the program to ALL submeshes

If you combine skeletal animation with vertex animation, then all techniques must be hardware accelerated (or none).

Implementing morph animation

Implement morph animation in hardware by

• Writing a vertex program that linearly blends between the first and second position keyframes passed as positions and the first free texture coordinate set.
• Binding the animation_parametric value to a parameter (which tells you how far to interpolate between the two).

Indicate hardware morph animation with the following attribute in the vertex program definition:

includes_morph_animation true

With this attribute, the Multiverse Client will not perform morph animation in software: any skeletally-animated entity using the material will forgo the usual software morph and expect the vertex program to do it.

If a model includes both skeletal animation and morph animation, they must both be implemented in the vertex program if either is to be hardware acceleration.

NOTE: You must assign a material which implements the program to ALL submeshes

If you combine morph animation with vertex animation, then all techniques must be hardware accelerated (or none).

Implementing pose animation

Pose animation performs blending between multiple poses based on weight. Implement pose animation in a vertex program with the following attribute:

includes_pose_animation n

Where n is the maximum number of poses the shader can blend. Pull in the original vertex data (bound to position), and n, which is in the first free texture unit upwards. Also use the animation_parametric auto-parameter value code to define the starting point of the constants containing the pose weights. The constants start at the parameter you define and fill n constants.

NOTE: You must assign a material which implements the program to ALL submeshes

If you combine pose animation with vertex animation, then all techniques must be hardware accelerated (or none).

Vertex texture fetching

Recent generations of video cards allow you to read from a texture in the vertex program rather than just the fragment program, as is traditional. This enables you to, for example, read the contents of a texture and displace vertices based on the intensity of the color contained within.

Specify the uses_vertex_texture_fetch attribute to use vertex texture fetching and check hardware support:

uses_vertex_texture_fetch true

Declaring the use of vertex texture fetching

Since hardware support for vertex texture fetching is not ubiquitous, use the uses_vertex_texture_fetch directive when declaring vertex programs that use vertex textures, so that if the client hardware does not support it, technique fallback can be enabled, as described above.

You do not have to use this attribute for DirectX-targeted shaders, since vertex texture fetching is only supported in vs_3_0, and you can state this as a required syntax in the shader definition.

Render system texture binding differences

Unfortunately the method for binding textures so that they are available to a vertex program is not standardized. Shader Model 3.0 (SM3.0) hardware under DirectX9 includes four separate sampler bindings for the purposes of vertex textures. DirectX may move to the GL model with the advent of DirectX 10, since a unified shader architecture implies sharing of texture resources between the two stages. As it is right now though, we're stuck with an inconsistent situation.

To reflect this, use the binding_type attribute in a texture unit to indicate which unit you are targetting with your texture - 'fragment' (the default) or 'vertex'. For render systems that don't have separate bindings, this actually does nothing. But for those that do, it will ensure your texture gets bound to the right processing unit.

While DirectX 9 has separate bindings for the vertex and fragment pipelines, binding a texture to the vertex processing unit still uses up a 'slot' which is then not available for use in the fragment pipeline. NVidia samples avoid binding a texture to the same index on both vertex and fragment units, and the textures may not appear correctly in the fragment unit, but may if it is moved into the next unit.

Texture format limitations

The types of texture you can use in a vertex program are limited to 1- or 4-component, full precision floating point formats. In code that equates to PF_FLOAT32_R or PF_FLOAT32_RGBA. No other formats are supported. In addition, the textures must be regular 2D textures (no cube or volume maps) and mipmapping and filtering is not supported, although you can perform filtering in your vertex program if you wish by sampling multiple times.

Hardware limitations

Not all GPUs may support vertex texture fetching. See the hardware manufacturer's specifications for information.

Vertex programs with shadows

When using shadows (see About Shadows), vertex programs can add some additional complexities, because the Multiverse Client can only automatically deal with everything when using the fixed-function pipeline. If you use vertex programs, and you are also using shadows, you may need to make some adjustments.

If you use stencil shadows, then any vertex programs which do vertex deformation can be a problem, because stencil shadows are calculated on the CPU, which does not have access to the modified vertices. If the vertex program is doing standard skeletal animation, this is OK (see section above) because the Multiverse Client knows how to replicate the effect in software, but any other vertex deformation cannot be replicated, and you will either have to accept that the shadow will not reflect this deformation, or you should turn off shadows for that object.

If you use texture shadows, then vertex deformation is acceptable; however, when rendering the object into a shadow texture (the shadow caster pass), the shadow has to be rendered in a solid color (linked to the ambient color for modulative shadows, black for additive shadows). You must therefore provide an alternative vertex program, so the Multiverse Client provides a way to specify one to use when rendering the caster. For more information, see Shadows and Vertex Programs.

Cg programs

Define a Cg program as follows:

fragment_program myCgFragmentProgram cg
{
    source myCgFragmentProgram.cg
    entry_point main
    profiles ps_2_0 arbfp1
}

The source parameter references a Cg source file.

The entry_point parameter specifies the name of a function in the Cg program which is the first one called as part of the fragment program. Cg programs can include multiple functions, so you must specify the initial "entry point."

Next, comes the profiles parameter that lists one or more low-level syntax codes (see table below). Cg uses the profiles to compile a program down to low-level assembler language. Because you can list more than one syntax, the program can be compiled down to multiple low-level syntaxes. For maximum compatibility, enter the simplest profiles under which your programs can be compiled. The ordering also matters; if a card supports more than one syntax then it will use the one listed first.

Lastly, the optional compile_arguments parameter specifies arguments as you would to the cgc command-line compiler, should you wish to.

Low-level syntaxes

The following table lists syntax codes available for high-level vertex and fragment programs.

Syntax Code	Syntax Description	Hardware Support
vs_1_1	DirectX vertex shader model 1.1	ATI Radeon 8500, nVidia GeForce 3.
vs_2_0	DirectX vertex shader model 2.0	ATI Radeon 9600, nVidia GeForce FX 5 series.
vs_2_x	DirectX vertex shader model 2.x	ATI Radeon X series, nVidia GeForce FX 6 series.
vs_3_0	DirectX vertex shader model 3.0	nVidia GeForce FX 6 series.
ps_2_0	DirectX pixel shader (fragment program)	ATI Radeon 9600, nVidia GeForce FX 5 series
ps_2_x	DirectX pixel shader (fragment program)--basically ps_2_0 with more instructions.	ATI Radeon X series, nVidia GeForce FX 6 series
ps_3_0	DirectX pixel shader (fragment program)	nVidia GeForce FX6 series.
ps_3_x	DirectX pixel shader (fragment program)	nVidia GeForce FX7 series.

DirectX9 HLSL

DirectX9 HLSL has a similar language syntax to Cg but is tied to the DirectX API. The only benefit over Cg is that it only requires the DirectX 9 render system plugin, not any additional plugins. Declaring a DirectX 9 HLSL program is very similar to Cg, for example:

vertex_program myHLSLVertexProgram hlsl
{
    source myHLSLVertexProgram.txt
    entry_point main
    target vs_2_0
}

As you can see, the main syntax is almost identical, except that instead of a profiles parameter with a list of assembler formats, there is a target parameter that specifies a single DirectX assembler format syntax.

Matrix multiplation

HLSL provides two different ways to multiply a vector by a matrix: mul(v,m) or mul(m,v). The only difference between them is that the matrix is effectively transposed. Use mul(m,v) with the matrices passed in from the Multiverse Client. This agrees with the shaders produced from tools like RenderMonkey, and is consistent with Cg too, but disagrees with the Dx9 SDK and FX Composer which use mul(v,m). With those shaders, switch the parameters.

If you use the float3x4 / matrix3x4 type in your shader, bound to an the Multiverse Client auto-definition (such as bone matrices) you should use the column_major_matrices = false option (discussed below) in your program definition. This is because the Multiverse Client passes float3x4 as row-major to save constant space (3 float4's rather than 4 float4's with only the top 3 values used) and this tells the Multiverse Client to pass all matrices like this, so that you can use mul(m,v) consistently for all calculations. the Multiverse Client will also to tell the shader to compile in row-major form (you don't have to set the /Zpr compile option or #pragma pack(row-major) option, the Multiverse Client does this for you). Note that passing bones in float4x3 form is not supported by the Multiverse Client, but you don't need it given the above.

Advanced parameters

preprocessor_defines defines

This parameter defines symbols which can be used ins the HLSL shader code to alter the behavior (through #ifdef or #if clauses). Separate definitions by ';' or ',' and may optionally have a '=' operator within them to specify a definition value. Those without an '=' will implicitly have a definition of one.

column_major_matrices [true|false]

The default for this parameter is 'true' so that the Multiverse Client passes auto-bound matrices in a form where mul(m,v) works. Setting this option to false does two things: it transpose auto-bound 4x4 matrices and sets the /Zpr (row-major) option on the shader compilation. This means you can still use mul(m,v), but the matrix layout is row-major instead. This is only useful if you need to use bone matrices (float3x4) in a shader since it saves a float4 constant for every bone involved.

sdk_mul_compat [true|false]

The default for this parameter is 'false'. Setting this value to 'true' enables a compatibility mode for shaders that are written to use the mul(v,m) form of matrix/vector multiplication. Set this value to 'true' when importing shaders from sources that use this form, such as the DirectX SDK, NVIDIA FX Composer, or Mental Mill.

Using unified high-level programs

NOTE: The following sectionis applicable to HLSL and Cg, but the Multiverse Client does not currently support GLSL.

It can often be useful to write both HLSL and GLSL programs to specifically target each platform, but if you do this via multiple material techniques this can cause a bloated material definition when the only difference is the program language. Well, there is another option. You can 'wrap' multiple programs in a 'unified' program definition, which will automatically choose one of a series of 'delegate' programs depending on the rendersystem and hardware support.

vertex_program myVertexProgram unified
{
	delegate realProgram1
	delegate realProgram2
	... etc
}

This works for both vertex and fragment programs, and you can list as many delegates as you like - the first one to be supported by the current rendersystem & hardware will be used as the real program. This is almost like a mini-technique system, but for a single program and with a much tighter purpose. You can only use this where the programs take all the same inputs, particularly textures and other pass / sampler state. Where the only difference between the programs is the language (or possibly the target in HLSL - you can include multiple HLSL programs with different targets in a single unified program too if you want, or indeed any number of other high-level programs), this can become a very powerful feature. For example, without this feature here's how you'd have to define a programmable material which supported HLSL and GLSL:

vertex_program myVertexProgramHLSL hlsl
{
	source prog.hlsl
	entry_point main_vp
	target vs_2_0
}
fragment_program myFragmentProgramHLSL hlsl
{
	source prog.hlsl
	entry_point main_fp
	target ps_2_0
}
vertex_program myVertexProgramGLSL glsl
{
	source prog.vert
}
fragment_program myFragmentProgramGLSL glsl
{
	source prog.frag
	default_params
	{
		param_named tex int 0
	}
}
material SupportHLSLandGLSLwithoutUnified
{
	// HLSL technique
	technique
	{
		pass
		{
			vertex_program_ref myVertexProgramHLSL
			{
				param_named_auto worldViewProj world_view_proj_matrix
				param_named_auto lightColour light_diffuse_colour 0
				param_named_auto lightSpecular light_specular_colour 0
				param_named_auto lightAtten light_attenuation 0
			}
			fragment_program_ref myFragmentProgramHLSL
			{
			}
		}
	}
	// GLSL technique
	technique
	{
		pass
		{
			vertex_program_ref myVertexProgramHLSL
			{
				param_named_auto worldViewProj world_view_proj_matrix
				param_named_auto lightColour light_diffuse_colour 0
				param_named_auto lightSpecular light_specular_colour 0
				param_named_auto lightAtten light_attenuation 0
			}
			fragment_program_ref myFragmentProgramHLSL
			{
			}
		}
	}
}

And that's a really small example. Everything you added to the HLSL technique, you'd have to duplicate in the GLSL technique too. So instead, here's how you'd do it with unified program definitions:

vertex_program myVertexProgramHLSL hlsl
{
	source prog.hlsl
	entry_point main_vp
	target vs_2_0
}
fragment_program myFragmentProgramHLSL hlsl
{
	source prog.hlsl
	entry_point main_fp
	target ps_2_0
}
vertex_program myVertexProgramGLSL glsl
{
	source prog.vert
}
fragment_program myFragmentProgramGLSL glsl
{
	source prog.frag
	default_params
	{
		param_named tex int 0
	}
}
// Unified definition
vertex_program myVertexProgram unified
{
	delegate myVertexProgramGLSL
	delegate myVertexProgramHLSL
}
fragment_program myFragmentProgram unified
{
	delegate myFragmentProgramGLSL
	delegate myFragmentProgramHLSL
}
material SupportHLSLandGLSLwithUnified
{
	// HLSL technique
	technique
	{
		pass
		{
			vertex_program_ref myVertexProgram
			{
				param_named_auto worldViewProj world_view_proj_matrix
				param_named_auto lightColour light_diffuse_colour 0
				param_named_auto lightSpecular light_specular_colour 0
				param_named_auto lightAtten light_attenuation 0
			}
			fragment_program_ref myFragmentProgram
			{
			}
		}
	}
}

At runtime, when myVertexProgram or myFragmentProgram are used, the Multiverse Client automatically picks a real program to delegate to based on what's supported on the current hardware / rendersystem. If none of the delegates are supported, the entire technique referencing the unified program is marked as unsupported and the next technique in the material is checked fro fallback, just like normal. As your materials get larger, and you find you need to support HLSL and GLSL specifically (or need to write multiple interface-compatible versions of a program for whatever other reason), unified programs can really help reduce duplication.