FX Effects

Effects Framework

To create a set of visual effects in a logical unit, Microsoft defined effects framework. Effect files are referred to as fx files with the extension .fx. It can be compiled as a .fxo file.

The structure of an fx file looks like the following:

Global Variables
  Effect States
    Pipeline,Texture,Sampler,Shader
  Functions
    HLSL, Assembly
  Effect File Parameters
  Technique T0
    Global Variables
    Effect States
    Pass P0
  ...

The effects framework allows effect developers to group similar graphics techniques in a single file. It also allows the content creators to create a single set of content that will work across multiple PC configurations while maintaining a consistent look and feel. The effects framework allows techniques to range from high-level shader, low-level shader, and fixed-function pipeline.

Here are some features of effects:

• The Global Variables of effects can be set by either the effect itself or by the application.

• The State Management of effects includes pipeline, texture, sampler, shader state management. The pipeline state management includes setting transformations, lighting, materials, and rendering options. The texture and sampler state management contains specifying texture files, initializing texture stages, creating sampler objects, and setting sampler state. The shader state management includes creating and deleting shader, setting shader constants, setting shader state, and rendering with shader.

• The Effects contain multiple rendering options called Techniques. Each technique encapsulates global variables, pipeline state, texture and sampler state, and shader state. A single style is implemented in a rendering pass. One or more passes can be encapsulated in a technique. All of the passes and techniques can be validated to see if the effect code will run on the hardware device. Effects will save and restore state, leaving the device in the same state as before the effect was run.

• The Function is defined by HLSL or Assembly program.

Effect Parameters

Effect parameters are all the non-static variables declared in an effect. This can include global variables and annotations. Effect parameters can be shared between different effects by declaring parameters with the shared keyword and then creating the effect with an effect pool. A global variable with the static keyword in front is not an effect parameter.

Annotations provide a mechanism for adding user information to effect parameters. It is used to exchange data between an application and a GPU program. Annotation declarations are delimited by angle brackets, <>. For example,

float3 LightVector < string UILightVector = "Light Vector"; >
= {1.0, -1.0, 0.0};

Usually an annotation starts with a data type (string), followed by an assign statement with variable name (UILightVector) and the value ("Light Vector"), and ended by a semicolon. The annotation is declared behind a variable and in front of the assignment of the initial value {1.0, -1.0, 0.0}.

Effect States

Effects simplify managing pipeline state. States of effects specify the render conditions of the pipeline. Effects render a given technique that may contain one or more passes. State is restored each time a technique finished. However there are no state changes between passes. Effect states can be divided into the following functional areas:

Notes: State description in details can be found in the DirectX 9.0 SDK documentation.

FX Effects in Ladybug Mixer

Ladybug Mixer can complete real-time video post-processing through programmable GPU. Users can apply image-processing techniques to video and implement these techniques with HLSL in GPU to achieve 10x+ performance than CPU computing. A video frame is treated as a texture and is rendered over a scene, and then the final result is processed through pixel shader. The pos-processing effects can achieve interesting results such as blur of a picture, bloom of the bright spots, or depth of field to blur a pixel based on its depth.

An effect file implemented all the post-process in Ladybug Mixer. More than one techniques that may have one or more passes have been defined in the .fxo file.

Declaration

In the effect file, there are four texture objects with samplers from which the postprocess pixel shader will sample. They are defined as a texture array.

texture SamplerTex0;
texture SamplerTex1;
texture SamplerTex2;
texture SamplerTex3;

sampler source

sampler g_sampler[MAXNUM_CHANNEL] = {
  sampler_state
  {
    Texture = ;
    AddressU = Clamp;
    AddressV = Clamp;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = None;
  };
  ...
}; 

Usually the post-process pixel shader will need to sample a source texture multiple times for a single destination pixel, each at a different location near the destination pixel. When this is the case, the effect file declares a kernel which is an array of texel offsets (float2) from the destination pixel. The effect file then defines another equally-sized array of float2. For example,

float2 PixelKernel[KernelSize] = { { -1, 0 },{ 0, 0 }, { +1, 0 } };
float2 TexelKernel[KernelSize]
<
  string ConvertPixelsToTexels = "PixelKernel";
>;

This second array has a string annotation, named ConvertPixelsToTexels, which contains the name of the first kernel array. When the sample loads the effect file and sees this annotation, it will translate the offsets from pixel coordinates to texel coordinates, which are compatible with the texture sampling functions such as tex2D. When the source texels are sampled, the offsets in the kernel are added to the texture coordinates to obtain the texel values near the destination pixel.

Parameters

Some effects use parameters that the user can adjust. For instance, the Bright Pass effect has parameters that the user can tweak to give different visual appearances. Each PostProcess technique can define one or more annotations for this purpose. The maximum number of parameters supported by a single technique is four.

Here is an example.

float2 PixelKernel[4] = { {0, 1}, {1, 0}, {0, -1}, {-1, 0} };
float2 TexelKernel[4]
  < string ConvertPixelsToTexels = "PixelKernel"; >;

convert pixel kernel to texel kernel.

float3 LuminanceConv = { 0.2125f, 0.7154f, 0.0721f };

specifies luminance converting constant.

float4 PostProcessPS( float2 Tex[MAXNUM_CHANNEL] : TEXCOORD0 ) : COLOR0
{
  float4 Orig = tex2D( g_sampler[0], Tex[0] );
  float4 Sum = 0;
  for( int i = 0; i < 4; i++ )
    Sum+=(abs(Orig-tex2D(g_sampler[0],Tex[0]+TexelKernel[i]))-0.5f)*1.2+0.5f;
  return saturate( dot( LuminanceConv, Sum ) ) * 5;
}

defines PostProcessPS function. Get the Sum value around the texel kernel, then computes the dot product.

technique PostProcess
{
  pass p0
  {
    VertexShader = compile vs_2_0 VertScene();
    PixelShader  = compile ps_2_0 PostProcessPS();
    CullMode = None;
    ZEnable = false;
  }
}

PostProcess technique specifies vertex and pixel shader. The vertex shader is the fixed function. The pixel shader is the compiled PostProcessPS function with PS2.0. The z enable is set to false.

Copyright© 2004-2016 Neatware. All Rights Reserved.