Neatware Header

Ladybug Mixer
   §  Introduction
   §  Architecture
   §  Document (PDF)
   §  FAQ
   §  Versions

   §  Video Processing
   §  Video Presentation
   §  Digital Signage
   §  Video Jockey VJ
   §  Surveillance
   §  Home Theater
   §  Screen Wall
   §  Stream Computing

   §  Configuration
   §  Script Control
   §  Download Demo

FX Effects

Effects Framework

To create a set of visual effects in a logical unit, Microsoft defined effects framework. Effect files are referred to as fx files with the extension .fx. It can be compiled as a .fxo file.

The structure of an fx file looks like the following:

Global Variables
  Effect States
    HLSL, Assembly
  Effect File Parameters
  Technique T0
    Global Variables
    Effect States
    Pass P0

The effects framework allows effect developers to group similar graphics techniques in a single file. It also allows the content creators to create a single set of content that will work across multiple PC configurations while maintaining a consistent look and feel. The effects framework allows techniques to range from high-level shader, low-level shader, and fixed-function pipeline.

Here are some features of effects:

  • The Global Variables of effects can be set by either the effect itself or by the application.

  • The State Management of effects includes pipeline, texture, sampler, shader state management. The pipeline state management includes setting transformations, lighting, materials, and rendering options. The texture and sampler state management contains specifying texture files, initializing texture stages, creating sampler objects, and setting sampler state. The shader state management includes creating and deleting shader, setting shader constants, setting shader state, and rendering with shader.

  • The Effects contain multiple rendering options called Techniques. Each technique encapsulates global variables, pipeline state, texture and sampler state, and shader state. A single style is implemented in a rendering pass. One or more passes can be encapsulated in a technique. All of the passes and techniques can be validated to see if the effect code will run on the hardware device. Effects will save and restore state, leaving the device in the same state as before the effect was run.

  • The Function is defined by HLSL or Assembly program.

Effect Parameters

Effect parameters are all the non-static variables declared in an effect. This can include global variables and annotations. Effect parameters can be shared between different effects by declaring parameters with the shared keyword and then creating the effect with an effect pool. A global variable with the static keyword in front is not an effect parameter.

Annotations provide a mechanism for adding user information to effect parameters. It is used to exchange data between an application and a GPU program. Annotation declarations are delimited by angle brackets, <>. For example,

float3 LightVector < string UILightVector = "Light Vector"; >
= {1.0, -1.0, 0.0};

Usually an annotation starts with a data type (string), followed by an assign statement with variable name (UILightVector) and the value ("Light Vector"), and ended by a semicolon. The annotation is declared behind a variable and in front of the assignment of the initial value {1.0, -1.0, 0.0}.

Effect States

Effects simplify managing pipeline state. States of effects specify the render conditions of the pipeline. Effects render a given technique that may contain one or more passes. State is restored each time a technique finished. However there are no state changes between passes. Effect states can be divided into the following functional areas:

Name States
Pipeline states Light states, Material states, Render states,
Vertex render states, Pixel render states
Sampler states Sampler states, Sampler stage states
Texture states Texture states, Texture stage states
Shader states Shader constant states, Vertex shader constant states
Pixel shader constant states
Transform states Transform states

Notes: State description in details can be found in the DirectX 9.0 SDK documentation.

FX Effects in Ladybug Mixer

Ladybug Mixer can complete real-time video post-processing through programmable GPU. Users can apply image-processing techniques to video and implement these techniques with HLSL in GPU to achieve 10x+ performance than CPU computing. A video frame is treated as a texture and is rendered over a scene, and then the final result is processed through pixel shader. The pos-processing effects can achieve interesting results such as blur of a picture, bloom of the bright spots, or depth of field to blur a pixel based on its depth.

An effect file implemented all the post-process in Ladybug Mixer. More than one techniques that may have one or more passes have been defined in the .fxo file.


In the effect file, there are four texture objects with samplers from which the postprocess pixel shader will sample. They are defined as a texture array.

texture SamplerTex0;
texture SamplerTex1;
texture SamplerTex2;
texture SamplerTex3;

sampler source

sampler g_sampler[MAXNUM_CHANNEL] = {
    Texture = ;
    AddressU = Clamp;
    AddressV = Clamp;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = None;

Usually the post-process pixel shader will need to sample a source texture multiple times for a single destination pixel, each at a different location near the destination pixel. When this is the case, the effect file declares a kernel which is an array of texel offsets (float2) from the destination pixel. The effect file then defines another equally-sized array of float2. For example,

float2 PixelKernel[KernelSize] = { { -1, 0 },{ 0, 0 }, { +1, 0 } };
float2 TexelKernel[KernelSize]
  string ConvertPixelsToTexels = "PixelKernel";

This second array has a string annotation, named ConvertPixelsToTexels, which contains the name of the first kernel array. When the sample loads the effect file and sees this annotation, it will translate the offsets from pixel coordinates to texel coordinates, which are compatible with the texture sampling functions such as tex2D. When the source texels are sampled, the offsets in the kernel are added to the texture coordinates to obtain the texel values near the destination pixel.


Some effects use parameters that the user can adjust. For instance, the Bright Pass effect has parameters that the user can tweak to give different visual appearances. Each PostProcess technique can define one or more annotations for this purpose. The maximum number of parameters supported by a single technique is four.

AnnotationName Type Description
Parameter0 string Contains the name of the variable that the user can adjust.
Parameter0Def float4 Default value for Parameter0. This is always a float4. If fewer than four floats are needed, use Parameter0Size to specify it.
Parameter0Size int Specifies how many components of Parameter0 are used. If this is 2, then only the first two components (x and y) are used.
Parameter0Desc string A string that describes this parameter.

Here is an example.

float2 PixelKernel[4] = { {0, 1}, {1, 0}, {0, -1}, {-1, 0} };
float2 TexelKernel[4]
  < string ConvertPixelsToTexels = "PixelKernel"; >;

convert pixel kernel to texel kernel.

float3 LuminanceConv = { 0.2125f, 0.7154f, 0.0721f };

specifies luminance converting constant.

float4 PostProcessPS( float2 Tex[MAXNUM_CHANNEL] : TEXCOORD0 ) : COLOR0
  float4 Orig = tex2D( g_sampler[0], Tex[0] );
  float4 Sum = 0;
  for( int i = 0; i < 4; i++ )
  return saturate( dot( LuminanceConv, Sum ) ) * 5;

defines PostProcessPS function. Get the Sum value around the texel kernel, then computes the dot product.

technique PostProcess
  pass p0
    VertexShader = compile vs_2_0 VertScene();
    PixelShader  = compile ps_2_0 PostProcessPS();
    CullMode = None;
    ZEnable = false;

PostProcess technique specifies vertex and pixel shader. The vertex shader is the fixed function. The pixel shader is the compiled PostProcessPS function with PS2.0. The z enable is set to false.