Finalizing the HBAO

This week I finished to fully implement the HBAO effect, following we have  the effect xml file:

<posteffect>
    <layer name="hbao" shader="/shader/postproc/HBAO/HBAO.xml" downsample="0">
        <parameter name="angle bias" type="float">0.52359877</parameter>
        <parameter name="sqr radius" type="float">0.01</parameter>
        <parameter name="inv radius" type="float">10</parameter>
        <parameter name="radius" type="float">0.1</parameter>
        <parameter name="num steps" type="float">25</parameter>
        <parameter name="num directions" type="float">32</parameter>
        <parameter name="contrast" type="float">2.3</parameter>
        <parameter name="attenuation" type="float">1.0</parameter>

        <input source="/data/posteffects/CosSinJitter.bmp" texname="tex csj"/>
        <output name="out" format="argb8" />
    </layer>

    <layer name="blurX" shader="/shader/postproc/HBAO/BilateralBlurX.xml">
        <parameter name="blur radius" type="float">7</parameter>
        <parameter name="blur falloff" type="float">0.010204081632</parameter> <!-- 1 / (2*radius*radius)-->
        <parameter name="sharpness" type="float">100000</parameter>

        <input layer="hbao.out" texname="tex diffuse"/>
        <output name="out" format="argb8" />
    </layer>

    <layer name="blurY" shader="/shader/postproc/HBAO/BilateralBlurY.xml">
        <parameter name="blur radius" type="float">7</parameter>
        <parameter name="blur falloff" type="float">0.010204081632</parameter> <!-- 1 / (2*radius*radius)-->
        <parameter name="sharpness" type="float">100000</parameter>

        <input layer="blurX.out" texname="tex diffuse"/>
        <output name="out" format="argb8" />
    </layer>

    <layer name="combine" shader="/shader/postproc/HBAO/combine.xml">
        <input layer="*screen" texname="tex diffuse"/>
        <input layer="blurY.out" texname="tex ao"/>
    </layer>
</posteffect>

So, it contains 4 layer:

• The HBAO - is the main layer, as the original effect uses a buffer(dx10) with precomputed random (cosine,sine,jitter) I baked it into a texture. In the parameters we have, the radius (in which the texture will be sampled), the number of directions and how much steps in each direction will be calculated. 
• Blur X and Y - is the bilateral blur, this is, a gaussian blur weigthed by the depth diferrence between the samples, the sharpnes parameter controls how much the difference affect the weight 
• Combine - this layers just combines the AO with the color, it can be done directly in the BlurY pass, but I keep it for now, just to make debug/test easy.

Now I will work on properly hook the depth buffer so that the effects can use it, I think there is some bug related to fbo's on intel gpus, because when I try to run the deferred rm on it the rm binds a D24S8 buffer and then the ShadowPSSM tries to bind a D32 buffer causing a invalid attachment error, but if I change the first to D32 it works fine. Maybe this bug was causing the invalid attachment when I tried to hook the depth buffer with a D24S8 format.

Other strange thing is that this same intel gpu supports dx10, but in the HBAO if I use "for" loops it says that I can't use "for", and also if I remove the "for"(unroll) it says that I exceded the max temporary registers! But the NVIDIA demo with almost the same shader runs fine.

Implementing HBAO

After improve the error and warnings notification I started to implement the HBAO, based on the NVIDIA HBAO  demo. I think that this ambient occlusion achieves better visual quality than the standard SSAO and also it can be implemented without having to store  the normals, thus being usable not only by the deferred rm.
Here is a small video showing the effect:

As there are various effects that uses the depth buffer, now I have to find a elegant way to hook the depth buffer and provide it to the effects without affecting the rm drawing. I also thought in provide some kind of depth buffer usage capabilities to post-effects.

Post-Effect XML Parser

This week I did some enhancements in  PostEffectLayersParser. An effect can be described by the following structure:

<posteffect>
    <layer name="name" shader="/shader.xml" downsample="0" mipmap="false" maxmipmap="-1">
        <input layer="" texname="tex diffuse" texcoord="texture coordinate 0" pixelsize="pixel size" />
         (...)
        <input source="/tex.png" texname="a texture" />


        <output name="out1" format="argb16_f" reusable="true">
        (...)
        <output ... />
        <parameter name="param_name" type="vector2">1,1<parameter />
        (...)
        <parameter (...)>(...)<parameter />
    </layer>
    <layer>
        (...)
    </layer>
</posteffect>

Based on that structure we have:
An effect can have multiple layers and each layer must specify at least its name and the shader used. Optionally can be specified its downsample, mipmap and maxmipmap.
Each layer can have one or more inputs and one or more outputs.
In the input attributes we have:

• layer - specifies which layer output will be used as input. If omitted will be used the first output of the previous layer (in case it's the first layer then will be used the input texture). The input format is <layername>.<outputname>, if <outputname> is omitted then is used the first output of <layername>.
• texname - shader variable name used to access this texture, if omitted the default value is "tex diffuse"
• texcoord - shader variable name used to access the texture coordinate, if omitted the default value is "texture coordinate 0"
• pixelsize - shader variable name used to access the pixel size for this texture, if omitted no shader variable is provided to the shader.
• source - specifies the path to the texture used as input.

Notice that layer and source are mutual exclusive, and if both attributes are provided layer takes precedence over source. If no input tag is provided then is used one with all default values.

In the output tag can be specified:

• name - the output name, used to access this output as input for other layers, if omitted this output will not be accessible unless this is the first output.
• format -the output texture format, if omitted the default value is "argb8".
• reusable - if false prevents the reuse of this output , if omitted the default value is "true".
• mipmap - enables mipmap generation, overrides the value specified in layer tag, optional.
• maxmipmap - max mipmap level to generate, overrides the value specified in layer tag, optional.

If output tag is omitted, the output used will use all default values.

The parameter tag is used to provide default shader variable values for the post effect.

Refactoring the PostEffect code

After porting the old postprocessing code I've started to refactor the code.
Starting by the PostEffectSupport class, first I removed some functions that no longer make sence. As PostEffectSupport contains an array of post effects and not Layers, the functions ClearLayers, AddLayers* and ClearIntermediates are removed. I've added the function SetPostEffectEnable so that we can enable/disable postprocessing at any time.

In the iPostEffect interface I've removed the function ClearIntemediates because all texture management was move to PostEffectManager. I also added the function LoadFromFile and Construct, which respectively, loads the effect from an external xml file and setup the effect (which include allocate textures, link layers and sets up shaders variables).

In this process I created some helper classes: DependencySolver and PrioriryShadeVariableContext.
The first given a graph of layers with edges linking a layer output to another layer input, this class calculates the minimum number of textures needed to draw the effect and assigns each texture to its respective layer.
The second stacks up shaders variable contexts with each one having a priority so when pushing the variables to the svStack the highest priority dominates over the lower priorities. Then we can have a svContext for default variables and one(or multiples) to user defined variables in a clean way.

For the PostEffectManager I added 3 new functions:

• RequestTexture
• SetupView
• RemoveUnusedTextures

Which respectively:

• Asks the manager to create or reuse a texture previously allocated based on the allocation info and the given number(identifier).
• Sets up the view, if resolution has changed it releases all textures so that PostEffectSupport forces the effects to reacquire the textures
• We keep an array of allocated textures (csRef), then when the reference of the texture reaches 1, it's no longer used in any post effect and we can safely remove it, yet this function isn't implemented because I'm having some trouble with the CS iterators, I think using std would be much more easy.

And finally, the HDR code is BROKEN!
I noticed that the hdr already no longer worked, but now given all these changes, the code is really broken and I will have to devote some time to fix it. It's good and bad because it consumes time but I think in the end the hdr code will be more simple more versatile and, of course, will work properly.

Crystal Space Postprocessing Requirements & Design

The postprocessing requirementes are:

• posteffect can have multiple passes (layers)
• each pass can uses multiples inputs
• the pass inputs can be others layers outputs, or custom textures
• each pass can have multiple outputs (render targets)
• output textures can have custom format, downsample and mipmap values
• # of output textures should be minimal
• output textures can be reused if not strictly specified the opposite
• a posteffect can have default shaders parameters
• it should be data-driven, that is, fully set up using the posteffects xml files

Implementation

As some part of these requirements has already been inherited from the previous code, then I'm doing small changes in the codebase to fit the rest of the requirements.

Texture usage optimization
I started to work in optimizing the texture usage, sharing common textures between layers. The old method (using ping-pong textures) would not work properly, imagine the case where we have 3 layers:

Layer 2 uses the output from layer 1 and layer 3 uses both outpus (1 and 2).

In such a case we clearly realize that we can't reuse the output of layer 1 until layer 3 finishes its work, the same occurs for layer 2.

From this example we conclude that:
- All inputs of layer i can't be reused until layer i is drawn
- All outputs of layer i can't be shared between themselves (obvious)

For that problem I wrote a simple algorithm that resolve the needed texture usage (pseudo code):

ResolveDependency( layers )
avail_tex : list;
used_tex : list;

for each output of last layer
    used_tex.put(output);

for i = last layer to  first layer
    //remove textures where assigned layer number > i
    // and put it on avail_tex
    updateLists(i);
    for each input of layer i
        //find if there is any texture  that match the
        //properties from the layer output referenced by input
        tex = avail_tex.find(format,mipmap,downsample);

        if tex = not_found then
            tex = CreateNewEntry(format,mipmap,downsample);

        asign tex to layer i
        used_tex.put(tex);
end

Of course it's a shallow overview of the implementation details.

I also did some minor changes in some structs related to the layer options. I will list the structs first then I explain them.

struct TextureAllocationInfo
{
    bool mipmap;
    int maxMipmap;
    int downsample;
    bool reusable;
    csString format;
    TextureAllocationInfo();
    bool operator==(const TextureAllocationInfo& other);
};

As the name sugests this struct defines the properties of the layer's outpus, and will also be used as input param by  a Texture cache that will be implemented to share textures (RT's) between posteffects.

struct PostEffectLayerOptions
{
    TextureAllocationInfo info;
    csRef<iTextureHandle> renderTarget;
    csRect targetRect;
    csString name;
    bool operator==(const PostEffectLayerOptions& other);
};

This struct defines the full output options, its name and a manual render target if desired.

enum LayerInputType { AUTO, STATIC, MANUAL };
struct PostEffectLayerInputMap
{
    LayerInputType type;
    csRef<iTextureHandle> inputTexture;
    csString sourceName;
    csString svTextureName;
    csString svTexcoordName;
    csString svPixelSizeName;
    csRect sourceRect;
    PostEffectLayerInputMap () : type (AUTO),
        svTextureName ("tex diffuse"),
        svTexcoordName ("texture coordinate 0") {}
};

The input map had small but important changes, first it now has a type that defines if it references a layer output, a texture resource or is manually seted up. The sourceName variable depending on the type can be: "layername.outputname" so that we can link layers input/output, or a path like "/data/lookuptex.png" for AUTO and STATIC type respectively. For manual textures the sourceName is unused.

For last but not least:

struct LayerDesc
{
    csArray<PostEffectLayerInputMap> inputs;
    csArray<PostEffectLayerOptions> outputs;
    csRef<iShader> layerShader;
    csString name;

    LayerDesc () {}
    LayerDesc (iShader* shader)
    LayerDesc (iShader* shader, const char * layerName)
    LayerDesc (iShader* shader, PostEffectLayerInputMap &inp)
    LayerDesc (iShader* shader, PostEffectLayerInputMap &inp, 
                   PostEffectLayerOptions &opt)
    LayerDesc (iShader* shader, 
                   csArray<PostEffectLayerInputMap> &inp,
                   PostEffectLayerOptions &opt)
    LayerDesc (iShader* shader, PostEffectLayerOptions &opt);
};

The layer descritor contain all the parameters needed to create and setup a layer, and of course, alot of constructors to easily create a layer.

The next step will be create a texture cache where all posteffects can request textures, link layers inputs to outputs and create a loader for STATIC type inputs.

Porting the postprocessing code

Porting the posteffect code from Crystal Space soc2011 branch to the newest(soc2013/postprocessing), wasn't too hard as I expected.
In the porting process I learned a bit more on how the current postprocessing code work, I found the critical parts to start the work and, of course, I found bugs too.

• hdr is buggy, something related to frame buffers deallocation
• Basic posteffect chaining is working
• Removing/adding the effcts is buggy, I think it's due to the PostEffectsSupport not chaining the rendertargets correctly
• The posteffect code doesn't yet reflect a design focused in the use of multiples effects

There are some minor bugs too, like wrongly offseted texture coordinates and pixelSize shader variable not setup correctly, for these I will commit fixes soon.

I also did a small demo to test posteffect chaining. Here is an image showing three simple posteffects chained

The next step in the work is to start improving the desing of the posteffect code to reflect the needs of the proposed effects.

Generating msvc projects for CrystalSpace

 Generate the msvc projects was tiring , everything that could go wrong went. Then I decided to make an entire tutorial explaining how to get things working.


First of all, download cygwin (if you are using windows) .

Installing ftjam
Then install ftjam, download it here, and copy the binary to  cygwin/bin or cygwin/usr/sbin folder.

Installing perl and Template Toolkit
Before installing it, you need to get gcc-4, if you doesn't already have it run the cygwin setup and in the devel packages select gcc-4 C/C++ package.
If you are using cygwin, also install mingw gcc C/C++ compiler or you will get a error when running ./configure script, because gcc-4 doesn't support a flag used by the configure script.

To install Template Toolkit go to the  cygwin/linux terminal:

# cpan install Template

Or the hardest way, download ActivePerl here, go to the activeperl/bin folder and run cpan.bat, and then in the prompt run the command:

# install Template

Run the configure script

# ./configure

Notice that if your  CS svn repository is configured to use CRLF windows line endings these scripts will fail to run, therefore change the line endings to LF.

Before running jam you will need to run (only once, to create a configuration file):

# ttree

it will asks you to create a configuration file, confirm and now you are ready to run:

# jam msvcgen

If you don't run the ttree before using jam, it will fail because jam uses ttree and ttree will asks to create the configuration file without execute the command passed by jam.