Finalizing the HBAO

This week I finished to fully implement the HBAO effect, following we have  the effect xml file:

<posteffect>
    <layer name="hbao" shader="/shader/postproc/HBAO/HBAO.xml" downsample="0">
        <parameter name="angle bias" type="float">0.52359877</parameter>
        <parameter name="sqr radius" type="float">0.01</parameter>
        <parameter name="inv radius" type="float">10</parameter>
        <parameter name="radius" type="float">0.1</parameter>
        <parameter name="num steps" type="float">25</parameter>
        <parameter name="num directions" type="float">32</parameter>
        <parameter name="contrast" type="float">2.3</parameter>
        <parameter name="attenuation" type="float">1.0</parameter>

        <input source="/data/posteffects/CosSinJitter.bmp" texname="tex csj"/>
        <output name="out" format="argb8" />
    </layer>

    <layer name="blurX" shader="/shader/postproc/HBAO/BilateralBlurX.xml">
        <parameter name="blur radius" type="float">7</parameter>
        <parameter name="blur falloff" type="float">0.010204081632</parameter> <!-- 1 / (2*radius*radius)-->
        <parameter name="sharpness" type="float">100000</parameter>

        <input layer="hbao.out" texname="tex diffuse"/>
        <output name="out" format="argb8" />
    </layer>

    <layer name="blurY" shader="/shader/postproc/HBAO/BilateralBlurY.xml">
        <parameter name="blur radius" type="float">7</parameter>
        <parameter name="blur falloff" type="float">0.010204081632</parameter> <!-- 1 / (2*radius*radius)-->
        <parameter name="sharpness" type="float">100000</parameter>

        <input layer="blurX.out" texname="tex diffuse"/>
        <output name="out" format="argb8" />
    </layer>

    <layer name="combine" shader="/shader/postproc/HBAO/combine.xml">
        <input layer="*screen" texname="tex diffuse"/>
        <input layer="blurY.out" texname="tex ao"/>
    </layer>
</posteffect>

So, it contains 4 layer:

• The HBAO - is the main layer, as the original effect uses a buffer(dx10) with precomputed random (cosine,sine,jitter) I baked it into a texture. In the parameters we have, the radius (in which the texture will be sampled), the number of directions and how much steps in each direction will be calculated. 
• Blur X and Y - is the bilateral blur, this is, a gaussian blur weigthed by the depth diferrence between the samples, the sharpnes parameter controls how much the difference affect the weight 
• Combine - this layers just combines the AO with the color, it can be done directly in the BlurY pass, but I keep it for now, just to make debug/test easy.

Now I will work on properly hook the depth buffer so that the effects can use it, I think there is some bug related to fbo's on intel gpus, because when I try to run the deferred rm on it the rm binds a D24S8 buffer and then the ShadowPSSM tries to bind a D32 buffer causing a invalid attachment error, but if I change the first to D32 it works fine. Maybe this bug was causing the invalid attachment when I tried to hook the depth buffer with a D24S8 format.

Other strange thing is that this same intel gpu supports dx10, but in the HBAO if I use "for" loops it says that I can't use "for", and also if I remove the "for"(unroll) it says that I exceded the max temporary registers! But the NVIDIA demo with almost the same shader runs fine.

Implementing HBAO

After improve the error and warnings notification I started to implement the HBAO, based on the NVIDIA HBAO  demo. I think that this ambient occlusion achieves better visual quality than the standard SSAO and also it can be implemented without having to store  the normals, thus being usable not only by the deferred rm.
Here is a small video showing the effect:

As there are various effects that uses the depth buffer, now I have to find a elegant way to hook the depth buffer and provide it to the effects without affecting the rm drawing. I also thought in provide some kind of depth buffer usage capabilities to post-effects.

Post-Effect XML Parser

This week I did some enhancements in  PostEffectLayersParser. An effect can be described by the following structure:

<posteffect>
    <layer name="name" shader="/shader.xml" downsample="0" mipmap="false" maxmipmap="-1">
        <input layer="" texname="tex diffuse" texcoord="texture coordinate 0" pixelsize="pixel size" />
         (...)
        <input source="/tex.png" texname="a texture" />


        <output name="out1" format="argb16_f" reusable="true">
        (...)
        <output ... />
        <parameter name="param_name" type="vector2">1,1<parameter />
        (...)
        <parameter (...)>(...)<parameter />
    </layer>
    <layer>
        (...)
    </layer>
</posteffect>

Based on that structure we have:
An effect can have multiple layers and each layer must specify at least its name and the shader used. Optionally can be specified its downsample, mipmap and maxmipmap.
Each layer can have one or more inputs and one or more outputs.
In the input attributes we have:

• layer - specifies which layer output will be used as input. If omitted will be used the first output of the previous layer (in case it's the first layer then will be used the input texture). The input format is <layername>.<outputname>, if <outputname> is omitted then is used the first output of <layername>.
• texname - shader variable name used to access this texture, if omitted the default value is "tex diffuse"
• texcoord - shader variable name used to access the texture coordinate, if omitted the default value is "texture coordinate 0"
• pixelsize - shader variable name used to access the pixel size for this texture, if omitted no shader variable is provided to the shader.
• source - specifies the path to the texture used as input.

Notice that layer and source are mutual exclusive, and if both attributes are provided layer takes precedence over source. If no input tag is provided then is used one with all default values.

In the output tag can be specified:

• name - the output name, used to access this output as input for other layers, if omitted this output will not be accessible unless this is the first output.
• format -the output texture format, if omitted the default value is "argb8".
• reusable - if false prevents the reuse of this output , if omitted the default value is "true".
• mipmap - enables mipmap generation, overrides the value specified in layer tag, optional.
• maxmipmap - max mipmap level to generate, overrides the value specified in layer tag, optional.

If output tag is omitted, the output used will use all default values.

The parameter tag is used to provide default shader variable values for the post effect.

Refactoring the PostEffect code

After porting the old postprocessing code I've started to refactor the code.
Starting by the PostEffectSupport class, first I removed some functions that no longer make sence. As PostEffectSupport contains an array of post effects and not Layers, the functions ClearLayers, AddLayers* and ClearIntermediates are removed. I've added the function SetPostEffectEnable so that we can enable/disable postprocessing at any time.

In the iPostEffect interface I've removed the function ClearIntemediates because all texture management was move to PostEffectManager. I also added the function LoadFromFile and Construct, which respectively, loads the effect from an external xml file and setup the effect (which include allocate textures, link layers and sets up shaders variables).

In this process I created some helper classes: DependencySolver and PrioriryShadeVariableContext.
The first given a graph of layers with edges linking a layer output to another layer input, this class calculates the minimum number of textures needed to draw the effect and assigns each texture to its respective layer.
The second stacks up shaders variable contexts with each one having a priority so when pushing the variables to the svStack the highest priority dominates over the lower priorities. Then we can have a svContext for default variables and one(or multiples) to user defined variables in a clean way.

For the PostEffectManager I added 3 new functions:

• RequestTexture
• SetupView
• RemoveUnusedTextures

Which respectively:

• Asks the manager to create or reuse a texture previously allocated based on the allocation info and the given number(identifier).
• Sets up the view, if resolution has changed it releases all textures so that PostEffectSupport forces the effects to reacquire the textures
• We keep an array of allocated textures (csRef), then when the reference of the texture reaches 1, it's no longer used in any post effect and we can safely remove it, yet this function isn't implemented because I'm having some trouble with the CS iterators, I think using std would be much more easy.

And finally, the HDR code is BROKEN!
I noticed that the hdr already no longer worked, but now given all these changes, the code is really broken and I will have to devote some time to fix it. It's good and bad because it consumes time but I think in the end the hdr code will be more simple more versatile and, of course, will work properly.