Environment Mapping with Cg and OpenGL

Environment Mapping with Cg

Environment Mapping

In this article I will demonstrate an effect called Environment Mapping. Environment mapping attempts to simulate the effect of reflective or refractive surfaces in a shader rasterizer. I assume the reader has a basic understanding of OpenGL and Cg. If you require an introduction in OpenGL, you can refer to my article titled [Introduction to OpenGL for Game Programmers]. And for an introduction to Cg, you can refer to my article titled [Introduction to Cg Runtime with OpenGL].

Introduction

If you have ever tried to create a ray tracer or a path tracer, then you should be familiar with the concept of reflections and refractions. Rendering methods such as ray tracing and path tracing can simulate these effects naturally however GPU rasterization cannot. It is the job of the shader programmer to come up with a method that can somehow simulate the same effect that can be achieved using a global illumination rendering method such as ray tracing and path tracing. The image below shows an example render using ray-tracing.

Arauna - Realtime Ray Tracing

Arauna - Realtime Ray Tracing

The reflection and refraction technique displayed in the image above is an example of the effect that can be achieved from global illumination rendering algorithms. Our goal is to reproduce this effect as closely as possible in real-time.

Using a global illumination rendering algorithm (ray tracing or path tracing), you can achieve effects such as self-reflection and self-refraction. Self-reflection is when parts of the same object are reflected onto itself (for example, the handle of a teapot is reflected in the body of the teapot), and self-refraction is when parts of the object that appear behind the object can be accurately refracted. In GPU shaders, these effects are harder to reproduce.

For an article that describes how to do multiple reflection and refraction in GPU shaders, I recommend you read chapter 17 of the “GPU Gems 3” book (“Robust Multiple Specular Reflections and Refractions” by Tamas Umenhoffer, Gustavo Patow, and Lazlo Szirmay-Kalos).

In this article, I will demonstrate a simple method to simulate environment reflection and refraction techniques using GPU shaders.

Dependencies

The demo shown in this article uses several 3rd party libraries to simplify the development process.

  • The Cg Toolkit (Version 3): The Cg Toolkit provides the tools and API needed to integrate the Cg shader programs in your application.
  • Boost (1.46.1): Boost has some very useful libraries that I use throughout my demo applications. In this demo, I use the Signals, Filesystem, Function, and Bind boost libraries to provide a generic, platform independent functionality that simplifies some of the features used in this demo.
  • Simple DirectMedia Layer (1.2.14): Simple DirectMedia Layer (SDL) is a cross-platform multimedia library that I use to create the main application window, initialize OpenGL, and handle keyboard, mouse and joystick input.
  • OpenGL Mathmatics (GLM): An OpenGL centric mathmatics library for 3D graphics applications.
  • Simple OpenGL Image Library (SOIL): SOIL is a tiny C library used primarily for uploading textures into OpenGL.

All of the dependencies described here are included in the source code example included at the end of this article.

The EnvironmentMapping Demo Application

I will first show how we can setup the application to use the shaders that will be shown later. I am using an effect framework which I created to simplify loading shaders, accessing and update shader parameters, and iterating through the passes of a technique. For a detailed explanation of this framework and the underlying Cg code, you can refer to my article titled [Introduction to Cg Runtime with OpenGL]

Globals and Headers

Let’s start by including the headers and defining the global variables that are used for this demo.

#include "EnvironmentMappingPCH.h"
#include "Application.h"
#include "PivotCamera.h"
#include "ElapsedTime.h"
#include "EffectManager.h"
#include "Material.h"
#include "Effect.h"
#include "EffectParameter.h"
#include "Technique.h"
#include "Pass.h"

#include "Events.h"

I’m using the effect framework library to create the initial OpenGL application window as well as to load shader effects and access shader parameters. These first headers (with the exception of the precompiled header) are all from the effect framework.

Then we will define a few global parameters that are used throughout the demo.

Application g_App( "Environment Mapping Demo", 512, 512 );
PivotCamera g_Camera;

glm::vec3   g_InitialCameraRotation( 0, 0, 0 );
glm::vec3   g_InitialCameraPiviot( 0, 0, 0 );
glm::vec3   g_InitialCameraPosition( 0, 0, -10 );

GLuint  g_TorusDisplayList = 0;

GLuint  g_EnvCubeMap = 0;
GLuint  g_BrushedMetalTexture = 0;
GLuint  g_GlassTexture = 0;

Material g_ReflectiveMaterial;
Material g_RefractiveMaterial;
Material g_ReflectRefractMaterial;

bool g_bAnimate = true;
float g_fRotatePrimitive = 0.0f;

glm::ivec2 g_CurrentMousePos(0);
bool g_bLeftMouseDown = false;
bool g_bRightMouseDown = false;

The g_App parameter is used to startup and run the application. To handle the application logic and rendering, callback functions will be registered with the application that will be invoked in the applications update loop.

The g_Camera parameter implements a simple arc-ball camera class that also supports zooming and panning of the camera. Following the camera parameter are several parameters that define an initial view that will be applied to the camera when the application starts.

Initially I was using glutSolidTorus to render a torus that can be used to demonstrate the effects used in this demo, but glutSolidTorus doesn’t generate texture coordinates. I needed to generate texture coordinates for the geometry so the alternative was to generate a procedural torus with correct texture coordinates and normals. The g_TorusDisplayList is used to store the ID of a display list that can be used to render the same torus multiple times without having to generate the vertex position, texture coordinate, and surface normal for every render frame.

We also need to define a few texture object ID’s for the textures that will be used for this demo. The g_EnvCubeMap parameter stores the ID of the 6-sided cube map texture. The g_BrushedMetalTexture parameter defines an ID for a brushed-metal 2D texture that will be applied to the reflective material and the g_GlassTexture parameter defines a 2D texture ID for a glass texture that will be applied to the refractive material.

The g_bAnimate and g_fRotatePrimitive parameters store some animation data that is used to rotate the scene objects. The animation can be toggled by pressing the [space] bar.

And the final three parameters define information about the mouse position and the state of the mouse buttons. These are used to pan and rotate the view of the camera.

Forward Declarations

The functions that will be used as callbacks for the application class must be forward declared before they can be used.

void OnKeyPressed( KeyEventArgs& e );
void OnMouseButtonPressed( MouseButtonEventArgs& e );
void OnMouseButtonReleased( MouseButtonEventArgs& e );
void OnMouseMoved( MouseMotionEventArgs& e );
void OnResized( ResizeEventArgs& e );
void OnUpdate( UpdateEventArgs& e );
void OnRender( RenderEventArgs& e );
void OnPreRender( RenderEventArgs& e );

void OnInitialize( EventArgs& e );
void OnEffectLoaded( EffectLoadedEventArgs& e );
void OnRuntimeError( RuntimeErrorEventArgs& e );
void OnTerminate( EventArgs& e );

void InitGL();
void LoadResources();

void DrawCubeMap( GLuint texID  );

The first group of methods will be registered as callbacks for the application to invoke when certain events occur.

In InitGL static OpenGL states will be configured.

The LoadResources method is used to load any textures and models that are used by the demo.

The DrawCubeMap method will draw the skybox using the cube map texture that is passed as an argument.

The Main Method

The entry point for our application is the main method. We will use it to register the callback functions that will be used for the application and startup the applications update loop.

int main( int argc, char* argv[] )
{
    g_Camera.SetRotate( g_InitialCameraRotation );
    g_Camera.SetPivot( g_InitialCameraPiviot );
    g_Camera.SetTranslate( g_InitialCameraPosition );

    // Register event callbacks
    g_App.KeyPressed += OnKeyPressed;
    g_App.MouseButtonPressed += OnMouseButtonPressed;
    g_App.MouseButtonReleased += OnMouseButtonReleased;
    g_App.MouseMoved += OnMouseMoved;
    g_App.Resize += OnResized;

    g_App.PreRender += OnPreRender;
    g_App.Render += OnRender;

    g_App.Update += OnUpdate;
    g_App.Initialized += OnInitialize;
    g_App.Terminated += OnTerminate;

    return g_App.Run();
}

The camera’s initial position an orientation are initialized and the callback functions are registered with the application class.

The applications processing loop is initialized by calling the Application::Run method.

The OnInitialize Method

The first thing that happens after the application has initialized is the Application::Initialized event is invoked causing the OnInitialize method to be invoked.

void OnInitialize( EventArgs& e )
{
    // Create an effect manager
    EffectManager::Create().Initialize();
    EffectManager& effectMgr = EffectManager::Get();

    g_TorusDisplayList = CreateTorusDisplayList( 0.75, 2.0, 64, 64 );

    LoadResources();

    effectMgr.EffectLoaded += OnEffectLoaded;
    effectMgr.RuntimeError += OnRuntimeError;

    // Load the effects
    effectMgr.CreateEffectFromFile( "Resources/Shaders/C7E1_reflection.cgfx", "C7E1_reflection" );
    effectMgr.CreateEffectFromFile( "Resources/Shaders/C7E3_refraction.cgfx", "C7E3_refraction" );
    effectMgr.CreateEffectFromFile( "Resources/Shaders/C7E3_refract_reflect.cgfx", "C7E3_refract_reflect" );

    // Setup refective material
    g_ReflectiveMaterial.Reflection = 0.5f;

    // Setup refractive material
    g_RefractiveMaterial.Refraction = 0.95f;
    g_RefractiveMaterial.Transmittance = 0.8f;

    // Setup reflective/refractive material
    g_ReflectRefractMaterial.Reflection = 1.0f;
    g_ReflectRefractMaterial.Refraction = 0.95f;
    g_ReflectRefractMaterial.Transmittance = 0.5f;

    InitGL();
}

The first thing we do in this function is create and initialize the EffectManager singleton instance and get a reference to that instance.

I decided to use a torus to demonstrate this effect so the CreateTorusDisplayList method will create an OpenGL display list that can be used to quickly render the torus geometry in the scene.

The LoadResources is responsible for loading the texture and model resources used by the demo. It will load the cube map texture and the 2D texture resources.

On line 192 we register a callback method with the effect manager which will be invoked when an effect is loaded. We will use this event to set the static parameters of the effect after they are loaded (or reloaded).

On lines 196-198 the effect files are loaded by the effect manager. The first effect loaded on line 196 is the reflection effect, the refraction shader effect is loaded on line 197, and a shader that combines both the reflection and refraction effect is loaded on line 198.

For this demo, I’ve added a few new parameters to the Material type to define the reflection, refraction, and diffusion parameters that are used to simulate the reflection and refraction phenomenon. I will discuss how these parameters are used in the section where the shader programs are discussed.

On line 212, the InitGL method will initialize a few OpenGL states and parameters.

Let’s first take a look at how the torus display list is generated.

The CreateTorusDisplayList Method

The CreateTorusDisplayList method will generate a display list that can be used to render a torus without having to compute the vertex positions, texture coordinates and normals each frame.

If you would like to know more about the math behind the torus, you can refer to the [Torus] Wikipedia page (http://en.wikipedia.org/wiki/Torus).

GLuint CreateTorusDisplayList( GLdouble innerRadius, GLdouble outerRadius, GLint sides, GLint rings )
{
    static const float _2_PI = 6.283185307179586476925286766559f;

    float stepU = _2_PI / sides;
    float stepV = _2_PI / rings;

    float u, v;

    GLuint displayList = glGenLists(1);

    glNewList(displayList, GL_COMPILE );
    {
        for ( u = 0; u < _2_PI; u += stepU )
        {
            glBegin( GL_QUAD_STRIP );

            for ( v = 0; v < _2_PI; v += stepV )
            {
                TorusVertx( u, v, innerRadius, outerRadius );
                TorusVertx( u+stepU, v, innerRadius, outerRadius );
            }

            TorusVertx( u, _2_PI, innerRadius, outerRadius );
            TorusVertx( u + stepU, _2_PI, innerRadius, outerRadius );

            glEnd();
        }
    }
    glEndList();

    return displayList;
}

Fist a new display list is created using the glGenLists method. The list definition is started with the glNewList method and the definition is finalized with the glEndList method.

This function will simply loop through each side of the torus and generate a “ring” that loops a full circle () around the torus body.

Torus Cycles

Torus Cycles

As shown in the image, the “sides” of the torus wrap around the outside of the torus (shown by the magenta ring on the torus body. The “rings” wrap around the inside of the torus body (show by the red ring on the torus loop).

For each step on the torus, a vertex of the torus is plotted using the TorusVertx inline function.

// Plot a single vertex of the torus
inline void TorusVertx( float u, float v, float innerRadius, float outerRadius )
{
    static const float sTexCoord[3] = { 0.5, 0, 0 };
    static const float tTexCoord[3] = { 0, 0.5, 0 };

    float x, y, z, s, t;
    float cu, su, cv, sv;

    cu = cosf( u );
    su = sinf( u );
    cv = cosf( v );
    sv = sinf( v );

    x = ( outerRadius + innerRadius * cv ) * cu;
    y = ( outerRadius + innerRadius * cv ) * su;
    z = innerRadius * sv;

    // Object planar texture mapping
    s = ( x * sTexCoord[0] ) + ( y * sTexCoord[1] ) + ( z * sTexCoord[2] );
    t = ( x * tTexCoord[0] ) + ( y * tTexCoord[1] ) + ( z * tTexCoord[2] );

    // Spherical texture mapping
    //float length = sqrtf( x * x + y * y + z * z );
    //s = ( x / length ) * sTexCoord[0] + ( y / length ) * sTexCoord[1] + ( z / length ) * sTexCoord[2];
    //t = ( x / length ) * tTexCoord[0] + ( y / length ) * tTexCoord[1] + ( z / length ) * tTexCoord[2];

    glTexCoord2f( s, t );
    glNormal3f( cu * cv, su * cv, sv );
    glVertex3f( x, y, z );
}

This function is based on the parametric equation of a torus:

Where,

  • are in the interval .
  • is the outer radius of the torus (indicated by the magenta line in the torus image above).
  • is the inner radius of the torus (indicated by the red line in the torus image above).

The texture coordinate is determined from a object-planar texture coordinate generation where the x, y, and z components of the position are directly mapped to the texture coordinate. The sTexCoord and tTexCoord static constant arrays determine the axis and scaling of the texture coordinates.

To enable spherical texture coordinate mapping, un-comment lines 142-144.

The LoadResources Method

The LoadResources method is used to load any textures or geometry that is used by the application. Texture loading is a large topic in itself, but I’m using the “Simple OpenGL Image Library” (SOIL) to simplify the image loading. SOIL can be used to load standard texture formats as well as cube maps.

void LoadResources()
{
    // This texture was downloaded from http://www.hazelwhorley.com/textures.html
    // These textures are protected by the Creative Commons license for use with non-commercial applications.
    // Special thanks to Hazel Whorley for creating these great textures.
    g_EnvCubeMap = SOIL_load_OGL_cubemap( "Resources/Textures/Mountain/mountain_west.bmp",
                                          "Resources/Textures/Mountain/mountain_east.bmp", 
                                          "Resources/Textures/Mountain/mountain_up.bmp",
                                          "Resources/Textures/Mountain/mountain_down.bmp",
                                          "Resources/Textures/Mountain/mountain_south.bmp",
                                          "Resources/Textures/Mountain/mountain_north.bmp",
                                          SOIL_LOAD_AUTO,
                                          SOIL_CREATE_NEW_ID,
                                          SOIL_FLAG_MIPMAPS );

    // To prevent artifacts at the edges, use clamping at texture bounds.
    glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

    // Make sure we unbind and disable the cube map texture.
    glBindTexture( GL_TEXTURE_CUBE_MAP, 0 );
    glDisable( GL_TEXTURE_CUBE_MAP );

    // Load a brushed metal texture for the reflection material
    g_BrushedMetalTexture = SOIL_load_OGL_texture( "Resources/Textures/brushed-metal.jpg", SOIL_LOAD_AUTO, SOIL_CREATE_NEW_ID, SOIL_FLAG_MIPMAPS );

    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT );
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT );

    // Load a glass texture for the refractive material
    g_GlassTexture = SOIL_load_OGL_texture( "Resources/Textures/glass-texture-2.jpg", SOIL_LOAD_AUTO, SOIL_CREATE_NEW_ID, SOIL_FLAG_MIPMAPS );

    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT );
    glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT );

    glBindTexture( GL_TEXTURE_2D, 0 );
    glDisable( GL_TEXTURE_2D );
}

Our cube map texture that will contain the environment that will be reflected and refracted in our shader is stored in the g_EnvCubeMap parameter. The cube map consists of six sides. The sides are named with the assumption that the cube map is rendered at the center of the viewer and the sides are relative to the axis of the world with no rotation.

  • The North side is the side that is in the positive Z axis.
  • The South side is the side that is in the negative Z axis.
  • The East side is the side that is in the positive X axis.
  • The West side is the side that is in the negative X axis.
  • The Up side is the side that is in the positive Y axis.
  • The Down side is the size that is in the negative Y axis.
Depending on the handedness of the coordinate system the definition of positive Z and negative Z may be reversed.

On line 96, and 97 the cube map’s texture wrapping mode is set to GL_CLAMP_TO_EDGE. This reduces the appearance of seams at the edge of the cube map.

Two 2D textures are also loaded. These texture are applied to the torus’s and we want the textures to repeat if the texture coordinate gets out of the range 0 to 1.

When we load a texture using SOIL, it keeps the texture bound to the first texture stage. So before we leave the function, we have to unbind the texture so that we don’t accidentally texture something we didn’t mean to texture.

The InitGL Method

We’ll use the InitGL method to initialize the OpenGL states that are used for this demo.

void InitGL()
{
    glClearDepth( 1.0f );
    glEnable( GL_DEPTH_TEST );
}

Since we will be drawing the cube map texture over the entire screen, we don’t need to define a clear color. And since we won’t be using any lights (because the fragment shader will be calculating all the colors we will see) we don’t need to initialize any lights or materials or anything of that sort.

We only need to set the value the depth buffer is cleared to and enable depth testing in the rendering pipeline.

The OnEffectLoaded Method

The OnEffectLoaded method is the event callback that gets invoked when an effect has been loaded. All of the effects used in this demo define a cube map sampler called “envSampler”. We’ll query the effect parameter and set the parameter to the value of the cube map texture that was loaded in the LoadResources method.

void OnEffectLoaded( EffectLoadedEventArgs& e )
{
    // Set the properties for the effects
    Effect& effect = e.Effect;

    // All of the effects in this demo have a samplerCUBE parameter called "envSampler".
    // Query that and assign the cube map to that sampler.
    EffectParameter& cubeMapParameter = effect.GetParameterByName( "envSampler" );
    cubeMapParameter.Set( g_EnvCubeMap );
}

The effect that generated the event is accessible from the event parameters. The “envSampler” parameter is quired from the effect and assigned the environment cube map texture object ID.

That should be everything we need to do to initialize the demo. Let’s now take a look at the update and render methods.

The OnUpdate Method

The OnUpdate method will be used to simply update the angle of rotation that will be used to rotate the tori.

Every two seconds the EffectManager will be asked to check all of it’s loaded effects to see if the effect file has changed on disc. If so, the effect will be reloaded.

void OnUpdate( UpdateEventArgs& e )
{
    static float fAnimTimer = 0.0f; 
    static float fReloadTimer = 0.0f;
    static float fRotationRate = 45.0f;

    // Every seconds, we'll check to see if the effects need to be reloaded.
    fReloadTimer += e.ElapsedTime;
    if ( fReloadTimer > 2.0f )
    {
        EffectManager::Get().ReloadEffects();
        fReloadTimer = 0.0f;
    }

    if ( g_bAnimate )
    {
        fAnimTimer += e.ElapsedTime;

        g_fRotatePrimitive = fmodf(fAnimTimer * fRotationRate, 360.0f );
    }
}

The fAnimTimer variable is used to keep track of how long the animation has been running thus far and the fRotationRate variable is used to control the speed of rotation of the torus objects.

Every two seconds, the EffectManager will check to see if any of the effects have been updated and disc and if so, the effect will be reloaded.

On line 358, the g_fRotatePrimitive is updated based on the fRotationRate variable which will be used later to rotate the primitives before being rendered.

The OnPreRender Method

The OnPreRender method is used to update any effect variables that can be shared with all other effects. The EffectManager defines a few predefined shared parameters and when an effect is loaded all effect parameters that have a semantic that matches the shared parameter semantic will be automatically connected to that shared parameter.

void OnPreRender( RenderEventArgs& e )
{
    // Update the shared parameters owned by the effect manager
    EffectManager& mgr = EffectManager::Get();

    mgr.SetViewMatrix( g_Camera.GetViewMatrix() );
    mgr.SetProjectionMatrix( g_Camera.GetProjectionMatrix() );

    mgr.SetElapsedTime( e.ElapsedTime );
    mgr.SetApplicationTime( e.TotalTime );
    mgr.SetMousePosition( g_CurrentMousePos );
    mgr.SetMouseButtonState( g_bLeftMouseDown, g_bRightMouseDown );

    g_Camera.ApplyViewTransform();
}

On line 367 and 368 the camera’s view matrix and projection matrix parameters are set. The EffectManager will automatically calculate any matrices that are dependent on the view and projection matrices (including the inverse, transpose, and inverse-transpose versions of those matrices) and if the effect assigns a matrix parameter with a matching semantic, that value will be automatically updated via the shared parameter.

Other shared parameters include the elapsed time since the previous frame, the total time the application has been running, the current position of the mouse, and the current state of the mouse buttons.

The OnRender Method

The OnRender method will render a sky box using the cube map texture that was loaded earlier and it will also render the three tori that demonstrates the reflection, refraction and reflection-refraction effects.

The first thing we’ll do is initialize some parameters and draw the sky box using the cube map we loaded earlier.

void OnRender( RenderEventArgs& e )
{
    EffectManager& mgr = EffectManager::Get();

    glm::mat4 viewMatrix = glm::inverse( g_Camera.GetViewMatrix() );
    glm::vec3 eyePos = glm::vec3( viewMatrix[3] );

    // (Optimization) We only need to clear the depth buffer but because 
    // the cube map will overdraw the entire color buffer.
    glClear( GL_DEPTH_BUFFER_BIT );

    // Draw the unit cube map around the camera.
    DrawCubeMap( g_EnvCubeMap ); 

    DrawAxis( 2.0f, g_Camera.GetPivot() );

    glm::mat4x4 worldMatrix(1.0f);

The eye position in world space is derived from the camera’s view matrix and stored in the eyePos variable.

Since the sky box will be overdrawing the entire screen, there is no benefit to clearing the color buffer so on line 420 we only need to clear the depth buffer.

The DrawCubeMap method will draw a sky box around the viewer. If we disable the lighting and disable writing to the depth buffer, we can draw a unit cube around the origin of the model-view matrix using the cube map as a texture for the cube. The effect is view that appears to be infinitely far away from the viewer.

The DrawAxis method will just draw some lines at the origin of the pivot camera’s view. This is useful to align the origin of rotation of our view.

On line 427, we also define a matrix parameter that will be used to position our objects in the world.

Next we’ll draw the three tori. The first torus is a reflective torus with the brushed metal

    // Draw a reflective torus
    {
        Effect& effect = mgr.GetEffect("C7E1_reflection");
        EffectParameter& eyeParameter = effect.GetParameterByName( "gEyePos" );
        eyeParameter.Set( eyePos );

        EffectParameter& baseTextureParam = effect.GetParameterByName( "baseSampler" );
        baseTextureParam.Set( g_BrushedMetalTexture );

        worldMatrix = glm::translate( glm::vec3( -6.0f, 0.0f, 0.0f ) );
        worldMatrix = glm::rotate( worldMatrix, g_fRotatePrimitive, glm::vec3( 1, 0, 0 ) );

        mgr.SetWorldMatrix( worldMatrix );
        mgr.SetMaterial( g_ReflectiveMaterial );

        mgr.UpdateSharedParameters();
        effect.UpdateParameters();

        Technique& technique = effect.GetFirstValidTechnique();
        foreach( Pass* pass, technique.GetPasses() )
        {
            pass->BeginPass();
            glCallList( g_TorusDisplayList );
            pass->EndPass();
        }
    }

First we get the C7E1_reflection that was loaded earlier and set some of the non-shared parameters like the eye position (which could be shared but there is currently no method for determining which space the eye positions should be expressed in) and the base texture of the object.

On line 438 and 439 the world matrix is built that places the torus 6 units to the left of the origin and rotates it about the X axis.

The world matrix is assigned to the EffectManager which ensures that any shared parameters that rely on the world matrix are updated (like the parameter defined with the WORLDVIEWPROJECTION semantic for example).

The EffectManager also defines a series of shared parameters that define the different components of the material that should be applied to the object. On line 442 the reflective material is assigned to the shared parameters that are managed by the EffectManager class.

On line 444, and 445, the effect parameters and shared parameters are sent to the GPU using the EffectManager::UpdateSharedParameters and Effect::UpdateParameters methods.

To render the geometry using the effect, we first need to query the technique that is associated with the effect and for each pass defined in the effect, we render the geometry.

We can render the torus display list using the glCallList OpenGL method passing as a parameter the ID to the torus display list we created earlier.

The next torus will be rendered using the reflection/refraction effect. There isn’t much difference in this code block so I won’t outline each step but I will just highlight the changed code.

    // Draw a reflective-refractive torus
    {        
        Effect& effect = mgr.GetEffect("C7E3_refract_reflect");
        EffectParameter& eyeParameter = effect.GetParameterByName( "gEyePos" );
        eyeParameter.Set( eyePos );

        EffectParameter& baseTextureParam = effect.GetParameterByName( "baseSampler" );
        baseTextureParam.Set( g_GlassTexture );

        worldMatrix = glm::translate( glm::vec3( 0.0f, 0.0f, 0.0f ) );
        worldMatrix = glm::rotate( worldMatrix, g_fRotatePrimitive, glm::vec3( 0, 1, 0 ) );

        mgr.SetWorldMatrix( worldMatrix );
        mgr.SetMaterial( g_ReflectRefractMaterial );

        mgr.UpdateSharedParameters();
        effect.UpdateParameters();

        Technique& technique = effect.GetFirstValidTechnique();
        foreach( Pass* pass, technique.GetPasses() )
        {
            pass->BeginPass();
            glCallList( g_TorusDisplayList );
            pass->EndPass();
        }
    }

The only difference here is the effect that is used to render the torus and the world transform of the torus object.

And the third torus is rendered using a purly refractive shader.

    // Draw a refractive torus
    {
        Effect& effect = mgr.GetEffect("C7E3_refraction");
        EffectParameter& eyeParameter = effect.GetParameterByName( "gEyePos" );
        eyeParameter.Set( eyePos );

        EffectParameter& baseTextureParam = effect.GetParameterByName( "baseSampler" );
        baseTextureParam.Set( g_GlassTexture );

        worldMatrix = glm::translate( glm::vec3( 6.0f, 0.0f, 0.0f ) );
        worldMatrix = glm::rotate( worldMatrix, -g_fRotatePrimitive, glm::vec3( 1, 0, 0 ) );

        mgr.SetWorldMatrix( worldMatrix );
        mgr.SetMaterial( g_RefractiveMaterial );

        mgr.UpdateSharedParameters();
        effect.UpdateParameters();

        Technique& technique = effect.GetFirstValidTechnique();
        foreach( Pass* pass, technique.GetPasses() )
        {
            pass->BeginPass();
            glCallList( g_TorusDisplayList );
            pass->EndPass();
        }
    }

Again, very little change from the previous object except the effect that is used to render the torus and the world transform of the object.

And finally, to swap the front and back render buffers and present the view we need to inform the application to present the back buffer.

    g_App.Present();
}

Now we’ve seen the application side of rendering our scene. Let’s take a look at the shader effect files.

The Shader Effects

This demo uses three different shaders.

  1. C7E1_reflection.cgfx: A reflection shader.
  2. C7E3_refraction.cgfx: A refraction shader.
  3. C7E3_refract_reflect.cgfx: A combination of refraction and reflection effect.

The Reflection Shader

The reflection shader works by calculating an incident vector () which is the vector from the viewer (the eye position) to the point we are shading. The incident vector is reflected from the surface we are shading about the surface normal () at the point we are shading.

Computing the Reflected Ray

Computing the Reflected Ray

In the image above, the red vector represents the incident vector () which is the vector from the viewer to the point we are shading. The green vector represents the surface normal () at the point we are shading. The yellow vector is the projection of onto and is computed by scaling the surface normal by the dot product of and . The blue vector () is computed by subtracting the yellow vector twice from .

If we make the following definitions:

  • is the world position of the point we are shading.
  • is the world position of the camera (or eye position).

Then the general formula for calculating the reflection vector is:

Fortunately, you don’t have to compute this reflection vector yourself in the shader program because Cg provides the reflect function to you.

  • float3 reflect( I, N ): Returns the reflected vector () from an incoming incident ray () and the surface normal (). The resulting vector has the same length as .

Now let’s take a look at the shader code.

The Vertex Program

The reflection vector is computed per-vertex instead of per-fragment. There is nothing preventing you from moving the reflection calculation to the fragment program but the difference in quality in doing the reflection calculation per-fragment might not be very noticeable if your model is highly tessellated.

First lets define a few global structs and parameters.

struct Material {
    float4 Ke           : EMISSIVE;
    float4 Ka           : AMBIENT;
    float4 Kd           : DIFFUSE;
    float4 Ks           : SPECULAR;
    float shininess     : SPECULARPOWER;
    float reflection    : REFLECTION;
    float refraction    : REFRACTION;
    float transmittance : TRANSMITTANCE;
};

texture baseTexture;
sampler2D baseSampler = sampler_state
{
    Texture = <baseTexture>;
    MinFilter = Linear;
    MagFilter = Linear;
};

texture cubeTexture;
samplerCUBE envSampler = sampler_state
{
    Texture = <cubeTexture>;
    MinFilter = Linear;
    MagFilter = Linear;
};

float4x4 gModelViewProj : WORLDVIEWPROJECTION;
float4x4 gModelToWorld  : WORLD;

float3 gEyePos;
Material gMaterial;

For this demo, I’ve added the reflection, refraction, and transmittance parameters to the material struct. For reflection, only the reflection parameter is used and it is a scalar value in the range 0 to 1 which indicates the intensity of the reflection. A value of 0 means the material is not reflective at all while a value of 1 indicates the material is completely reflective.

The baseSampler parameter is used to apply a diffuse texture to the object and the envSampler parameter is used to store the cube map that is used as the environment sampler that will be applied to the objects. These parameters were set in the application before the objects are rendered.

And we also need to store the matrices that are used to transform our object space position into clip space and world space.

The gEyePos vector defines the position of the viewer in world space.

If you followed the previous articles about lighting, you may have noticed that the light positions, eye positions, and vertex positions were all defined in objects space. For the cube map implementation, the vectors need to be in world space because the environment cube map is also defined in world space. If the reflection or refraction vectors are not in the same space as the environment map, then the wrong color value will be returned when sampling the cube map texture.
// This is C7E1v_reflection from "The Cg Tutorial" (Addison-Wesley, ISBN
// 0321194969) by Randima Fernando and Mark J. Kilgard.  See page 177.
void C7E1v_reflection(float4 position : POSITION,
                      float2 texCoord : TEXCOORD0,
                      float3 normal   : NORMAL,

                  out float4 oPosition  : POSITION,
                  out float2 oTexCoord  : TEXCOORD0,
                  out float3 R          : TEXCOORD1,

              uniform float3   eyePositionW,
              uniform float4x4 modelViewProj, 
              uniform float4x4 modelToWorld)
{
  oPosition = mul(modelViewProj, position);
  oTexCoord = texCoord;

  // Compute position and normal in world space
  float3 positionW = mul(modelToWorld, position).xyz;
  float3 N = mul((float3x3)modelToWorld, normal);
  N = normalize(N);
  
  // Compute the incident and reflected vectors
  float3 I = positionW - eyePositionW;
  R = reflect(I, N);
}

The incoming parameters supplied by the application are the position parameter with the POSITION semantic, the texCoord parameter with the TEXCOORD0 semantic, and the normal with the NORMAL semantic.

The out parameters which are passed to the fragment progarm are the oTexCoord parameter with TEXCOORD0 semantic and the reflection vector R parameter with TEXCOORD1 semantic.

The oPosition parameter with POSITION semantic is clip-space position of the vertex and it must be computed in the vertex program but it is not bound to any input parameter in the fragment program.

On line 48, the vertex position is transformed from object space to clip space by multiplying the vertex position by the world, view, projection matrix. The texture coordinate is imply passed-through to the fragment program.

On lines 52, and 53 the world-space position and surface normal is computed by multiplying by the world matrix of the object. This is a necessary step before the reflection vector can be computed because the reflection vector must be accurately computed in world-space. This is a requirement for the cube map texture sampler that the (direction) vector that is used is expressed in the same space as the map is defined.

On line 57 the incident direction vector is computed and passed to the reflect function to produce the reflection vector.

The Fragment Program

The only responsibility of the fragment program is to sample the base texture and the cube map and compute the final color of the fragment based on the amount of reflection defined in the material.

// This is C7E2f_reflection from "The Cg Tutorial" (Addison-Wesley, ISBN
// 0321194969) by Randima Fernando and Mark J. Kilgard.  See page 180.
void C7E2f_reflection(float2 texCoord : TEXCOORD0,
                      float3 R        : TEXCOORD1,

                  out float4 color : COLOR,

              uniform Material material,
              uniform sampler2D decalMap,
              uniform samplerCUBE environmentMap)
{
  // Fetch reflected environment color
  float4 reflectedColor = texCUBE(environmentMap, R);

  // Fetch the decal base color
  float4 decalColor = tex2D(decalMap, texCoord);

  color = lerp(decalColor, reflectedColor, material.reflection);
}

The first two parameters texCoord and R are input parameters passed from the vertex shader.

The only output parameter that the fragment program must compute is the color parameter with the COLOR semantic. This is the final color that will be blended with the current fragment in the framebuffer.

On line 73 the environment map is sampled passing the reflection vector R as the second parameter. This direction vector does not need to be normalized before it is used in a cube sampler.

On line 76, the base texture is sampled using the texture coordinate passed from the vertex program.

The final color is a linear interpolation between the base texture and the value of the cube map in the direction R based on the reflection parameter defined in the Material struct. If the reflection parameter is 0, then only the decalColor will be visible and if the reflection parameter is 1, then only the reflectedColor will be visible.

Techniques and Passes

Each effect only defines a single technique with a single pass.

technique main
{
    pass p0
    {
        VertexProgram = compile latest C7E1v_reflection( gEyePos, gModelViewProj, gModelToWorld );
        FragmentProgram = compile latest C7E2f_reflection( gMaterial, baseSampler, envSampler );
    }
}

We define the VertexProgram and the FragmentProgarm by telling Cg to compile the two entry points for each program using the latest platform supported.

Now let’s take a look at refraction.

The Refraction Shader

Refraction occurs when light enters a medium that has a different density than the medium the light originated from. For example, when light passes through glass, the light will appear to “bend” at the boundary of the two mediums (air, and glass).

Refraction is defined by Snell’s law which states that the ratio of the sines of the angles of incidence and refraction is equivalent to the ratio of phase velocities in the two media, or equivalent to the opposite ratio of the indices of refraction.

If we define the following:

  • and are the wave velocities in the separate media.
  • and are the refractive indices of the two medium.

Then,

Fortunately, we don’t have to worry too much about how the refraction is calculated because Cg provides the refract function to calculate the refractive vector.

  • float3 refract( float3 i, float3 n, float eta ): Computes the refraction vector from the incident ray i, the surface normal n and the ratio of the indices of refraction between the two medium (eta). The incident vector (i) and the surface normal (n) should be normalized.

Although it is the discretion of the vendor regarding how this method is implemented, but it is possible that this method is implemented in this way:

float3 refract( float3 i, float3 n, float eta )
{
  float cosi = dot(-i, n);
  float cost2 = 1.0f - eta * eta * (1.0f - cosi*cosi);
  float3 t = eta*i + ((eta*cosi - sqrt(abs(cost2))) * n);
  return t * (float3)(cost2 > 0);
}

The Vertex Program

The only difference between the reflection shader and the refraction shader is the computation of the direction vector that is passed to the fragment program.

// This is C7E3v_refraction from "The Cg Tutorial" (Addison-Wesley, ISBN
// 0321194969) by Randima Fernando and Mark J. Kilgard.  See page 187.
void C7E3v_refraction(float4 position : POSITION,
                      float2 texCoord : TEXCOORD0,
                      float3 normal   : NORMAL,
   
                  out float4 oPosition  : POSITION,
                  out float2 oTexCoord  : TEXCOORD0,
                  out float3 T          : TEXCOORD1,

              uniform Material material,
              uniform float3 eyePositionW,
              uniform float4x4 modelViewProj, 
              uniform float4x4 modelToWorld)
{
  oPosition = mul(modelViewProj, position);
  oTexCoord = texCoord;

  // Compute position and normal in world space
  float3 positionW = mul(modelToWorld, position).xyz;
  float3 N = mul((float3x3)modelToWorld, normal);
  N = normalize(N);
  
  // Compute the incident and refracted vectors
  float3 I = normalize(positionW - eyePositionW);
  T = refract(I, N, material.refraction);
}

On lines 58 and 59 the refract vector is computed from the normalized incident vector and the surface normal and the ratio of indices of refraction.

A table of refractive indices can be found at http://en.wikipedia.org/wiki/List_of_indices_of_refraction.

Air at sea-level has an index of refraction of 1.000277 and water at room temperature has an index of refraction of 1.333. So the if the incident vector (I) is going from air to water, the ratio of the indicies would be:

Different types of glass have different index of refraction values but generally a good index of refraction for glass is 1.5. So the ratio between air and glass would be:

And this is the value that is used for the refraction parameter.

The Fragment Program

The fragment program for the refractive texture is also almost identical to that of the reflection program. The only difference is that instead of using the material’s reflect parameter, the transmittance value determines how much of the light is transmitted through the object. You may have noticed that the amount of light that passes through a medium changes with the thickness of the material. We are not concerned with dispersion and attenuation factors in this simple shader program so this is something that you can investigate for yourself.

// This is C7E4f_refraction from "The Cg Tutorial" (Addison-Wesley, ISBN
// 0321194969) by Randima Fernando and Mark J. Kilgard.  See page 188.
void C7E4f_refraction(float2 texCoord : TEXCOORD0,
                      float3 T        : TEXCOORD1,

                  out float4 color : COLOR,

              uniform Material    material,
              uniform sampler2D   decalMap,
              uniform samplerCUBE environmentMap)
{
  // Fetch the decal base color
  float4 decalColor = tex2D(decalMap, texCoord);

  // Fetch refracted environment color
  float4 refractedColor = texCUBE(environmentMap, T);

  // Compute the final color
  color = lerp(decalColor, refractedColor, material.transmittance);
}

If the transmittance value is 0, then no light is transmitted through the material and only the base texture is visible. If the transmittance value is 1, then the material is completely transparent and only refracted light is used to color the fragment.

There is one more shader that does both the reflection and refraction effects then blends the final color based on the values of reflection and transmittance parameters but I will leave it up to the reader to implement this.

If everything works well, then the resulting application should look something like this:

References

The Cg Tutorial

The Cg Tutorial

The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics (2003). Randima Fernando and Mark J. Kilgard. Addison Wesley.

The cube map images used for this demo were downloaded from http://www.hazelwhorley.com/textures.html. Special thanks to Hazel Whorley for creating these great cube maps.

Download the Source

The source code example for this article can be downloaded from the link below.

zip1 EnvironmentMapping.zip

One thought on “Environment Mapping with Cg and OpenGL

Leave a Reply

Your email address will not be published. Required fields are marked *