Lighting models for games III

Last time we had just made the move to per pixel lighting which made a huge visual difference without changing our formulas even the tiniest bit. You might wonder why I didn't start using it and it's a valid question as the per pixel vs. per normal really doesn't matter for the lighting model. The reason I did so was to shoe the huge difference of applying high quality content can do to the look of an object. No matter how much work we do with our lighting models and formulas it doesn't matter unless we have a skilled artist by our side.



Last time we had just made the move to per pixel lighting which made a huge visual difference without changing our formulas even the tiniest bit. You might wonder why I didn't start using it and it's a valid question as the per pixel vs. per normal really doesn't matter for the lighting model. The reason I did so was to shoe the huge difference of applying high quality content can do to the look of an object. No matter how much work we do with our lighting models and formulas it doesn't matter unless we have a skilled artist by our side.

So we start with looking at what we had the last time.

That specular is still looking awfully plastic but we remember that we didn't had a material functions for the specular yet. The one thing we want to control is how shiny the specular gets on different parts of the surface for example pieces of iron in the rock should have a bright specular while dirt shouldn't when in doubted simply look around and trust your eyes. However it is the artist’s eyes that should spot this difference all we need to do is to give them a way to control it. So in the same way as we use a colormap to control the diffuse light and the ambient light we will have a specular map to control the specular light. But this one can be a simple gray scale telling  how much specular should be applied at that point with 1 as its greatest value for full spec and 0 as its lowest for no spec.

So taking our final code from last time


specucularStrength=pow(reflectionDifference,SurfaceConstant)
pixelColor+=lightSource.LightIntensity*specucularStrength;

And then just add our specular map and we end up with this


pixelColor+=lightSource.LightIntensity*pow(reflectionDifference,SurfaceConstant)*specularMap;

And here it is in action

This final step was quite simple and we are left with an impressive looking image this can be considered the base line in lighting using a normal map and a spec map. But looking at it doesn’t the shadow side looks just as ugly as in out first image with ambient light. Well it does because it is the same all formulas we have been working on so far only affects the surfaces facing towards the light to some degree so it's time to get back to our ambient lighting model.

Do you remember that I wrote that ambient light was sometimes occluded because of the access of rays being blocked by other surfaces, If you are unsure go back to the first part and look at the image of the ball without direct lighting. This effect of less rays of ambient light hitting a point because of other surfaces being in the way is called ambient occlusion. And this has the power to create forms in the ambient side because deep crevices etc won’t receive much ambient light because there is just a really small hole for the light to go into.  For how you are going to calculate this ambient occlusion there are a lot of different methods with varying speed and accuracies. The basic idea is that you throw around some rays from the point and see how many are blocked. But if you decide to calculate this in real time or try to precalculate it doesn't matter for this test. Remember that ambient light is supposed to be directionless so therefore the ambient occlusion of an object upon itself can be precalculated.

However when you place objects in an environment the different objects in the environment will start occluding each other and the ambient occlusion will be based upon where in the world the object is standing and therefore can't be precalculated (though you can make some pretty good cheats, but then again games like crysis nowadays calculates the ambient occlusion in real time). This however won't matter for our discussing but our ambient occlusion is as usual stored as a map. And we simply add this to our ambient lighting equation so it looks like this.

pixelColor+=Ambient.LightIntensity*ambientOcclusionMap*colorMap;

Let's take a look at how one of these maps looks.

 
The brighter the map the more ambient light hits it. The darker it is the less light hits it. You can see at the top of the light house how the windows edges and the top blocks of the ambient light leaving dark edges. We can also clearly see how the different bulges and indents in the base shows up clearly. With this we would get a lot more life in the dark side as we would still see shapes and we will even get improvements in the light side as ambient now will behave more correctly and leave darker areas in corners for example.

So it is now time to re-examine one of our assumptions. We have assumed that ambient light is directionless because it represents the light being bounced from all surfaces in all directions. And this is true however the light that comes from the different directions will in reality have different colours because of the different materials it has bounced off. This means that there should be some sort of colour difference between different facings even in the area that is only hit by ambient light. For obvious reasons we can't calculate this for real so we have to select a fake method for it. How you fake it is a lot depending on your type of game. If you are in an outdoor world on a grassy landscape the reflected light from below will be tinted slightly green and the light from above having bounced at clouds etc will have a slightly blue tint. However by now we are cheating so much that I can't say that what we are doing is based in physics. The idea of what we want to simulate is sound and based on physics however the methods are not.  So for simplicity we are going to use an environmental map for selecting the colours of our ambient light. it is nice because the artist have total control and you can blend between different maps as you move from an area to an area but it is in no way the best method but for a single object like ours it gives a god enough approximation. This map should have a pretty similar lightness value over most of it and also only contain subtle colour variations. But in the end it is down to the artists.

The differences towards the version that only used the ambient occlusion is subtle but also clear it is obviously darker but you also detect subtle colour differences depending on facing that gives us something that starts to look as a decent lighting model already on its own without all the other stuff.

So we once again has a new formula for our ambient light

pixelColor+=AmbientEnviromentalMap*ambientOcclusionMap*colorMap;

 You sample from the ambient environmental map by using the world space normals of the object. This should be done with the per pixel normals as a matter of choice because they are more accurate. To be honest on this map it doesn't make that much of a visual difference because the effect is so subtle but it is a good idea to get in the habit of doing everything per pixel because the artists will add a lot of detail in those maps and anything that aren't using them will help to flatten the entire surface visually imagine if a surface looks like it has tones of detail and then suddenly there is a reflection that doesn't follow the details, this kind of things can kill your looks really quickly if you are going for high fidelity looks. And if you aren't well then you can obvious fake everything but you still need to know how it should work because hardware is just getting faster and faster and someday you want to do it for real.

But now I am going to stick this lighting on our object and see what happens when we add in the colormap.

 

As you can see the final contribution of the effect might feel subtle and you might even wonder if it is all worth it the lighting looked much better and sharper before we applied the colormap but now it looks kind of bland. And well selecting how strong ambient light you have in the world is a tough choice but it is also the artist’s choice and not yours. And while this might look subtle it makes big difference when everything is combined together for the final step. So before we move onto that let’s combine this with our normal mapped and spec mapped object to see how it looks.

I think you will agree that those subtle effects make a huge difference now when the rest of the light is applied. The parts that was formless and boring earlier now suddenly has form and shape and feels real even though our ambient is very weak. And even the parts that were lighted by our diffuse and specular light feels like they have more depth and detail than earlier. So I believe we can say that we have improved greatly on the base model we started this post with. But we also have the last part of our series of components left we still have to add reflections.

During this last step I will gladly admit I am exaggerating the effect greatly when doing the visuals. This is to make sure that it is visible in these static and compress low res images.  In actual running code the effect can be toned down a lot and still add a lot of life to the image. I might release a video compendium to this lecture series showing the different effects life however compression artifacts makes that questionable as some of our effects just disappears unless exaggerated.

I'll just reiterate the basics in case someone has forgotten since the first article. The brightness of the reflection has nothing to do with the amount of light that the object it is being in reflected is under. Instead it is only affected about the lighting conditions on the reflected object (In the real world all light is just bounced photons no matter if they are reflections or not of course but for our in game model we try to simplify things). This means that the reflections will be most clearly visible in the shadow side of the object because it is a larger percentage of the light hitting that area than it is in the lit side.

This is exactly what we need as the dark side still looks dead (this is much more visible in motion but not in YouTube quality motion so you just have to trust me)  but it will also cause slight effects to add more complexity to the looks of the shading in the bright side. And of course it will allow us to simulate highly reflective objects.

Since we can't simulate reflections perfectly, (well we could in certain circumstances like a limited number of reflecting items or as a special case effect for mirrors water etc, but for this we are still talking about a general lighting equation that we can use pretty much on anything.) we just want to try to get a visual change in the surfaces that says we are reflecting something for true reflective objects obviously this something would have to representative for the world around it. To do this we will use an environmental map as a representation of the world that can be reflected and for most objects we will just use a fixed map. But for truly reflective materials where it matters we can generate a new map on the fly every frame (or more likely every 10 frames or something like that to keep costs down)

But we want to use the environmental map as a view of the environment around us.


The look up this time will be done differently for the ambient light we could just use the world space normal to make a lookup in a cube map, but here what we want to see is what objects light would have bounced at this spot and then hit our eye.  You surely remember that to get an outgoing vector after a reflection from an ingoing vector you just reflect it around the surface normal so that the angles between the normal and the ingoing vector and the normal and the outgoing vector match.  We are going to do it pretty much the same as that except the vector from our eye to the point on the surface is going to be the incoming vector and the outgoing vector is the vector that we are going to use to look up into our cube map (if the process of doing environmental mapping with a cube map feels unfamiliar this is explained in detail in the DirectX SDK).

Since the reflected light doesn't come from any of our other light sources we just add it together to get the total light amount for our object.


So the light hitting a surface is Ambient+Diffuse+Specular+Reflection. Putting this in equation form will give us.

pixelColor=0
pixelColor+=(Ambient.LightStrength*AmbientOcclusionFactor)*ColorMapValue;
pixelColor+= EnviromentalImage*ReflectivityMap

For(i=0;i
{
  PixelColor+=FalloffFunction(lightStrength[i],DistancetoLight[i])*colorMap* dot(surfaceNormal,lightDirection[i])));

  pixelColor+=            pow((Dot(HalfVector,Normal)*FalloffFunction(lightStrength[i],DistancetoLight[i])SmoothnessFaktor)*specularMap);
}


If you look closely at the above formula you see that reflections and ambient light are the same no matter how many light sources you are using this is because they aren't direct effects of the light rather they are indirect effects by bounced light and as we can simulate it correctly we just fake it. But the rest of the lighting equation is calculated per light. Let’s first look at what we have before we delve into that.

You should see the clear reflected light in the shadow side now. And you should also realize why I told you I exaggerated the effect, obviously it shouldn’t be this strong but it should be there to bring out life in the dark sides.

We have one last step to talk about now shadows. The reason this is important is that I have seen so many games that just throw a generic shadow behind an object cast from some light and are so happy about it. This isn't what a shadow is in real life. If something looks like being in shadow it is because some objects are blocking the way of the photons from a light source. What’s important to ember is that it should only remove the light from that single light source as all other photons will still be hitting it just fine. If that light source is weak or its light is fading out then the shadow should also be weak or fading out.  An interesting part about reality is that we have objects that allow some parts of the photons to pass through. This means that their shadows darkness should be relative to how many photons that are actually getting through the material. So for the moment let’s consider adding a new function to our system LightOcclusionFunction() it returns a value from 0 to 1 with 0 meaning total occlusion and 1 meaning no occlusion of the photons.
pixelColor=0
pixelColor+=(Ambient.LightStrength*AmbientOcclusionFactor)*ColorMapValue;
pixelColor+= EnviromentalImage*ReflectivityMap

For(i=0;i
{
  PixelColor+=FalloffFunction(LightOcclusionFunction()*lightStrength[i],DistancetoLight[i])*colorMap* dot(surfaceNormal,lightDirection[i])));

  pixelColor+=            pow((Dot(HalfVector,Normal)*FalloffFunction(LightOcclusionFunction()*lightStrength[i],DistancetoLight[i])SmoothnessFactor)*specularMap);
}

This is all fine and dandy but in most games the value of the lightOcclusionfunction is either 0 or 1 so it is a blocker and you have to perform a lot of code for each light to determine this which leads to rendering the world with a multi pass model. First you render the ambient and reflective contribution then you render each light by its own adding it's light to the pixel value this gives exactly the same result as before but allows the computer to do a ton of calculations on where light is hitting etc for each light.


This changes our code into this.

Pass 1
     pixelColor=0
     pixelColor+=(Ambient.LightStrength*AmbientOcclusionFactor)*ColorMapValue;
     pixelColor+= EnviromentalImage*ReflectivityMap

For(i=0;i
{
Pass 2+i
        PixelColor+=FalloffFunction(LightOcclusionFunction()*lightStrength[i],DistancetoLight[i])*colorMap*   dot(surfaceNormal,lightDirection[i])));

  pixelColor+=            pow((Dot(HalfVector,Normal)*FalloffFunction(LightOcclusionFunction()*lightStrength[i],DistancetoLight[i])SmoothnessFactor)*specularMap);
}


This is of course from the view of a single pixel if we looked at it from a rendering function of view it would work out like this.

Pass 1
For each Instance Render with following PS
        pixelColor=0
        pixelColor+=(Ambient.LightStrength*AmbientOcclusionFactor)*ColorMapValue;
        pixelColor+= EnviromentalImage*ReflectivityMap
        screenColor=pixelColor;

    For(i=0;i
    {
    Pass 2+i
        For Each Instance Render With Following PS
            pixelColor=0;
            PixelColor+=FalloffFunction(LightOcclusionFunc(lightStrength[i]),DistancetoLight[i            ])*colorMap* dot(surfaceNormal,lightDirection[i])));
            pixelColor+= pow((Dot(HalfVector,Normal)*FalloffFunction(LightOcclusionFunc             (lightStrength[i]),DistancetoLight[i])SmoothnessFaktor)*specularMap);
            screenColor+=pixelColor
    }

This will allow us to block light realistically on each object from each light source allowing some real subtle effects. One interesting thing that  can happen when doing rendering with a lot of lights is that the colour value of a pixel may surpass 255 which is the maximum colour a monitor can display. We might later talk about how to fix this using High Dynamic Range rendering techniques. But for now you just have to be aware so that you don't over light the scene.

I know this series of articles has been heavy and there is tons of stuff I have left out and things I have assumed from the reader that’s the only way to do this without writing an entire book. But I hope that by starting with grounding it clearly in the real world and using visual examples that I have made it easier to follow along.