Texture Modification using Render Targets, with some Stencil Buffer Action
Sometimes you need to modify a texture while your game is running, and there are a number of ways to do this. One of the first things newer game programmers often try to do is use Texture2D.GetData to copy the texture data from the GPU to an array on the CPU, modify the bytes, and then send it back to the GPU with Texture2D.SetData.
This is a bad idea on many levels. Beyond issues with pipeline stalls, GetData and SetData can be slow, especially when working with a large texture. Any time you’re tempted grab data from the GPU for use on the CPU you should very carefully consider all of your options. There are often other solutions that let you keep the data entirely on the GPU and accomplish the same thing.
This tutorial will use an example that could be solved with GetData and SetData, and show you another alternative using render targets and the stencil buffer that will let you perform the same function entirely on the GPU.
CPU Craters
Let’s pretend you want to draw 2D planet, and periodically add a crater to it. You want a hole to appear somewhere on the planet, so it looks like part of it was removed.
You could do this using the GetData/SetData method by getting the data from a texture into an array, setting the color to the background (or alpha to 0) in the shape of the crater, then writing the data back to the texture. Or you could be a little cleverer and eliminate GetData by always keeping the data in the array, but you still have to do the SetData to get it into the texture on the GPU each time it’s changed.
GPU Craters
The method we’ll use to do this entirely on the GPU involves several steps. First, we need a couple of resources. We’ll use a simple textured circle for a planet, and a “crater” shaped texture for the crater.
It’s important to note that the black areas on these have an alpha value of 0, meaning completely transparent. For the planet this just lets us draw the round shape over the background without looking like a square image. But for the crater image the alpha value is very important since it will control what part of the crater image is removed from the planet.
Next, we need to set up two render targets (these will be referred to later as Render Target A, and Render Target B). When we need to add a crater, one of these will be used as a target for drawing to, while the other used as a texture. The next time we add a crater they will swap roles – the texture will become the target, and the target will become the texture. This is called “ping-ponging” and will be discussed more fully later.
Once we have these resources ready to go, the method for adding a crater goes like this:
- Activate Render Target A using GraphicsDevice.SetRenderTarget.
- Clear the graphics device, setting the color to solid black, and the stencil buffer to 0.
- Set up the stencil buffer state so whatever we draw writes a value of 1 to the stencil buffer.
- Set up the alpha test state so we only draw where the alpha value is zero.
- Draw the crater texture. Because of the way we’ve set up the graphics device, only the parts of the crater texture that have alpha = 0 will be drawn, and those parts will write a 1 to the stencil buffer. So what we have at this point is a “mask” in the stencil buffer that we can use in the next step. The white area in the following image represents the stencil mask we’ve set up – the stencil buffer contains “1” in the white area, and “0” everywhere else.
- Set up the stencil buffer so when we draw, anything that has a value of 1 in the stencil buffer will be masked out – meaning it won’t draw.
- Draw the “planet texture”. Because of the way we’ve set up the graphics device, anything with a 1 in the stencil buffer won’t be drawn – since these 1’s are in the shape of a crater, that shape will be masked out of the planet texture, leaving holes that look like craters.
- Set the render target to the backbuffer. We can now access Render Target A as a texture, and that texture contains the planet texture with a crater-shaped hole in it.
Step 5
Step 7
From now on, until we need to add another crater, we can treat Render Target A as a texture and draw it using SpriteBatch, and we’ll have a nice crater. Now, what if we need to add another crater? This is where the ping-ponging comes in. Since Render Target A is now the “planet texture”, we need to be able to draw somewhere else when we’re filling in the stencil buffer with our crater shape. It just so happens that we set up another place to draw to, Render Target B.
So now, in Step 1, instead of activating Render Target A we need to activate Render Target B and draw the crater shapes into that. But what happens when we get to Step 7? Well, the “planet texture” is now in Render Target A, so we draw that. And in Step 8, Render Target B now contains our new planet texture with two craters.
And if we add a third crater then we’re back to where we started – drawing to Render Target A, and using Render Target B as the source texture. In other words, we “ping-pong” between the two render targets – each time we need to modify the texture, one is used for a texture, and one is used for drawing to, and then those roles are swapped.
You may have noticed that there’s one issue here. The first time through, Render Target B has nothing in it, so we can’t use it as the planet texture. This can be handled by using the actual planet texture the first time, and the render target thereafter.
The Code
Now let’s walk through the code involved, using XNA 4.0. You can do this in 3.1, but you’ll have to make significant changes when creating the render targets and setting the render states.
The complete code is in the downloadable project linked at the end of the tutorial. We’ll just go through the highlights here, referring to the steps mentioned above as we go.
The XNA 4.0 API has been changed substantially where render states are concerned, and for the better. Render states have been grouped by functionality into several classes. You create instances of these classes to represent the state you want, then set them on the graphics device, or pass them to SpriteBatch. So first we need to create these render state objects.
Set Up Render State Objects
For Step 3, we need to use the DepthStencilState class to set up the device to always set the stencil buffer to 1. We enable the stencil buffer, set the stencil function to Always, the pass operation to Replace, and ReferenceStencil to 1. This means that as we’re drawing, each pixel will Always pass, and the value in the stencil buffer will be Replaced with 1.
stencilAlways = new DepthStencilState(); stencilAlways.StencilEnable = true; stencilAlways.StencilFunction = CompareFunction.Always; stencilAlways.StencilPass = StencilOperation.Replace; stencilAlways.ReferenceStencil = 1; stencilAlways.DepthBufferEnable = false;
And for Step 4 we need to use the standard AlphaTestEffect so we can draw the asteroid texture only where the alpha value is 0.
Matrix projection = Matrix.CreateOrthographicOffCenter(0, PlanetDataSize, PlanetDataSize, 0, 0, 1); Matrix halfPixelOffset = Matrix.CreateTranslation(-0.5f, -0.5f, 0); alphaTestEffect = new AlphaTestEffect(GraphicsDevice); alphaTestEffect.VertexColorEnabled = true; alphaTestEffect.DiffuseColor = Color.White.ToVector3(); alphaTestEffect.AlphaFunction = CompareFunction.Equal; alphaTestEffect.ReferenceAlpha = 0; alphaTestEffect.World = Matrix.Identity; alphaTestEffect.View = Matrix.Identity; alphaTestEffect.Projection = halfPixelOffset * projection;
We first set up an orthographic projection matrix that matches SpriteBatch. We set AlphaFunction to Equal, and ReferenceAlpha to 0. This means the alpha test will pass whenever the alpha value we’re drawing is equal to 0. In our crater texture, the crater area has an alpha value of 0, while the surrounding area has 1, so only the crater area will be drawn.
For Step 6 we need a stencil buffer state that allows drawing only where the stencil buffer contains a 0. We enable the stencil buffer, set the stencil function to Equal, the pass operation to Keep, and the reference stencil to 0. This means that when we’re drawing, each pixel will pass if the value in the stencil buffer is Equal to 0.
stencilKeepIfZero = new DepthStencilState(); stencilKeepIfZero.StencilEnable = true; stencilKeepIfZero.StencilFunction = CompareFunction.Equal; stencilKeepIfZero.StencilPass = StencilOperation.Keep; stencilKeepIfZero.ReferenceStencil = 0; stencilKeepIfZero.DepthBufferEnable = false;
Create Render Targets
Now that we have the render state objects created, it’s time to create the render targets. Both are the same, so just one is shown here. This creates a render target with a Color format, and a depth format that includes a stencil buffer.
renderTargetA = new RenderTarget2D(GraphicsDevice, PlanetDataSize, PlanetDataSize, false, SurfaceFormat.Color, DepthFormat.Depth24Stencil8, 0, RenderTargetUsage.DiscardContents);
Draw the Crater Mask
Next up is drawing the crater masks (Steps 2-5). First we activate the render target, clear it to solid black, and clear the stencil buffer to 0.
GraphicsDevice.SetRenderTarget(activeRenderTarget); GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.Stencil, new Color(0, 0, 0, 1), 0, 0);
Next we begin a SpriteBatch, passing in the stencilAlways and alphaTestEffect objects that we created earlier. Calculate some random rotation, size the crater texture using a Rectangle, and call SpriteBatch.Draw to draw the crater.
spriteBatch.Begin(SpriteSortMode.Immediate, BlendState.Opaque, null, stencilAlways, null, alphaTestEffect); Vector2 origin = new Vector2(craterTexture.Width * 0.5f, craterTexture.Height * 0.5f); float rotation = (float)random.NextDouble() * MathHelper.TwoPi; Rectangle r = new Rectangle((int)position.X, (int)position.Y, 50, 50); spriteBatch.Draw(craterTexture, r, null, Color.White, rotation, origin, SpriteEffects.None, 0); spriteBatch.End();
Draw the Planet Texture
Now we need to draw the latest planet texture, using the stencil buffer to mask out the craters (Steps 6-7). We begin a SpriteBatch, passing in the stencilKeepIfZero object we created earlier. Note that the first time we draw the actual planet texture, but subsequently we draw using the texture from the previous iteration.
spriteBatch.Begin(SpriteSortMode.Immediate, BlendState.Opaque,
null, stencilKeepIfZero, null, null);
if (firstTime)
{
spriteBatch.Draw(planetTexture, Vector2.Zero, Color.White);
firstTime = false;
}
else
spriteBatch.Draw(textureRenderTarget, Vector2.Zero, Color.White);
spriteBatch.End();
Swap Render Targets
Finally we activate the backbuffer render target.
GraphicsDevice.SetRenderTarget(null);
And then swap the render targets as discussed previously.
RenderTarget2D t = activeRenderTarget; activeRenderTarget = textureRenderTarget; textureRenderTarget = t;
In the main Draw function, you draw the latest cratered planet using the textureRenderTarget. Of course, you need to deal with using the planet texture the first time through though. The downloadable code shows one simple way to do that.
GraphicsDevice.Clear(Color.CornflowerBlue); spriteBatch.Begin(); spriteBatch.Draw(textureRenderTarget, planetPosition, Color.White); spriteBatch.End();
Conclusion
And there you have it, a powerful technique for altering textures during your game. Doing this entirely on the GPU is quite a bit more complex than GetData/SetData, but is well worth the extra trouble.
There are some things you can do to improve this technique. If you need to add a lot of craters, rather than adding them one at a time you can batch them up for a while, then in Step 5 draw all of them at once.
I hope you found this tutorial informative. Learning about render targets and stencil buffers opens up a whole new world of possibilities beyond just making craters. What other uses can you think of?
Download the sample XNA 4.0 project
Download the sample XNA 3.1 project
Past Lives
I’ve taken a bit of a programming hiatus so far this summer, with the exception of the WordPress plugin I mentioned last time which I just spent a couple hours a week on over the past several months. I’m kind of taking a deep breath before plunging into making a game for the upcoming Windows Phone 7. During my break I’ve spent some time thinking about the projects I’ve worked on over the years – some that I’ve hated, and some that I’ve loved. One that always stands out for me is a little something called MariusNet, and I thought I’d share the story.
Back in the days before Bungie was famous for Halo, they made this little game called Myth. I thought it was intriguing, but never played it much. My friend Conner (to use his Myth gamer name) was really into it though, and I followed his exploits from a distance. I did eventually create an account, and since I had recently seen the musical Les Miserables, “Marius” seemed like a good name for a fantasy game.
One of the really cool things about Myth was the central server everyone logged into for chatting and starting games. A large community formed through that, and lifelong friendships were born. Many people formed into gaming clans. Conner was one of the early members of Clan Plaid, which remains an active gaming clan to this day, although they’ve long since branched out to other games. I was recruited later (9/27/2001 to be exact, as I sit here looking at my Marius #CP# beer mug) but was never very active, to my regret.
Sometime after Myth II was released, Bungie started working on some really cool new technology, and a PC game they were going to call Halo. One of the ideas that was talked about early on was the ability to go anywhere on the entire ringed planet. Sometime thereafter, Bungie was purchased by Microsoft, and they moved their offices from Chicago out to the west coast. The “go anywhere” thing obviously never panned out.
Bungie’s move ended up being the driving force behind what ended up becoming MariusNet. It seems that somehow they had lost the source code to their Myth I game server, and being the mad hackers they were, they had hard coded internal network addresses into the server (or something like that), and were unable to get it running again in their new offices. Suddenly several hundred people were no longer able to play the game they loved. My friend Conner approached me and asked if I’d be willing to help him reverse engineer a replacement Myth game server, working on the assumption that the Myth I server functioned very similarly to the still running Myth II server. I eventually agreed.
We found some packet sniffing software, a disassembler, and I abandoned my wife and kids for a couple weeks while we worked. Somewhere along the line we started calling it MariusNet (kind of similar to Bungie.net, get it?) To make a long story short, we eventually (with a couple of key bits of info from one of the Bungie guys) hacked together a more or less stable server that allowed Myth I players to connect, chat, start and play games. Over time we improved the server, re-engineered the login system, added game stats and ranking – the whole works. We even eventually added support for Myth II, Myth III, and even Bungie’s original game Marathon.
Once Bungie realized we had something permanent, they officially discontinued their Myth I server support – which we took as their official blessing that they were fine with our reverse-engineering efforts. They even found time to interview Conner and I. And as thanks for our efforts, we were invited to the original Xbox and Halo launch party in Chicago. We played in the very first public game of Halo, and ate breakfast with the Bungie guys afterwards (yes, it ran that late).
I dropped out of supporting the server quite a few years ago, but some good people took over. I don’t think about MariusNet very often, but every now and then I’ll go back for a visit to see that my namesake is still happily chugging a long. As I write this, there are 31 people playing Myth II, and 12 playing Marathon. “Thriving” may be too strong of a word for the community, especially compared to its glory days, but it’s a good feeling to think that I had some small part in helping people continue playing a game they loved, and that some of those people are still playing even years later.
Anyway, hope you don’t mind my little trip down memory lane. Tomorrow I think I’ll start making some new memories on the Windows Phone 7 simulator.
Visit MariusNet: www.mariusnet.com
WordPress Plugin Diversion
Earlier this year I started doing some daily walking and hiking. I would be remiss if I didn’t tie some technology into it somehow, so in my spare time I’ve been working on a WordPress plugin to track my progress.
The plugin is now showing up in the WordPress Plugin Directory for anyone who’s interested.
Uninstalling Windows Phone Developer Tools CTP
Like many others it seems, when uninstalling the Windows Phone Dev Tools CTP I uninstalled each of the individual components before getting to the actual “Microsoft Windows Phone Developer Tools CTP – ENU”. When this happens the uninstall doesn’t work, unless you first re-install everything else.
Or, thanks to Nick Berardi, you can just navigate to the install folder (C:\Program Files (x86)\Microsoft Visual Studio 10.0\Microsoft Visual Studio 2010 Express for Windows Phone CTP – ENU is the default), right click on vs_setup.msi and select Uninstall from the menu.
It works great, and is much better than redownloading the CTP and reinstalling first.
Now, back to installing the XNA Game Studio 4.0 Beta!
Procedural Planet Video
Here’s a video showing the space-to-surface transition, as well as the tank driving around a bit. I’m pretty happy with how it looks, but there are a lot of things that still need to be fixed:
- Seams on neighboring nodes when they’re at a different level of detail.
- Lighting on node edges is wrong since the normal map generation isn’t currently taking into account neighboring height values, resulting in very visible node edges.
- Need to add some noise to the texture generation so the transitions are less obvious. This should also help minimize the repeating patterns.
- Normals are too “strong” at high altitudes. I think I need to change the normal strength based on the quad tree level.
- Need to add morphing between LOD changes to eliminate popping.
I think those changes will improve things quite a bit. But first I’m going to plug the atmospheric scattering shader back in. I’m currently using Sean O’Neil’s method, and that’s what I’m going to add back for now. But I’m not really happy with it and plan to give Precomputed Atmospheric Scattering a go.
So, here’s the video…
Of Tanks and Quad Trees
I needed a bit of a diversion from the planet rendering itself, into something that would give some purpose behind it. Why is the planet there? Well, what better use is there for a planet than driving a tank on it?
The tank model from the XNA Heightmap Collision with Normals sample is the perfect tank to drive on a lonely, empty, and possibly dangerous new world. I grabbed the model and texture resources, and the Tank class, added them to my project, and added some “spawn tank” functionality to get it drawing, resulting in a beautiful flying tank that seemed to defy gravity. Since nobody should be able to defy gravity, it was time to bring the tank down to earth.
The basic concept is to find the triangle underneath the tank and the exact position within that triangle. This will give us the exact height at that location, and the normal so we can orient the tank correctly. When using a single height map that’s not too bad, but when there are potentially hundreds of height maps, organized in a quad tree, there’s some more work involved.
The first thing I do is find the quad tree node that’s under the tank. There isn’t a regular grid like with a height map, so I had to come up with a different way to find it. The first thing that came to mind was to do traverse the quadtree with a simple point-in-bounding-box check, but that would only work if the tank is already very close to the ground. I want to be able to have a position 6,000 km above the surface and be able to find the proper node.
The next idea was to create a view frustum from the planet center looking outward at each node. To create the frustum I’d have to calculate the proper field of view so the frustum planes would go through the edges of the quad tree nodes. I actually tried this method but had trouble getting the field of view calculations to work so things were aligned properly. I still think this method would work if I spent enough time on it, but there’s actually quite a bit of code that has to be executed for each check, so it’s not the best method anyway.
The final idea, and the one I have working, is to do a simple ray-bounding-box intersection test. I was initially going to create the ray at the planet center, pointing out towards the position we’re looking at, but that would very likely run into floating point precision issues, so instead the ray starts 1km below the planet surface, which gives plenty of precision to work with.
So, I traverse the quad tree, doing the ray-bounding-box intersection test. If I find a hit, I perform the check against the children, and so on, until I reach a leaf node. Once I have a leaf node I need to find the triangle within the leaf, using ray-triangle intersection tests. Now, I understand the concept, but coding one with my current knowledge is beyond me, so I downloaded the XNA triangle picking example, which has a nice ray-triangle intersection method.
Now, the original ray-bounding-box intersection can give false hits since we’re working with a spherical surface because the bounding boxes can overlap a bit. So the ray-triangle intersection tests might fail to find a triangle. If that happens I just continue on traversing the tree with the bounding box checks until I eventually find the proper node, as well as the proper triangle. This works, 99% of the time. There is still a bug where the code fails to find a node at all. It’s very rare, but something I’ll need to figure out someday.
So, in order to get the proper height value I need to get the exact point within the triangle. When using a height map it’s common to use bi-linear interpolation to find this height, and I spent some time trying to get that to work with partial success. I finally stepped back and realized that the ray-triangle intersect was returning the distance along the ray of the intersect, so it was a simple matter of multiplying the normalized ray-direction by the intersect distance to give me the exact position I needed. Treating that position as a vector from the planet center, the magnitude is the height value that’s needed for the tank position.
That leaves the normal so the tank can be oriented properly. When using a height map the normal is also found using bi-linear interpolation. This presented a problem since I couldn’t ever get that to work completely for the height. Instead of mucking with it too much I chose to average the normals of the triangle, which seems to fit well within my “good enough” expectations at this time.
So, my planet has gained a purpose, and in so doing I now have some code that’s going to be very useful for things like placing trees and walking and crashing and shooting. I’ll try to put up a movie in the next day or so.