Tuesday, February 20, 2018
Terrain Texture Splatting and Three js
Terrain Texture Splatting and Three js
So there comes a time in every developers life when theyre unable to use the tools they know and love to get the job done. Sometimes the tools arent a good fit for the task and other times there are political reasons. Maybe you work for the man and the man says use tool "x".
Thats a bummer but thats reality. On the bright side, youll have an opportunity to learn and gain experience you might not have had before. So heres my story:
One day I was in this situation where I had to make a terrain demo in a web page WITHOUT using any plugins. If you havent already seen my video about this, check it out here. This wouldnt be an issue with Unity 5 and above (due to export to asm.js) but at this time, version 4 was the latest.
So lets back up a moment. What is texture splatting and why would we want to use it?
Simply put, texture splatting is a technique for combining different textures. And not only that, it allows you to change how those textures are combined (blended) at every texel. That means you dont have to blend those textures together the same way over the whole object, the way they are blended can vary.
For instance, check out the terrain below. It is blending a combination of 4 different textures in different ways across the terrain.
What youre seeing is a combination of the following:
You can see that the mix of textures is applied in different ways across the terrain. For instance, some areas have more snow, more dirt, more grass, or more rock.
What youre not seeing in those images is the instructions for how to combine them at every point. You just see the result. So whats the secret sauce here?
Check out this image:
Immediately you can see that the crazy colors in this image are associated with the terrain you saw above. Thats because each color channel in the image (RGBA) corresponds to how much a certain texture should be used. You might get something like this:
Red channel: dirt
Green channel: grass
Blue channel: cliff rock
Alpha channel: snow
Note that alpha is not a color but an extra spot usually reserved for transparency. Each color channel is multiplied by the texture its associated with and added up to get the final result. This process is done using pixel shaders (executes in code that runs directly on the GPU) which makes it very fast to compute. Heres some shader code that actually does exactly that:
vec4 pixelColor1 = texture2D(texture1, UV1);
vec4 pixelColor2 = texture2D(texture2, UV2);
vec4 pixelColor3 = texture2D(texture3, UV3);
vec4 pixelColor4 = texture2D(texture4, UV4);
vec4 alphaMap = texture2D(tAlphaMap, alphaUV);
gl_FragColor = pixelColor1 * alphaMap.r + pixelColor2 * alphaMap.g +
pixelColor3 * alphaMap.b + pixelColor4 * alphaMap.a;
A quick explanation: texture2D is function that looks up a color from texture using a texture coordinate and tAlphaMap refers to the splat texture shown above. Each color channel (.r .g .b .a) is a number between 0 and 1 which makes multiplying color by it the same as taking a percentage of that color (ex .25 = 25%). The idea is that your percentages will all add up to one so you get an appropriate contribution for each texture based on your values. For example, check out a couple examples with just dirt and grass:
.5 * dirt + .5 * grass
.75 * dirt + .25 * grass
So great, texture splatting is a convenient and fast technique for texturing terrain. Its also what Unity uses along with a bunch of nice brushes and tools for painting with them. Again, unfortunately we cant use Unity... at least for rendering in the final application.
No problem, nothing is stopping us from authoring our splat texture ("alpha map") from Unity and using it elsewhere. Well, nothing other than the fact that Unity doesnt provide a convenient way to get data out. So lets see whats accessible to us via script.
Great, we can access the data. Now how do we get it out? Like this:
void SaveSplat() {
// Use the selected texture if it exists or else bring up a dialog to choose
string assetPath;
if (alphaMap) {
assetPath = AssetDatabase.GetAssetPath(alphaMap);
} else {
assetPath = EditorUtility.SaveFilePanelInProject("Save texture",
"mySplatAlphaMap.asset", "asset", "Please enter a file name to save the texture to.");
}
// If a valid location was chosen
if (assetPath.Length != 0) {
// Get the terrain and its data, my script thats being used here is TerrainTool
Terrain terrain = (target as TerrainTool).transform.GetComponent;();
TerrainData data = terrain.terrainData;
float[,,] maps = data.GetAlphamaps(0, 0, data.alphamapWidth, data.alphamapHeight);
int numSplats = maps.GetLength(2);
Color32[] image = new Color32[data.alphamapWidth * data.alphamapHeight];
for (int y = 0; y < data.alphamapHeight; ++y) {
// Flip the image if desired such as when planning to export the texture to use elsewhere
int vertical = (invertY) ? data.alphamapHeight - y - 1: y;
for (int x = 0; x < data.alphamapWidth; ++x) {
int imageIndex = y * data.alphamapWidth + x; // The colors are in the range from 0 to 1 but an image file is expected to be from 0 to 255
image[imageIndex].r = (byte)(maps[vertical, x, 0] * 255.0f);
image[imageIndex].g = (numSplats > 1) ? (byte)(maps[vertical, x, 1] * 255.0f) : (byte)0;
image[imageIndex].b = (numSplats > 2) ? (byte)(maps[vertical, x, 2] * 255.0f) : (byte)0;
image[imageIndex].a = (numSplats > 3) ? (byte)(maps[vertical, x, 3] * 255.0f) : (byte)0;
}
}
// make a texture to store our image colors in
Texture2D finalSplatTexture = new Texture2D(data.alphamapWidth, data.alphamapHeight);
finalSplatTexture.SetPixels32(image);
alphaMap = finalSplatTexture;
// Save this out as a .asset
AssetDatabase.CreateAsset(finalSplatTexture, assetPath); } }So this goes through the whole alphamap and saves it as a .asset. Unfortunately ".asset" is a Unity format which wont help us much so we need to take one more step to convert that into something we want.
And, frequently when we need something, someone else has already done it out there and has made their work publicly available. For this step, we are in luck. A quick search turns up this:
import System.IO;
@MenuItem("Assets/Export Texture")
static function Apply () {
var texture : Texture2D = Selection.activeObject as Texture2D;
if (texture == null)
{
EditorUtility.DisplayDialog("Select Texture", "You Must Select a Texture first!", "Ok");
return;
}
var bytes = texture.EncodeToPNG();
File.WriteAllBytes(Application.dataPath + "/exported_texture.png", bytes);
}That adds a handy menu item that we can use once weve selected our splat .asset to generate a .png file. Mission accomplished!
So now onto the next piece, putting it into a three.js web application. Unfortunately, thats an entirely different blog post unto itself. If youre interested in it feel free to leave a comment for me. I read all of them.
Oh, and you can get a Unity package with everything weve talked about HERE.
If you wanna see this in action, make sure you have a WebGL enabled browser and check out the result HERE
To see the source for that demo, hop on over to my github repo where its waiting for you HERE.
