Logo 3.5Cats_AndHalfAFish


Walls (part 1).

April 16, 2020 Shade, Blender : baking textures
What I've learned about modeling and normal map baking ...

Note : Godot 3.2.1, Blender 2.82.

These are some of my observations gathered from when I was trying to model a simple wall that was to be used for baking textures.
As always : I don't claim to know what I'm doing ... =)

I decided to start my adventure using some of the repeating pieces that are 6x9 meters.

Castle blockout.
Blocking out the castle.

Naturally, I decided that the wall model for baking textures should be the same size ...

As it turns out, that wasn't really the way to do it :
1) the surface is too large (takes too long to model + too many vertices for my poor computer),
2) textures in games are usually 'powers of two' (= square).
Of course, I only realized these things after spending a lot of time ...

Oh well.

I then switched to a wall model of 3x3 meters, and 1K textures for baking (faster to test things out).

Let's see ... What are all the things I tried ?

Well, I tried : 1) modeling a high-poly wall from loose bricks, and baking that onto a low-poly wall (cube), 2) using that same high-poly wall to bake onto a plane, 3) various combinations of loopcuts, bevels, and insets to model a wall starting with a plane, and baking that onto another plane.

The types of textures I tried baking are : normal map, ambient occlusion, cavity, and diffuse. Some older tutorials also mention specular maps, but I haven't seen these used in the PBR workflow ... Two other map types that are used often, are 'metal' and 'roughness'.

Workflow : modeling.

The look I'm going for, for this part of the castle, is based on this (pretty bad) reference image :

Reference image for wall.

As you can see, it's a pretty flat wall.
Ideally, I would have liked to just find a nice texture somewhere, and slap it on a mesh. Since that didn't happen, plan B was to either : 1) create all textures in Krita/Gimp/..., or, 2) bake the textures from a 3d model in Blender.
Since I wanted to learn how to bake textures, I went for option 2.

This is what the modeled wall (default Blender material) looks like when tiled. (I still have no idea what I'm doing with the lighting, so yeah, it's bad.)
The wall was created using a combination of loopcuts, bevels, and insets. The shading is 'smooth', except for the big stone faces, which are 'flat'. The total number of verts is about 1400.

Render of wall with default material.

Note : The reason I went for bevels and smooth shading despite the mostly flat wall, is that normal maps can only be baked for sloping or smooth faces. If you would try to bake a normal map for a mesh like the one below, the resulting map would be completely blue.

Normal map bake failure.

Note : I'm not sure if I mentioned this in a previous post, but I find it helpful to turn 'auto merge' on. The setting can be found in the Active Tools tab (but only in Edit Mode).

Auto merge settings.

Note : these are the settings I used for the insets.
Also notice that the bricks at the edges of the wall are 2 halves of the same brick (tiling). For the shading to work correctly, it's important to delete the 2 outermost faces !

Inset settings and deleted faces.

Workflow : normal map bake.

The first thing to do, is to create a low-poly copy of the object you want to bake.
Often this will be a duplicate of the high-poly piece with a Decimate Modifier applied. If you don't know what I'm talking about, watch the Grant Abbitt videos I mention at the end of this post (and especially number 3 : quick retopology).
Since my high-poly wall is pretty flat, I'm going to use a plane as my low-poly copy.

The bake-plane is 3x3 meters and has its origin set to the same corner as my high-poly wall. It's a good idea to apply rotation and scale to both pieces (Ctrl+A), and to make sure their normals are pointing in the same direction.

You can check the orientation in Object Mode by clicking the 'overlays' dropdown button, and selecting 'face orientation'. You want to see blue (no red). Blue means we are looking at the front of the individual faces of the mesh. This is especially important when you import the mesh into a game engine that uses backface culling (meaning it doesn't render faces that show their backs to the camera). Watch the video created by BailyDesign for a better explanation.

Checking the orientation of the meshes.
Checking the orientation of the meshes.

The 'classical' way to check face orientation is, of course, by going into Edit Mode and activate the normals buttons (see the BailyDesign video).

If all of that is fine, you'll want to duplicate the high-poly wall by using Alt+D (not : Shift+D) a couple of times, so that the original wall is surrounded on all sides. This is to prevent artifacts.
Next, select all high-poly objects, Shift-select the middle piece (= active object), and press Ctrl+J to join all pieces while keeping the origin of the middle piece.

Merged high-poly meshes.

The correct positioning of the low-poly (lp) object relative to the high-poly (hp) object is still a bit of a mystery. Should I put my plane in front or behind the high-poly plane ? Or do the origins have to be in exactly the same location?
Tutorials where eg. a high-poly 3d brick is baked onto a low-poly cube state that the hp model has to be completely surrounded by the lp object. So, similarly to those tutorials, I have been placing my lp wall plane in front of my hp wall (but as closely together as they can be without intersection). For instance, I put my lp plane at y=0.005, and my hp wall at y=0.

Relative positioning of the 2 meshes.

On the other hand, it is also ok to put the low-poly plane behind or on top of the high-poly mesh and compensate by using the 'ray distance' in the Bake section. It needs to be set to a value in the same range as the one used above.
If you pick a distance that's too small, the normal map will be completely blue, or it will miss the higher details that are further away from the bake-plane.

Correct normal map. Incorrect normal map.
Good normal map vs. map obtained with incorrect ray distance.

Now, we're almost ready to start baking textures.

Select the bake-plane and give it a material (the default is ok). Next, uv-unwrap the plane (Edit Mode > select all > press 'U' while the mouse is inside the 3d view > smart unwrap / project from view / ...).
At this point, it's important to check that the orientation of the uv map is correct. If the uv map is rotated, you'll have problems later, when you import the model into the game engine !
I think (but I might be wrong here !) that the map might also be incorrect : a normal map stores XYZ information for all normals as RGB color info. So, if the uv map is rotated, I would guess that XYZ will be mapped to GRB rather than RGB ... (see the image further down this post where I show 2 wall pieces side by side)

To check the orientation, select 1 edge of the plane and see if its position on the uv map is correct. If not, rotate the entire uv map.

Checking the uv map orientation.
Checking the uv map orientation (here : incorrect).

With the bake-plane selected, move to the Shading Workspace. You should see something like this :

Shading  with bake-plane selected.

In order to bake the normal map, you need to add an 'image texture' node to this shader (Add > Texture > Image Tx). The node doesn't need to be connected to anything.
In the image texture node, press the 'new' button to create a 1K texture. This is just a test bake, to make sure everything is working as it should. Later on, you can always use bigger textures if you feel you need the extra resolution.

Setting up for normal map bake.
Setting up for the normal map bake.

Normal map data contains light information ; not color (albedo) information. That's why it's important to set the color space to 'non-color' in the image texture node !

All baking needs to be done from within the Render Properties panel (looks like a tv) and with the Cycles engine.

Setting up for normal map bake (2).

In the Bake section, set the bake type to Normal. Check 'selected to active' and 'clear image' (so that we can continue to bake to the same texture). I've also changed the output margin from 16px to 0px (not necessary, see note below).
Next, select the image texture node in the shader graph to activate it (it now has a white border), click the merged high-poly wall in the 3d view, and Shift-select the bake-plane. The order is important : we want to bake from the seleced object (high-poly, dark orange border) onto the active object (bake-plane, light orange border) image texture !
Finally : click the Bake button. Error messages are displayed below the shader graph. Also remember to save the normal map ; I used 8-bit RGB with 15% compression (default) for my test bake.

Normal map bake with correct uv. Normal map bake with rotated uv.
Normal map bake comparison : correct vs rotated uv map.

And here are 2 test wall pieces. The right one had its uv map baked while it was rotated 90 degrees. The resulting textured wall piece had to be rotated 90 degrees in the other direction.
Also remember to check the face orientation (remember: blue is good, red is bad).

Normal maps applied to 2 walls.
Normal maps applied to 2 walls (correct uv vs rotated uv).

The material node setup is like this :

Material node setup.

Finally, compare the wall with the baked normal map (left) with the original high-poly wall (right) :

Normal map vs high-poly.
Normal mapped plane (left) vs high-poly wall (right).

Note : the bake panel has a setting called Output Margin. Its default value is 16px. According to the Blender documentation this is how many pixels beyond the border of each uv “island” the baked result is extended (to soften the seams in the texture).
In case of an unsubdived plane there's only 1 island, so, this setting does not influence the uv map.

Note : if your normal map looks 'weird' (funny colors), you can play with the 'ray distance setting', or even use a baking cage (see Grant Abbitt video mentioned in my previous post).

Note : if your normal map looks 'banded' or blurry, you can try baking onto a higher resolution texture. Also check to make sure you're using the non-color space (as opposed to sRGB) !

Note : some people suggest that normal maps should be saved in a loss-less format such as 'BMP' or 'Targa RAW' ...

Workflow : baking other maps.

The other maps can be obtained in exactly the same way. Just select the bake type you're interested in, and remember to adjust the color space as well. As far as I know, only the diffuse texture bake has the sRGB color space ...

I think I'll postpone talking about the other maps until the next post, as this one is already getting a bit long.

Workflow : import into Godot.

I think it's important to test your game assets as soon as possible in the game engine.
So, I imported the high-poly wall and the normal-mapped plane into Godot to see if there was a noticeable difference between the two.

First, I tried exporting as OBJ. The Blender settings were : 'selection only' (checked) and 'write material' (unchecked). I also needed to manually copy the normal map (.png) into Godot.
In Godot, I added 2 MeshInstance nodes to the scene tree, and loaded the .obj files (in the 'mesh' section). Next, I created a Spatial Material for each (default settings). Finally, for the normal-mapped mesh, I activated the 'normal map' option and loaded the .png file.

Next, I tried exporting as 'glTF separate' (default settings). This made things even easier : all textures are automatically copied to Godot, and a correct material is assigned to the mesh.
Only thing left to do in Godot, is double click the .gltf file, select 'open anyway', save as a scene, and add an instance node to the scene tree.

These are the walls in game :

Walls : normal-mapped vs high-poly.
Walls : normal-mapped (4 verts) vs high-poly (1400 verts).

Note : you may have noticed that the normal map texture looks weirdly yellow in Godot :

Godot normal map looks yellow.

I found this 2018 Issue on Github talking about just that : #16490.
Apparently, this has to do with texture compression in Godot. The default import method for the normalmap.png is 'video ram' with a 'lossy quality' setting of 0.7. As I understand it, Godot now imports only the red and green channels, and rebuilds the depth (blue channel) in the shader at runtime. This is an optimization to reduce memory usage (but it also slightly increases processing usage).
If you change the compression mode to 'lossless' or 'uncompressed' and reimport the texture, the blue channel will be imported as well, and the texture will look 'normal' (no pun intended).
So : the yellow color is fine. Don't worry about it. =)

Godot normal map looks blue.

In the next post, I'll continue documenting the workflow for this wall mesh. See you then !

List of relevant tutorials.