Today I want to give you a few tips if you're planning to use tilesets in Unreal Engine 4. The tool could use some updates and quality of life improvements, but it does the job, and it sure is more than enough to ship a demo.
If you're not familiar with the UE4 assets involved in the process here's a quick breakdown:
- Textures are image files.
- Tilesets are used to split a texture in tiles and add properties (e.g. collision) to said tiles.
- Tilemaps group together tiles from different tilesets and can be placed in the game world to be interacted with.
With the terminology out of the way, let's see how you can use tilesets efficiently in the engine.
Leverage the layers
Tilemaps are made for textures and Paper2D, but they're actually 3D objects. Compared to old consoles that just had a couple of layers for scrolling backgrounds and tiles, UE4 makes it really easy to add depth to a 2D scene.
In the first section of the tilemap properties you can create as many layers you want, and they will be stacked in front of each other. This is useful to draw parts of the tilemap that need to be separated for programming reasons, but they also allow you to stack your tiles and save a lot of drawing space because you don't need to draw new tiles when a plant overlaps a stone wall.
If you look at about halfway through your tilemap properties, under the "Setup" section, you'll notice the "separation per layer" property. This setting can increase or reduce the distance between the single layers of your tilemap, allowing you more room to place your actors and add details to your scene.
The origin for the tileset is always placed at the top-left corner of the first layer. Other layers below that will increase in depth accordingly. In the example, the tilemap is placed at the origin (0,0,0) and the layers are automatically placed at -10, -20, -30 and so on on the Y axis.
Increasing the separation of the layers can also be used to avoid compenetrations of the tilemap with 3D models if you're going for the "New Super Mario Bros" look.
Simplify the collisions
The layers also give us another benefit: we can implement collisions in a single layer and save us the pain of adding collisions to every single tileset.
You may have seen from a recent twitter post that I have redrawn my collision tileset. That tileset is the only one with collisions set up and helped me save a lot of time when designing levels. You can even define collisions larger than the tile they belong to, to create larger shapes and improve the performance of the game.
If you look at this example, the selected tile actually has a very large collision, which spans 4 tiles in total, but it's totally fine as long as you remember to also place its non-colliding tiles to help you visualize the actual collision in-game.
This is the same tilemap but with collisions enabled. Of course the collision layer must be hidden in the final build of your game, however, as of today, setting the layer to "Hidden in game" doesn't seem to affect any compiled build, but setting the layer alpha to 0 is a workaround.
When you place your tilemap in the level I suggest you shift it forward so the collision layer is at Y=0. This will help you organize your scenes and coding. In Nexatli, Kevin always moves on the Y=0 plane.
Use secondary textures
The real downside of layers in tilemaps is that they share the same material. Soon you'll need more textures and effects on the same tile to add glow, wind, transparencies and whatever your game might need. Exactly like sprites, tilesets can have additional textures that can be integrated in materials to enhance your tileset.
Below, I have loaded a texture to apply glow to the golden portions of the banner.
Packing RGBA textures properly is an art in and of itself but with some effort and a few materials tailored to your needs the one-material limitation won't be much of an hassle.
Avoid mixing tilesets
Is it better to make a large tileset with most of your graphics or many smaller ones suited to smaller sections of your game?
Each approach has pros and cons, but I've been leaning towards the former lately. This is due to how the engine handles and displays tilemaps. Look at the number highlighted in red in the image below.
When you mix tiles belonging to different tilesets in the same layer that number rises considerably, degrading rendering performance. The ideal approach, in this case, is having the entire layer (or better, the entire tilemap) composed of one tileset. If you build your tilesets in large textures (256px wide and up) you'll be able to pack many more tiles and add lots of details to your maps, keeping the sections count low.
If you can't really use large tilesets or you need to mix multiple tilesets in the same tilemap, you can split them between layers when possible, or even other tilemaps. From personal experience, two layers or two tilemaps are better than a single tilemap with mixed tiles, with equal tile count.
Fix the seams
Once you have moved tiles around and found the perfect look for your tilemaps it's not the time to fix one last glitch of the engine. That is, when you're not going to touch your tilesets again for the next few months.
You may have noticed seams or thin likes between the tiles of the tilemap when playing. that happens because the tiles are actually rendered with precise UV manipulations. To avoid rendering artifacts you can "condition the texture". Right-click on your tileset asset and select "Condition tile sheet texture". The engine will create a new texture with the same tiles expanded by just 2 pixels and all tile coordinates updated accordingly. Having this 2-pixel buffer around your tiles will prevent rendering artifacts but if you need to update the tiles again you'll need to revert the link to your hand-painted pixelart texture, updated it and condition it again.
I hope these snippets of info have been useful to you. If you don't know how to build your tileset for your scene just send me a DM on twitter.
If you need technical advice on Paper2D instead, ask @Heavy_Bullets. He knows a lot more about the inner workings of the engine than me and could give you a pointer in case something in Paper2D goes horribly wrong.
The texture's palette in this examples is AAP64, made by @AdigunPolack.