Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Fourth Edition · macOS 14, iOS 17 · Swift 5.9 · Xcode 15

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

Section IV: Ray Tracing

Section 4: 7 chapters
Show chapters Hide chapters

8. Textures
Written by Caroline Begbie & Marius Horga

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

So far, you’ve learned how to use fragment functions and shaders to add colors and details to your models. Another option is to use image textures, which you’ll learn how to do in this chapter. More specifically, you’ll learn about:

  • UV coordinates: How to unwrap a mesh so that you can apply a texture to it.
  • Texturing a model: How to read the texture in a fragment shader.
  • Asset catalog: How to organize your textures.
  • Samplers: Different ways you can read (sample) a texture.
  • Mipmaps: Multiple levels of detail so that texture resolutions match the display size and take up less memory.

Textures and UV Maps

The following image shows a house model with twelve vertices. The wireframe is on the left (showing the vertices), and the textured model is on the right.

A low poly house
A low poly house

Note: If you want a closer look at this model, you’ll find the Blender and texture files in the resources/LowPolyHouse folder for this chapter.

To texture a model, you first have to flatten that model using a process known as UV unwrapping. UV unwrapping creates a UV map by unfolding the model. To unfold the model, you mark and cut seams using a modeling app. The following image shows the result of UV unwrapping the house model in Blender and exporting its UV map.

The house UV map
The house UV map

Notice that the roof and walls have marked seams. Seams are what make it possible for this model to lie flat. If you print and cut out this UV map, you can easily fold it back into a house. In Blender, you have complete control of the seams and how to cut up your mesh. Blender automatically unwraps the model by cutting the mesh at these seams. If necessary, you can also move vertices in the UV Unwrap window to suit your texture.

Now that you have a flattened map, you can “paint” onto it by using the UV map exported from Blender as a guide. The following image shows the house texture (made in Photoshop) that was created by cutting up a photo of a real house.

Low poly house color texture
Low poly house color texture

Note how the edges of the texture aren’t perfect, and the copyright message is visible. In the spaces where there are no vertices on the map, you can add whatever you want since it won’t show up on the model.

Note: It’s a good idea to not match the UV edges exactly, but instead to let the color bleed, as sometimes computers don’t accurately compute floating-point numbers.

You then import that image into Blender and assign it to the model to get the textured house that you saw above.

When you export a UV mapped model from Blender, Blender adds the UV coordinates to the file. Each vertex has a two-dimensional coordinate to place it on the 2D texture plane. The top-left is (0, 1) and the bottom-right is (1, 0).

The following diagram indicates some of the house vertices with some matching coordinates listed.

UV coordinates
UV coordinates

One of the advantages of mapping from 0 to 1 is that you can swap in lower or higher resolution textures. If you’re only viewing a model from a distance, you don’t need a highly detailed texture.

This house is easy to unwrap, but imagine how complex unwrapping curved surfaces might be. The following image shows a UV map of the train (which is still a simple model):

The train's UV map
The train's UV map

Photoshop, naturally, is not the only solution for texturing a model. You can use any image editor for painting on a flat texture. In the last few years, several other apps that allow painting directly on the model have become mainstream:

  • Blender (free)
  • Procreate on iPad ($)
  • Substance Designer and Substance Painter by Adobe ($$): In Designer, you can create complex materials procedurally. Using Substance Painter, you can paint these materials on the model.
  • 3DCoat by 3Dcoat.com ($$)
  • Mari by Foundry ($$$)

In addition to texturing, using Blender, 3DCoat or Nomad Sculpt on iPad, you can sculpt models in a similar fashion to ZBrush and then remesh the high poly sculpt to create a low poly model. As you’ll find out later, color is not the only texture you can paint using these apps, so having a specialized texturing app is invaluable.

The Starter App

➤ Open the starter project for this chapter, and build and run the app.

The starter app
Pga zdohvef ebn

1. Loading the Texture

A model typically has several submeshes that reference the same texture. Since you don’t want to repeatedly load this texture, you’ll create a central TextureController to hold your textures.

import MetalKit

enum TextureController {
  static var textures: [String: MTLTexture] = [:]
}

static func loadTexture(texture: MDLTexture, name: String) -> MTLTexture? {
  // 1
  if let texture = textures[name] {
    return texture
  }
  // 2
  let textureLoader = MTKTextureLoader(device: Renderer.device)
  // 3
  let textureLoaderOptions: [MTKTextureLoader.Option: Any] =
    [.origin: MTKTextureLoader.Origin.bottomLeft]
  // 4
  let texture = try? textureLoader.newTexture(
    texture: texture,
    options: textureLoaderOptions)
  print("loaded texture from USD file")
  // 5
  textures[name] = texture
  return texture
}

Loading the Submesh Texture

Each submesh of a model’s mesh has a different material characteristic, such as roughness, base color and metallic content. For now, you’ll focus only on the base color texture. In Chapter 11, “Maps & Materials”, you’ll look at some of the other characteristics. Conveniently, Model I/O loads a model complete with all the materials and textures. It’s your job to extract them from the loaded asset in a form that suits your engine.

asset.loadTextures()

struct Textures {
  var baseColor: MTLTexture?
}

var textures: Textures

// 1
private extension Submesh.Textures {
  init(material: MDLMaterial?) {
    baseColor = material?.texture(type: .baseColor)
  }
}

// 2
private extension MDLMaterialProperty {
  var textureName: String {
    stringValue ?? UUID().uuidString
  }
}

// 3
private extension MDLMaterial {
  func texture(type semantic: MDLMaterialSemantic) -> MTLTexture? {
    if let property = property(with: semantic),
       property.type == .texture,
       let mdlTexture = property.textureSamplerValue?.texture {
      return TextureController.loadTexture(
        texture: mdlTexture,
        name: property.textureName)
    }
    return nil
  }
}

textures = Textures(material: mdlSubmesh.material)
The render hasn't changed
Hbu loyjah gelm'h xbasxac

2. Passing the Loaded Texture to the Fragment Function

In a later chapter, you’ll learn about several other texture types and how to send them to the fragment function using different indices.

typedef enum {
  BaseColor = 0
} TextureIndices;

extension TextureIndices {
  var index: Int {
    return Int(self.rawValue)
  }
}

encoder.setFragmentTexture(
  submesh.textures.baseColor,
  index: BaseColor.index)

3. Updating the Fragment Function

➤ Open Fragment.metal, and add the following new argument to fragment_main, immediately after VertexOut in [[stage_in]],:

texture2d<float> baseColorTexture [[texture(BaseColor)]]

constexpr sampler textureSampler;

float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv).rgb;
return float4(baseColor, 1);
The textured house
Gho vezladog loebu

The Ground Plane

It’s time to add some ground to your scene. Instead of loading a USD model, you’ll create a ground plane using one of Model I/O’s primitive types, just as you did in the first chapters of this book.

lazy var ground: Model = {
  Model(name: "ground", primitiveType: .plane)
}()

ground.scale = 40
ground.rotation.z = Float(90).degreesToRadians
ground.rotation.y = sin(timer)
ground.render(
  encoder: renderEncoder,
  uniforms: uniforms,
  params: params)
The ground plane
Rsi hpeexj bnaqi

The Asset Catalog

When you write your full game, you’re likely to have many textures for the different models. If you use USD format models, the textures will generally be included. However, you may use different file formats that don’t hold textures, and organizing these textures can become labor-intensive. Plus, you’ll also want to compress images where you can and send textures of varying sizes and color gamuts to different devices. The asset catalog is where you’ll turn.

The grass texture
Vri snewx farpehi

static func loadTexture(name: String) -> MTLTexture? {
  // 1
  if let texture = textures[name] {
    return texture
  }
  // 2
  let textureLoader = MTKTextureLoader(device: Renderer.device)
  let texture: MTLTexture?
  texture = try? textureLoader.newTexture(
    name: name,
    scaleFactor: 1.0,
    bundle: Bundle.main,
    options: nil)
  // 3
  if texture != nil {
    print("loaded texture: \(name)")
    textures[name] = texture
  }
  return texture
}

extension Model {
  func setTexture(name: String, type: TextureIndices) {
    if let texture = TextureController.loadTexture(name: name) {
      switch type {
      case BaseColor:
        meshes[0].submeshes[0].textures.baseColor = texture
      default: break
      }
    }
  }
}

lazy var ground: Model = {
  let ground = Model(name: "ground", primitiveType: .plane)
  ground.setTexture(name: "grass", type: BaseColor)
  return ground
}()
Dark grass texture
Patc vjifl zekhowe

sRGB Color Space

The rendered texture looks much darker than the original image because ground.png is an sRGB texture. sRGB is a standard color format that compromises between how cathode ray tube monitors work and what colors the human eye sees. As you can see in the following example of grayscale values from 0 to 1, sRGB colors are not linear. Humans are more able to discern between lighter values than darker ones.

sRGBcolor = pow(linearColor, 1.0/2.2);

metalView.colorPixelFormat = .bgra8Unorm_srgb
View with sRGB color pixel format
Cuew sezw sYRL watix fepel lajhul

Capture GPU Workload

There’s an easy way to find out what format your texture is in on the GPU, and also to look at all the other Metal buffers currently residing there: the Capture GPU workload tool (also called the GPU Debugger).

A GPU trace
E JBE yhuku

Resources on the GPU
Metaowyoh av bve LKE

Texture info
Dilvigo alxu

Convert texture to data
Leqcaty pawdato da seto

Linear workflow
Dageid vegntzuw

Samplers

When sampling your texture in the fragment function, you used a default sampler. By changing sampler parameters, you can decide how your app reads your texels.

constexpr sampler textureSampler(filter::linear);
A smoothed texture
U njuandot mongeku

Filtering
Naffiyaxq

constexpr sampler textureSampler(
  filter::linear,
  address::repeat);
float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * 16).rgb;
The sampler address mode
Vje tanmhij apykeyh dega

Texture tiling
Bergeji yoqusw

uint tiling;

var tiling: UInt32 = 1

params.tiling = tiling

lazy var ground: Model = {
  let ground = Model(name: "ground", primitiveType: .plane)
  ground.setTexture(name: "grass", type: BaseColor)
  ground.tiling = 16
  return ground
}()

float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;
Corrected tiling
Jucwettuh wogakp

A moiré example
O zaijé ejevvji

Mipmaps

Check out the relative sizes of the roof texture and how it appears on the screen.

Size of texture compared to on-screen viewing
Qafe ov letpuce baxzawur qa ay-zqboid xeeluty

Mipmaps
Xurtipj

Mipmap comparison
Bozrek zodxaniric

let textureLoaderOptions: [MTKTextureLoader.Option: Any] =
  [.origin: MTKTextureLoader.Origin.bottomLeft,
   .generateMipmaps: true]

mip_filter::linear
Mipmaps added
Gaksent ewqih

House mipmaps
Qeayu woscezp

Asset Catalog Attributes

Perhaps you were surprised, since you only changed the USD texture loading method, to see that the ground render improved. The ground is a primitive plane, and you load its texture from the asset catalog.

Texture attributes in the asset catalog
Qamfopo eqvfamisur in two omhib heyobel

Mipmap slots
Nohhiv qvupp

The Right Texture for the Right Job

Using asset catalogs gives you complete control over how to deliver your textures. Currently, you only have one color texture for the grass. However, if you’re supporting a wide variety of devices with different capabilities, you’ll likely want to have specific textures for each circumstance. On devices with less RAM, you’d want smaller graphics.

Custom textures in the asset catalog
Jefgeg pabqulam ap zle alyon yayozuv

Anisotropy

Your rendered ground is looking a bit muddy and blurred in the background. This is due to anisotropy. Anisotropic surfaces change depending on the angle at which you view them, and when the GPU samples a texture projected at an oblique angle, it causes aliasing.

max_anisotropy(8)
Anisotropy
Uxinanrirc

Challenge

In the resources folder for this chapter, you’ll find two textures:

Barn textures
Toml nokxocog

Key Points

  • UVs, also known as texture coordinates, match vertices to the location in a texture.
  • During the modeling process, you flatten the model by marking seams. You can then paint on a texture that matches the flattened model map.
  • You can load textures from model files, the asset catalog, or with a bit of extra work, images held in the bundle.
  • A model may be split into groups of vertices known as submeshes. Each of these submeshes can reference one texture or multiple textures.
  • The fragment function reads from the texture using the model’s UV coordinates passed on from the vertex function.
  • The sRGB color space is the default color gamut. Modern Apple monitors and devices can extend their color space to P3 or wide color.
  • Capture GPU workload is a useful debugging tool. Use it regularly to inspect what’s happening on the GPU.
  • Mipmaps are resized textures that match the fragment sampling. If a fragment is a long way away, it will sample from a smaller mipmap texture.
  • Asset catalogs give you complete control of your textures without having to write cumbersome code. Customization for different devices is easy using the asset catalog.
  • Topics such as color and compression are huge. In the resources folder for this chapter, in references.markdown, you’ll find some recommended articles to read further.
7. The Fragment Function
9. Navigating a 3D Scene
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now