Difference between revisions of "Arcane University:DDS Data Format"

The Beyond Skyrim Wiki — Hosted by UESP
Jump to: navigation, search
(→‎Export in GIMP)
(dxt -> bc)
 
(33 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
=== The Texture Data Type Microsoft Direct Draw Surface (DDS) ===
+
Almost all textures used by Skyrim are DDS (Direct Draw Surface) files. This article contains information and advice on the different types of DDS files, and how to use them.
  
−
=== Texture Types and the DTX format ===
+
== Texture Types and the DXT/BC format ==
  
−
*'''What is the DDS file format?'''
+
===What is the DDS file format?===
−
**DXT is a set of compression algorithms or codecs applied to raster/bitmap images. The set is also known as S3 Texture Compression (S3TC). They all convert 4×4 blocks of pixels to either 64-bits or 128-bits depending on the codec. All are lossy algorithms. The original codecs were created by S3 Graphics, but many non-encumbered alternatives exist now. There are many different codecs: DXT1, DXT3, DXT5, etc. I’ll talk more about them in a moment. DDS (Direct Draw Surface) is an image file format, rather like a container for storing image data compressed using one of the DXT codecs. It was developed by Microsoft and introduced with DirectX 7. Together, a DDS file containing data organized using a DXT code creates an image file that can be used in most graphical applications, at least those that support DDS natively or via a plug-in. DDS files are very common in the game industry, where advantages in loading speed and video memory savings outweigh disadvantages.  
+
*Block Compression (BC), formerly known as DXT, is a set of compression algorithms or codecs applied to raster/bitmap images. The set is also known as S3 Texture Compression (S3TC). They all convert 4×4 blocks of pixels to either 64-bits or 128-bits depending on the codec. All are lossy algorithms. The original codecs were created by S3 Graphics, but many non-encumbered alternatives exist now. There are many different codecs: BC1, BC2, BC3, etc. I’ll talk more about them in a moment. DDS (Direct Draw Surface) is an image file format, rather like a container for storing image data compressed using one of the DXT codecs. It was developed by Microsoft and introduced with DirectX 7. Together, a DDS file containing data organized using Block Compression creates an image file that can be used in most graphical applications, at least those that support DDS natively or via a plug-in. DDS files are very common in the game industry, where advantages in loading speed and video memory savings outweigh disadvantages.  
  
−
*'''How does it work?'''
+
===How does it work?===
−
**DXT sees images as collections of 4×4 blocks of pixels called “texels”. For every texel, DXT selects two colors from the texel, each determining one end of a color range of 4 colors. The middle two colors are interpolated. The sixteen pixels of the texel are then assigned a 2-bit index (0-3) that maps them to the color range. The two representative colors are stored as 16-bit RGB values (5:6:5). So each texel requires 2×16 bits for the colors, plus 16×2 bits for the indices, giving a total of 64 bits for each texel, which equates to 4 bits per pixel. So for any set of images having the same dimensions, compressed size will always be the same.  
+
*Block Compression sees images as collections of 4×4 blocks of pixels called "texels". For every texel, the codec selects two colors from the texel, each determining one end of a color range of 4 colors. The middle two colors are interpolated. The sixteen pixels of the texel are then assigned a 2-bit index (0-3) that maps them to the color range. The two representative colors are stored as 16-bit RGB values (5:6:5). So each texel requires 2×16 bits for the colors, plus 16×2 bits for the indices, giving a total of 64 bits for each texel, which equates to 4 bits per pixel. So for any set of images having the same dimensions, compressed size will always be the same.  
  
−
*'''What if the image has an alpha channel?'''
+
===What if the image has an alpha channel?===
−
** How DXT handles this depends on the codec used. In DXT5, the alpha channel is encoded using a second set of 64 bits for each texel. DXT5 stores alpha information in a way that is almost the same as color information. Two alpha values are selected and *used as the extremes for a range of transparency values. The alpha values are represented by 8 bits each, and the range indices by 3 bits each, allowing for gradients of up to 8 shades. DXT3 handles the alpha channel a little differently. Each pixel gets 4 bits to represent its alpha, for a total of 16 unique values of transparency. This allows the alpha channel to be represented more accurately than DXT5, but with less subtle transitions. So, for an image with no alpha using DXT1, compression results in an image using 4 bpp (bits per pixel). For an image using DXT3 or DXT5 incorporating an alpha channel, the requirements will be 8 bpp. Note that the actual image size will likely be larger, as it will frequently include mipmap data.
+
* This depends on the codec used. In BC3, the alpha channel is encoded using a second set of 64 bits for each texel. BC3 stores alpha information in a way that is almost the same as color information. Two alpha values are selected and used as the extremes for a range of transparency values. The alpha values are represented by 8 bits each, and the range indices by 3 bits each, allowing for gradients of up to 8 shades. BC2 handles the alpha channel a little differently. Each pixel gets 4 bits to represent its alpha, for a total of 16 unique values of transparency. This allows the alpha channel to be represented more accurately than BC3, but with less subtle transitions. So, for an image with no alpha using DXT1, compression results in an image using 4 bpp (bits per pixel). For an image using BC2 or BC3 incorporating an alpha channel, the requirements will be 8 bpp. Note that the actual image size will likely be larger, as it will frequently include mipmap data.
  
−
*'''What are the advantages of DDS files and DXT compression?'''
+
===What are the advantages of DDS files and Block Compression?===
−
**Fast load times. DDS files are ready to be used by the graphics system and can be read straight into graphics memory with little overhead. In situations where many files are being constantly swapped in/out of the graphics unit, this can be a substantial savings and can reduce “lag”, especially with big texture files. Mipmaps can be pre-generated and included in the DDS files. This is another savings in load times and gives the graphic designer control over mipmap construction. More on mipmaps in a moment. Data remains compressed in video memory. All image formats except DDS/DXT are loaded into graphics memory in flat, uncompressed state. (And uncompressing them takes time and resources.) DDS/DXT files remain in their compressed state in video RAM, using special algorithms on the video card to retrieve data on demand. Compression ratio is 6:1 if no alpha channel is used, or 4:1 if an alpha channel is used. This can result in huge video memory savings.
+
*Fast load times. DDS files are ready to be used by the graphics system and can be read straight into graphics memory with little overhead. In situations where many files are being constantly swapped in/out of the graphics unit, this can be a substantial savings and can reduce "lag", especially with big texture files. Mipmaps can be pre-generated and included in the DDS files. This is another savings in load times and gives the graphic designer control over mipmap construction. More on mipmaps in a moment. Data remains compressed in video memory. All image formats except DDS are loaded into graphics memory in flat, uncompressed state. (And uncompressing them takes time and resources.) DDS files remain in their compressed state in video RAM, using special algorithms on the video card to retrieve data on demand. Compression ratio is 6:1 if no alpha channel is used, or 4:1 if an alpha channel is used. This can result in huge video memory savings.
  
−
*'''What are the disadvantages?'''
+
===What are the disadvantages?===
−
**DXT codecs are lossy. What is stored as compressed data is not the same as the original image, and on a fine level it may not even be close. Images with high contrast regions such as print or cartoon-like colors and borders will likely generate visible artifacts, particularly with smaller resolutions. For this reason DXT can be problematic when used with normal maps, though there are work-arounds. I’ll come back to this issue. Never use DDS files for editing and archiving. The compression degrades the original colors. It is not a good format where retention of the true color is critical, particularly in those situations where fine differences may have a large impact. DXT generates images using a 16 bit color depth. Where formats like JPG or PNG use 8 bits per channel per pixel (RGB 8:8:8) for a 24-bit color depth (32 bits with PNGs having an alpha channel), DXT reduces the spectrum down to 5:6:5 bits using an interpolation algorithm to arrive at the new color values. In short, you lose much of the original color range. But many graphic applications in games and sims don’t require tight control over the colors, and a good choice of the color palette can eliminate any visible results of this DXT effect. File size on disk can be large for DDS. For example, a 1024×1024 image with an alpha channel and mipmaps will result in a 1.37MB file. But file size is a relatively small consideration these days. Game/simulation performance matters far more.
+
*Block Compression codecs are lossy. What is stored as compressed data is not the same as the original image, and on a fine level it may not even be close. Images with high contrast regions such as print or cartoon-like colors and borders will likely generate visible artifacts, particularly with smaller resolutions. For this reason DDS can be problematic when used with normal maps, though there are work-arounds. I’ll come back to this issue. Never use DDS files for editing and archiving. The compression degrades the original colors. It is not a good format where retention of the true color is critical, particularly in those situations where fine differences may have a large impact. Block Compression generates images using a 16 bit color depth. Where formats like JPG or PNG use 8 bits per channel per pixel (RGB 8:8:8) for a 24-bit color depth (32 bits with PNGs having an alpha channel), Block Compression reduces the spectrum down to 5:6:5 bits using an interpolation algorithm to arrive at the new color values. In short, you lose much of the original color range. But many graphic applications in games and sims don't require tight control over the colors, and a good choice of the color palette can eliminate any visible results of this effect. File size on disk can be large for DDS. For example, a 1024×1024 image with an alpha channel and mipmaps will result in a 1.37MB file. But file size is a relatively small consideration these days. Game/simulation performance matters far more.
  
−
*''' Are there any image dimension limitations? '''
+
===Are there any image dimension limitations?===
−
**DDS/DXT images can be of any dimension expressed in powers of two up to the limits of your application or hardware, but no dimension can be less than 4. So 1024×512 is fine, but 2048×2 is not. Since DXT compression works with texels and each texel is a 4×4 block of pixels, it follows that DXT can’t work with an image with a dimension smaller than 4.
+
*DDS images can be of any dimension expressed in powers of two up to the limits of your application or hardware, but no dimension can be less than 4. So 1024×512 is fine, but 2048×2 is not. Since Block Compression works with texels and each texel is a 4×4 block of pixels, it follows that it can’t work with an image with a dimension smaller than 4.
  
−
*'''DDS and Mipmaps'''
+
==Mipmaps==
−
**An object seen up close needs a relatively high-resolution texture so that it doesn’t look pixelated. But an object seen in the distance needs much less resolution to look reasonable, and smaller resolutions require less resources. A mipmap is a set of pre-calculated versions of the same image progressively decreasing in size until the image reaches a dimension of 1×1. The idea is to have a handy set of smaller images that can be used to increase rendering speed and reduce aliasing effects while allowing the creator to optimize the appearance of the image at each stage. With mipmaps, the render engine can use the smaller, pre-processed version. This speeds things up significantly when you consider a scene of many objects at varying distances from the viewer. With most image file formats, the render engine must generate the mipmaps when the image is loaded. This takes time, and the render engine may not make the best choices for the appearance of the image. DDS files give you control over this process, allowing you to pre-generate mipmaps and store them ready-to-go along side the source image in the same DDS file. There is of course a penalty. Mipmaps take up space, increasing data size by 33% beyond that required to store the source image. This increases both size on disk and size in video memory. Unless your primary concern is file size, the benefits of mipmapping more than outweigh the costs.
+
An object seen up close needs a relatively high-resolution texture so that it doesn’t look pixelated. But an object seen in the distance needs much less resolution to look reasonable, and smaller resolutions require less resources. Mipmaps are lower resolution versions of the same image, progressively decreasing in size. These low-res images can then be used for objects that are far away. DDS files can have pre-generated mipmaps included, so that the renderer doesn't have to create them on the fly. The downside is the additional file size: this process increases data size by 33%, both size on disk and in video memory.
  
−
*'''DXT Flavors Egad! There are tons of DXT codecs! Which should I use?'''
+
A texture without mipmaps will not crash the game, but may cause significant drops in framerate, and will look "grainy" at a distance. All Skyrim textures should have mipmaps, except for facegen textures and UI art.
−
**DXT comes in a lot of flavors. Fortunately most are specialty applications. For common situations you’ll probably need to consider only three: DXT1, DXT3 and DXT5. Let’s briefly overview each one to get an idea when they might be used.
+
−
*** '''DXT1 (BC1)'''
+
−
**** RGB, 4 bits per pixel, no alpha or 1 bit (black or white) alpha
+
−
**** DXT1 is a fixed 8:1 compression ratio
+
−
***:If your image does not require an alpha channel, use the no-alpha DXT1. It uses the same compression algorithm as DXT3/5 for color data, and will get you half the file size. I’ve never used this one with an alpha channel. Sometimes referred to as BC1 (Block Compression 1).
+
−
*** '''DXT3 (BC2)'''
+
−
**** ARGB, 8 bits per pixel, explicit alpha
+
−
**** DXT3 is a fixed 4:1 compression ratio
+
−
***:DXT3’s method for storing alpha is better for files that have clearly delineated defined alpha regions and values. It may result in banding artifacts if used on images with smooth blended alpha regions– use DXT5 for these cases. I rarely use DXT3.
+
−
*** '''DXT5 (BC3)'''
+
−
**** ARGB, 8 bits per pixel, interpolated alpha
+
−
**** DXT5 is same 4:1 compression ratio as DXT3
+
−
***:DXT5 is the go-to codec for most images that include an alpha channel. The cost is double the file size of a DXT1 image. If you do not need an alpha channel (and I suggest that you not include an alpha channel unless you absolutely must), use DXT1.
+
  
−
'''DDS Files: editing and archiving'''
+
==There are tons of compression codecs! Which should I use?==
−
* DDS images are just regular images with or without an alpha channel. They can be created in GIMP or Photoshop or whatever you have. To export your image as a DDS file, you’ll need a plugin. NVIDIA offers a free tool for importing/exporting DDS files into Photoshop:
+
Block Compression (BC) comes in a lot of flavors. Fortunately most are specialty applications. For common situations you’ll probably need to consider only two: BC1 and BC3 (for Skyrim LE), or BC1 and BC7 (for SSE).
−
[https://developer.nvidia.com/nvidia-texture-tools-adobe-photoshop]
+
===BC1===
−
* A plugin for GIMP can be found here: [http://code.google.com/p/gimp-dds/]
+
Formerly known as DXT1, BC1 provides the smallest file size, at the cost of havinga 1-bit alpha channel. This can be useful for diffuse textures with no transparency, or that use alpha testing but no blending (this is sometimes called "cutout alpha").
−
Learning to use these plugins is beyond the scope of this article, but I have a few observations. It depends on the nature of the image.
+
* RGB, 4 bits per pixel, no alpha or 1 bit (black or white) alpha
−
* I’ve sometimes worked with images that simply refuse to export to DXT1.
+
* BC1 is a fixed 8:1 compression ratio
−
* Exporting to DXT5 tends to be much faster and less problematic.
+
===BC3===
−
* Don’t include an alpha channel if you don’t absolutely need one. An alpha channel will double the file size of your DDS file and significantly add to game or simulation video processing workload. I commonly find images created by others that include an unnecessary, fully-transparent alpha channel. Don’t do that.
+
Formerly known as DXT5, BC3 was the go-to codec for most images that include an alpha channel in Skyrim LE (diffuse with transparency, and normal maps with specular). The cost is double the file size of a BC1 image. If compressing textures for Skyrim SE, use BC7 instead.
−
* Since DDS files are lossy and rather badly so, you should not use them as an archive format. Don’t edit DDS files if there is any possibility of using a non-lossy source, especially if the DDS file is serving as a normal map. Always begin your edits by opening a file saved in a non-lossy format. If you must edit a DDS file as your only option, first save the file in a non-lossy format, thus preserving at least that version as a standard, and create all subsequent versions from that source. If you do not, subsequent saves and editing will quickly corrupt the image into something unusable.
+
* ARGB, 8 bits per pixel, interpolated alpha
 +
* BC3 is a fixed 4:1 compression ratio
 +
===BC4===
 +
BC4 is a single-channel compression codec. It is ideal for textures like environment masks that contain only one color channel. Does not work in Skyrim LE.
 +
* grayscale, 8 bits per pixel, no alpha
 +
* BC4 is a fixed 2:1 compression ratio
 +
===BC7===
 +
BC7 is a more recent compression algorithm that is supported by Skyrim SE, but not LE. BC7 is preferred over BC3 in nearly all cases, because it is the same size but at a higher quality. BC1 is still preferred for textures without alpha channels, or 1-bit ("cutout") alpha, since it is much smaller. When exporting as BC7, always use the slowest compression algorithm for the highest quality.
 +
* RGB or RGBA, 8 bits per pixel, interpolated alpha
 +
* BC7 is same 4:1 compression ratio as BC3 and BC5
  
−
'''DXT and Normal Maps'''
+
{{note|Images that are already compressed in .dds format cannot be increased in quality by reformatting them, unless you use the original ''uncompressed'' texture as a base.}}
 +
 
 +
==Converting to DDS file format==
 +
In order to convert your .png/tga textures to the Skyrim friendly DDS format, you must have the [[Arcane_University:3D_Art#Convert_Maps_to_DDS|necessary software]].
 +
 
 +
Second, you must make sure your textures conform to the [[Arcane_University:NIF_Data_Format#Texture_Slots|NIF Texture Slots]]; that means you must have at the very least a diffuse map and a normal map. How you achieve this differs depending on your workflow and texturing program.
 +
 
 +
===Automatically Combine and Export Maps===
 +
If you use '''Substance Painter''' you should download the [https://www.nexusmods.com/skyrimspecialedition/mods/44400 Substance Painter to Skyrim] plugin. This plugin mimics Skyrim's renderer as close as possible, and allows you to easily export the textures in the correct maps. So instead of exporting separate images for base color, height, metallic, normal and roughness textures, the plugin combines these into the PNG/TGA diffuse, normal, glow and environment maps for you.
 +
 
 +
===Manually Combining Textures===
 +
If you work in '''Quixel Mixer''' or '''Blender''', a plugin such as the one for Substance Painter does not exist (yet). That means you must export specific textures and combine them in a [[AU:3D_Art#2D_texturing_program|2D software]] such as Gimp or Photoshop. It's important that the 2D software has the Layer Modes feature (such as Multiply) as we'll need that to combine our textures.
 +
 
 +
[[File:Manually_Combining_Maps_for_DDS_Diffuse.png|thumb|600px|Albedo, AO and Curvature Map combined to create a diffuse.]]
 +
====Textures for the Diffuse====
 +
*Albedo/Base Color (Texturing)
 +
*Ambient Occlusion (AO) (Baking)
 +
*Curvature (Baking)
 +
 
 +
In your conversion software, you place Base Color (albedo) as bottom solid layer - no layer modes needed. Then place the Ambient Occlusion image on 100% Multiply Layer Mode above the Base Color Layer. And finally, place the Curvature image on 40% Soft Light above AO and Base Colour.
 +
 
 +
Tada! Your diffuse is now ready to be exported as DDS.
 +
 
 +
====Textures for the Normal Map====
 +
*Normal Map (Baking)
 +
*Specular (Texturing)
 +
 
 +
Your normal map is a bit different than the diffuse, as we will place the specular map in the alpha channel.
 +
 
 +
 
 +
'''Photoshop Instructions'''
 +
*Step 1: Open normal map
 +
*Step 2: Import specular map or inverted roughness map
 +
*Step 3: Select (ctrl+a) and copy (ctrl+c) specular map
 +
*Step 4: Create the alpha channel
 +
*Step 5: Paste specular onto alpha channel
 +
*Step 6: Delete specular map layer and enable all channels
 +
 
 +
Confused? [https://wiki.beyondskyrim.org/w/images/e/ed/Specular_to_Normal_Alpha_Channel.gif Watch this video].
 +
 
 +
 
 +
'''GIMP Instructions'''
 +
 
 +
Visit our [https://wiki.beyondskyrim.org/wiki/Arcane_University:Gimp_for_Skyrim GIMP for Skyrim] wiki page for instructions.
 +
 
 +
===Converting PNG to DDS===
 +
Now that you know how to export textures and combine the maps, it's time to convert them to DDS file format. Regardless of software, this is a very straightforward process. Most meshes must always have a diffuse and a normal map, but there are additional optional maps you can generate should your asset benefit from it.
 +
 
 +
 
 +
<center>'''Simply open the PNG/TGA of your diffuse, normal map, environment, and/or glow map in the software, and File > Save/Export As > DDS.'''</center>
 +
 
 +
<center>{{note|'''Always generate mipmaps.'''}}</center>
 +
 
 +
 
 +
For information on what BC/DXT to choose, read more above in [[Arcane_University:DDS_Data_Format#There_are_tons_of_DXT_codecs.21_Which_should_I_use.3F|What to Choose?]] or on the [[AU:NIF_Data_Format|NIF]] page. On the NIF page you'll also find information on where and how to attach the DDS textures to your NIF, and naming conventions for your maps. PS: This is the unique asset workflow. If you're using [https://www.youtube.com/watch?v=uUJShalzWy8 trim sheets] you might have to use a different workflow.
 +
 
 +
Although the export process is similar for most software, each one uses a different compressor. Some result in better quality than others. These are some exporters, ordered from highest quality to lowest, with a brief summary of their advantages and disadvantages:
 +
 
 +
{| class="wikitable"
 +
|+ <span id="Comparison of DDS Exporters"></span>Comparison of DDS Exporters
 +
|-
 +
| 1. [https://developer.nvidia.com/nvidia-texture-tools-exporter '''Nvidia Texture Tools (NVTT) (Standalone and/or Photoshop Plugin)''']
 +
| These first two are tied in terms of quality. NVTT does require an Nvidia graphics card, but it has a nice GUI and outputs excellent quality. It comes as either a standalone application, which supports command-line use and batch scripts, or a Photoshop plugin.
 +
|-
 +
| 1. [https://github.com/Unity-Technologies/crunch/tree/unity '''Crunch''']
 +
| Crunch is designed for more advanced compression, and is a command-line tool, but it also outputs excellent quality DDS when configured properly.
 +
|-
 +
| 2. [https://gpuopen.com/compressonator/ '''Compressonator''']
 +
| Requires an AMD graphics card, but outputs good quality and has a nice GUI. Also supports command-line use.
 +
|-
 +
| 3. [https://www.getpaint.net/ '''Paint.NET''']
 +
| Very easy to use, supports all DDS formats. Quality is middle-of-the-road, but usually fine for Skyrim's purposes. It can be used with any hardware, including integrated graphics, and does not rely on proprietary software like Photoshop.
 +
|-
 +
| 4. [https://software.intel.com/content/www/us/en/develop/articles/intel-texture-works-plugin.html '''Intel Texture Works Plugin for Photoshop''']
 +
| Has support for all DDS formats, but suffers from discoloration and blocky gradients.
 +
|-
 +
| 5. [https://www.gimp.org '''GIMP''']
 +
| Free, does not require special graphics cards or proprietary software, but has inferior results and lacks support for BC7 entirely.
 +
|}
 +
 
 +
For more detailed information, see [https://w3dhub.com/forum/topic/417101-dds-files-and-dxt-compression/?tab=comments#comment-671198 this post].
 +
 
 +
=== DXT and Normal Maps ===
 
A normal map stores information that allows lighting calculations to be done at the per-pixel level rather than interpolating between vertices. This can make lighting much smoother and more graduated, or it can make significant changes how lighting affects a given point. The common use of a normal map is to tweak surface data to give the illusion of having extra detail. It’s not real detail– it doesn’t actually change the geometry, but it does change the lighting and can give very realistic results without adding polygonal complexity.
 
A normal map stores information that allows lighting calculations to be done at the per-pixel level rather than interpolating between vertices. This can make lighting much smoother and more graduated, or it can make significant changes how lighting affects a given point. The common use of a normal map is to tweak surface data to give the illusion of having extra detail. It’s not real detail– it doesn’t actually change the geometry, but it does change the lighting and can give very realistic results without adding polygonal complexity.
−
Normal map data is essentially a normal vector for each pixel. The normal gives an indication of the “facing” of the pixel. Normal maps use the RGB color space to store the data needed for the effect– the 24 bits for each pixel are used to encode the pixel’s vector. Given that a color in this context is really a vector, changing a color can significantly alter the scale and direction of that vector, thus altering the effects at any given location. Since DXT compression does not exactly preserve the colors, DXT images can make a real mess of normal maps.
+
Normal map data is essentially a normal vector for each pixel. The normal gives an indication of the "facing" of the pixel. Normal maps use the RGB color space to store the data needed for the effect– the 24 bits for each pixel are used to encode the pixel's vector. Given that a color in this context is really a vector, changing a color can significantly alter the scale and direction of that vector, thus altering the effects at any given location. Since DXT compression does not exactly preserve the colors, DXT images can make a real mess of normal maps.
  
−
You can still use DXT compression for normal maps. In fact it’s common to do so in games and sims, where resolutions are fairly low and resources come at a premium. Just be aware that the end result is likely to be rather mangled. For low-res situations it’s usually not a big problem. But consider saving the normal map in a non-lossy format. You’ll lose the load-time and video RAM advantages of DDS, but you can compensate by going to a quarter-size resolution. In general for normal maps, you’ll get better appearance with a non-lossy format one-quarter the resolution of the DXT image. For example, a 512×512 PNG normal map will likely look better than a 1024×1024 DDS/DXT normal map.
+
You can still use DXT compression for normal maps. In fact it’s common to do so in games and sims, where resolutions are fairly low and resources come at a premium. Just be aware that the end result is likely to be rather mangled. For low-res situations it's usually not a big problem. Using BC3 or BC7 compression mitigates this somewhat. Using uncompressed textures is not recommended.
  
−
== Export in GIMP ==
+
== See Also ==
 +
*[https://www.reedbeta.com/blog/understanding-bcn-texture-compression-formats Understanding BCn Texture Compression Formats] by Nathan Reed
 +
*[https://w3dhub.com/forum/topic/417101-dds-files-and-dxt-compression/?tab=comments#comment-671198 Comparison of different DDS compressors]
 +
*[[wikipedia:DirectDraw Surface|DirectDraw Surface]] on Wikipedia
 +
*[[wikipedia:S3 Texture Compression|S3 Texture Compression]] on Wikipedia
 +
*[[Arcane University:NIF Data Format]]
  
−
* Exporting DDS texture files in Gimp via free Export Plugin with mipmaps for LOD and without an alpha map, so only Red Green Blue, also used for mono-color ("grey") maps and most commonly, diffuse maps (see texture sets, Arcane_University:Nif_Data_Format.
 
−
* [[File:Gimp1.PNG]]
 
−
* [[File:Gimp2.PNG]]
 
−
*
 
  
−
[[Category:Arcane_University]]
+
[[Category:Arcane University-3D Art]]
 
[[Category:Data_Format]]
 
[[Category:Data_Format]]

Latest revision as of 23:44, 8 September 2023

Almost all textures used by Skyrim are DDS (Direct Draw Surface) files. This article contains information and advice on the different types of DDS files, and how to use them.

Texture Types and the DXT/BC format[edit]

What is the DDS file format?[edit]

  • Block Compression (BC), formerly known as DXT, is a set of compression algorithms or codecs applied to raster/bitmap images. The set is also known as S3 Texture Compression (S3TC). They all convert 4×4 blocks of pixels to either 64-bits or 128-bits depending on the codec. All are lossy algorithms. The original codecs were created by S3 Graphics, but many non-encumbered alternatives exist now. There are many different codecs: BC1, BC2, BC3, etc. I’ll talk more about them in a moment. DDS (Direct Draw Surface) is an image file format, rather like a container for storing image data compressed using one of the DXT codecs. It was developed by Microsoft and introduced with DirectX 7. Together, a DDS file containing data organized using Block Compression creates an image file that can be used in most graphical applications, at least those that support DDS natively or via a plug-in. DDS files are very common in the game industry, where advantages in loading speed and video memory savings outweigh disadvantages.

How does it work?[edit]

  • Block Compression sees images as collections of 4×4 blocks of pixels called "texels". For every texel, the codec selects two colors from the texel, each determining one end of a color range of 4 colors. The middle two colors are interpolated. The sixteen pixels of the texel are then assigned a 2-bit index (0-3) that maps them to the color range. The two representative colors are stored as 16-bit RGB values (5:6:5). So each texel requires 2×16 bits for the colors, plus 16×2 bits for the indices, giving a total of 64 bits for each texel, which equates to 4 bits per pixel. So for any set of images having the same dimensions, compressed size will always be the same.

What if the image has an alpha channel?[edit]

  • This depends on the codec used. In BC3, the alpha channel is encoded using a second set of 64 bits for each texel. BC3 stores alpha information in a way that is almost the same as color information. Two alpha values are selected and used as the extremes for a range of transparency values. The alpha values are represented by 8 bits each, and the range indices by 3 bits each, allowing for gradients of up to 8 shades. BC2 handles the alpha channel a little differently. Each pixel gets 4 bits to represent its alpha, for a total of 16 unique values of transparency. This allows the alpha channel to be represented more accurately than BC3, but with less subtle transitions. So, for an image with no alpha using DXT1, compression results in an image using 4 bpp (bits per pixel). For an image using BC2 or BC3 incorporating an alpha channel, the requirements will be 8 bpp. Note that the actual image size will likely be larger, as it will frequently include mipmap data.

What are the advantages of DDS files and Block Compression?[edit]

  • Fast load times. DDS files are ready to be used by the graphics system and can be read straight into graphics memory with little overhead. In situations where many files are being constantly swapped in/out of the graphics unit, this can be a substantial savings and can reduce "lag", especially with big texture files. Mipmaps can be pre-generated and included in the DDS files. This is another savings in load times and gives the graphic designer control over mipmap construction. More on mipmaps in a moment. Data remains compressed in video memory. All image formats except DDS are loaded into graphics memory in flat, uncompressed state. (And uncompressing them takes time and resources.) DDS files remain in their compressed state in video RAM, using special algorithms on the video card to retrieve data on demand. Compression ratio is 6:1 if no alpha channel is used, or 4:1 if an alpha channel is used. This can result in huge video memory savings.

What are the disadvantages?[edit]

  • Block Compression codecs are lossy. What is stored as compressed data is not the same as the original image, and on a fine level it may not even be close. Images with high contrast regions such as print or cartoon-like colors and borders will likely generate visible artifacts, particularly with smaller resolutions. For this reason DDS can be problematic when used with normal maps, though there are work-arounds. I’ll come back to this issue. Never use DDS files for editing and archiving. The compression degrades the original colors. It is not a good format where retention of the true color is critical, particularly in those situations where fine differences may have a large impact. Block Compression generates images using a 16 bit color depth. Where formats like JPG or PNG use 8 bits per channel per pixel (RGB 8:8:8) for a 24-bit color depth (32 bits with PNGs having an alpha channel), Block Compression reduces the spectrum down to 5:6:5 bits using an interpolation algorithm to arrive at the new color values. In short, you lose much of the original color range. But many graphic applications in games and sims don't require tight control over the colors, and a good choice of the color palette can eliminate any visible results of this effect. File size on disk can be large for DDS. For example, a 1024×1024 image with an alpha channel and mipmaps will result in a 1.37MB file. But file size is a relatively small consideration these days. Game/simulation performance matters far more.

Are there any image dimension limitations?[edit]

  • DDS images can be of any dimension expressed in powers of two up to the limits of your application or hardware, but no dimension can be less than 4. So 1024×512 is fine, but 2048×2 is not. Since Block Compression works with texels and each texel is a 4×4 block of pixels, it follows that it can’t work with an image with a dimension smaller than 4.

Mipmaps[edit]

An object seen up close needs a relatively high-resolution texture so that it doesn’t look pixelated. But an object seen in the distance needs much less resolution to look reasonable, and smaller resolutions require less resources. Mipmaps are lower resolution versions of the same image, progressively decreasing in size. These low-res images can then be used for objects that are far away. DDS files can have pre-generated mipmaps included, so that the renderer doesn't have to create them on the fly. The downside is the additional file size: this process increases data size by 33%, both size on disk and in video memory.

A texture without mipmaps will not crash the game, but may cause significant drops in framerate, and will look "grainy" at a distance. All Skyrim textures should have mipmaps, except for facegen textures and UI art.

There are tons of compression codecs! Which should I use?[edit]

Block Compression (BC) comes in a lot of flavors. Fortunately most are specialty applications. For common situations you’ll probably need to consider only two: BC1 and BC3 (for Skyrim LE), or BC1 and BC7 (for SSE).

BC1[edit]

Formerly known as DXT1, BC1 provides the smallest file size, at the cost of havinga 1-bit alpha channel. This can be useful for diffuse textures with no transparency, or that use alpha testing but no blending (this is sometimes called "cutout alpha").

  • RGB, 4 bits per pixel, no alpha or 1 bit (black or white) alpha
  • BC1 is a fixed 8:1 compression ratio

BC3[edit]

Formerly known as DXT5, BC3 was the go-to codec for most images that include an alpha channel in Skyrim LE (diffuse with transparency, and normal maps with specular). The cost is double the file size of a BC1 image. If compressing textures for Skyrim SE, use BC7 instead.

  • ARGB, 8 bits per pixel, interpolated alpha
  • BC3 is a fixed 4:1 compression ratio

BC4[edit]

BC4 is a single-channel compression codec. It is ideal for textures like environment masks that contain only one color channel. Does not work in Skyrim LE.

  • grayscale, 8 bits per pixel, no alpha
  • BC4 is a fixed 2:1 compression ratio

BC7[edit]

BC7 is a more recent compression algorithm that is supported by Skyrim SE, but not LE. BC7 is preferred over BC3 in nearly all cases, because it is the same size but at a higher quality. BC1 is still preferred for textures without alpha channels, or 1-bit ("cutout") alpha, since it is much smaller. When exporting as BC7, always use the slowest compression algorithm for the highest quality.

  • RGB or RGBA, 8 bits per pixel, interpolated alpha
  • BC7 is same 4:1 compression ratio as BC3 and BC5
Ambox important.png NOTE: Images that are already compressed in .dds format cannot be increased in quality by reformatting them, unless you use the original uncompressed texture as a base.

Converting to DDS file format[edit]

In order to convert your .png/tga textures to the Skyrim friendly DDS format, you must have the necessary software.

Second, you must make sure your textures conform to the NIF Texture Slots; that means you must have at the very least a diffuse map and a normal map. How you achieve this differs depending on your workflow and texturing program.

Automatically Combine and Export Maps[edit]

If you use Substance Painter you should download the Substance Painter to Skyrim plugin. This plugin mimics Skyrim's renderer as close as possible, and allows you to easily export the textures in the correct maps. So instead of exporting separate images for base color, height, metallic, normal and roughness textures, the plugin combines these into the PNG/TGA diffuse, normal, glow and environment maps for you.

Manually Combining Textures[edit]

If you work in Quixel Mixer or Blender, a plugin such as the one for Substance Painter does not exist (yet). That means you must export specific textures and combine them in a 2D software such as Gimp or Photoshop. It's important that the 2D software has the Layer Modes feature (such as Multiply) as we'll need that to combine our textures.

Albedo, AO and Curvature Map combined to create a diffuse.

Textures for the Diffuse[edit]

  • Albedo/Base Color (Texturing)
  • Ambient Occlusion (AO) (Baking)
  • Curvature (Baking)

In your conversion software, you place Base Color (albedo) as bottom solid layer - no layer modes needed. Then place the Ambient Occlusion image on 100% Multiply Layer Mode above the Base Color Layer. And finally, place the Curvature image on 40% Soft Light above AO and Base Colour.

Tada! Your diffuse is now ready to be exported as DDS.

Textures for the Normal Map[edit]

  • Normal Map (Baking)
  • Specular (Texturing)

Your normal map is a bit different than the diffuse, as we will place the specular map in the alpha channel.


Photoshop Instructions

  • Step 1: Open normal map
  • Step 2: Import specular map or inverted roughness map
  • Step 3: Select (ctrl+a) and copy (ctrl+c) specular map
  • Step 4: Create the alpha channel
  • Step 5: Paste specular onto alpha channel
  • Step 6: Delete specular map layer and enable all channels

Confused? Watch this video.


GIMP Instructions

Visit our GIMP for Skyrim wiki page for instructions.

Converting PNG to DDS[edit]

Now that you know how to export textures and combine the maps, it's time to convert them to DDS file format. Regardless of software, this is a very straightforward process. Most meshes must always have a diffuse and a normal map, but there are additional optional maps you can generate should your asset benefit from it.


Simply open the PNG/TGA of your diffuse, normal map, environment, and/or glow map in the software, and File > Save/Export As > DDS.
Ambox important.png NOTE: Always generate mipmaps.


For information on what BC/DXT to choose, read more above in What to Choose? or on the NIF page. On the NIF page you'll also find information on where and how to attach the DDS textures to your NIF, and naming conventions for your maps. PS: This is the unique asset workflow. If you're using trim sheets you might have to use a different workflow.

Although the export process is similar for most software, each one uses a different compressor. Some result in better quality than others. These are some exporters, ordered from highest quality to lowest, with a brief summary of their advantages and disadvantages:

Comparison of DDS Exporters
1. Nvidia Texture Tools (NVTT) (Standalone and/or Photoshop Plugin) These first two are tied in terms of quality. NVTT does require an Nvidia graphics card, but it has a nice GUI and outputs excellent quality. It comes as either a standalone application, which supports command-line use and batch scripts, or a Photoshop plugin.
1. Crunch Crunch is designed for more advanced compression, and is a command-line tool, but it also outputs excellent quality DDS when configured properly.
2. Compressonator Requires an AMD graphics card, but outputs good quality and has a nice GUI. Also supports command-line use.
3. Paint.NET Very easy to use, supports all DDS formats. Quality is middle-of-the-road, but usually fine for Skyrim's purposes. It can be used with any hardware, including integrated graphics, and does not rely on proprietary software like Photoshop.
4. Intel Texture Works Plugin for Photoshop Has support for all DDS formats, but suffers from discoloration and blocky gradients.
5. GIMP Free, does not require special graphics cards or proprietary software, but has inferior results and lacks support for BC7 entirely.

For more detailed information, see this post.

DXT and Normal Maps[edit]

A normal map stores information that allows lighting calculations to be done at the per-pixel level rather than interpolating between vertices. This can make lighting much smoother and more graduated, or it can make significant changes how lighting affects a given point. The common use of a normal map is to tweak surface data to give the illusion of having extra detail. It’s not real detail– it doesn’t actually change the geometry, but it does change the lighting and can give very realistic results without adding polygonal complexity. Normal map data is essentially a normal vector for each pixel. The normal gives an indication of the "facing" of the pixel. Normal maps use the RGB color space to store the data needed for the effect– the 24 bits for each pixel are used to encode the pixel's vector. Given that a color in this context is really a vector, changing a color can significantly alter the scale and direction of that vector, thus altering the effects at any given location. Since DXT compression does not exactly preserve the colors, DXT images can make a real mess of normal maps.

You can still use DXT compression for normal maps. In fact it’s common to do so in games and sims, where resolutions are fairly low and resources come at a premium. Just be aware that the end result is likely to be rather mangled. For low-res situations it's usually not a big problem. Using BC3 or BC7 compression mitigates this somewhat. Using uncompressed textures is not recommended.

See Also[edit]

Retrieved from "https://wiki.beyondskyrim.org/w/index.php?title=Arcane_University:DDS_Data_Format&oldid=4751"