How to limit the number of mipmaps, or limit their width / height not to be lower than 2?

Developer Tools Other Tools Texture Tools
1

Hi. I’m modding and old RTS game, and I’m attempting to convert .dds textures to the format that the game understands (and works faster with it, possibly because of pre-defined header).

The problem is, that this game doesn’t like when a texture or mipmaps’s width/height is different than 2, 4, 8, 16, 32, 64, 128, 256, 1024, 2048, etc.

I tried to convert a .tga texture: 1024x128. Unfortunately, nvtt_export created 11 mipmaps, where 7th mipmap is 8x1 and causes the game crash:

mip_maps
mip_maps423×862 14.5 KB

I’m converting textures in batch with options --quality production --mip-filter kaiser -f bc2.

Is there any solution for it, to only allow mipmaps to keep the original aspect ratio, limit at 2, or limit their number?

2

I solved the problem :)

I didn’t know that there’s NVIDIA Texture Tools Exporter and NVIDIA Texture Tools (separate installers).
The NVIDIA Texture Tools has nvcompress.exe, so I could call it with arguments such as:

-bc2 -highest -mipfilter kaiser -min-mip-size 2

And it works perfectly fine.

1 Like
3

Glad to hear you found the solution!

(P.S. We include nvcompress in both the Exporter installer and in the Linux NVTT-only installers – so for anyone reading this in the future, if you have the Exporter or NVTT, you likely also have nvcompress. Nvcompress’ source code is also available at https://github.com/nvpro-samples/nvtt_samples/blob/main/compress/compress.cpp!)

Thanks,

–Neil

1 Like
4

Thank you for the reply.

Just one more thing I noticed. The nvcompress.exe doesn’t support batch processing like nvtt_export.exe.

In nvtt_export.exe it’s easy to convert all textures and utilize 100% GPU/GPU with option:

-b,–batch,–batch-file TEXT
Runs a series of arguments from a batch file (ignores all other flags)

I wonder if this could be added to nvcompress.exe as well?

In the source code I noticed there are a few mentions of FileList. Is this option already supported, just text is missing in -help?

EDIT:
Ahhh! It accepts directories as input, so I just need to set the input/output directory and it gets done itself. Only at the end I need to rename files by the .bat scripts, but that’s not a big deal.

EDIT:
Nope. Converting entire directories gives the input I/O errors and creates empty (few bytes) files :( I have to loop it one-by-one.

5

Hi @Dr.Mona - could you try a command line like this? This seems to work for me:

"C:\Program Files\NVIDIA Corporation\NVIDIA Texture Tools\nvcompress.exe" -bc2 -highest -mipfilter kaiser -min-mip-size 2 path/to/exampleInputFolder path/to/exampleOutputFolder

You should see output like this:

NVIDIA Texture Tools 3.2.3 - Copyright NVIDIA Corporation 2007 - 2023
CUDA acceleration ENABLED. nvtt::Context() time: 0.157 seconds

Compressing the following files:
tests\2K Cube Map.png

Compressing the following files:
tests\4K Cube Map.png
tests\Cubemap_cloudy_text_cube.png
tests\pebbles-nvtt2-div4.png
tests\pebbles-nvtt2.png

Total processing time: 10.394 seconds
6

I can confirm that sometimes it works when filenames are short (I think). It’s most likely some path length limit:

@echo off
setlocal enabledelayedexpansion
cd /d "%~dp0"
set "base_path=%~dp0"

"E:\Program Files\NVIDIA Corporation\NVIDIA Texture Tools\nvcompress.exe" -bc2 -highest -mipfilter kaiser -min-mip-size 2 "E:\DESKTOP\tgaaa\001_Enhance Big UI Textures\004_Enhanced_Upscaled_PNG\Topaz_Low_Res" "E:\DESKTOP\TestOutputFolder" > nvcompress_LOG.txt

This path length was “only” 146, so not even close to MAX_PATH.

nvcompress_LOG.txt:

NVIDIA Texture Tools 3.2.3 - Copyright NVIDIA Corporation 2007 - 2023
CUDA acceleration ENABLED. nvtt::Context() time: 0.100 seconds

NVTT Error: I/O error opening the file at E:\DESKTOP\tgaaa\001_Enhance Big UI Textures\004_Enhanced_Upscaled_PNG\Topaz_Low_Res\icon_formation_convoy_stone-gigapixel-low_res-scale-4_00x.png

The output becomes 509 files with a gigantic size of 63,6 KB (65 152 bytes).

In this example I used the absolute path, but it’s exactly the same effect when using a relative path.

Generally, I workarounded it by using my C++ program that accepts arguments and does other operations for me. It calls nvcompress.exe in loop, using threads pool, so I utilize 100% of CPU. With C++20 it’s very easy to set up multithreading. Additionally I also check if the input texture size is valid for the game, auto-rename the output file, etc.

EDIT:
Same thing with a shorter path:

"E:\Program Files\NVIDIA Corporation\NVIDIA Texture Tools\nvcompress.exe" -bc2 -highest -mipfilter kaiser -min-mip-size 2 "E:\DESKTOP\input" "E:\DESKTOP\output" > nvcompress_LOG.txt

NVTT Error: I/O error opening the file at E:\DESKTOP\input\icon_formation_convoy_stone-gigapixel-low_res-scale-4_00x.png

And same thing with just:

"E:\Program Files\NVIDIA Corporation\NVIDIA Texture Tools\nvcompress.exe" -bc2 -highest -mipfilter kaiser -min-mip-size 2 "input" "output" > nvcompress_LOG.txt

So most likely filenames.

I uploaded this PNG in case you wanted to give it a try:

file.zip (162.2 KB)

EDIT: I just noticed that a new version has been released. However, the same issue persists.

7

That’s interesting! This still seems to work for me:

"C:\Program Files\NVIDIA Corporation\NVIDIA Texture Tools\nvcompress.exe" -bc2 -highest -mipfilter kaiser -min-mip-size 2 "file" "file-out"
NVIDIA Texture Tools 3.2.4 - Copyright NVIDIA Corporation 2007 - 2023
CUDA acceleration ENABLED. nvtt::Context() time: 0.176 seconds

Compressing the following files:
file\icon_formation_convoy_stone-gigapixel-low_res-scale-4_00x.png

Total processing time: 0.202 seconds

So, the PNG seems to be OK. (In C++ terms, I/O error opening the file at ... means that calling fopen_s(filename, "rb") failed). I tried a few variants (paths with spaces, “\0”, and so on); characters outside the ANSI code page made it fail, but all the characters in the command line above are in ASCII. Do you know if there’s any chance your system might be configured with a code page where characters in ASCII might be mapped to multi-byte sequences, say? If so, I’ll have a go at testing that out.

Thanks! (P.S. if you have a custom C++ program, you can use the source code of nvcompress directly – it’s licensed under Apache 2.0 – and that’ll probably speed up your program a bit since you won’t have to start a process and initialize CUDA for each image!)

Thanks,

–Neil

8

Okay… It seems like the problem is something else.

It works fine with 1 file in folder, or even with 240 of them.

However, I have > 2600 files. I noticed that the output folder gets exactly 509 empty files created.

When I have 509 files in the input folder, it still manages to convert correctly. Just 1 file more (510) causes the crash :D

I suppose it’s related to some max file handles on Windows. Nvcompress seem to be creating all files first, before starting compression: for(; i < FileList.size(); i++).

I think the solution would be an additional loop with max ~500 files (probably less better).

Or maybe:
https://learn.microsoft.com/en-us/cpp/c-runtime-library/file-handling?view=msvc-170
“The C run-time libraries have a 512 limit for the number of files that can be open at any one time. Attempting to open more than the maximum number of file descriptors or file streams causes program failure. Use _setmaxstdio to change this number.”

Thanks! (P.S. if you have a custom C++ program, you can use the source code of nvcompress directly – it’s licensed under Apache 2.0 – and that’ll probably speed up your program a bit since you won’t have to start a process and initialize CUDA for each image!)

Thanks! I thought it depends on some additional libraries which would complicate compilation, but maybe I can give it a try.

9

Ah! And the 3 other open files would be stdout, stderr, and stdin for a total of 509 + 3 == 512.

Thanks for finding this bug! I’ll have a go at fixing it; you should see the fix first on GitHub - nvpro-samples/nvtt_samples: NVIDIA Texture Tools samples for compression, image processing, and decompression., and then in the next NVTT version.

Thanks!

–Neil

1 Like
10

I might have detected another bug, opposite to what I asked in the topic title.

It seems like I’m unable to convert images to have 1x1 mipmaps now, from 256x256 tga file.

I tried using many configuration options, such as:

-bc1

-bc1 -min-mip-size 1

-bc1 -max-mip-count 11

tga.zip (124.0 KB)

Every time, at the end the file only has 7 (well, 8 counting from 0) mipmaps, where the last one is 2x2.

Maybe it’s something with the tga file itself, but I thought it’s good to verify.

11

Ah, thanks for pointing that out! It looks like there’s an off-by-one error here. I’ve made an internal bug ticket – thanks for finding this!

–Neil

1 Like
12

Hi @Dr.Mona!

We just released Texture Tools Exporter 2024.1.0 and NVTT 3.2.5, which include fixes for a number of things in this thread! I’ve posted the release notes here. Here’s a quick summary:

  • The off-by-one error is now fixed in nvcompress.
  • nvcompress now uses batches of at most 500 files at once, which works around the 509 file handle limit.
    • I’ll add both of these changes to GitHub shortly.
  • We’ve also added “Minimum Mipmap Size” and “Maximum Mipmap Count” options (--min-mip-size and --max-mip-count) to the Exporter, so it’s now usable from there as well.

Thanks again!

1 Like