Nif Optimization

Created by Razorwing from Silgrad Tower

Background

My name is Razorwing from the Silgrad Tower team. If you're a 3D Studio Max modeller like me and haven't tried amorilia's script before you'll most likely want to read this information.

First I'll make a run-of-the-mill disclaimer that you're using this method at your own risk. It's not tested 100% and could cause you problems. All I can say is that I've used it quite a lot myself and haven't experienced any problems at all.

The 3D Studio Max exporter created by the Niftools group has a ton of perks. Whom can't love the ability to set up arrangements of box collision straight in the program, or why not the amazing ability to create convex shapes. But the exporter does have one flaw, and unfortunately it's a very serious one. It bloats your model with hundreds, sometimes thousands, of redundant and invisible faces. If you haven't noticed the effect before, select your model in 3D Studio Max's viewport then right-click and choose 'Properties...'. Note how many faces Max says the model has. Now open the nif in Nifskope and check out how many "Num Triangles" are reported on the NiTriStripsData block (use a calculator if you have multiple NiTriStrips). You can also see the number reported in the status bar of the Construction Set by selecting it in the world window.

File:PythonScriptBeforeOptimize.jpg

My wooden cup exported from Max - 779 faces

In the case of my wooden cup Max says it has 219 faces while Nifskope says it has 779 faces -- in other words, the export process has bloated the model by 350% (!). You can imagine the problem becomes dire if you place ten wooden cups in a home since you'll lose ~5,500 faces from the facecount budget. If you use a lot of custom models like Silgrad Tower does then you can imagine how the problem affects an interior. For instance development ceased on one of our taverns because the facecount budget had been maxed out, tallying 452,349 faces, and the only remedy would have been to divide it into two cells. But after running amorilia's script on our model library the tavern's facecount shrunk to an astonishing 284,257 faces in one fell swoop!

The problem with bloated faces can't be remedied in either 3D Studio Max nor Nifskope. You can achieve a slight optimization by restripifying the model through Nifskope, but it's not a substantial improvement and half the time the process actually increase the number of faces. It would also seem you can save a lot of faces by not stripifying your model -- unticking the "Generate Strips" option on the 3D Studio Max export panel -- but that is a really bad idea, so I won't go into detail on how those methods work. I just wanted to mention them in case the question came up as to why the methods weren't used.

File:PythonScriptAfterOptimize.jpg

My wooden cup after amorilia's script - 219 faces

But as it happens, the Blender exporter does not have the same problem with bloating as the 3D Studio Max exporter does. And thankfully amorilla, head of Blender script development over at Niftools, has created a script that optimizes nif files outside of Blender. As a sweet bonus the script also updates the tangent spaces of your models, which is a real timesaver. The algorhitm for updating tangent spaces seem on par with Nifskopes, while the 3D Studio Max exporter's UTS function is slightly worse (but better than no UTS at all, obviously).

The script is written in a different language than Nifskope is, so it's not likely it will be incorporated into the program in the foreseeable future. But one of the great things about using the script vs. importing/re-exporting through Blender is that it can do batch operations.

Installation

To get the script to work you need Python and PyFFI. Python is an open source programming language, while PyFFI is an interface between Python and nif files.

• Install Python 2.6 (in this article it will be assume that you use the default installation path, C:\Python26).
• Install PyFFI.

Using the Script from the Windows Explorer

Running the Script

File:Nifoptimize1.png

For simple usage you can use it directly from the Windows explorer.

• Right click on the nif you wish to optimize.
• Select "Optimize with PyFFI".
• A DOS window will open, asking for confirmation. Type the letter Y and hit enter.
• You will get to see the results of the optimization.
• Test your optimized nif file with nifskope and in the game to make sure that nothing has been lost. If the script makes a visual change to your nif file, report it on the niftools forum.

File:Nifoptimize2.png

Configuration

Below, in the section on command line options, you can read all about how to tweak various settings in the script if you run it from the command line. You can also configure the script as run from the context menu.

Open a folder, and go to "Tools > Folder Options...". On the File Types tab, scroll down to the NIF entry and click on the Advanced button. Select "Optimize with PyFFI" and click the "Edit..." button.

In the text box under "Application used to perform action", you see this text string:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --pause optimize "%1"

You can add new options between --pause and "%1", for instance:

"C:\Python26\python.exe" "C:\Python26\Scripts\niftoaster.py" --pause -x NiMaterialProperty optimize "%1"

if you do not want the script to optimize NiMaterialProperty blocks when you run it from the context menu.

Using the Script from the Command Prompt

To use the more advanced features of the nif optimizer, such as batch processing of many nifs with a single command, and excluding particular block types from optimization, you need to use the script through the command prompt. If you are uncomfortable using the command prompt, then do not worry, all the steps are explained.

Running the Script

• Create a folder called "nifoptimize" on your C: drive (the name is not crucial, you can choose whichever folder name you like).
• Make a full backup of all nifs you plan to optimize before proceeding. If the script runs into a bug then it may corrupt your nif and thereby destroy it.
• Copy the nifs you want to optimize to "C:\nifoptimize".
• Open the start menu in Windows and select Run (img). In the popup, write cmd and hit enter (img). Now a Windows shell with command prompt will appear.
  File:PythonScriptInDos.jpg

  This is what the DOS window should look like if using the default commands.
• Type

cd C:\Python26

followed by enter, to tell the Windows shell to go to your Python install folder. If you did not install Python in the default location then just replace the folder with your Python install folder. If you are unsure about where Python is installed, open the start menu then go to "Programs > Python 2.6", right-click any of the shortcuts and choose properties. Open the Shortcut tab, where the Python path is displayed in the "Target:" textbox. Write that url in your DOS window in the same format as the example.

• Next, type

python.exe Scripts\niftoaster.py optimize C:\nifoptimize

followed by enter, to run the niftoaster.py script, which will process all nif files in the folder C:\nifoptimize including the nifs in all subfolders, subsubfolders, ... If you optimize a lot of nif files at the same time, it may be useful to store the output in a file instead of having to browse through the console. You can do this using the following command:

python.exe Scripts\niftoaster.py optimize C:\nifoptimize > C:\log.txt

which will store all output in the file C:\log.txt.

• If everything works like it should, you should get a warning and a confirmation prompt. Write the letter Y and hit return. Now the program will work it's way through all the nifs in the folder.
• If the script comes across a nif it doesn't understand it will display an error, but it should continue working. The problematic nif should be left untouched, but be sure to check with NifSkope, just to avoid any risk of using a corrupted nif.
• Don't forget to doublecheck that all nifs work afterwards. If it opens in Nifskope and looks alright then it is probably not corrupted.

Command Line Options

The most important command line options are listed here. For a complete list use the '-h' option.

Simple Options

For most use cases, these options will suffice.

-x blocktype

Exclude block type blocktype from optimization. For example, if you type

python.exe Scripts\niftoaster.py -x NiTriStrips optimize C:\nifoptimize

then the script will not attempt to optimize NiTriStrips blocks (that is, removing duplicate vertices, restripifying, reskinning, and so on). You can accumulate this option to exclude multiple block types.

python.exe Scripts\niftoaster.py -x NiTriStrips -x NiMaterialProperty optimize C:\nifoptimize

will prevent NiTriStrips to be optimized, and will also prevent duplicate NiMaterialProperty blocks to be merged.

-i blocktype

Only optimize block type blocktype. For example, if you type

python.exe Scripts\niftoaster.py -i NiTriStrips optimize C:\nifoptimize

then the script will only attempt to optimize NiTriStrips blocks (that is, removing duplicate vertices, restripifying, reskinning, and so on). You can accumulate this option to include multiple block types. If not specified, then all block types are included except those listed with the -x option.

-h

Show all command line options with brief help.

Advanced Options

These options are mainly useful for debugging.

-r

Raise exception on errors during optimization. The desired behaviour is simply to print a warning on errors, but continue with the next file nevertheless. If you want the script to halt when it encounters an error, use this option. Mainly useful for debugging.

--noninteractive

Do not ask for confirmation before optimizing all files. Usually, you do not want to use this option, but when you have to run the optimizer many times on the same file for debugging purposes, then this might save you some time.

--dry-run

Run the optimizer but save the result to a temporary file instead of overwriting the original files. Again, only useful for debugging.

Good luck, and happy optimizing!

FAQ

• Where do I report bugs? http://sourceforge.net/tracker/?group_id=199269
• Is there a visual difference/loss? No. If you notice a visual difference between a nif and its optimized version, please report this as a bug and include the original nif in your report.
• If the optimizer bails out with an error message like *** TEST FAILED ON ... *** what does that mean? Is my nif corrupted? What you should do is report this as a bug and post the nif with your bug report. Most likely your nif is not corrupted. Whenever the optimizer gives this error, then it will try to restore the original file. As far as is known, this restoration always succeeds. If you need to be absolutely sure, check your nif afterwards and compare it with the original.
• Does the optimizer act on kf files? The current version (as of 19 September 2008) should simply leave them alone as kf files usually do not contain geometry. Future versions might include optimizations also for nif blocks related to animation, and hence, kf files.
• This is great! Can I run the optimizer on my complete meshes folder? The optimizer was not written for this purpose, however some users have reported significant frame rate improvements by doing so. The risk is that it is impossible to manually check each optimized nif, leaving you with possibly corrupted files. So you're on your own. From the official meshes which are stored in the bsa, in these folders no corrupted nifs have yet been found in them after optimizing (starred means making a big difference, so you might want to try these first). ALWAYS MAKE A BACKUP.
  • architecture *
  • dungeons *
  • clutter
  • furniture
  • landscape
  • plants
  • rocks
• So, are there any meshes that I should not run with the optimizer? In principle, the optimizer is intended to work with any nif, but of course not every nif has been tested and there might be corner cases where the optimizer will not work, or even corrupt your nif. These are known not to work:
  • hair
  • helmets
  • some creatures
  • some clothes
  • any nif that has an .egm associated with it