Creating Large Worldspaces

From the Oblivion ConstructionSet Wiki
Revision as of 15:00, 16 January 2011 by imported>Bruneauinfo (Major edit of the first 2/3 of the article with many notes and readability improvements.)
Jump to navigation Jump to search

Before You Begin

As opposed to small worldspaces (4 quads or less), working on large worldspaces (4 quads or more) requires almost a completely different method from the very start. Before starting your grand project, you should first take a moment and think over what it is you are hoping to achieve, and how much of it is doable by you or your team. Large worldspaces, like the large projects which require them, require rather excessive amounts of hours of work, however unlike with that quest mod that takes place in mostly vanilla spaces, work on a large worldspace often requires several weeks of work and preparation before you can even START on placing buildings, NPCs, and quests. As such, it is not something which should be taken lightly.

Notes

  • A quad equals 32 x 32 cells. 4 quads would be 64x64 cells. See Oblivion Units for more on what is meant by a quad.
  • As an aid to deciding how large to make your new worldspace consider looking at the Tamriel worldspace in the CS. Open it in the Heightmap editor and generate an overview. Tamriel uses 16 quads, however, much of the space can not be traversed in Vanilla Oblivion.
  • This article is written with the assumption you are creating a worldspace larger than a single quad.

Required Tools

First, you should ensure you have the following 3rd party programs installed and working as you will need to make use of each of them at some point.

  • Oblivion Mod Manager - load order management (Wrye Bash also accepted)
  • TES4Gecko - Needed to move worldspaces, remove records, convert to/from .esm and merge.
  • TES4Edit - Needed to remove records and remove VWD flags from objects.
  • TES4LODGen - Needed to generate distant objects after VWD flags have been removed.

Notes

  • In particular TES4Gecko can be tough to get working right with the shortcut, but the additional memory which is setup through this shortcut is NECESSARY for merging.
  • TES4Gecko will work fine without the Shortcut on very small files. However, with large world space files the Shortcut is required.
  • An example shortcut is included with the program. Consider making a copy of it before altering it so you can easily start fresh again if you have trouble getting it right the first time.
  • After right clicking on the original shortcut select Properties. You want the Shortcut Tab.
  • For a particularly long Pathname try typing it out in a text editor so you can see the whole thing at once and make sure you have formatted it correctly. Then copy and paste it into the appropriate location.
  • For Windows 7 users you need to put quotes around the Path name to get TES4Gecko's shortcut to work properly. Example:
"C:\Program Files (x86)\Java\jre1.6.0_18\bin\javaw.exe" -Xmx1200m -jar "T:\(install_path)\TES4Gecko 15.2\TES4Gecko.jar"
  • The above example is what you enter as the Target in Windows 7. And the example below is the portion you would use as the Start in: entry. Note the example below is the same as the latter portion you entered from the above example.
"T:\(install_path)\TES4Gecko 15.2\TES4Gecko.jar"
  • The readme file included with TES4Gecko contains additional instructions. Also, if you have trouble you can always search the internet.

Essential Skills

Before continuing, this article assumes that you have already briefed and familiarized yourself with using both the heightmap editor and the region editor as well as have considerable experience in working with worldspaces and the CS. If you have not done this, you should probably take a few days to play around with those two tools to see how they work on a worldspace you aren't particularly tied to completing. The following links should be of particular interest.

Once you have a handle of the related tools, you can begin. Even if you already have significant aspects of your mod completed, you should ALWAYS start a new worldspace within a new .esp. The reason for this is simple, if you're only loading your worldspace in the CS and not some other mod, the CS has more memory free so crashes are less frequent. The added benefit of not having some chance of screwing up the mod you already worked on is pure bonus.

Heightmap Generation

Start a new .esp. You do not need to load Oblivion.esm at this time. You are creating a new worldspace. You will not need Oblivion.esm until you get to region generation. Not loading Oblivion.esm here at the beginning will save you time by not having to process it. And you will need all the time you can get on this type of project.

Save this .esp with a name you will recognize later. Now define your worldspace in the CS (World > World Spaces > New, give it a name and adjust the settings as desired within that window (you can change climate and water settings later, map will come last (none of these are covered in this primer))). Don't create any terrain for this worldspace quite yet, just have it defined. Save the mod again. Although this may seem a bit redundant, some aspects of the heightmap editor (notorious for being problematic as you probably know) don't work particularly well when working on a worldspace which has not been saved (it's a memory thing); saving the mod records the definition of the worldspace within the .esp, so it is more stable (or something). Now you can begin within the heightmap editor.

The Invaluable Navigation Method

Before going any further something needs to be said about navigating with the landscape editor in the Render Window. Yes, using the landscape editor is not the next step in this process. However, since the reader is contemplating working on a very large project where time can make the difference between completing a project and abandoning it, knowing the most efficient workflow can be a game saver.

It is very important for large world creation and editing to learn how to navigate with the landscape editor most efficiently. The method is contained in the points below.

  • When using the landscape editor it is extremely helpful to arrange the cells by cell number in the Cell View Window. This makes it much easier to find a particular cell that needs correction. You do this by clicking on the header titled "Location" in the Cell View window. This will order the cells numerically so you can find cells most efficiently.
  • Press 'A' to turn on enhanced lighting. This makes it easier to see what you're doing.
  • Navigate the map from a bird's eye view using the arrow keys to move the camera when hunting for errors and making corrections instead of the mouse, spacebar, shift key, etc. Details:
1 - Start by pressing 'T' to get the camera oriented properly for using the arrow keys. The arrow keys do NOT work relative to the camera position. Rather the 'Up' arrow always moves the camera north, the 'Down' arrow always moves the camera South, etc. Pressing 'T' will orient the camera to a bird's eye view perspective where up will be North, down will be South, etc. This will make the arrow keys match the direction the camera is moving on-screen. You can press 'T' at any time to restore this position. 

2 - After pressing 'T' to get the camera oriented properly use the mouse wheel to zoom out. Zoom out until you see the edge of the currently viewable cells. Navigating from this perspective will allow you to see errors and rips which will usually appear as purple just as long as the land is above water. Rips under water will require zooming in closer to find. 

3 - Make the mental connection between Cell coordinates and how the coordinate system is related to the world you created in the heightmap editor. To do this start by finding Cell 0,0 in the landscape editor and loading it. Then follow steps 1 and 2 directly above to get the camera oriented properly. Now go to the heightmap editor and figure out where 0,0 is. Note how the Cell coordinates change on the heightmap editor as you go North, South, East, or West. You should be able to pick a cell from the heightmap editor and navigate to it manually in the landscape editor using only the arrow keys. 

4 - Find errors in your landscape textures in the heightmap editor and then locate them in the landscape editor using the error keys. Correct them in the landscape editor using 'Soften Vertices' or 'Flatten Vertices' then save your corrections in the CS - BUT NOT IN THE HEIGHTMAP EDITOR. 

5 - Avoid using the 'Shift' key and 'Space Bar' for navigating except when making corrections that would be difficult to make otherwise. Once these types of corrections are made press 'T' again to restore proper camera perspective and zoom out.

Notes

  • When navigating with the landscape editor set your keyboard over to your right (if you're right handed) so that your left hand can easily work the arrow keys while your right hand works the mouse. Left handed folks won't need to do this.

Now on to heightmap editing!

Using the Heightmap Editor

Below are some notes and methods to help make your new worldspace creation happen as efficiently as possible. The following took several weeks of testing and trials to discover and was written here to save you a lot of time. Keeping these points in mind will help get your mod finished with the least amount of time and frustration.

  • During initial heightmap generation only work on 4 quads at a time. Blending between sets of quads can be done later. Plan your work as you go so that you can match your heightmap elevations between work sessions. But perfection is not required at this point.
  • The greatest number of errors (rips) on a texture are created in the hightmap editor where land is not properly blended before saving the texture. Any large, abrupt changes in elevation are prone to tearing. These are indicated by hard lines between two colors. There should be no hard lines left on your texture in the hightmap editor when you save it. Make sure all hard lines are blended with either the blending tool or the noise tool. Your finished work should look like it was applied with a paint sprayer.
  • Additionally, in reference to the above note tears can be prevented at the exterior perimeter of your New World by blending the last few cells into the water rather than having the land just suddenly stop and drop off. Blending can be done with the blending tool or with the noise tool depending on the terrain type you want at these locations.
  • If your worldspace is 4 quads or less you may not need to worry too much about blending the perimeter and so can neglect the edges of your worldspace. The edges can be hidden by leaving a buffer of about 3 cells between the edge of your quads and playable regions. However, you should still take note you may want to come back and generate dummy terrain so your world will have LOD terrain extending into the distance beyond your created worldspace. Also, when working on your worldspace with the landscape editor you may notice decreased performance as the Render Window tries to load terrain at the edges of your world where there are rips and other errors. It might be worth your time to go ahead and at least fix any errors that occur outside your playable worldspace with the Landscape Editor.
  • A common location for errors to occur in the heightmap error is at 0,0 as well as at the centers like 32,32 -32,-32 64,64 etc. These errors can show themselves repeatedly each time their heightmap area is saved.
  • When working on large Worldspaces you should convert the .esp to a .esm regardless of how you intend to release your mod. There are two reasons for this:
1 - Due to limitations in the CS, and potentially your system, working on too many cells can lead to severe instability and/or frequent crashing. The more cells a worldspace has, the more it needs to load into memory while you're working on it. .esp files larger than 40mb (8 quads) usually cannot be loaded in the CS without some sort of error popping up. Even if you are only using 4 quads for your worldspace and plan to release as a .esp, you will still want to use a .esm during creation just so that you can take advantage of the benefits of this format.  

2 - By using an .esm you always retain a backup of the worldspace at the last point of merging. Your .esm file is after all just a series of .esp files merged together. So if you screw something up horribly while heightmapping, you don't have to start from scratch. This doesn't mean you shouldn't make backups of your .esm files. It's just a handy feature of .esm files.
  • For World Spaces larger than 4 quads: When blending the borders between quads in the landscape editor it isn't necessary to to blend these perfectly to your finished specifications. Getting a good finished product can be just as time consuming in the landscape editor as it is in the heightmap editor with all of its seeming random error generation. Here is the suggested methodology keeping in mind the rest of the article's instructions:
1 - Design and finish the layout of 4 quads 

2 - Save in the heightmap editor 

3 - Make error corrections in the landscape editor, save in the CS, and create .esm with TES4Gecko

4 - Start new .esp and design/finish the layout of the next 4 quads 

5 - Save in the heightmap editor 

6 - Make error corrections in the landscape editor, save, and merge with the .esm 

7 - Repeat above steps designing additional quads following this pattern and merge 

8 - After all heightmaps are created and merged start a new esp 

9 - Edit the borders in the heightmap. Multiple esps will be required for large worlds. Merge each "border-edit" esp with the esm as they are created. Also as you edit the borders scan the heightmap you're working on for errors and fix them as you go. These will appear as sharply contrasted squares/rectangles. 

10 - After all borders are edited use the landscape editor to correct errors in conjunction with the heightmap editor. DO NOT make corrections with the heightmap editor at this time, and DO NOT save any more textures in the heightmap editor. At this time you will only use the heightmap editor for finding errors in your textures. Make the actual corrections using the landscape editor. Merge with the esm.

The World You Have Created

When testing your .esm, always have it loaded in the 01 index so that extra steps (in particular moving worldspaces and removing vwd references from trees) aren't needed in order to get an idea of what the terrain really looks like. If you don't know how to do this, you may wish to take a few moments with [Oblivion Mod Manager] or Wrye Bash to learn how to manage your load order. If placing your .esm in the 01 index interferes with other mods, you may just want to create a new character with only your worldspace active and just cheat your way to this space.

As a sort of brief summary of this whole article you will use TES4Gecko to convert the worldspace.esp mod to an .esm and make modifications and changes to that world in the form of additional .esp files that are then cleaned and merged with the worldspace.esm file. When it comes time to create the content (quests, buildings, NPCs, items, dungeons, etc) this information will be saved in an .esp that is based on and requires both your worldspace.esm file as well as the oblivion.esm file loaded. When you have finished work on the content .esp you will merge the content.esp into the worldspace.esm with TES4Gecko for the release of your mod.

Notes

  • For a particularly large worldspace, you have no option other than using .esm format for release, and will need to make adjustments related to this as far as connections, quests, and LOD data is concerned.
  • Keep in mind that within an .esm you cannot have any changes to a vanilla worldspace (even placing a door in a vanilla exterior) without causing the landscape to get screwed up.
  • Once you have trees and other content, you will also need to clear out all the VWD flags for your worldspace using TES4Edit in order for your worldspace to show correctly when loaded anywhere other than the 01 index. You would need to do this cleaning after every change which adds more trees, but thankfully, you only need to clean out those things which were added/moved.

Region Generation

Before doing region generation make sure you have a backup of the .esm and have merged all terrain that you planned to do. From this point on you will need to have Oblivion.esm loaded as well as your new worldspace .esm file. You're now going to need content from Oblivion.esm.

As you setup your regions, it is suggested that you do some small test patches (4 cells 2x2) with that region to see how it will texture and place things. Once you have a region to your liking, remove it and all objects generated by that region within the same CS session, and move on to testing the next region. DO NOT SAVE YOUR MOD WITH ANY GENERATED ITEMS IN YOUR WORLDSPACE AT THIS TIME, ALWAYS CLEAN BEFORE SAVING. Before defining actual region borders, save your mod, open it with TES4Gecko, and remove any entries which refer to edited cells. Merge this region mod so that the settings are contained within your .esm and can be reverted to later. Make a backup. Now, start a new .esp for your region borders. Only define the region borders DO NOT GENERATE THEM. This mod will have some small cell changes, so is not as easy to clean, which is why you did this as a separate .esp. Make sure it has no placed objects, then merge it.

It can take a great deal of time to generate regions. You will get "Not Responding" messages for the CS while generating regions. Note that these are different than crashes. In Windows 7 when the CS crashes all CS windows become highlighted and a warning box pops up. However, when the CS is terribly busy you will see Not Responding messages at the top of the CS window. One way to check to see if the CS is locked up or not during a Not Responding message is to open the Windows Task Manager and check to see if the CS is still using CPU cycles and to see if the amount of memory it is using is still climbing (keep in mind memory use will climb very slowly). With a quad core Intel i7 at 2.6 GHz 12 - 13% of CPU is used and several Kilobytes of data are added every few seconds.

There are two very important things to keep in mind when generating regions: 1) Larger regions mean not only more time spent generating, but also larger .esp size, and depending on system specs and density of items, can lead to frequent crashing (why we're using a .esm). The only thing you should be doing while working on region generation is generating objects for that region. Leave ALL other structures and statics until after you've generated EVERYTHING as these structures can often get buried during generation.

2) Once an object has been placed in a cell from region generation, and once that session has been saved and ended, the object can no longer be removed by that region. Meaning that if you want to change any settings related to region items, or have overlapping regions, you would have to manually remove them cell by cell if you've already saved the mod and started a new session. To make life easier, you should adopt a rather simple method of avoiding this problem all together. After every region you generate, select every bordering region and select "remove objects for this region". Even if you haven't generated that region yet, the region you generated placed objects slightly outside its border, and once saved, these objects may be doubled when that bordering region is generated later. This will leave only the space which exists only within the one region you generated as having items (the remaining parts will be mostly generated with those other regions. Save the mod, merge, and work on the next. For small regions, you can do more that one region per .esp, but should avoid generating regions which are near each other within the same .esp. For larger regions, you should keep it to a single .esp per region.

No doubt you're staring at this a bit confused. Just take your time and work through it one region at a time. For a 4 quad worldspace which is entirely generated, it is to be expected that you will be spending no less than a week doing little else other than generating objects if you are using several regions and region types (took me a little over 2 weeks when I did one 4 quad worldspace simply because I had to figure out most of this stuff and deal with crashes when the CS ran out of memory). For larger worldspaces, significantly longer. The downside however is that in order to make the landscape work right as a .esm without requiring it to be loaded in the 01 index, you will have to go through a few thousand tree entries in TES4Edit and remove the vwd flags. It's not as bad as it sounds since most of the work is just lots of clicking on the little + boxes and expanding cell information, but there isn't any sort of automated script which does this to my knowledge. You would also need to use TES4LodGen to generate distant objects afterward, but that's a fairly quick thing.

Dummy Terrain

Often overlooked is the purpose of using dummy terrain instead of just making your worldspace larger in order to have those neat LOD vistas. Dummy terrain is simply terrain which exists as a LOD, but does not exist as actual terrain within the mod people have. The purpose of this is to give the impression of distant landmasses without adding the additional filesize. Keep in mind that in order for trees and LOD terrain to show up on this dummy terrain, you would need to include the LOD meshes, textures, and Distant Object files with your mod (as opposed to having people generate their own).

The way this works is to use the .esm containing your finished worldspace, and create a new .esp based on that .esm. As LOD meshes are generated 4 quads at a time, you will only be able to work on 4 quads at a time, and each .esp for dummy terrain should only contain 4 quads regardless of how large your worldspace may be. NEVER MERGE THESE .ESP FILES WITH YOUR .ESM. Do a 4 quad area, generate LOD regions (just trees, textures and rocks) for these areas, save the .esp, generate the LOD mesh for those 4 quads, save it, and start on the next dummy terrain. It is a good idea to use descriptive naming with these .esp files. Something like "MyWorldspaceTerrainDummyEast.esp" is more useful than "DummyTerrainMod01.esp" should you ever need to fix any problems with that terrain or the related LOD files.

Try to change as little of your completed worldspace landscape as possible. Some blending will still happen along borders, but the better you can match these dummy terrains with the existing terrain that exists within your .esm, the better. Although nothing you do within the .esp files containing the dummy terrain will take effect on the actual worldspace of your mod, less difference along borders means closer LOD approximation.

When you've done all of this, you should now have LOD meshes and textures for the quads surrounding your worldspace. Now we need to generate the records for distant objects in these quads. Have your .esm and ALL of those .esp files containing your dummy terrain marked as active in your load order, Then use TES4LodGen to generate the distant objects for your worldspace. This may take longer than normal since you may be working with 4-10 times the number of cells, be patient. Once it has a message saying that the process is done, close the program, and test it in-game, being sure to uncheck those dummy .esp files. If all went well, you should now have distant mountains and terrain which extends well beyond your worldspace border, which vanishes if you go through your border regions or get too close. If not, you may need to regenerate/fix some of the LOD meshes for those quads.

When you have a working dummy terrain, package the LOD models, the LOD textures, and the entire distant objects folder in a .rar or other container, make a backup of the file, and release it as an optional download for your mod. You should still file away your dummy terrain .esp files in case you need to make changes to one of those sections, or decide to expand into one of those quads with playable terrain (forcing you to release as a .esm, and have a larger file size). People will not need the .esp files to see the terrain, but you will if you need to fix anything in those areas.

This not the only way of generating dummy terrain, but is probably the most straight-forward. Another method likely exists if you are using several playable worldspaces within a non-playable parent worldspace, as a means of unifying them all together, but this can become rather difficult to manage.

Retrieved from "https://cs.uesp.net/w/index.php?title=Creating_Large_Worldspaces&oldid=14231"