Creating Large Worldspaces

From the Oblivion ConstructionSet Wiki
Revision as of 23:56, 20 January 2011 by imported>Bruneauinfo (→‎Testing the New Worldspace)
Jump to navigation Jump to search

Before You Begin

As opposed to small worldspaces (4 quads or less), working on large worldspaces (4 quads or more) requires a different approach. Before starting your grand project consider the practicality of it. Take a moment and think over what you are hoping to achieve. Will you and/or your team be able to complete the project you're envisioning? Large worldspaces require a lot of hours of work. The process of creating them can get very tedious at times. Unlike a basic quest mod that takes place in a vanilla worldspace, a large worldspace can require several weeks if not months of work and preparation before you can even start placing buildings, NPCs, dungeons, etc.

The purpose of this warning is not to discourage you. Creativity is what the modding community is all about. What is being suggested here is that before you start your project and as you work on it try to do two things: 1 - Keep it practical. If it's not practical it probably won't get finished. 2 - Know what you're doing. If you know how to use the tools you're using your work will go much more efficiently. When your project is moving forward it will help motivate you to continue. If your project keeps running into delays because you skipped a step, or didn't read a useful tip and have to go back and start from an earlier point - this is what kills a project.

Notes
  • A quad equals 32 x 32 cells. 4 quads would be 64x64 cells. See Oblivion Units for more on what is meant by a quad.
  • As an aid to deciding how large to make your new worldspace consider looking at the Tamriel worldspace in the CS. Open it in the Heightmap editor and generate an overview. Tamriel uses 16 quads, however, much of the space can not be traversed in Vanilla Oblivion.
  • This article is written with the assumption you are creating a worldspace larger than a single quad.

Essential Skills

This article assumes you have considerable experience working with worldspaces and the CS. This article contains some very useful pointers in these areas, but if you've never used the CS before, or you've never worked within a worldspace you should attempt some of the tutorials listed before getting started on this project.

It is of course expected you have familiarized yourself with using both the Heightmap Editor and the Region Editor. If you haven't used these tools it is recommended you start a test worldspace and learn at least the basics of how to use them. This article contains some pointers on using them, but if you haven't used them before expect to do a lot of trial and error work at first.

Important references and tutorials:

The Invaluable Navigation Method

As has already been said time can make the difference between completing a project and abandoning it. Knowing the most efficient workflow can be a game saver.

It is very important for large world creation and editing to learn how to navigate with the landscape editor most efficiently. The method is contained in the points below.

  • When using the landscape editor it is extremely helpful to arrange the cells by cell number in the Cell View Window. This makes it much easier to find a particular cell that needs correction or inspection. You do this by clicking on the header titled "Location" in the Cell View window. This will order the cells numerically so you can find cells most efficiently with the mouse wheel.
  • Press 'A' to turn on enhanced lighting. This makes it easier to see what you're doing.
  • Navigate the map from a bird's eye view using the arrow keys to move the camera when hunting for errors and making corrections instead of the mouse, spacebar, shift key, etc. Details:
1 - Start by pressing 'T' to get the camera oriented properly for using the arrow keys. The arrow keys do NOT work relative to the camera position. Rather the 'Up' arrow always moves the camera north, the 'Down' arrow always moves the camera South, etc. Pressing 'T' will orient the camera to a bird's eye view perspective where up will be North, down will be South, etc. This will make the arrow keys match the direction the camera is moving on-screen. You can press 'T' at any time to restore this position. 

2 - After pressing 'T' to get the camera oriented properly use the mouse wheel to zoom out. Zoom out until you see the edge of the currently viewable cells. Navigating from this perspective will allow you to see errors and rips which will usually appear as purple just as long as the land is above water. Rips under water will require zooming in closer to find. 

3 - Make the mental connection between Cell coordinates and how the coordinate system is related to the world you created in the heightmap editor. To do this start by finding Cell 0,0 in the landscape editor and loading it. Then follow steps 1 and 2 directly above to get the camera oriented properly. Now go to the heightmap editor and figure out where 0,0 is. Note how the Cell coordinates change on the heightmap editor as you go North, South, East, or West. You should be able to pick a cell from the heightmap editor and navigate to it manually in the landscape editor using only the arrow keys. 

4 - Find errors in your landscape textures in the heightmap editor and then locate them in the landscape editor using the error keys. Correct them in the landscape editor using 'Soften Vertices' or 'Flatten Vertices' then save your corrections in the CS - BUT NOT IN THE HEIGHTMAP EDITOR. 

5 - Avoid using the 'Shift' key and 'Space Bar' for navigating except when making corrections that would be difficult to make otherwise. Once these types of corrections are made press 'T' again to restore proper camera perspective and zoom out.
Notes
  • When navigating with the landscape editor set your keyboard over to your left(if you're right handed) so that your left hand can easily work the arrow keys while your right hand works the mouse. Left handed folks won't need to do this.

Required Tools

You should ensure you have the following 3rd party programs installed and working as you will need to make use of each of them at some point.

  • Oblivion Mod Manager - load order management (Wrye Bash also accepted)
  • TES4Gecko - Needed to move worldspaces, remove records, convert to/from .esm and merge.
  • TES4Edit - Needed to remove records and remove VWD flags from objects.
  • TES4LODGen - Needed to generate distant objects. TES4LODGen no longer uses VWD flags and ignores them completely.
TES4Gecko Notes
  • In particular TES4Gecko can be tough to get working right with the shortcut, but the additional memory setup through this shortcut is NECESSARY for merging.
  • TES4Gecko will work fine without the Shortcut on very small files. However, with large worldspace files the Shortcut is required.
  • An example shortcut is included with the program. Consider making a copy of it before altering it so you can easily start fresh again if you have trouble getting it right the first time.
  • After right clicking on the original shortcut select Properties. You want the Shortcut Tab.
  • For a particularly long Pathname try typing it out in a text editor so you can see the whole thing at once and make sure you have formatted it correctly. Then copy and paste it into the appropriate location.
  • For Windows 7 users you need to put quotes around the Path name to get TES4Gecko's shortcut to work properly. Example:
"C:\Program Files (x86)\Java\jre1.6.0_18\bin\javaw.exe" -Xmx1200m -jar "T:\(install_path)\TES4Gecko 15.2\TES4Gecko.jar"
  • The above example is what you enter as the Target in Windows 7. And the example below is the portion you would use as the Start in: entry. Note the example below is the same as the latter portion you entered from the above example.
"T:\(install_path)\TES4Gecko 15.2\TES4Gecko.jar"
  • The readme file included with TES4Gecko contains additional instructions. Also, if you have trouble you can always search the internet.

Ready to Go

Once you have a handle of the related tools and basic methods you should begin. You should always start a new worldspace with a new .esp. This is true even if you already have significant aspects of your mod completed. The reason for this is simple. More RAM available for the CS means crashes will be less frequent.

Notes
  • An added benefit of starting a new .esp is that it decreases the chance of corrupting your other mods and vice versa.

Heightmap Generation

Start a new .esp. You do not need to load Oblivion.esm at this time. You are creating a new worldspace. You will not need Oblivion.esm until you get to region generation. Not loading Oblivion.esm here at the beginning will save you time by not having to process it. It will also lower demands on RAM.

Save this .esp. You will probably want to name this .esp something similar to the name of the new worldspace. Now define your worldspace in the CS (World > World Spaces > New, give it a name and adjust the settings as desired within that window (you can change climate and water settings later, map will come last (none of these are covered in this primer))). Don't create any terrain for this worldspace quite yet, just have it defined. Save the mod again. Although this may seem a bit redundant, some aspects of the heightmap editor (notorious for being problematic as you probably know) don't work particularly well when working on a worldspace which has not been saved (it's a memory thing); saving the mod records the definition of the worldspace within the .esp, so it is more stable (or something). Now you can begin within the heightmap editor.

Using the Heightmap Editor Part 1

Below are some notes and methods to help make your new worldspace creation happen as efficiently as possible. The following took several weeks of testing and trials to discover and was written here to save you a lot of time. Keeping these points in mind will help get your mod finished with the least amount of time and frustration.

  • During heightmap generation only work on 4 quads at a time. If your worldspace will be larger than 4 quads the blending between sets of quads can be done later. Still you should try your best to plan your work as you go so that you can match your heightmap elevations between 4-quad work sessions. Take notes on edge elevations if your heightmap will extend beyond the edge of a quad, where rivers will meet up in bordering quads, and where mountain ranges begin and end. Perfection is not required at this point as most errors can be resolved later. But the more care you take now the easier it will be to make these corrections later.
  • The greatest number of errors (rips) on a texture are created in the hightmap editor where land is not properly blended before saving the texture. Any large, abrupt changes in elevation are prone to tearing. These are indicated by hard lines between two colors. There should be no hard lines left on your texture in the hightmap editor when you save it. Make sure all hard lines are blended with either the blending tool or the noise tool. Your finished work should look like it was applied with a paint sprayer.
  • Additionally, in reference to the above note tears can be prevented at the exterior perimeter of your New World by blending the last few cells into the water rather than having the land just suddenly stop and drop off. Blending can be done with the blending tool or with the noise tool depending on the terrain type you want at these locations.
  • If your worldspace is 4 quads or less you may not need to worry too much about blending the perimeter and so can neglect the edges of your worldspace. The edges can be hidden by leaving a buffer of about 3 cells between the edge of your quads and playable regions. However, you should still take note you may want to come back and generate dummy terrain so your world will have LOD terrain extending into the distance beyond your created worldspace. Also, when working on your worldspace with the landscape editor you may notice decreased performance as the Render Window tries to load terrain at the edges of your world where there are rips and other errors. It might be worth your time to go ahead and at least fix any errors that occur outside your playable worldspace with the Landscape Editor. So to prevent these errors altogether just follow the previous bullet note and blend those edges in the Heightmap editor.
  • A common location for errors to occur in the heightmap is at 0,0 as well as at the centers - like 32,32 -32,-32 64,64 etc. These errors can regenerate themselves repeatedly each time their heightmap area is saved.
  • After finishing work on your first 4 quad heightmap you should save your mod in the CS and follow the directions in the following two sections.

Converting Your ESP to an ESM

When working on large Worldspaces you should convert your .esp to an .esm regardless of how you intend to release your mod. There are two reasons for this:

  • Due to limitations in the CS, and potentially your system, working on too many cells can lead to instability and frequent crashing. The more cells a worldspace has, the more the CS has to load into memory while you're working. ESP files larger than 40mb (8 quads) usually cannot be loaded in the CS without some sort of error popping up. Even if you are only using 4 quads for your worldspace and plan to release an .esp, it is still suggested you use a .esm during creation so you can take advantage of the benefits this format offers.
  • By using an .esm you always retain a backup of the worldspace at the last point of merging (more on merging in the next section). Your .esm file is after all just a series of .esp files merged together. So if you screw something up horribly while heightmapping, you don't have to start from scratch. This doesn't mean you shouldn't make backups of your .esm files. It's just a handy feature of .esm files.

Keep in mind that if your new worldspace project will make changes to a worldspace located within another .esm - like for example the vanilla Oblivion worldspaces located in Oblivion.esm - those changes must be kept in an .esp file rather than an esm. Otherwise odd errors and undesirable behaviors will occur in the game. (Essentially, not following this guideline wipes out all non-persistent objects in the edited worldspace.) So, if your worldspace will be saved in an .esm file, and your mod will makes changes in a worldspace in another .esm, your mod will require a separate and additional .esp for any changes you intend to make to any .esm-based worldspaces. An .esm CAN safely link to another .esm's interior cells, and it CAN include scripts to place objects/NPCs into another .esm's worldspace without using an .esp - as long as these changes do not actually make edits to the worldspace directly. Still to be safe you should try to make all these types of changes in an .esp that will remain an .esp at release time.

  • Once you have trees and other content, you will also need to clear out all the VWD flags for your worldspace using TES4Edit in order for your worldspace to show correctly when loaded anywhere other than the 01 index. You would need to do this cleaning after every change which adds more trees, but thankfully, you only need to clean out those things which were added/moved. (Note: This final note has still not be verified in recent testing.)

Using the Heightmap Editor Part 2

For worldspaces larger than 4 quads when blending the borders between 4-quad sections in the landscape editor it isn't necessary to to blend these perfectly to your finished specifications. Your only goal is to fix tears - i.e. the errors. Getting a good finished product can be just as time consuming or more in the landscape editor as it is in the heightmap editor - despite all of its seeming random error generation. Here is the suggested methodology: 

1 - Design and finish the layout of 4 quads 

2 - Save in the heightmap editor 

3 - Make error corrections in the landscape editor, save in the CS, and create .esm with TES4Gecko

4 - Start new .esp and design/finish the layout of the next 4 quads 

5 - Save in the heightmap editor 

6 - Make error corrections in the landscape editor, save, and merge with your worldspace.esm 

7 - Repeat above steps designing additional quads following this pattern and merge 

8 - After all heightmaps are created and merged start a new esp 

9 - Edit the borders in the heightmap. Multiple esps will be required for large worlds. Merge each "border-edit" esp with the esm as they are created. Also as you edit the borders scan the heightmap you're working on for errors and fix them as you go. These will appear as sharply contrasted squares/rectangles. 

10 - After all borders are edited use the landscape editor to correct errors in conjunction with the heightmap editor. DO NOT make corrections with the heightmap editor at this time, and DO NOT save any more textures in the heightmap editor. At this time you will only use the heightmap editor for finding errors in your textures. Make the actual corrections using the landscape editor. Merge with the esm.
Notes
  • Using the above methodology to create world spaces larger than 4 quads will eventually generate a great number of .esp files as well as .esm backups. It's a good idea to create a folder to hold all of your original .esp files as well as another to store your .esm backups. This will reduce the clutter in your Oblivion Data folder. Within your .esp folder it will also be a good idea to create separate folders to hold the different types of .esp files. Then if you need to backup to an earlier part of the creation process you will have multiple avenues to getting your worldspace to a previous point.
  • It is also a good idea to number the .esp files you merge as you create them. For example: MyWorld_Heightmap_Part1.esp, MyWorld_Heightmap_Part2.esp etc. You can do the same thing for your heightmap blending files and region generation files.

Testing the New Worldspace

When first testing your new worldspace.esm you should always have it loaded in the 01 Mod Index - in other words directly after oblivion.esm. If you don't know how to do this you should take a few moments and read up on Oblivion Mod Manager or Wrye Bash and learn how to manage your load order.

The reason you do this is because for one, other mods may interfere with your new worldspace.esm. These problems are rooted in some common issues that have a relatively easy fix. So since it's likely that placing your worldspace.esm in the 01 Mod Index will interfere with other mods you should employ the simplest temporary fix and have only oblivion.esm and your worldspace.esm active. Start a new game with a new character for testing and then "cheat" your way into your worldspace. There several ways to do this. See below for two.

Method 1: Create an Exploration Plugin

Create a door in the Imperial Prison cell where you start the vanilla game. Set the door up so it teleports you to a door in your new worldspace. Just make sure you never merge this .esp file with your esm. It might also be handy to drop a horse in your new worldspace using this same temporary plugin so you have something to explore the terrain with. Make sure you give ownership to the Player so the horse doesn't run off when you dismount. Keeping this plugin handy can also help when testing regions later in this article. You can move your door and horse to various locations around your worldspace as you generate your region test areas.

Method 2: Use the Console

Open the console (~) while in the game.

Type: COW MyWorldSpace 0,0

Replace 0,0 with whatever cell location you want to visit.

Replace MyWorldSpace with the ID of your WorldSpace.

Distant LOD Issues

This is a quick explanation of the "problems" mentioned in the above section Testing the New Worldspace. This information has been inserted here to help you understand the "problem" so you can find the solution later when you start creating the DistantLOD for your worldspace. This was taken from the Landscape LOD Tutorial.

If your distant LOD land does not show up, it is recommended that you use TES4Gecko to add your World Space to the Master Index, which will make it appear in the game regardless of other moderations and regardless of load order.

To do this: Download TES4Gecko (http://www.tesnexus.com/downloads/file.php?id=8665)
Install TES4Gecko according to the instructions included with it.
Once you start TES4Gecko, go to the "Move Worldspaces" function.
Then select the .esp that contains your worldspace.

This will edit your .esp so that the game sees your landscapes as part of the original game rather than as a moderation. It also changes the names of the files in \meshes\landscape\lod and \textures\landscapelod\generated to match the changes made to the .esp.

Region Generation

Before doing region generation make a backup of your new worldspace .esm file. Before starting you should have all the terrain you planned to use merged with the newworld.esm. To create regions you're going to need data from from Oblivion.esm. So from this point on you will need to have Oblivion.esm loaded as well as your new worldspace .esm file.

There are at least two methods for creating regions. One is to start from scratch. The other is to use existing regions from Oblivion and edit them to your liking. In fact, some regions in the Oblivion worldspace may be exactly what you're looking for and require no editing at all. Keep this in mind since using existing region content can save you a lot of time.

How To Re-Use Existing Regions

1 - Open the Region Editor

2 - Below the words Region Editor you'll see a select box for the current world spaces. 

3 - Select Tamriel. The fastest way to get to Tamriel is to click on the selection box and press 'T'. This will show you a list of Regions used in the Tamriel world space. 

4 - Click on a region name in the list and find it on the region map. You may need to navigate around the map to find where the corresponding region was used. Just hover your cursor over the map and press and hold the space bar while using the mouse to move the map around. Also use the mouse wheel to zoom out. The corresponding region will have a region border around it - a dark red line outlining where the region data has been applied. 

5 - Right-click on the region map within the region border and click on 'View World Here' in the pop-up. This will show you what effect the Region Data had on Tamriel in the Render Window. 

6 - If you like the region right-click on it in the list on the left and select 'Create Data Copy'. The copy is created at the bottom of the list and given a generic name that includes a number. Select this Region in the list and give it a new name on the right under the 'General' tab. 

7 - Now find your world space in the select list. You will see your copied regions there. Once you apply a region to a worldspace it will no longer appear under all worldspace region lists - just the worldspace where you used it. You may repeat these steps above as many times as you need to borrow region data from Tamriel and other world spaces.

Create Your Regions from Scratch

See the topic Regions for more on creating Regions from scratch.

Notes

  • For either custom or recycled regions you should consider doing some small test patches (4 cells 2x2) with that region to see how it will texture and place things. Once you have a region to your liking, remove it and all objects generated by that region within the same CS session, and move on to testing the next region. DO NOT SAVE YOUR MOD WITH ANY GENERATED ITEMS IN YOUR WORLDSPACE AT THIS TIME, ALWAYS CLEAN BEFORE SAVING. Before defining actual region borders, save your mod, open it with TES4Gecko, and remove any entries which refer to edited cells. Merge this region mod so that the settings are contained within your .esm and can be reverted to later. Make a backup. Now, start a new .esp for your region borders. Only define the region borders DO NOT GENERATE THEM. This mod will have some small cell changes, so is not as easy to clean, which is why you did this as a separate .esp. Make sure it has no placed objects, then merge it.
  • It can take a great deal of time to generate regions. You will get "Not Responding" messages for the CS while generating regions. Note that these are different than crashes. In Windows 7 when the CS crashes all CS windows become highlighted and a warning box pops up. However, when the CS is terribly busy you will see Not Responding messages at the top of the CS window. One way to check to see if the CS is locked up or not during a Not Responding message is to open the Windows Task Manager and check to see if the CS is still using CPU cycles and to see if the amount of memory it is using is still climbing (keep in mind memory use will climb very slowly). With a quad core Intel i7 at 2.6 GHz 12 - 13% of CPU is used and several Kilobytes of data are added every few seconds.

There are two very important things to keep in mind when generating regions: 1) Larger regions mean not only more time spent generating, but also larger .esp size, and depending on system specs and density of items, can lead to frequent crashing (why we're using a .esm). The only thing you should be doing while working on region generation is generating objects for that region. Leave ALL other structures and statics until after you've generated EVERYTHING as these structures can often get buried during generation.

2) Once an object has been placed in a cell from region generation, and once that session has been saved and ended, the object can no longer be removed by that region. Meaning that if you want to change any settings related to region items, or have overlapping regions, you would have to manually remove them cell by cell if you've already saved the mod and started a new session. To make life easier, you should adopt a rather simple method of avoiding this problem all together. After every region you generate, select every bordering region and select "remove objects for this region". Even if you haven't generated that region yet, the region you generated placed objects slightly outside its border, and once saved, these objects may be doubled when that bordering region is generated later. This will leave only the space which exists only within the one region you generated as having items (the remaining parts will be mostly generated with those other regions. Save the mod, merge, and work on the next. For small regions, you can do more that one region per .esp, but should avoid generating regions which are near each other within the same .esp. For larger regions, you should keep it to a single .esp per region.

No doubt you're staring at this a bit confused. Just take your time and work through it one region at a time. For a 4 quad worldspace which is entirely generated, it is to be expected that you will be spending no less than a week doing little else other than generating objects if you are using several regions and region types (took me a little over 2 weeks when I did one 4 quad worldspace simply because I had to figure out most of this stuff and deal with crashes when the CS ran out of memory). For larger worldspaces, significantly longer. The downside however is that in order to make the landscape work right as a .esm without requiring it to be loaded in the 01 index, you will have to go through a few thousand tree entries in TES4Edit and remove the vwd flags. It's not as bad as it sounds since most of the work is just lots of clicking on the little + boxes and expanding cell information, but there isn't any sort of automated script which does this to my knowledge. You would also need to use TES4LodGen to generate distant objects afterward, but that's a fairly quick thing.

Dummy Terrain

Often overlooked is the purpose of using dummy terrain instead of just making your worldspace larger in order to have those neat LOD vistas. Dummy terrain is simply terrain which exists as a LOD, but does not exist as actual terrain within the mod people have. The purpose of this is to give the impression of distant landmasses without adding the additional filesize. Keep in mind that in order for trees and LOD terrain to show up on this dummy terrain, you would need to include the LOD meshes, textures, and Distant Object files with your mod (as opposed to having people generate their own).

The way this works is to use the .esm containing your finished worldspace, and create a new .esp based on that .esm. As LOD meshes are generated 4 quads at a time, you will only be able to work on 4 quads at a time, and each .esp for dummy terrain should only contain 4 quads regardless of how large your worldspace may be. NEVER MERGE THESE .ESP FILES WITH YOUR .ESM. Do a 4 quad area, generate LOD regions (just trees, textures and rocks) for these areas, save the .esp, generate the LOD mesh for those 4 quads, save it, and start on the next dummy terrain. It is a good idea to use descriptive naming with these .esp files. Something like "MyWorldspaceTerrainDummyEast.esp" is more useful than "DummyTerrainMod01.esp" should you ever need to fix any problems with that terrain or the related LOD files.

Try to change as little of your completed worldspace landscape as possible. Some blending will still happen along borders, but the better you can match these dummy terrains with the existing terrain that exists within your .esm, the better. Although nothing you do within the .esp files containing the dummy terrain will take effect on the actual worldspace of your mod, less difference along borders means closer LOD approximation.

When you've done all of this, you should now have LOD meshes and textures for the quads surrounding your worldspace. Now we need to generate the records for distant objects in these quads. Have your .esm and ALL of those .esp files containing your dummy terrain marked as active in your load order, Then use TES4LodGen to generate the distant objects for your worldspace. This may take longer than normal since you may be working with 4-10 times the number of cells, be patient. Once it has a message saying that the process is done, close the program, and test it in-game, being sure to uncheck those dummy .esp files. If all went well, you should now have distant mountains and terrain which extends well beyond your worldspace border, which vanishes if you go through your border regions or get too close. If not, you may need to regenerate/fix some of the LOD meshes for those quads.

When you have a working dummy terrain, package the LOD models, the LOD textures, and the entire distant objects folder in a .rar or other container, make a backup of the file, and release it as an optional download for your mod. You should still file away your dummy terrain .esp files in case you need to make changes to one of those sections, or decide to expand into one of those quads with playable terrain (forcing you to release as a .esm, and have a larger file size). People will not need the .esp files to see the terrain, but you will if you need to fix anything in those areas.

This not the only way of generating dummy terrain, but is probably the most straight-forward. Another method likely exists if you are using several playable worldspaces within a non-playable parent worldspace, as a means of unifying them all together, but this can become rather difficult to manage.

Retrieved from "https://cs.uesp.net/w/index.php?title=Creating_Large_Worldspaces&oldid=14249"