Difference between revisions of "Arcane University:World space synchronization"

The Beyond Skyrim Wiki — Hosted by UESP
Jump to: navigation, search
(WIP)
 
Line 10: Line 10:
 
The copy happens in two passes that are independent and can be done in any order: copying of '''landscape''', and copying of '''references'''. Both methods work on rectangular regions in a world space, so you must figure out what those rectangular regions are (there can be multiple that combine into a more complex shape, even just one cell large if necessary). Write down the cell coordinates of the northeast and southwest corners of every rectangle according to the origin world space's coordinate system. Then, also calculate what those coordinates amount to in [https://www.creationkit.com/index.php?title=Unit '''Units'''] (precise formulas forthcoming). It helps to have a grid map or similar on hand.
 
The copy happens in two passes that are independent and can be done in any order: copying of '''landscape''', and copying of '''references'''. Both methods work on rectangular regions in a world space, so you must figure out what those rectangular regions are (there can be multiple that combine into a more complex shape, even just one cell large if necessary). Write down the cell coordinates of the northeast and southwest corners of every rectangle according to the origin world space's coordinate system. Then, also calculate what those coordinates amount to in [https://www.creationkit.com/index.php?title=Unit '''Units'''] (precise formulas forthcoming). It helps to have a grid map or similar on hand.
  
Next, find out the difference between the coordinate systems of the two world spaces. Pick a location that can be pinpointed in both world spaces, note its cell coordinates in the origin world spaces, and its cell coordinates in the destination world space. Subtract the X and Y coordinates in the origin world space from the X and Y coordinates in the destination world space, respectively. This will give you the '''offset''', or the number of cells to move the copy by along the X and Y axes. Also calculate the X and Y offsets in units by multiplying them with 4096 (as that is the number of units comprising the edge of a Skyrim world space cell). If the offset is not a clean multiple of a cell, then the method for copying landscape will not work, as the method for that works cell by cell. The method for copying references is unit-based, but will be of little use if the landscape cannot be accurately copied.
+
Next, find out the difference between the coordinate systems of the two world spaces. Pick a location that can be pinpointed in both world spaces, note its cell coordinates in the origin world spaces, and its cell coordinates in the destination world space. Subtract the X and Y coordinates in the origin world space from the X and Y coordinates in the destination world space, respectively. This will give you the '''offset''', or the number of cells to move the copy by along the X and Y axes. Also calculate the X and Y offsets in units by multiplying them with 4096 (as that is the number of units comprising the edge of a Skyrim world space cell). If the offset is not a clean multiple of a cell, then the method for copying landscape will not work, as the method for that works cell by cell. The method for copying references is unit-based, but will be of little use if the landscape cannot be accurately copied. Lastly, if the sea level is not at the same height in the two world spaces, write down the difference between them in units (this does not have to be a multiple of 4096).
  
If the two world spaces belong to different masters, and the destination world space's file is not a master of the origin world space's file, then the copied area must be free of land textures and references whose base objects come from the origin world space's file. This can be resolved in advance (such as by re-implementing any non-vanilla assets in a shared master, and switching all references to use those shared base objects; for Beyond Skyrim, this is BSAssets) but can also be arranged at the end of the process; this article will show how to contain the copy in a patch plugin which can be tweaked before being sent off to be merged into the destination world space's master.
+
If the two world spaces belong to different masters, and the destination world space's file is not a master of the origin world space's file, then the copied area must be free of land textures and references whose base objects come from the origin world space's file. This can be resolved in advance (such as by re-implementing any non-vanilla assets in a shared master, and switching all references to use those shared base objects; for Beyond Skyrim, this is BSAssets) but can also be arranged at the end of the process; this article will show how to contain the copy in a patch plugin which can be tweaked before being sent off to be merged into the destination world space's master. If it is too difficult to integrate land textures, these can optionally not be copied as detailed further below.
  
 
To redirect all references of a certain base object in the origin file to a base object that is compatible with the destination file, one can use the "Search and replace" function in the Creation Kit, or the following xEdit script by Zilav, which has the advantage of also covering land textures (just replace OldFormID and NewFormID as needed):
 
To redirect all references of a certain base object in the origin file to a base object that is compatible with the destination file, one can use the "Search and replace" function in the Creation Kit, or the following xEdit script by Zilav, which has the advantage of also covering land textures (just replace OldFormID and NewFormID as needed):
Line 32: Line 32:
 
=== Landscape ===
 
=== Landscape ===
  
 +
Here, landscape refers to the LAND record in each cell, containing land height (i.e. heightmap data) and the land textures used (which refer to LTEX base objects).
  
 +
If you wish to copy land textures, and some of them still come from the origin world space's file, then you must temporarily make the origin file a master of the destination file for the copying to work; the copying will also not happen in a patch plugin, but is applied directly on the destination file. However, a later step will extract the edits into a patch plugin after all.
 +
 +
Apply a script on the origin world space (right click on the world space in the origin file, then Apply script). Choose the built-in script "Worldspace copy landscape area to another worldspace". If you want to copy land textures, edit the script so that <code>bCopyLayers = True</code>.
 +
 +
A pop-up window appears. Configure the two world spaces; enter the cell coordinates of the southwest corner of the current rectangle, and sizes of the rectangle along both axes, which is the difference between the two X and Y coordinates respectively. A rectangle comprised of one cell will have sizes 1 and 1; a square of 2x2 cells will be 2 and 2, and so on. Then enter the southwest cell coordinates in the destination world space by summing the cell offsets with the origin world space's coordinates. You must leave "Create patch plugin" unchecked ''unless you have no problematic land textures or forego copying them entirely'', and if you check it, every rectangle will go in its own patch plugin which you will want to merge afterward.
 +
 +
Then allow the script to run. LAND records are heavy and so this will take some time. For a large rectangle, xEdit may run out of memory, so consider using the 64bit version. After a rectangle has been copied, you can save to free the memory back up.
 +
 +
Once you are done with all rectangles, and if you didn't have "Create patch plugin" checked, save and close xEdit just in case so you can make a back-up of your destination esm. Put a fresh copy of your destination esm in your data folder, one free of any land copying. In xEdit, load only this fresh esm. Once loaded, right click it, select "Compare to..." and select your modified esm. This will make a new load order in xEdit where the modified esm appears to override the records of the fresh one. Right click anywhere and select "Apply filter"; in the pop-up, check "by Base record signature" and check "LAND". Also make sure "by conflict status overall" is still checked, and deselect the top three options. Also check "Flatten Blocks/Sub-Blocks" and "Flatten Cell Children" further below. Run the filter and wait a (long) while. After it's done, only the changed LAND records should still be visible in xEdit. In your modified esm, Alt+Click the expand icon next to the destination world space to instantly open all cells, select one LAND record, press Ctrl+A to select all other LAND records as well, right click and select "Copy as override into...". Make it a new plugin. This plugin now contains all the results of the process, and you can test it alongside the fresh esm.
  
  

Revision as of 17:32, 4 November 2021

In this article, world space synchronization refers to the process of copying a part of one world space into another, so that the borders match up in both. This is needed for Beyond Skyrim projects whose provinces border each other. Once a project has their border region more or less finalized, it needs to be copied over to the other project so that they can level design their border region while accounting for what's directly on the other side; and once that's done, this border region needs to be copied back the other way.

This article will demonstrate how to do one such copying operation using xEdit, without the larger-scale project management involved; it is also useful for other situations where this may have to be done, and not just in Beyond Skyrim (many scripts discussed here were originally created for Skywind).


Preparation

You need two world spaces, one containing the landscape to copy (henceforth origin world space) and one to copy to (destination world space). For clarity's sake, this article will assume that they are contained in masters (.esm) to distinguish them from plugins (.esp), but the process is the same if they are in plugins. The two world spaces may be part of the same master file, which makes the process a bit simpler, but often will be in separate masters.

The copy happens in two passes that are independent and can be done in any order: copying of landscape, and copying of references. Both methods work on rectangular regions in a world space, so you must figure out what those rectangular regions are (there can be multiple that combine into a more complex shape, even just one cell large if necessary). Write down the cell coordinates of the northeast and southwest corners of every rectangle according to the origin world space's coordinate system. Then, also calculate what those coordinates amount to in Units (precise formulas forthcoming). It helps to have a grid map or similar on hand.

Next, find out the difference between the coordinate systems of the two world spaces. Pick a location that can be pinpointed in both world spaces, note its cell coordinates in the origin world spaces, and its cell coordinates in the destination world space. Subtract the X and Y coordinates in the origin world space from the X and Y coordinates in the destination world space, respectively. This will give you the offset, or the number of cells to move the copy by along the X and Y axes. Also calculate the X and Y offsets in units by multiplying them with 4096 (as that is the number of units comprising the edge of a Skyrim world space cell). If the offset is not a clean multiple of a cell, then the method for copying landscape will not work, as the method for that works cell by cell. The method for copying references is unit-based, but will be of little use if the landscape cannot be accurately copied. Lastly, if the sea level is not at the same height in the two world spaces, write down the difference between them in units (this does not have to be a multiple of 4096).

If the two world spaces belong to different masters, and the destination world space's file is not a master of the origin world space's file, then the copied area must be free of land textures and references whose base objects come from the origin world space's file. This can be resolved in advance (such as by re-implementing any non-vanilla assets in a shared master, and switching all references to use those shared base objects; for Beyond Skyrim, this is BSAssets) but can also be arranged at the end of the process; this article will show how to contain the copy in a patch plugin which can be tweaked before being sent off to be merged into the destination world space's master. If it is too difficult to integrate land textures, these can optionally not be copied as detailed further below.

To redirect all references of a certain base object in the origin file to a base object that is compatible with the destination file, one can use the "Search and replace" function in the Creation Kit, or the following xEdit script by Zilav, which has the advantage of also covering land textures (just replace OldFormID and NewFormID as needed):

unit UserScript;

const
  OldFormID = $01012345;
  NewFormID = $01054321;

function Process(e: IInterface): integer;
begin
  CompareExchangeFormID(e, OldFormID, NewFormID);
end;

end.


Landscape

Here, landscape refers to the LAND record in each cell, containing land height (i.e. heightmap data) and the land textures used (which refer to LTEX base objects).

If you wish to copy land textures, and some of them still come from the origin world space's file, then you must temporarily make the origin file a master of the destination file for the copying to work; the copying will also not happen in a patch plugin, but is applied directly on the destination file. However, a later step will extract the edits into a patch plugin after all.

Apply a script on the origin world space (right click on the world space in the origin file, then Apply script). Choose the built-in script "Worldspace copy landscape area to another worldspace". If you want to copy land textures, edit the script so that bCopyLayers = True.

A pop-up window appears. Configure the two world spaces; enter the cell coordinates of the southwest corner of the current rectangle, and sizes of the rectangle along both axes, which is the difference between the two X and Y coordinates respectively. A rectangle comprised of one cell will have sizes 1 and 1; a square of 2x2 cells will be 2 and 2, and so on. Then enter the southwest cell coordinates in the destination world space by summing the cell offsets with the origin world space's coordinates. You must leave "Create patch plugin" unchecked unless you have no problematic land textures or forego copying them entirely, and if you check it, every rectangle will go in its own patch plugin which you will want to merge afterward.

Then allow the script to run. LAND records are heavy and so this will take some time. For a large rectangle, xEdit may run out of memory, so consider using the 64bit version. After a rectangle has been copied, you can save to free the memory back up.

Once you are done with all rectangles, and if you didn't have "Create patch plugin" checked, save and close xEdit just in case so you can make a back-up of your destination esm. Put a fresh copy of your destination esm in your data folder, one free of any land copying. In xEdit, load only this fresh esm. Once loaded, right click it, select "Compare to..." and select your modified esm. This will make a new load order in xEdit where the modified esm appears to override the records of the fresh one. Right click anywhere and select "Apply filter"; in the pop-up, check "by Base record signature" and check "LAND". Also make sure "by conflict status overall" is still checked, and deselect the top three options. Also check "Flatten Blocks/Sub-Blocks" and "Flatten Cell Children" further below. Run the filter and wait a (long) while. After it's done, only the changed LAND records should still be visible in xEdit. In your modified esm, Alt+Click the expand icon next to the destination world space to instantly open all cells, select one LAND record, press Ctrl+A to select all other LAND records as well, right click and select "Copy as override into...". Make it a new plugin. This plugin now contains all the results of the process, and you can test it alongside the fresh esm.


References

Final checks