Talk:Long Integer

Dragoon Wraith TALK 12:45, 15 August 2006 (EDT): I'm having issues with longs that are above 10^8 (or less than -10^8). They are longs on an object, and so these numbers should work. My math looks something like this:

         13
+      2500
       2513
+     70000
      72513
+  17000000
   17072513
+1000000000
 1017072517

As you can see, when I add the last number in, it changes the last digit for no apparent reason. It does not change it consistently, so I can't just account for the change. I assume this is due to memory issues, but the maximum number I reach is 1326262626 and the minimum is -1326262626, which is well within the limitations written on this page. What gives?

I know that in esm/esp/ess files all variables and gmst's are stored as floats, even if they're defined as integers. Maybe it stores integer script variables as floats in game too, and you're seeing the normal rounding errors suffered by floats? Timeslip 15:11, 15 August 2006 (EDT)

Ignore me. I didn't read the article... Timeslip 15:13, 15 August 2006 (EDT)

Actually, don't ignore me. I just did my own testing, and as far as I can tell longs are stored as floats in object scripts too. Maybe the article is wrong? Timeslip 15:20, 15 August 2006 (EDT)

Dragoon Wraith TALK 22:19, 15 August 2006 (EDT): That was my decision, as well. Well, damn. That sucks. Oh, well, I'll just cut it off at four instead of five. Annoying, but not overly so. Four seems to be perfectly accurate in all situations. I'll change the article.

Some more tests

I've done some more tests, and long seems to have more accuracy then what's been found so far. I needed to store Actor Value data together with Effect Code data. At first, I tried

long TempEffectCode
long TempActorValue
...
set TempEffectCode to (GetNthEffectItemCode pItem 0)
set TempActorValue to (GetNthEffectItemActorValue pItem 0)
set TempActorValue to (TempActorValue + TempEffectCode)
...
set TempActorValue to (TempEffectCode - TempActorValue)
printc "AV is %g", TempActorValue

I always got "AV is 46" no matter which Actor Value or Effect Code was used, so in that case it didn't have a high enough precision.

So, I went back to the drawing board and decided to cut out as many digits as possible. Effect Codes of Actor Value effects always end with AT or SK. This means they always have an effect code of 14135----- or 13308----- (respectively). So I chopped off the first 5 digits with a modulus of 100000 and used the rest to store a combination of Effect Code and Actor Value:

set TempEffectCode to (GetNthEffectItemCode pItem 3)
set TempActorValue to (TempEffectCode%100000) + (GetNthEffectItemActorValue pItem 3)
printc "TempAV is %g", TempActorValue
...
set TempActorValue to (TempActorValue%100)
printc "AV is %g", TempActorValue

Now the results seem to use every one of the 10 digits. For Restore Agility I get "TempAV is 63733; AV is 33". Restore agility is a Restore Attribute effect (REAT) so its effect code is 1413563730, Agility's code is 3, and 30+3=33. Similarily, Damage Agility = 64231 (1413564228), Fortify Endurance = 66283 (1413566278), and Damage Luck = 64235 (1413564228).

So, it seems that the entire 10 digits of the effect code can be used, but I'm not sure why it seems to be 10 digits in this case and fewer in my previous case. As well, I don't doubt the above commentary. In 2006 only the first version was available, so maybe it's something that was (slightly) fixed?--Haama 03:14, 30 March 2008 (EDT)

A bit more information in this thread - you can't accurately multiply large numbers by -1 and modulus seems to work with double precision.--Haama 21:26, 30 March 2008 (EDT)

More Precision tests

From what the obse guys have decoded of the scripting system, it seems that long variables are stored as double-precision floating point numbers. The double format has something like 52 bits in the mantissa, so a long should be quite able to store any valid 32-bit number. My own tests with a custom script function (added by an obse plugin) confirm this. There are a couple caveats:

• This only covers assigning to, and retrieving from, the variable. I've not tested the arithmetic operations, but from all the previous posts it seems likely they are more limited.
• The sign bit in a double isn't quite the same as the sign bit in an integer, so only carefully written script functions will be able to access this bit correctly (what would be the most significant bit in an unsigned integer).

--JRoush 17:19, 29 June 2010 (EDT)