Difference between revisions of "Floating Point"
imported>DragoonWraith (thanks JOG for fixing that up, I'm not exactly an expert on these things. Anyway, is the sorting thing desirable? That can easily be set up without renaming them if you like.) |
imported>DragoonWraith m (thanks for adding - using the Wiki List syntax is recommended, however.) |
||
| (5 intermediate revisions by 4 users not shown) | |||
| Line 9: | Line 9: | ||
In school we learned how to write large numbers | In school we learned how to write large numbers as scientific notations to save some space on paper and to make calculations more easily readable: "1000000000" is the same as "10^9" and "1234000000" is "1.234 * 10^9" or "1.234E9". The same applies to small numbers: "0.000001234" is "1.234E-6" | ||
When we have a limited length for the first floating point number (the "mantissa") very large or very small numbers will be truncated: Oblivion uses a precision of about 7 decimal-digits for the mantissa: | When we have a limited length for the first floating point number (the "mantissa") very large or very small numbers will be truncated: Oblivion uses a precision of about 7 decimal-digits for the mantissa: | ||
| Line 18: | Line 18: | ||
You can't display the numbers between 1,000,000,000 and 1,000,000,100 this way, because they would require 9 digits but only 7 are stored. | You can't display the numbers between 1,000,000,000 and 1,000,000,100 this way, because they would require 9 digits but only 7 are stored. | ||
==See Also== | |||
* [[Short Integer]] | |||
* [[Long Integer]] | |||
[[Category:Variables]] | [[Category:Variables]] | ||
Latest revision as of 17:16, 15 October 2008
Float is a real number format that can store very high and very small values. The range goes from 1.18E-38 to 3.40E38
The format isn't always the best choice, though, because the technical implementation of this variable-format may cause some inaccuracies.(e.g all numbers from 2000000000 to 2000000064 are stored as 2000000000)
Why does this happen?
(In real-life floating point numbers are to the base 10. The computer uses the base 2, which makes things a bit more complicated, but the basics are the same, so I use the base 10 for this explanation).
In school we learned how to write large numbers as scientific notations to save some space on paper and to make calculations more easily readable: "1000000000" is the same as "10^9" and "1234000000" is "1.234 * 10^9" or "1.234E9". The same applies to small numbers: "0.000001234" is "1.234E-6"
When we have a limited length for the first floating point number (the "mantissa") very large or very small numbers will be truncated: Oblivion uses a precision of about 7 decimal-digits for the mantissa:
1.0000000E9 = 1,000,000,000
1.0000001E9 = 1,000,000,100
You can't display the numbers between 1,000,000,000 and 1,000,000,100 this way, because they would require 9 digits but only 7 are stored.