Difference between revisions of "Floating Point"
imported>JOG |
imported>JOG |
||
Line 7: | Line 7: | ||
''(In real-life floating point numbers are to the base 10. The computer uses the base 2, which makes things a bit more complicated, but the basics are the same, so I use the base 10 for this explanation).'' | ''(In real-life floating point numbers are to the base 10. The computer uses the base 2, which makes things a bit more complicated, but the basics are the same, so I use the base 10 for this explanation).'' | ||
In school we learned how to write large numbers with exponents to save some space on paper and to make a calculation better readable: "1000000000" is the same as "10^9" and "1234000000" is "1.234 * 10^9" or "1.234E9". The same applies to small numbers: "0.000001234" is "1.234E-6" | In school we learned how to write large numbers with exponents to save some space on paper and to make a calculation better readable: "1000000000" is the same as "10^9" and "1234000000" is "1.234 * 10^9" or "1.234E9". The same applies to small numbers: "0.000001234" is "1.234E-6" | ||
When we have a limited length for the first floating point number (the "mantissa") very large or very small numbers will be truncated: Oblivion uses a precision of about 7 decimal-digits for the mantissa: | When we have a limited length for the first floating point number (the "mantissa") very large or very small numbers will be truncated: Oblivion uses a precision of about 7 decimal-digits for the mantissa: |
Revision as of 08:13, 24 March 2006
Float is a real number format that can store very high and very small values. The range goes from 1.18E-38 to 3.40E38
The format isn't always the best choice, though, because the technical implementation of this variable-format may cause some inaccuracies.(e.g all numbers from 2000000000 to 2000000064 are stored as 2000000000)
Why does this happen?
(In real-life floating point numbers are to the base 10. The computer uses the base 2, which makes things a bit more complicated, but the basics are the same, so I use the base 10 for this explanation).
In school we learned how to write large numbers with exponents to save some space on paper and to make a calculation better readable: "1000000000" is the same as "10^9" and "1234000000" is "1.234 * 10^9" or "1.234E9". The same applies to small numbers: "0.000001234" is "1.234E-6"
When we have a limited length for the first floating point number (the "mantissa") very large or very small numbers will be truncated: Oblivion uses a precision of about 7 decimal-digits for the mantissa:
1.0000000E9 = 1,000,000,000
1.0000001E9 = 1,000,000,100
You can't display the numbers between 1,000,000,000 and 1,000,000,100 this way, because they would require 9 digits but only 7 are stored.