On Apr 27, 2005, at 12:56 PM, Eirik Karlsen wrote:
No, doesn't work.
The Format won't handle floating point number 'properly' because
of the x.xxxxxxx notation. Also there are rounding errors....
y=-99.9990000 'for testing only
So I did a small test.
dim y as Double
y = -99.9990000
self.Title = Format(y, "-0.00000000000000")
Yields -99.99900000000000 as it should, but
dim y as Double
y = -99.9990000
self.Title = Format(y, "-0.000000000000000") // One more zero
Has accuracy problems and returns something like -99.99852.......
I wonder how the IDE's debugger handles it, because it shows -99.999
and that's it. If I use
y = -99.999999999999912345
The IDE shows -99.99999999999991, and Format shows y =
-99.999999999999914735
The IDE appears to know something. :)
Seth Willits
----------------------------------------------------------
Freak Software - http://www.freaksw.com/
ResExcellence - http://www.resexcellence.com/realbasic/
----------------------------------------------------------
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
|