| To: | REALbasic NUG <realbasic-nug at lists dot realsoftware dot com> |
|---|---|
| Subject: | Re: Math Error? |
| From: | Norman Palardy <npalardy at great-white-software dot com> |
| Date: | Fri, 28 Jul 2006 10:08:01 -0600 |
| Delivered-to: | listarchive at realsoftware dot com |
| Delivered-to: | realbasic-nug at lists dot realsoftware dot com |
| References: | <380-22006742718323878 at M2W019 dot mail2web dot com> <F665B7DE-FA16-4F85-9085-4DCA4BF9908F at great-white-software dot com> <4e876156bf99eb94e0231e9a75129892 at satx dot rr dot com> <17537608-DB15-4E71-87AA-3B14CE95720B at great-white-software dot com> <577F15C8-B613-4FC8-B432-5CBBB289C4DB at mobleybros dot com> <77124270607280735o5557bae0m43f7eaed10bc6b3d at mail dot gmail dot com> <249055E1-7CBA-4AB1-9EFB-E06802C4F2D6 at mobleybros dot com> <77124270607280753u7757538fu894d9b3842d3285f at mail dot gmail dot com> |
On Jul 28, 2006, at 8:53 AM, Peter K. Stys wrote: In[26]:= N[72/2000*750,100]Out[26]= 27.0000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000In[27]:= N[72/2000*750] Out[27]= 27. Still suggests all these numbers and intermediate results are representable exactly at machine precision. I'm quite certain that 72/2000 is not. Write it as the simple sum of powers of 2 and you find the problem.72/2000 = 2^-2 + 2^-4 + 2^-5 + 2^-6 + 2^-11 + 2^-15 ..... and so on and you never get exactly the result you get from 72/2000. In the machine all you have is powers of 2 _______________________________________________ Unsubscribe or switch delivery mode: <http://www.realsoftware.com/support/listmanager/> Search the archives of this list here: <http://support.realsoftware.com/listarchives/lists.html> |
| <Prev in Thread] | Current Thread | [Next in Thread> |
|---|---|---|
| ||
| Previous by Date: | Re: Is there a way to get xml pretty printed in RB?, Seth Duke |
|---|---|
| Next by Date: | Re: SQL Select, Norman Palardy |
| Previous by Thread: | Re: Re: Math Error?, Peter K. Stys |
| Next by Thread: | Re: Re: Math Error?, Peter K. Stys |
| Indexes: | [Date] [Thread] [Top] [All Lists] |