Wiki Home

Serious Floating Point Bug


Namespace: DotNet

Hello again,

today i found a crazy bug in vfp.

SET DECIMALS TO 2
MessageBox(Ceiling(145.8*100)) && returns the correct value of 14580
SET DECIMALS TO 10
MessageBox(Ceiling(145.8*100)) && returns the wrong value of 14581

Replacing "ceiling" with "round" works fine.
As I guess it has sth to do with the limitation of 15digits (pentium based machines) in vfp. See "Numeric Data Type" in VFP-Help.

Alexander Lagler
( Topic last updated: 2007.12.21 04:20:06 AM )