OS: Debian GNU/Linux 11 (bullseye) (XFCE/lightdm-xsession)
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.19.
Build type: Release
Python version: 3.9.7
Qt version: 5.15.2
Coin version: 4.0.0
OCC version: 7.5.1
Locale: English/United States (en_US)
On Sketcher, when I enter a division, it can display a wrong value, see image bellow where it displays 4.83 instead of 4.825:
Also, after entering a value and editing the formula it can display a wrong value:
The value I entered was 8.79mm / 2, and now it's showing 8.789999999999999mm / 2.
Sketcher formula displays incorrect values
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Be nice to others! Respect the FreeCAD code of conduct!
Sketcher formula displays incorrect values
Last edited by anthonyg on Thu Oct 28, 2021 8:32 am, edited 1 time in total.
- Shalmeneser
- Veteran
- Posts: 9545
- Joined: Wed Dec 23, 2020 12:04 am
- Location: Fr
Re: Sketcher formula displays incorrect values
Please follow the banner in Help forum : version, ...
Re: Sketcher formula displays incorrect values
How many decimals did you set in the preferences?
Re: Sketcher formula displays incorrect values
OK, I though there would be something wrong because on the drawing itself you can see it displays 4.825.So how surprising it is that is displays 4.83 instead of 4.825 ???
Re: Sketcher formula displays incorrect values
Yep. There are some inconsistencies/weirdness in FreeCAD where user setting is sparsely applied.
Especially for Sketcher, it has a hardcoded precision that value is "6 significant digits". We discussed that already but no decision was made about a change : https://forum.freecadweb.org/viewtopic.php?f=3&t=61740
Regarding your other issue, it is due to some classical floating-point number representation. It's a bit weird but not so easy to tackle.
Re: Sketcher formula displays incorrect values
Python has some nice and detailed explanation: https://docs.python.org/3/tutorial/floa ... tion-errorRegarding your other issue, it is due to some classical floating-point number representation. It's a bit weird but not so easy to tackle.