Decimal's problem

Have some feature requests, feedback, cool stuff to share, or want to know where FreeCAD is going? This is the place.
Forum rules
Be nice to others! Read the FreeCAD code of conduct!
Post Reply
G-VAR
Posts: 24
Joined: Wed Dec 28, 2016 8:23 am

Decimal's problem

Post by G-VAR »

Hi everyone,

I have encountered a strange thing after changed the number of decimals to 12 in the preferences dialog, .
When i set the value of Density to 7850 kg/m^3 in the material settings dialog, i got 7850.000000000001 kg/m^3.

Does anyone have any idea?

Thanks for help!

OS: Windows 10
Word size of OS: 64-bit
Word size of FreeCAD: 32-bit
Version: 0.18.16093 (Git)
Build type: Release
Branch: releases/FreeCAD-0-18
Hash: 690774c0effe4fd7b8d2b5e2fb2b8c8d145e21ce
Python version: 2.7.14
Qt version: 4.8.7
Coin version: 4.0.0a
OCC version: 7.2.0
kisolre
Veteran
Posts: 4163
Joined: Wed Nov 21, 2018 1:13 pm

Re: Decimal's problem

Post by kisolre »

chrisb
Veteran
Posts: 53926
Joined: Tue Mar 17, 2015 9:14 am

Re: Decimal's problem

Post by chrisb »

Seems that with 12 decimals you are touching the precision of the internal calculation / representation.
A Sketcher Lecture with in-depth information is available in English, auf Deutsch, en français, en español.
G-VAR
Posts: 24
Joined: Wed Dec 28, 2016 8:23 am

Re: Decimal's problem

Post by G-VAR »

Thanks, kisolre.
So, is there any way to avoid this?
G-VAR
Posts: 24
Joined: Wed Dec 28, 2016 8:23 am

Re: Decimal's problem

Post by G-VAR »

chrisb wrote: Tue Apr 23, 2019 11:23 am Seems that with 12 decimals you are touching the precision of the internal calculation / representation.
Thanks, chrisb.

I think it's a double-converion problem.
Is there any way to avoid this?
kisolre
Veteran
Posts: 4163
Joined: Wed Nov 21, 2018 1:13 pm

Re: Decimal's problem

Post by kisolre »

G-VAR wrote: Tue Apr 23, 2019 11:31 am Is there any way to avoid this?
Avoid what? Displaying numbers with such precision? Why do you even need that? According to https://hypertextbook.com/facts/1996/Mi ... llip.shtml that is the size of an atom nucleus :D
Just set the number of decimals to 5-6 and you are good to go for almost any real life situation :)
openBrain
Veteran
Posts: 9034
Joined: Fri Nov 09, 2018 5:38 pm
Contact:

Re: Decimal's problem

Post by openBrain »

Just notice that changing the decimal number that is displayed doesn't change the internal precision used for computations. ;)
Post Reply