Obsoleting material handling bash scripts with python

A forum to discuss the implementation of a good Materials system in FreeCAD
User avatar
PrzemoF
Posts: 2655
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Obsoleting material handling bash scripts with python

Postby PrzemoF » Thu Feb 21, 2019 5:17 pm

bernd, can you point to the code you mentioned in PR [1]? I want to see what was your idea. I think about yaml format for the material cards, csv/ods for mass editing. But you're the one to make the final decision :D

[1] https://github.com/FreeCAD/FreeCAD/pull/1983
User avatar
bernd
Posts: 8486
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Obsoleting material handling bash scripts with python

Postby bernd » Thu Feb 21, 2019 5:32 pm

PrzemoF wrote:
Thu Feb 21, 2019 5:17 pm
I think about yaml format for the material cards
me too :D https://github.com/berndhahnebach/FreeC ... aterialdev but still far from finished ... :oops:

PrzemoF wrote:
Thu Feb 21, 2019 5:17 pm
But you're the one to make the final decision :D
and Yorik for sure.
User avatar
bernd
Posts: 8486
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Obsoleting material handling bash scripts with python

Postby bernd » Thu Feb 21, 2019 5:35 pm

in conjuction with these methods https://github.com/FreeCAD/FreeCAD/comm ... 4548e0de3e merged in master already you can write all cards to yaml ...
User avatar
bernd
Posts: 8486
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Obsoleting material handling bash scripts with python

Postby bernd » Thu Feb 21, 2019 5:36 pm

I am curious about your opinion ... but as mentioned it is far from finished. The commit "dev" is more a proof of concept.
User avatar
PrzemoF
Posts: 2655
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: Obsoleting material handling bash scripts with python

Postby PrzemoF » Thu Feb 21, 2019 9:17 pm

Tested & (mostly) works. The yaml format looks better than plain "config" type file. Side effect for me: vim is highlighting yaml nicely :D
It probably makes sense to wait for python3 because of uft8 to avoid this:

Code: Select all

UltimateTensileStrength: !!python/unicode '38.8 MPa'
Also yaml.load might be replaced with safe_load (paranoid safety mode :D ).

What's the plan for mass editing? csv/odt or a separate editor?

[1] https://pyyaml.org/wiki/PyYAMLDocumentation
User avatar
bernd
Posts: 8486
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Obsoleting material handling bash scripts with python

Postby bernd » Fri Feb 22, 2019 9:22 am

is there a way to handle the Py2 Py3 str unicode problems not inside the yaml file but inside the code instead? We should use "clean" material card files.
User avatar
PrzemoF
Posts: 2655
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: Obsoleting material handling bash scripts with python

Postby PrzemoF » Fri Feb 22, 2019 7:22 pm

I don;t know. I tried allow_unicode option, but no luck. I think that one of the big changes between python2 i 3 is string handling. I see strange things :shock:

Code: Select all

- General:
    Description: !!python/unicode 'Generic soda-lime glass'
    DescriptionDE: !!python/unicode 'Normalglas'
    DescriptionPL: Szkło sodowe <-- Polish ł but no !python/unicode 
    Father: !!python/unicode 'Glass'
    Name: !!python/unicode 'Glass'
I'll try to play around with the yaml module to get consistent utf-8 encoding for python2 and 3.

Edit: Looks promising: https://stackoverflow.com/questions/289 ... de-objects
Edit2: Another one https://stackoverflow.com/questions/396 ... aml-output
I think it can be done...
User avatar
bernd
Posts: 8486
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Obsoleting material handling bash scripts with python

Postby bernd » Fri Feb 22, 2019 8:21 pm

great I take a look too.

At least you and my origin language are perfectly made for testing :ugeek: äöüß ... damn no polish keyboard on my mobil ... :o
User avatar
PrzemoF
Posts: 2655
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: Obsoleting material handling bash scripts with python

Postby PrzemoF » Fri Feb 22, 2019 8:42 pm

zażółcić gęślą jaźń - that's a typical test. "Make ego/self yellow using gustle" is the meaning :shock:

OK, so the plan is to have all FCMat files in utf-8, but no !!python/unicode marker. Both python2 and 3 must work. If we can make it, the code could be included in the master immediately, maybe with a comment "to be enabled in v0.19"?

Edit: I'm reading the material reader/writer code in the master branch and the first idea that came to my brain is: do not touch it, start from scratch, make a new module and once it works switch over :D
User avatar
bernd
Posts: 8486
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Obsoleting material handling bash scripts with python

Postby bernd » Sat Feb 23, 2019 8:02 am

sounds like a plan! I would not start from scratch, but on top of this commit https://github.com/berndhahnebach/FreeC ... b6271967c8