Win gitpython & add-ons installer tool

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: Win gitpython & add-ons installer tool

Post by sgrogan »

triplus wrote: Wed Mar 07, 2018 7:02 pm Test this branch on Windows. I added some changes that might help with the issue when un-installing a workbench.
Will do, Thanks for looking.
"fight the good fight"
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: Win gitpython & add-ons installer tool

Post by sgrogan »

triplus wrote: Wed Mar 07, 2018 7:02 pm Test this branch on Windows. I added some changes that might help with the issue when un-installing a workbench.
No luck

Code: Select all

Traceback (most recent call last):
  File "C:\Users\Chris\GitHub\FCAutoBuild\var\FreeCAD_0.17.13354_x64_dev_win\Mod\AddonManager\AddonManager.py", line 340, in remove
    shutil.rmtree(clonedir)
  File "C:\Users\Chris\GitHub\FCAutoBuild\var\FreeCAD_0.17.13354_x64_dev_win\bin\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\Chris\GitHub\FCAutoBuild\var\FreeCAD_0.17.13354_x64_dev_win\bin\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\Chris\GitHub\FCAutoBuild\var\FreeCAD_0.17.13354_x64_dev_win\bin\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\Chris\GitHub\FCAutoBuild\var\FreeCAD_0.17.13354_x64_dev_win\bin\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\Chris\GitHub\FCAutoBuild\var\FreeCAD_0.17.13354_x64_dev_win\bin\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: 'C:\\Users\\Chris\\AppData\\Roaming\\FreeCAD\\\\Mod\\assembly2\\.git\\objects\\pack\\pack-7ee4ad020df47103a0499d7fde18b51cc1079f5e.idx'
I think the problem is not "releasing the handle", but rather the problem that the files have a read only property. I still haven't tested on Ubuntu. Are these files read only? Or it's not a problem to be read only on Linux?

Godd answer here: https://stackoverflow.com/questions/482 ... in-windows
I will try to implement it.
"fight the good fight"
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Win gitpython & add-ons installer tool

Post by triplus »

sgrogan wrote: Thu Mar 08, 2018 1:11 am I think the problem is not "releasing the handle", but rather the problem that the files have a read only property. I still haven't tested on Ubuntu. Are these files read only? Or it's not a problem to be read only on Linux?
On Linux the directory gets removed and shutil.rmtree() doesn't care about if any files are read only in it. On Windows a directory with read only file will produce the mentioned error. This works differently yes. On this two platform.
Godd answer here: https://stackoverflow.com/questions/482 ... in-windows
I will try to implement it.


I did a few more tests with Git directly. Indeed there will likely always be some read only files involved. I updated the branch. Pressing on the Remove button should now work on Windows?
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: Win gitpython & add-ons installer tool

Post by sgrogan »

triplus wrote: Fri Mar 09, 2018 12:00 am I did a few more tests with Git directly. Indeed there will likely always be some read only files involved. I updated the branch. Pressing on the Remove button should now work on Windows?
Works good on Windows, thanks.
AddonManager doesn't use the python git module for uninstall so my earlier hack for the environment isn't need for uninstall.
"fight the good fight"
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Win gitpython & add-ons installer tool

Post by triplus »

sgrogan wrote: Fri Mar 09, 2018 10:15 pm Works good on Windows, thanks.
You're welcome.
AddonManager doesn't use the python git module for uninstall so my earlier hack for the environment isn't need for uninstall.
Correct. I took a quick look and there is one more occurrence of shutil.rmtree() and a shutil.move(). Both related to i guess backup folder functionality. It would likely make sense to investigate on how to invoke this functionality and test this area too. On what happens when read only files are involved. As for shutil.rmtree() that will likely fail. I can add the same fix. As for shutil.move() i am not sure.

In addition what are your plans. To provide GitPython by default in Windows installer? And what about Git executable. User would have to install it manually? Will end users do that or just continue to use "ZIP download" method? And a bit unrelated question. Based on your info on Windows the path to Mod folder isn't guaranteed to be in ASCII character set. Can some users on Windows expect issues with FreeCAD modules and macros because of that?
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: Win gitpython & add-ons installer tool

Post by sgrogan »

triplus wrote: Sat Mar 10, 2018 1:49 am In addition what are your plans. To provide GitPython by default in Windows installer? And what about Git executable. User would have to install it manually? Will end users do that or just continue to use "ZIP download" method?
This was my original intention, but now I'm not so sure because of the unicode problem. I can add an option to the installer to install the git executable, like we do with the VS re-distributable package, only make it optional. The zip option works well on Windows, but update will re-install regardless if an update is available.
triplus wrote: Sat Mar 10, 2018 1:49 am Based on your info on Windows the path to Mod folder isn't guaranteed to be in ASCII character set. Can some users on Windows expect issues with FreeCAD modules and macros because of that?
It comes down to the users Windows %username% It looks like FreeCAD pre-pends sys.path to PATH which ends up unicode encoded. I don't know if encoding in utf-8 like my hack does will cause problems somewhere else. I believe the root cause is when unicode is passed to Popen (or subprocess).
"fight the good fight"
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Win gitpython & add-ons installer tool

Post by triplus »

OK the first thing i did is i created the PR:

https://github.com/FreeCAD/FreeCAD/pull/1357

Mentioned additional shutil.rmtree() and a shutil.move() lines are related to ZIP download. And there read only files are not expected. Therefore i didn't change anything there for now. In theory if user would use GitPython to install a module. And after remove GitPython/Git executable. ZIP download option would be used again. And then likely read only files would be involved in the existing module folder. Pressing on Remove button would still work though. And after using the ZIP download option would succeed again.
sgrogan wrote: Sat Mar 10, 2018 7:16 pm I can add an option to the installer to install the git executable, like we do with the VS re-distributable package, only make it optional.
Nice.
This was my original intention, but now I'm not so sure because of the unicode problem.
FreeCAD Python 3 build might remove this particular issue (maybe it would introduce others). As for Python 2. What you could test (if you are interested) is to add a small patch in GitPython. Around the line 700:

https://github.com/gitpython-developers ... git/cmd.py

You will find a line:

Code: Select all

    env = os.environ.copy()
Additional line after it:

Code: Select all

    env['PATH'] = str(env['PATH'])
Should likely fix the mentioned issue on your side.

P.S. Don't know though what to expect in scenarios where for example user name contains non-ASCII characters. You can i guess test that too. If you find the proposed solution feasible.
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: Win gitpython & add-ons installer tool

Post by sgrogan »

triplus wrote: Sat Mar 10, 2018 10:26 pm Should likely fix the mentioned issue on your side.

P.S. Don't know though what to expect in scenarios where for example user name contains non-ASCII characters. You can i guess test that too. If you find the proposed solution feasible.
Thanks. I haven't had time to test yet, but I will. I need to find a way to safely setup other usernames/langs/locales without borking my dev machine.
"fight the good fight"
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Win gitpython & add-ons installer tool

Post by triplus »

Take your time.

P.S. Generally speaking likely it will make sense for you to be able to easily test for possible issues (Py2/Py3) in this area. Not just GitPython related.
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: Win gitpython & add-ons installer tool

Post by sgrogan »

triplus wrote: Thu Mar 15, 2018 11:45 pm P.S. Generally speaking likely it will make sense for you to be able to easily test for possible issues (Py2/Py3) in this area
I can do this now, on Win (PY3.6 + QT5), thanks to @looo. Not so easy to distribute yet, but maintainable locally. I can also cross against Ubuntu 14.04 (PY3.4+QT4), again not so easy to distribute, but maintainable locally.

I do it locally and not in a VM, so I must be careful ;)
"fight the good fight"
Post Reply