Calling python.exe using asyncio

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Post Reply
User avatar
chennes
Veteran
Posts: 3868
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Calling python.exe using asyncio

Post by chennes »

The Addon Manager uses pip to automatically install dependencies, and a couple of weeks ago I also added some new code to use pip to check for available updates to those dependencies. Everything worked fine, I used subprocess.run to launch python -m pip and everything seemed fine. But that's a blocking call, and I'd rather run it asynchronously. I tried doing:

Code: Select all

           proc = await asyncio.create_subprocess_exec(
                python_exe,
                "-m pip --disable-pip-version-check list -o"
                )
            stdout, stderr = await proc.communicate()
            await proc.wait()
(where python_exe is the path to the python executable that was working just fine with subprocess). What I am getting out is:

Code: Select all

C:\\Users\\chennes\\Documents\\GitHub\\FreeCAD-Build\\bin\\python.exe: No module named  pip --disable-pip-version-check list -o'
OK, so I tried to run it more like subprocess, where the args are a list,

Code: Select all

           proc = await asyncio.create_subprocess_exec(
                python_exe,
                ["-m","pip","--disable-pip-version-check", "list", "-o"]
                )
            stdout, stderr = await proc.communicate()
            await proc.wait()
and I get:

Code: Select all

expected str, bytes or os.PathLike object, not list
What am I doing wrong? The documentation isn't clear to me: I expected it to work like subprocess, but apparently it does not.
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
agren
Posts: 40
Joined: Sat Apr 20, 2019 7:37 am

Re: Calling python.exe using asyncio

Post by agren »

You can search for "var-positional" here: https://docs.python.org/3/glossary.html

In the documentation create_subprocess_exec(program, *args) means create_subprocess_exec has the parameter program. Followed by 0 or more parameters which will be put in the tuple args.

So try something like:

Code: Select all

await asyncio.create_subprocess_exec(
    python_exe, "-m", "pip", "--disable-pip-version-check", "list", "-o"
)
ickby
Veteran
Posts: 3116
Joined: Wed Oct 05, 2011 7:36 am

Re: Calling python.exe using asyncio

Post by ickby »

also be aware that you need to have a special event loop running to use asyncio. That is not trival when using a QT application, so you need to use a extra package: https://github.com/CabbageDevelopment/qasync
User avatar
chennes
Veteran
Posts: 3868
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Calling python.exe using asyncio

Post by chennes »

Thanks both of you for the pointers. @ickby do you think I should just put the subprocess call into a dedicated QThread instead? I have had some difficulties getting those to die gracefully on Linux.
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
ickby
Veteran
Posts: 3116
Joined: Wed Oct 05, 2011 7:36 am

Re: Calling python.exe using asyncio

Post by ickby »

hm not sure. I think using QT if possible is the easiest, as no other libraries are required. But I have no experiance with it.
Annother idea would be the python subprocess module, but I'm not sure if this works with FreeCAD. The asyncio route is a bit involved for a rather small functionality, but in the end rather elegant. And if provided by FreeCAD I would not need to start the event loop for my collaboration module ;)
wmayer
Founder
Posts: 20203
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Calling python.exe using asyncio

Post by wmayer »

You can use QProcess instead. The class offers the methods start() and startDetached(). The latter won't terminate the child process if FreeCAD is terminated.
Post Reply