Text output (mainly stderr) from macros

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Text output (mainly stderr) from macros

Post by ian.rees »

I find the text output situation from macros in FreeCAD to be a bit confusing and would like to rationalise it, but don't want to break existing desired behaviour, so would like to get your thoughts.

We currently have 3 different places that a script's stderr and stdout can go from a macro; the executable itself, the Report View, and the Python console. There are two main ways to invoke a macro; the Execute Macro dialog (or the equivalent(?) green arrow button), or the Python console via execfile().

There are two settings to redirect Python output/errors to the Report View, but from what I can tell these only affect macros that are run via Execute Macro - if execfile() is used from the Python Console, then stdout and stderr both only go to the Python Console. Any Python output shown in the Report View also appears as stdout from the executable.

So, I've got some questions/proposals:

1 - Currently, both stderr and stdout from Python's are sent out of the FreeCAD executable's stdout. I think it should not merge stderr in to stdout implicitly.

2 - Wouldn't it be a bit more conventional if Python's stderr were sent out from the FreeCAD executable (via stderr), regardless of GUI settings?

3 - Is there a practical scenario where someone would want the Report View to not show Python stderr? It currently shows Qt warnings, for instance, and if the redirect option is turned off then AFAICT there is no visible error output from a macro run via Execute Macro. I'm just wondering if we shouldn't remove the "Redirect internal Python errors to Report View" preference, and always behave like it's turned on.

4 - If the option(s) to redirect Python output/errors to the Report View is selected, shouldn't the redirection work regardless of whether the macro was run from the Python console or Execute Macro dialog? I would expect to see the stdout/stderr shown both in the Python Console and Report View if the redirect option(s) were selected.

5 - I think it would be good if running a macro via the Execute Macro dialog (or the green button) would cause an equivalent Python command to appear in the Python Console, to make REPL style interaction a bit more natural. Right now, exectfile() is a bit of a hidden feature.
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Text output (mainly stderr) from macros

Post by simonvanderveldt »

Good points. I think where to show error messages originating from Python depends on which place Python has in FreeCAD.
If it's seen as something that's fully encapsulated a case can be made that error messages should only be shown inside FreeCAD's UI and not trickle down to FreeCAD's stderr.
If it's seen as a part of FreeCAD on the same level as the rest of the codebase then it's errors should trickle through to FreeCAD's stderr.
But should the same then be true for any stdout output?

I was surprised to see that none of the Python modules use Python's logging facility, instead custom Msg, Err, etc global functions seem to be used.
Seems like something that should be changed, which would then easily allow one to show/hide certain types of messages.
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

simonvanderveldt wrote:Good points. I think where to show error messages originating from Python depends on which place Python has in FreeCAD.
If it's seen as something that's fully encapsulated a case can be made that error messages should only be shown inside FreeCAD's UI and not trickle down to FreeCAD's stderr.
If it's seen as a part of FreeCAD on the same level as the rest of the codebase then it's errors should trickle through to FreeCAD's stderr.
But should the same then be true for any stdout output?
My feeling is that the Python interpreter is a key part of FreeCAD - it's an easy argument since substantial parts of the functionality we provide are written in Python. Even without the philosophical discussion though, it seems like our current implementation isn't self-consistent.
simonvanderveldt wrote: I was surprised to see that none of the Python modules use Python's logging facility, instead custom Msg, Err, etc global functions seem to be used.
Seems like something that should be changed, which would then easily allow one to show/hide certain types of messages.
That's a higher level than what I'm referring to - indeed Python's logging is a nice tool, but it's not going to be as helpful if the underlying stdout/stderr handling isn't done well.
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Text output (mainly stderr) from macros

Post by simonvanderveldt »

ian.rees wrote:
simonvanderveldt wrote:Good points. I think where to show error messages originating from Python depends on which place Python has in FreeCAD.
If it's seen as something that's fully encapsulated a case can be made that error messages should only be shown inside FreeCAD's UI and not trickle down to FreeCAD's stderr.
If it's seen as a part of FreeCAD on the same level as the rest of the codebase then it's errors should trickle through to FreeCAD's stderr.
But should the same then be true for any stdout output?
My feeling is that the Python interpreter is a key part of FreeCAD - it's an easy argument since substantial parts of the functionality we provide are written in Python. Even without the philosophical discussion though, it seems like our current implementation isn't self-consistent.
Sorry, I didn't want to start a philosophical discussion ;) But I do think it's important to make sure that there's a shared view on a high level, that makes it a lot easier to make these lower level/technical decisions.

Regarding your points:
ian.rees wrote:1 - Currently, both stderr and stdout from Python's are sent out of the FreeCAD executable's stdout. I think it should not merge stderr in to stdout implicitly.

2 - Wouldn't it be a bit more conventional if Python's stderr were sent out from the FreeCAD executable (via stderr), regardless of GUI settings?
When Python is seen as an integral part of FreeCAD it makes sense to always output errors to FreeCAD's stderr.
Merging of stdout and stderr should of course never happen.
ian.rees wrote:3 - Is there a practical scenario where someone would want the Report View to not show Python stderr? It currently shows Qt warnings, for instance, and if the redirect option is turned off then AFAICT there is no visible error output from a macro run via Execute Macro. I'm just wondering if we shouldn't remove the "Redirect internal Python errors to Report View" preference, and always behave like it's turned on.

4 - If the option(s) to redirect Python output/errors to the Report View is selected, shouldn't the redirection work regardless of whether the macro was run from the Python console or Execute Macro dialog? I would expect to see the stdout/stderr shown both in the Python Console and Report View if the redirect option(s) were selected.
If a user can do something that fails/causes an error and that user isn't notified of that error that seems like bad UX to me.
Your suggestion to always enable forwarding Python errors to the report view would make sense to fix this. On the other hand it's also already shown in the Python console.

It seems like no normal output from the Python console (including error messages) gets posted to the Report view.
Only when calling the FreeCAD specific functions like Msg the output is only shown in the Report View, and only in the Report View it's not shown in the Python console in that case.
Personally I don't really see the need for the Report View, I prefer to have my output only in the console. Though I guess opinions will differ on that :)
ian.rees wrote:5 - I think it would be good if running a macro via the Execute Macro dialog (or the green button) would cause an equivalent Python command to appear in the Python Console, to make REPL style interaction a bit more natural. Right now, exectfile() is a bit of a hidden feature.
+1 for consistency
Though it seems like Execute macro actually does something different than just a simple execfile()
https://github.com/FreeCAD/FreeCAD/blob ... o.cpp#L248
https://github.com/FreeCAD/FreeCAD/blob ... r.cpp#L318
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Text output (mainly stderr) from macros

Post by triplus »

ian.rees wrote:I would expect to see the stdout/stderr shown both in the Python Console and Report View if the redirect option(s) were selected.
It would be bit strange wouldn't it. If the code not interpreted in the Python console would redirect the output to the Python console?
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

triplus wrote:
ian.rees wrote:I would expect to see the stdout/stderr shown both in the Python Console and Report View if the redirect option(s) were selected.
It would be bit strange wouldn't it. If the code not interpreted in the Python console would redirect the output to the Python console?
I don't think it would be any stranger than the Report View.

But, that's not the main point of #4. I was suggesting that if the setting to redirect errors/output to report view is turned ON, then errors/output from the Python Console should appear in the Python Console AND in the report view. As things are now, the redirect settings are effectively ignored for code executed via the Python Console, which I think is inconsistent.
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

simonvanderveldt wrote: If a user can do something that fails/causes an error and that user isn't notified of that error that seems like bad UX to me.
Your suggestion to always enable forwarding Python errors to the report view would make sense to fix this. On the other hand it's also already shown in the Python console.
Right, it's a silent failure, which is no good, if the redirection settings aren't enabled. But, output is not already shown in the Python console when run via Execute Macro (I forgot to put that in the OP).
simonvanderveldt wrote:
ian.rees wrote:5 - I think it would be good if running a macro via the Execute Macro dialog (or the green button) would cause an equivalent Python command to appear in the Python Console, to make REPL style interaction a bit more natural. Right now, exectfile() is a bit of a hidden feature.
+1 for consistency
Though it seems like Execute macro actually does something different than just a simple execfile()
I think it effectively does the same thing, barring the redirection stuff which I think needs some attention. But, the implementation details don't really matter - the point is that the Python Console should show the user (who's likely working with Python anyways) how to run their macro via the console, just like we show how to do other operations in the Python console.
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Text output (mainly stderr) from macros

Post by triplus »

ian.rees wrote:I don't think it would be any stranger than the Report View.

But, that's not the main point of #4. I was suggesting that if the setting to redirect errors/output to report view is turned ON, then errors/output from the Python Console should appear in the Python Console AND in the report view. As things are now, the redirect settings are effectively ignored for code executed via the Python Console, which I think is inconsistent.
I see. Well to be honest there was a similar discussion about this area about a month back. And thinking about it again i don't really have any strong opinion on the matter. Improvements if possible in consistency and/or in just making sense. Both work for me.
wmayer
Founder
Posts: 20241
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Text output (mainly stderr) from macros

Post by wmayer »

There are two settings to redirect Python output/errors to the Report View, but from what I can tell these only affect macros that are run via Execute Macro - if execfile() is used from the Python Console, then stdout and stderr both only go to the Python Console.
Which perfectly makes sense because having output in the Report view and Python console doesn't have any benefit and I would find it rather annoying.
Currently, both stderr and stdout from Python's are sent out of the FreeCAD executable's stdout. I think it should not merge stderr in to stdout implicitly.
What exactly do you consider the executable's stdout? And where is stderr merged to stdout?
Is there a practical scenario where someone would want the Report View to not show Python stderr?
Well, actually the redirection is intended for developers to make functions written in Python robust enough. For a user it shouldn't happen that Python raises any errors.
It currently shows Qt warnings, for instance
But only in debug mode, not in release mode. Also aimed for developers.
I would expect to see the stdout/stderr shown both in the Python Console and Report View if the redirect option(s) were selected.
See above. I don't see any benefit and find this annoying behaviour.
Only when calling the FreeCAD specific functions like Msg the output is only shown in the Report View, and only in the Report View it's not shown in the Python console in that case.
Because this is a completely different mechanism.
Though it seems like Execute macro actually does something different than just a simple execfile()
Internally it uses PyRun_File of C API of Python.
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Text output (mainly stderr) from macros

Post by simonvanderveldt »

wmayer wrote:Well, actually the redirection is intended for developers to make functions written in Python robust enough. For a user it shouldn't happen that Python raises any errors.
What should happen when an error does happen?
Post Reply