Text output (mainly stderr) from macros

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

wmayer wrote:
Currently, both stderr and stdout from Python's are sent out of the FreeCAD executable's stdout. I think it should not merge stderr in to stdout implicitly.
What exactly do you consider the executable's stdout? And where is stderr merged to stdout?
If I make a script called aaa.FCMacro:

Code: Select all

from __future__ import print_function
import sys
print("This is to stdout")
print("This is to stderr", file=sys.stderr)
Then run it (with both of the redirect preferences enabled) like this:

Code: Select all

bin/FreeCAD /path/to/aaa.FCMacro > output.stdout 2> output.stderr
output.stdout will contain both lines from the script, and output.stderr will be empty.

However, if I run FreeCADCmd instead, then the stderr line ends up in output.stderr. So, there's something about the redirection in the GUI executable that is merging stderr in to stdout.

Will try to respond to the rest of your post later, but need to run now. Thanks! -Ian-
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

wmayer wrote:
There are two settings to redirect Python output/errors to the Report View, but from what I can tell these only affect macros that are run via Execute Macro - if execfile() is used from the Python Console, then stdout and stderr both only go to the Python Console.
Which perfectly makes sense because having output in the Report view and Python console doesn't have any benefit and I would find it rather annoying.
In that scenario, why would you have both open? Maybe I'm not understanding the value of the Report View - to me it's just a read-only version of the Python Console (which has the additional features of dumping Qt errors in debug mode, and of producing output from the FreeCAD executable). The only reason I usually have the Report View open, is because it's the only way to get output (including errors) from a macro run via Execute Macro.

At any rate, I think that change would provide a benefit, because it makes the redirect preferences do what their labels say they do. As things are now, it's confusing because the redirect settings have effects that depend on subtleties in how Python code is executed. So, I can't for example test a line from a script that's meant to produce stderr output, by running it from the Python Console.

Perhaps a better approach to solve my problem: What if the "Off" state of the redirect options caused Python output/errors to go to the Python Console? Right now, the checked/unchecked means Report View/NULL, but it would instead mean Report View/Python Console. -Ian-
Jee-Bee
Veteran
Posts: 2566
Joined: Tue Jun 16, 2015 10:32 am
Location: Netherlands

Re: Text output (mainly stderr) from macros

Post by Jee-Bee »

ian.rees wrote:In that scenario, why would you have both open? Maybe I'm not understanding the value of the Report View - to me it's just a read-only version of the Python Console (which has the additional features of dumping Qt errors in debug mode, and of producing output from the FreeCAD executable). The only reason I usually have the Report View open, is because it's the only way to get output (including errors) from a macro run via Execute Macro.
I have the same...
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Text output (mainly stderr) from macros

Post by wmayer »

output.stdout will contain both lines from the script, and output.stderr will be empty.
OK, I see. This is caused by the implementation of ConsoleObserverStd::Error. So, instead of using printf it should use fprintf(stderr). When doing so the redirection works as expected.
In that scenario, why would you have both open? Maybe I'm not understanding the value of the Report View - to me it's just a read-only version of the Python Console (which has the additional features of dumping Qt errors in debug mode, and of producing output from the FreeCAD executable).
The Report View has basically nothing to do with the Python console. The class Base::Console has the methods Message, Warning, Error and Log and it internally holds a list of instances of subclasses of ConsoleObserver. One of these subclasses is ReportOutput that prints the messages to the Report View.

When the Python redirection is enabled then all issued messages go through the corresponding methods of Base::Console and also prints them to the Report View.
The only reason I usually have the Report View open, is because it's the only way to get output (including errors) from a macro run via Execute Macro.
It's not the main purpose of the Report View to get Python messages, it's only one of its purposes.
So, I can't for example test a line from a script that's meant to produce stderr output, by running it from the Python Console.
When you run a line from the Python console then it always prints messages and errors to the Python console, too.
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Text output (mainly stderr) from macros

Post by wmayer »

Perhaps a better approach to solve my problem: What if the "Off" state of the redirect options caused Python output/errors to go to the Python Console? Right now, the checked/unchecked means Report View/NULL, but it would instead mean Report View/Python Console. -Ian-
There is another redirection done at startup which was needed to fix a bug on Windows. See issue #831

You will find the relevant code in https://github.com/FreeCAD/FreeCAD/blob ... i.cpp#L176
So, for system other than Windows it might be OK to disable this initial redirection. If this code is disabled for macOS and the redirection in the preferences are switched off then the default Python behaviour is active.
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

wmayer wrote:
In that scenario, why would you have both open? Maybe I'm not understanding the value of the Report View - to me it's just a read-only version of the Python Console (which has the additional features of dumping Qt errors in debug mode, and of producing output from the FreeCAD executable).
The Report View has basically nothing to do with the Python console.
Right, this is one of the issues I'm talking about. The Report View is currently the only way to get output from a large portion of our Python usage. So as a practical matter, there is a relationship between Python output and the Report View. Separately, there is a relationship between Python output and the Python Console, but it is quite different.

For example - why can certain Python actions (optionally) generate output in the Report View, but never in the Python Console? Or, why do some things generate output in only the Python Console, when I've explicitly requested that FreeCAD forward them to the Report View (perhaps because that is the only way for them to get to the executable's stdout/stderr for some reason).

I understand (more or less) how/why this works at a code level. However, from a design/user/etc perspective, I think this is situation is quite unintuitive. So I'd like to consider some changes that I think are individually small, but would help things fit together better.

Thanks for your responses so far. I'll try to put together some prospective changes so there's something a bit more concrete to discuss. -Ian-
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Text output (mainly stderr) from macros

Post by triplus »

Average end user isn’t interested in the verbose output. And for such user Python console has no real value beyond recording a macro or typing in a code snippet or two from the forum. Output therefore needs to happen there as it happens. In the Python console.

Developer on the other hand is interested in verbose output. Qt, Python ... related. Report view is the place for that.

Does it make sense? To me it does. Can behaviour be changed? It always can be changed. Does it need to change as current behaviour doesn't make sense? No. As current behaviour makes sense.

Therefore if you plan to change the behaviour don't focus too much on current behaviour as the reason for introducing the change. Rather base it on some other aspect. For example something is missing ATM and you will implement that. Or something like that.
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Text output (mainly stderr) from macros

Post by wmayer »

For example - why can certain Python actions (optionally) generate output in the Report View, but never in the Python Console?
The Python console is a medium which a user interactively works with. Here he types in some code and here he expects an eventual feedback.

Now when running a function that is written in Python (by pressing a button) that creates some output then why would you expect it to be shown in the Python console while at this point the user does not interact with it? Doing so would cause a lot of confusion when suddenly the output appears in the Python console when e.g. the user first entered some code into the console, then pressed a button somewhere and now sees an error in the console. Then he would assume that the entered code caused the error while the function is the culprit.
Or, why do some things generate output in only the Python Console, when I've explicitly requested that FreeCAD forward them to the Report View.
Besides user interaction with the Python console the only other way that adds some content to Python console is when logging of Python scripts is enabled. This is only done to add each code line to the history of the console. But the logging facility has absolutely nothing to do with stdout/stderr or Python output.
I understand (more or less) how/why this works at a code level. However, from a design/user/etc perspective, I think this is situation is quite unintuitive. So I'd like to consider some changes that I think are individually small, but would help things fit together better.
I don't see why the current behaviour should be unintuitive. IMO, if we start to mix up the clear separation between Report view and Python console then things start to be confusing and unusable.
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Text output (mainly stderr) from macros

Post by ian.rees »

wmayer wrote: Now when running a function that is written in Python (by pressing a button) that creates some output then why would you expect it to be shown in the Python console while at this point the user does not interact with it?
Well, I could turn that around and ask why they would expect it to appear in the Report View, which is also not being interacted with?
wmayer wrote:Doing so would cause a lot of confusion when suddenly the output appears in the Python console when e.g. the user first entered some code into the console, then pressed a button somewhere and now sees an error in the console.
That is a valid hypothesis, but mine is different.
wmayer wrote:This is only done to add each code line to the history of the console. But the logging facility has absolutely nothing to do with stdout/stderr or Python output.
I understand that is how it is, my point is that the way it is, is not intuitive to me.
wmayer wrote: I don't see why the current behaviour should be unintuitive. IMO, if we start to mix up the clear separation between Report view and Python console then things start to be confusing and unusable.
I understand that you don't understand what I understand ;). That's why I am trying to explain! And, why I would like to make up some prototypes that people could try.
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Text output (mainly stderr) from macros

Post by triplus »

ian.rees wrote:And, why I would like to make up some prototypes that people could try.
I would imagine average FreeCAD user doesn’t really care and would likely perceive such output as spam. Developer needs to look in the Report view regardless as there is other information outputted there. Therefore Report view would continue to be the correct place to look for such output and Python console would likely just create more (sometimes unrelated) spam.

P.S. You said you detected some output that gets lost. If that happens and you can redirect it successfully. That would highly likely count as an improvement.
Post Reply