Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Discussions about the wiki documentation of FreeCAD and its translation.
User avatar
Kunda1
Posts: 6223
Joined: Thu Jan 05, 2017 9:03 pm

Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby Kunda1 » Mon Jan 13, 2020 12:00 pm

Background
The wiki is where we refer users to for documentation. We need to have a way to keep it up to date and accurate. My intention is to create some scripts that can report back information from the multitude that exists on the wiki, making sure its accuracy and level of completion.

Note: I'm also posting this out as necessity to record the research I've done digging in to this and figure out what method to use to script this functionality. Since pywikibot is set up and available to use, I figure we could use it for this task. Unfortunately, the pywikibot documentation is not very extensive and my coding skills are bare minimum.

Research
The GUICommand template AKA https://www.freecadweb.org/wiki/Template:GuiCommand theoretically should be used on every FreeCAD command page. It looks like this in wikimarkup:

Code: Select all

{{GuiCommand
|Name=SampleCommand
|MenuLocation=Menu → Command
|Workbenches=Wb1, Wb2
|Shortcut=DefaultShortcut
|SeeAlso=OtherTopic
|Version=0.7
}}
This code generates a table/box on the right side of the wiki page featuring relevant information about the FC command.

I'm trying to write a script that cycles through all the FreeCAD command wiki pages and checks for the completeness and accuracy of this template (for example some wiki pages may be missing template all together, whereas some may just be missing a section from it)

In the HTML source the template creates a table with the classes: fcinfobox and ct
Lets randomly examine the Draft Wire wiki page, if we look at the source we see:

Code: Select all

<table class="fcinfobox wikitable ct" width="100%" style="float: right; width: 230px; margin-left: 1em">
<tr>
<td class="ctTitle" style="font-weight: bold; font-size: 125%;">
<p><a href="/wiki/File:Draft_Wire.svg" class="image"><img alt="Draft Wire.svg" src="/wiki/images/1/10/Draft_Wire.svg" width="32" height="32" /></a><span> </span>Draft Wire
</p>
</td></tr>
<tr>
<th class="ctOdd">Menu location
</th></tr>
<tr>
<td class="ctEven">Draft → Polyline
</td></tr>
<tr>
<th class="ctOdd">Workbenches
</th></tr>
<tr>
<td class="ctEven"><a href="/wiki/Draft_Module" title="Draft Module">Draft</a>, <a href="/wiki/Arch_Module" title="Arch Module">Arch</a>
</td></tr>
<tr>
<th class="ctOdd">Default shortcut
</th></tr>
<tr>
<td class="ctEven">P L
</td></tr>
<tr>
<th class="ctOdd">Introduced in version
</th></tr>
<tr>
<td class="ctEven">0.7
</td></tr>
<tr>
<th class="ctOdd">See also
</th></tr>
<tr>
<td class="ctEven"><a href="/wiki/Draft_Line" title="Draft Line">Draft Line</a>, <a href="/wiki/Draft_BSpline" title="Draft BSpline">Draft BSpline</a>
</td></tr>
We see then that the table data classes are categorized as
  • ctTitle for the command name (in this case 'Draft Wire')
  • ctOdd for the category names like 'Default shortcut' and 'Introduced in version' etc...
  • ctEven for the variables relating to the above categories i.e. '0.7' was the FC version Draft Wire was introduced in etc...
To be continued....
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
openBrain
Posts: 2835
Joined: Fri Nov 09, 2018 5:38 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby openBrain » Mon Jan 13, 2020 12:28 pm

You can get the list of concerned pages with https://www.freecadweb.org/wiki/index.p ... imit=10000 (10000 is arbitrary).
Then IMO, better tamper the URL to get the pages in edit mode (MediaWiki source). Then it's only a few sed/awk to get everything needed. A pity I don't have a PC, it would be only some minutes to set up a bash script. :/
vocx
Posts: 2452
Joined: Thu Oct 18, 2018 9:18 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby vocx » Tue Jan 14, 2020 1:13 am

Just for completeness "GUI Commands" are those which the user can launch from the graphical interface, that is, from clicking on a button, opening the context menu (right click) and choosing an option, or from a menu (File, Edit, Tools, etc.).

This means that not every page in the wiki should have a GUI Command template, only those which are defined in the software as such. In the case of Python workbenches, these are defined by the Gui.addCommand instruction, which should be placed in a module which is imported from its InitGui.py. Without this, the command is not registered in FreeCAD.

For example, the Arch Workbench loads a bunch of commands into menus and toolbars in its InitGui.py. These are imported from the Arch.py file, which imports the individual modules (ArchWall.py, ArchBuildingPart.py, etc.).

https://github.com/FreeCAD/FreeCAD/blob ... py#L30-L46

https://github.com/FreeCAD/FreeCAD/blob ... 1240-L1242
To support the documentation effort, and code development, your donation is appreciated: paypal.
vocx
Posts: 2452
Joined: Thu Oct 18, 2018 9:18 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby vocx » Tue Jan 14, 2020 1:17 am

Kunda1 wrote:
Mon Jan 13, 2020 12:00 pm
... Since pywikibot is set up and available to use, ...
Did you update the WikiRobots page? I never felt that I needed it, but I remember the information was a bit cryptic.

It's best if you describe exactly how you managed to make it work.
To support the documentation effort, and code development, your donation is appreciated: paypal.
openBrain
Posts: 2835
Joined: Fri Nov 09, 2018 5:38 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby openBrain » Tue Jan 21, 2020 2:14 pm

Kunda1 wrote:
Mon Jan 13, 2020 12:00 pm
Just in case it may help, below is a bash line that checks on all pages using the GuiCommand template how many properties are listed.
It prints 'OK' if all properties of the template are defined in the page, and 'NOK' otherwise. This can be enhanced if more info needed :

Code: Select all

for page in $(wget --no-check-certificate "https://www.freecadweb.org/wiki/index.php?title=Special:WhatLinksHere/Template:GuiCommand&limit=10000" -O - 2>/dev/null | sed -ne '/<ul id="mw\-whatlinkshere\-list"/,/<\/ul/p' | sed -e 's/^.*<ul[^>]*>//g'| sed -e 's/<\/ul.*$//g' | sed -e 's/^.*[^(]<a href="\/wiki\/\([^"]*\).*$/https\:\/\/www\.freecadweb\.org\/wiki\/index.php\?action=edit\&title=\1/g') ; do echo $page ; if [ $(wget "$page" -O - 2>/dev/null | sed -ne '/{{GuiCommand/,/}}/p' | wc -l) = 8 ] ; then echo "OK" ; else echo "NOK" ; fi ; done
I can give further explanation on the line if one is interested in. :)
User avatar
Kunda1
Posts: 6223
Joined: Thu Jan 05, 2017 9:03 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby Kunda1 » Tue Jan 21, 2020 2:57 pm

openBrain wrote:
Tue Jan 21, 2020 2:14 pm
I can give further explanation on the line if one is interested in. :)
Yes!
Can we make this epic 1 liner more readable, please!
Thank you :D
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
openBrain
Posts: 2835
Joined: Fri Nov 09, 2018 5:38 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby openBrain » Tue Jan 21, 2020 4:18 pm

Kunda1 wrote:
Tue Jan 21, 2020 2:57 pm
Yes!
Can we make this epic 1 liner more readable, please!
Thank you :D
Lets hope following can help making it a bit less cryptic. :)
Notice this bash-like thing won't work. You need to use the one-liner. (EDIT: could also be converted to a bash script file)

Code: Select all

for page in \ # for each page in ...
    $( \ # ... the result of the following command
        wget --no-check-certificate "https://www.freecadweb.org/wiki/index.php?title=Special:WhatLinksHere/Template:GuiCommand&limit=10000" -O - 2>/dev/null \ # download said page, print to stdout, do not display stderr
        | sed -ne '/<ul id="mw\-whatlinkshere\-list"/,/<\/ul/p' \ # roughly keep lines contained in the unordered list with said ID (this is the useful data)
        | sed -e 's/^.*<ul[^>]*>//g'\ # trim what is before the list beginning as well as the list opening tag
        | sed -e 's/<\/ul.*$//g' \ # trim what is after the list end as well as the list closing tag                                                                 
        | sed -e 's/^.*[^(]<a href="\/wiki\/\([^"]*\).*$/https\:\/\/www\.freecadweb\.org\/wiki\/index.php\?action=edit\&title=\1/g' \ # starting from the hyperlink to page, forge the 'edit mode' page URL
    ) ; \ # at this point, we have a list of URLs of all pages using GuiCommand in their 'edit mode'
do echo $page ;\ # print page URL
    if [ # if ... 
        $( \ # the result of the following command ...
            wget "$page" -O - 2>/dev/null \ # download the page, print source to stdout, do not print stderr
            | sed -ne '/{{GuiCommand/,/}}/p' \ # keep only what is in the GuiCommand template
            | wc -l \ # count lines
        ) \
        = 8 ] ; \ # ... contains 8 lines
    then echo "OK" ; \ # print "OK"
    else echo "NOK" ; \ # print "NOK"
    fi ; \
done
vocx
Posts: 2452
Joined: Thu Oct 18, 2018 9:18 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby vocx » Wed Jan 22, 2020 5:30 am

openBrain wrote:
Tue Jan 21, 2020 4:18 pm
Lets hope following can help making it a bit less cryptic. :)
...
Make it a regular script. No need to be one line. And there is no need to comment each line. It's better to explain full blocks of code when necessary, not individual lines. Think of Python, "readability counts".

Also, about the GUI command template, not every field is mandatory. This means that the template could be OK even with missing fields. I suggest three results, OK full, OK partial, Not OK.

See https://www.freecadweb.org/wiki/Template:GuiCommand

The mandatory fields are only these 3 fields.

Code: Select all

{{GuiCommand
|Name=SampleCommand
|MenuLocation=Menu → Command
|Workbenches=Wb1, Wb2
}}
This is a more complete version with 6 fields. But many, most in fact, don't have a shortcut, so there is no need to add this field. Also, most commands didn't record the version in which they appeared so they may not have version field. It's good practice to add the SeeAlso field, I always do it, but it's not mandatory either.

Code: Select all

{{GuiCommand
|Name=SampleCommand
|MenuLocation=Menu → Command
|Workbenches=Wb1, Wb2
|Shortcut=DefaultShortcut
|SeeAlso=OtherTopic
|Version=0.7
}}
Some commands don't have an icon, so they may have 7 fields.

Code: Select all

{{GuiCommand
|Name=SampleCommand
|Empty=1
|MenuLocation=Menu → Command
|Workbenches=Wb1, Wb2
|Shortcut=DefaultShortcut
|SeeAlso=OtherTopic
|Version=0.7
}}
Some commands may have a special icon. It would be nice to find these pages in order to remove this special icon. Adding such icon instruction is not necessary if the name of the icon is the same name as the wiki page without the .svg or .png ending.

Code: Select all

{{GuiCommand
|Name=SampleCommand
|Icon=my_icon.svg
|MenuLocation=Menu → Command
|Workbenches=Wb1, Wb2
|Shortcut=DefaultShortcut
|SeeAlso=OtherTopic
|Version=0.7
}}
To support the documentation effort, and code development, your donation is appreciated: paypal.
openBrain
Posts: 2835
Joined: Fri Nov 09, 2018 5:38 pm

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby openBrain » Wed Jan 22, 2020 9:40 am

vocx wrote:
Wed Jan 22, 2020 5:30 am
Make it a regular script.
Stop giving orders. You aren't any kind of boss here.
No need to be one line.
Ease of testing is a good reason to do it ATM. No need to download a file and make it executable. Just copy-paste
And there is no need to comment each line. It's better to explain full blocks of code when necessary, not individual lines. Think of Python, "readability counts".
As said, goal is to explain the way the script goes to people that (maybe) aren't bash aware. This doesn't pretend to be code commenting whose goal is far different.
Also, about the GUI command template, not every field is mandatory. This means that the template could be OK even with missing fields. I suggest three results, OK full, OK partial, Not OK.
As said, it could be improved. But good to agree on what is the desired output. ;)
blablablablablablablablah
I saw you complaining about people writing long post with a heap of text. Why don't you apply your own advices ? Find it a pity because you surely have valuable knowledge and insights but all drowned into useless blabla. ;)

Let's start a simple discussion. Following list presents possible fields and script display reaction :
  • Name => Error ('NOK') if not present
  • MenuLocation => Error ('NOK') if not present
  • Workbenches => Error ('NOK') if not present
  • Shortcut => Ignored ? Warning message if not present ?
  • SeeAlso => Ignored
  • Icon => Warning message (according vocx's post above)
  • Empty => Ignored ? Warning message ? Error message if used together with 'Icon' ?
  • Version => Ignored ? Warning message ?
  • Name/AA (localization) => Ignored
Thx for feedback. I'll update accordingly. ;)
chrisb
Posts: 20759
Joined: Tue Mar 17, 2015 9:14 am

Re: Detecting if a FreeCAD command page has a complete {{GUICommand}} template

Postby chrisb » Wed Jan 22, 2020 7:48 pm

openBrain wrote:
Wed Jan 22, 2020 9:40 am
Let's start a simple discussion. Following list presents possible fields and script display reaction :
  • Name => Error ('NOK') if not present
  • MenuLocation => Error ('NOK') if not present
  • Workbenches => Error ('NOK') if not present
  • Shortcut => Ignored ? Warning message if not present ?
  • SeeAlso => Ignored
  • Icon => Warning message (according vocx's post above)
  • Empty => Ignored ? Warning message ? Error message if used together with 'Icon' ?
  • Version => Ignored ? Warning message ?
  • Name/AA (localization) => Ignored
Thx for feedback. I'll update accordingly. ;)
Is this documented somewhere? It would be interesting for people extending or translating the documentation, and I still hope for a 0.19 boost in terms of docs.