FEM Workbench Testing - GSoC Proposal

Contributions from the participants, questions and answers to their projects.
Discussions of proposals for upcoming events.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
m42kus
Posts: 159
Joined: Tue Sep 02, 2014 9:41 am

Re: FEM Workbench Testing - GSoC Proposal

Post by m42kus »

Kunda1 wrote: Sat Jul 27, 2019 1:22 pm lets split off this tangent in to another thread. Is there one already or can someone suggest a name?
How about "Consensus on API Documentation" or "Consensus on Source Documentation"?
My GSoC Devlog: ceeli.github.io/gsoc
m42kus
Posts: 159
Joined: Tue Sep 02, 2014 9:41 am

Re: FEM Workbench Testing - GSoC Proposal

Post by m42kus »

yorik wrote: Sat Jul 27, 2019 9:17 pm About sphinx/ doxygen:

Some years ago I tried to use sphinx to generate the python API docs because, indeed, they look nicer. However, they fall short at the same place as Doxygen: They are unable to document C++-generated modules and classes. So at that time we made some "fake" modules (that you can still find in TemplatePyMod). that would allow sphinx to crawl through them. But obviosuly it was a PITA to maintain and got quickly obsolete.

Also, maintaining two systems is also annoying, once doxygen is able to parse python code too. So I myself basically gave up of sphinx. The code is still there but it's not used anywhere anymore.

At some point, I think we should build some tools that automatically build fake python files from selected FreeCAD modules.. It would actually not be too hard, python has many tools to do that.

The real problem with doxygen is that the html it produces is rather ugly. But there are several other possibilities that we could try, that I saw on the net, such as generating markdown files...
Thanks for the response. I have to say that I don't agree with the maintenance point you are making. In my experience it requires very little set up and maintenance work as long as tools are used for what they are intended. I suggested using two tools with the taught that it would be much easier than pushing one tool to its limits where the real time consuming work normally starts. With that said I of course agree that having one system that handles C++ and Python very well would be the best option.

I'd also like to repeat what I wrote earlier in the thread. The most important aspect is to produce a end result that is actually helpful to us developers and newcomers. If we fail to meet this goal it was all for nothing. And I don't think that we'll be able to reach this goal for Python by using doxygen.
My GSoC Devlog: ceeli.github.io/gsoc
User avatar
Kunda1
Veteran
Posts: 13434
Joined: Thu Jan 05, 2017 9:03 pm

Re: FEM Workbench Testing - GSoC Proposal

Post by Kunda1 »

Heads up: the discussion on frameworks regarding documenting the c++ and python sourcecode for FC is continuing in:
Discussion: Consensus on API Documentation (link)
Alone you go faster. Together we go farther
Please mark thread [Solved]
Want to contribute back to FC? Checkout:
'good first issues' | Open TODOs and FIXMEs | How to Help FreeCAD | How to report Bugs
m42kus
Posts: 159
Joined: Tue Sep 02, 2014 9:41 am

Re: FEM Workbench Testing - GSoC Proposal

Post by m42kus »

Kunda1 wrote: Sun Jul 28, 2019 7:48 pm Heads up: the discussion on frameworks regarding documenting the c++ and python sourcecode for FC is continuing in:
Discussion: Consensus on API Documentation (link)
Thanks! Can I help you migrate the posts in any way? :)
My GSoC Devlog: ceeli.github.io/gsoc
m42kus
Posts: 159
Joined: Tue Sep 02, 2014 9:41 am

Re: FEM Workbench Testing - GSoC Proposal

Post by m42kus »

I just committed stuff that makes it very easy to execute a simulation and verify the results using only the VTK Result Object as a source. That means that all solvers of FreeCAD are supported. If the solver doesn't directely output a VTK Result one is automatically generated from the Result Object. Have a look at the following unittest module to get an idea of how such tests are written: https://github.com/ceeli/FreeCAD_bhb/bl ... roblems.py
The library that enables that is the new fem test support library: https://github.com/ceeli/FreeCAD_bhb/bl ... support.py. Documentation will be added soon ;)
My GSoC Devlog: ceeli.github.io/gsoc
m42kus
Posts: 159
Joined: Tue Sep 02, 2014 9:41 am

Re: FEM Workbench Testing - GSoC Proposal

Post by m42kus »

Hi, the third coding phase is over and this is a summery of the current state of my testing project. This summery is divided into three parts: Python testing, C++ testing and source documentation (Python and C++).

The python changes are managed in a pull request on github. It contains two new modules to ease the development of FreeCAD tests. One for App tests (Mod/Test/AppTestSupport.py) and on for Gui tests (Mod/Test/GuiTestSupport.py). It also introduces a new way of managing collections of tests using the test packages directly (via the load_tests protocol). This is a little simpler in my opinion and avoids listing all tests again. There are now two subpackages femtest.app and femtest.gui containing tests that can be executed without the gui and those that require the gui respectively.

Test coverage of the python source can now be measured by using the new --cov option together with the -t option. This generates a report summery at the end of the test output and a html coverage report in the ./coverage directory. The html report shows which lines were executed for each file in detail. Optionally a package or module can be specified after the --cov option to limit the source files that are measured for coverage to the package or module. The test collections (e.g. femtest.app) specify which source files should be covered themselfs so no package or module must be specified to generate a useful report. See the documentation of src/App/FreeCADTest.py for more detail. The pull request also contains python source documentation and new test modules. A new very interesting kind of test can be found in femtest/app/test_problems.py. The tests in this module execute textbook examples and verify the results afterwards. The heavy lifting of these tests is done by the femtest/app/support_solver.py support module. All new files are documented so if you want to know more just look at the source ;).

I was less successful on the C++ side of things. There is a prototype which successfully tests the FemAnalysis and FemConstraint modules but I encountered a lot of problems when I tried to integrate the C++ unit tests into the test workflow of FreeCAD. My goal was to make it possible to execute C++ tests just like Python tests out of a running instance of FreeCAD but I could not produce a satisfying result by using googletest. The problem is that googletest doesn't expose it's internals. The only public interface is the command line interface which can be configured exactely once per program execution. This makes it impossible to do test case selection in a running instance of FreeCAD. I think we have to look at alternative unit test frameworks. I already had a look at boost.Test and Cpputest both of which expose a public API other than the command line interface. However I did create a pull request to incooperate the source documentation I created on github.

The last part concerns the source documentation and API documentation generation (html docs). I spent much more time on this part than I originally expected. I created three different prototypes:

Simpified C++ Doxygen
Simpified Python Doxygen
Sphinx Python

There are two related questions which need to be answered: Which tools to use and how to do source documentation? A discussion about this has started in this thread which will be continued in Consensus on API Documentation.
My GSoC Devlog: ceeli.github.io/gsoc
vocx
Veteran
Posts: 5197
Joined: Thu Oct 18, 2018 9:18 pm

Re: FEM Workbench Testing - GSoC Proposal

Post by vocx »

Why are you committing to Bernd's repository? Is the idea that he reviews the stuff, and then a separate pull request is made against the main FreeCAD repository?
m42kus wrote: Mon Aug 26, 2019 2:14 pm ...

The last part concerns the source documentation and API documentation generation (html docs). I spent much more time on this part than I originally expected. I created three different prototypes:
...
About the documentation, what exactly did you do? Did you just create the HTML templates, or did you set up an automated way to create the documentation in some clever way? As I said before, it would be great if you could write a report on how you used Sphinx. Then I could pick up that faster. I think we should really investigate the Sphinx+Breathe route which basically uses Doxygen for parsing C++ sources.

As for the current method of just using Doxygen for everything, I don't dislike the documentation of Python, actually. It's okay as long as the docstrings use a proper style, like numpy. See Python codeformating Draft, Arch in the regard of pep8 etc.

Now what I see more troublesome, is all those Python commands that are created by C++, or that are in XML files. I'm not sure how that all works. In some cases they seem to be documented inside the C++ sources, so they aren't available to be parsed by Doxygen or Sphinx. Yorik has mentioned that maybe we should create some fake Python sources, just so they are picked up by Doxygen (or Sphinx). I think it would make sense to investigate this.

For example

Code: Select all

class Module : public Py::ExtensionModule<Module>
{
public:
    Module() : Py::ExtensionModule<Module>("Drawing")
    {
        add_varargs_method("project",&Module::project,
            "[visiblyG0,visiblyG1,hiddenG0,hiddenG1] = project(TopoShape[,App.Vector Direction, string type])\n"
            " -- Project a shape and return the visible/invisible parts of it."
        );
        add_varargs_method("projectEx",&Module::projectEx,
            "[V,V1,VN,VO,VI,H,H1,HN,HO,HI] = projectEx(TopoShape[,App.Vector Direction, string type])\n"
            " -- Project a shape and return the all parts of it."
        );
        add_keyword_method("projectToSVG",&Module::projectToSVG,
            "string = projectToSVG(TopoShape[, App.Vector direction, string type, float tolerance, dict vStyle, dict v0Style, dict v1Style, dict hStyle, dict h0Style, dict h1Style])\n"
            " -- Project a shape and return the SVG representation as string."
        );
This all defines functions like Drawing.project(), Drawing.projectEx(), Drawing.projectToSVG(), and that's it. That's the entire documentation.

So a Python module with proper docstrings could be used to properly document the function.

Code: Select all

def project(shape, direction=Vector(1,0,0), type="Unknown"):
    """Project a shape and return the visible/invisible parts of it.

    The long and extensive docstring would be here.

    Parameters
    -------------
    shape : some shape

    direction : Base::Vector3, optional

    type : str, optional
    """
    return Drawing.project(shape, direction, type)
Always add the important information to your posts if you need help. Also see Tutorials and Video tutorials.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
User avatar
bernd
Veteran
Posts: 12849
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland
Contact:

Re: FEM Workbench Testing - GSoC Proposal

Post by bernd »

vocx wrote: Tue Aug 27, 2019 6:31 pm Why are you committing to Bernd's repository? Is the idea that he reviews the stuff, and then a separate pull request is made against the main FreeCAD repository?
In case of FEM only PRs this has been the work flow in recent years. Like a prereview.
User avatar
Kunda1
Veteran
Posts: 13434
Joined: Thu Jan 05, 2017 9:03 pm

Re: FEM Workbench Testing - GSoC Proposal

Post by Kunda1 »

How is the merge on this going?
m42kus wrote: Mon Aug 26, 2019 2:14 pm
@m42kus has yorik contacted you about wrapping things up?
Alone you go faster. Together we go farther
Please mark thread [Solved]
Want to contribute back to FC? Checkout:
'good first issues' | Open TODOs and FIXMEs | How to Help FreeCAD | How to report Bugs
m42kus
Posts: 159
Joined: Tue Sep 02, 2014 9:41 am

Re: FEM Workbench Testing - GSoC Proposal

Post by m42kus »

Kunda1 wrote: Thu Sep 12, 2019 9:55 pm How is the merge on this going?
m42kus wrote: Mon Aug 26, 2019 2:14 pm
@m42kus has yorik contacted you about wrapping things up?
No but this is a good reminder, I was on holiday directly after the GSoC but I’ll do some wrapping up now.
My GSoC Devlog: ceeli.github.io/gsoc
Post Reply