Late introduction, question on tests

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
mumme
Posts: 35
Joined: Sun Jun 21, 2015 4:52 pm

Late introduction, question on tests

Postby mumme » Wed Jul 08, 2015 9:07 pm

Hi, I haven't formally introduced myself.
My name is Fredrik Johansson, live in the south of Sweden. English is not my native tounge so please excuse me if I write/do something awkward.

I have done some small contributions to the Path workbench because I am interested in this, although I don't have much knowledge about machining.
I do some coding as a hobby when I find time for it, my activity in open source project can be very sporadic because of lack of time. But I hope I can do my small part in making FreeCAD become even more awesome than it is today, that is when I do have the time.


I have a question, if I were to try and write some regression tests for the different objects in Path module. Where (in what dir) should i put those, and how would those be called from the test workbench? I mean how does the test mechanism work?
wmayer
Site Admin
Posts: 14614
Joined: Thu Feb 19, 2009 10:32 am

Re: Late introduction, question on tests

Postby wmayer » Thu Jul 09, 2015 1:51 pm

Hi Fredrik,

welcome to the community.
I have a question, if I were to try and write some regression tests for the different objects in Path module. Where (in what dir) should i put those, and how would those be called from the test workbench? I mean how does the test mechanism work?
Currently we use a unit test framework based on Python.

For a new test you have to write a class. The class must inherit from unittest.TestCase, provide one or more test functions which start with "test", a setUp and a tearDown method.

Code: Select all

import unittest

class MyTest(unittest.TestCase):
  def setUp(self):
    # prepare some stuff in order to run the test
    ...
    
  def testCase1(self):
    # check something
    expr = ... # is True or False
    self.failUnless(expr)
    
  def testCase2(self):
    # check something else
    ...
    
  def tearDown(self):
    # free the stuff you did in setUp
Save your tests into a file e.g. TestPathApp.py. This file should be located in the Path module directory. Finally, add a line

Code: Select all

suite.addTest(unittest.defaultTestLoader.loadTestsFromName("TestPathApp") )
into TestApp.py
mumme
Posts: 35
Joined: Sun Jun 21, 2015 4:52 pm

Re: Late introduction, question on tests

Postby mumme » Thu Jul 09, 2015 6:44 pm

Ok, thanks!

Will look into it.
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Late introduction, question on tests

Postby ian.rees » Sat Jul 11, 2015 6:59 am

Hi Fredrik, thanks for volunteering to add unit tests! -Ian-
User avatar
teobo
Posts: 405
Joined: Fri Feb 21, 2014 11:23 am

Re: Late introduction, question on tests

Postby teobo » Thu Mar 03, 2016 11:27 pm

Hi,
me too, I got questions on testing, so I add them here.

Code: Select all

suite.addTest(unittest.defaultTestLoader.loadTestsFromName("TestPathApp") )
into TestApp.py
1) How is the above test run and how can I run it manually?
EDIT: just found out for starting from the embedded python console(1): all(1) Tests:
1.1.1:

Code: Select all

>>> import sys, unittest, FreeCAD, TestApp;
>>> unittest.TextTestRunner().run(TestApp.All())
Polynomial: f(x,y)=0.333333*x^2+0.333333*y^2+0.000000*x*y+0.000000*x+0.000000*y-0.444444
Polynomial: f(x,y)=-1.775872*x^2+0.000000*y^2-0.000000*x*y+0.255864*x+0.000000*y+0.523170
Polynomial: f(x,y)=0.500000*x^2+0.500000*y^2+0.000000*x*y+0.000000*x+0.000000*y-0.666667
<unittest.runner.TextTestResult run=91 errors=0 failures=0>
...........................................................................................
----------------------------------------------------------------------
Ran 91 tests in 124.629s
1.1.2 and for a single test:

Code: Select all

>>> suite = unittest.TestSuite()
>>> suite.addTest(unittest.defaultTestLoader.loadTestsFromName("TestFem"))
>>> unittest.TextTestRunner().run(suite)
<unittest.runner.TextTestResult run=1 errors=0 failures=0>
.
----------------------------------------------------------------------
Ran 1 test in 0.323s

OK
Is it really ok?
and yes running them from outside freecad, cases 1.2.1,1.2.2: do not know yet.
EDIT END
2) Given one is writing a unittest module for a macro. What would be the best practice of running it?

Experimented running it from the bash console, clearly without success, then from the freecad python console with some command lines randomly gathered from the internet. But it does not seem well done to me.

Code: Select all

foo = imp.load_source("Testpointtopost",os.environ["FEMProjScripts"]+"sourcesfc/Testpointtopost.py")
#suite.addTest(unittest.defaultTestLoader.loadTestsFromName("Testpointtopost")
#unittest.TextTestRunner().run(suite)
If someone could help me how the frequent unittests running for a freecad macro is done best, I would be thankful.
Tia
Last edited by teobo on Fri Mar 04, 2016 1:32 pm, edited 3 times in total.
User avatar
teobo
Posts: 405
Joined: Fri Feb 21, 2014 11:23 am

State of my investigations: runnning test suites

Postby teobo » Fri Mar 04, 2016 12:03 pm

hi, with this post I want to systematically collect the trigger cases of unit tests from my newbie point of view, for others to help me or to make use of it. There are around 2*2*2=8 cases plus some details. As well I put
success state +/-.
works against simplified module:*
blocking or hindering: errors messages: fat
The unittest file used:
https://www.dropbox.com/s/vjdgjfbn1nyu1 ... st.py?dl=0
The simplified module file tested against:
https://www.dropbox.com/s/kh6vl3qmgoly1 ... pp.py?dl=0

Fazit: At the moment I found workable: all ways go. Except:launch Gui tests from bash command line: Cannot load Gui module in console application.

1.1.1 system workbench tests,python console, all tests +, see above
1.1.2 python console, single tests +, see above
1.2.1 system workbench tests, bash console, all tests -
1.2.1 system workbench tests, python console, single tests -

2.1.1 +macro, python console, execfile: annoying "System exit: The application is still running. Do you want to exit without saving your data?" pops up

Code: Select all

##### 
myfile="../pub/sourcesfc/Testpointtopost.py"
execfile(myfile)
Answer 1:
replacing snipped in test_*.py and pop up disappears.

Code: Select all

#if __name__ == '__main__':
    #unittest.main()
suite = unittest.TestLoader().loadTestsFromTestCase(ReportTestCase)
unittest.TextTestRunner(verbosity=2).run(suite)
2.1.2 + macro, python console, suite:UnboundLocalError

Code: Select all

suite = unittest.TestSuite()
suite.addTest(unittest.defaultTestLoader.loadTestsFromName("Testpointtopost"))
r = unittest.TextTestRunner(stream=sys.stdout, verbosity=2)
r.run(suite)

UnboundLocalError: local variable 'os' referenced before assignment
Solution state:+
def setUp(self):
#import os


2.2.1 +macro, bash console, path.append in header: "ImportError"
snippet to head of Testfile:

Code: Select all

import sys
sys.path.append("/usr/lib/freecad/lib/")
import FreeCAD as App
import FreeCADGui as Gui

Code: Select all

python2.7 /home.../Testpointtopost.py

Traceback (most recent call last):
  File "/home/.../Testpointtopost.py", line 21, in <module>
    import FemGui
ImportError: Cannot load Gui module in console application.
Solution state:+without Gui

Code: Select all

#import FemGui
#import pointtopost
2.2.2 +macro, bash console, -m unittest

Code: Select all

:~$ export PYTHONPATH=$PYTHONPATH:/home...sourcesfc/
:~$python -m unittest Testpointtopost
python -m unittest Testpointtopost.ReportTestCase.test_prepares_resultdata_valid_report_interface_files
Ran 1 test in 0.038s
python -m unittest discover -p Test*py -v