Testing and Quality Assurance

Have some feature requests, feedback, cool stuff to share, or want to know where FreeCAD is going? This is the place.
Forum rules
Be nice to others! Read the FreeCAD code of conduct!
Post Reply
Jason C. Wells
Posts: 32
Joined: Fri May 06, 2016 7:43 pm
Location: Burien, WA

Testing and Quality Assurance

Post by Jason C. Wells »

Is there any formal testing or quality assurance associated with the project?

Regards,
Jason
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: Testing and Quality Assurance

Post by yorik »

Hi,

You should explain a bit more what you have in mind, but if you mean: Is there an external body in charge of assessing the quality of FreeCAD, the answer is no. However, that doesn't mean FreeCAD has no quality :) Basically I guess the quality comes more from the fact that there is an awesome, very dedicated community of developers and users that tests things thoroughly than from following any particular method. We have a couple of tools of course (test suite, travis CI, bug tracker, etc) but I suppose that doesn't qualify as quality assurance. Also, I suppose that doing that formally costs a lot of money, which places it beyond the reach of an open-source project like ours.

I'm no software specialist, but it seems to me that "software quality" is a pretty vague concept, no? How does someone judge that?

Also, out of curiosity, why do you ask? Would there be any need for that?
Jason C. Wells
Posts: 32
Joined: Fri May 06, 2016 7:43 pm
Location: Burien, WA

Re: Testing and Quality Assurance

Post by Jason C. Wells »

I think the test suite qualifies.

Not being a developer by trade, I was thinking along the lines of a series of use cases that get tested with each release. Some of the use cases would be straight forward. Some of the use cases would pathologically difficult things like a mobius strips or a torus with a toroid radius smaller than the swept profile radius.

If there was such a thing, I'd be glad to run through the use cases and make reports.

Regards,
Jason C. Wells
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: Testing and Quality Assurance

Post by PrzemoF »

Depending how you define "formal". Every single travis-ci build runs a set of tests. Failed test --> failed build. An example: https://travis-ci.org/FreeCAD/FreeCAD/j ... 8313#L6399

If you want to get involved adding new tests or extending existing tests would be really appreciated.
User avatar
wandererfan
Veteran
Posts: 6268
Joined: Tue Nov 06, 2012 5:42 pm
Contact:

Re: Testing and Quality Assurance

Post by wandererfan »

Jason C. Wells wrote:I think the test suite qualifies.
Strictly, the "Test Framework" is a regression test suite. It can only check if changes break existing functionality.
Quality Assurance involves steps taken in the development process - Code standards, process steps, design reviews, etc,
Quality Control is mostly testing - developing use cases, test data, etc and performing the actual testing, including regression testing. Also inspections.

Our QA and QC are fairly informal. Some designs are reviewed in the forums, werner and yorik do some inspections of delivered code, the early adopters test & report, ...

We would need a larger and more formally organized operation to implement real QA & QC.

Sorry, past life came bubbling up. :D

wf
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: Testing and Quality Assurance

Post by PrzemoF »

OK, from that angle that also qualifies: viewtopic.php?f=18&t=12833

Also we tried to use https://en.wikipedia.org/wiki/Phabricator for code reviews (with code pre-testing using flake8, approval and so on), but it never really took off the ground. It was used mostly by bernd & me for FEM wb development.
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: Testing and Quality Assurance

Post by yorik »

Jason C. Wells wrote:If there was such a thing, I'd be glad to run through the use cases and make reports.
It is easy to use, in FreeCAD switch to the Test framework workbench, then self-test -> Run
This will run all the available tests one after the other. But as wandererfan says, this only tests if everything still works correctly with the latest changes in the code.

As Prezmo says, these test suites could be expanded a lot, to include more subtle cases, etc.
Post Reply