Is there any formal testing or quality assurance associated with the project?
Regards,
Jason
Testing and Quality Assurance
Forum rules
Be nice to others! Read the FreeCAD code of conduct!
Be nice to others! Read the FreeCAD code of conduct!
-
- Posts: 32
- Joined: Fri May 06, 2016 7:43 pm
- Location: Burien, WA
Re: Testing and Quality Assurance
Hi,
You should explain a bit more what you have in mind, but if you mean: Is there an external body in charge of assessing the quality of FreeCAD, the answer is no. However, that doesn't mean FreeCAD has no quality Basically I guess the quality comes more from the fact that there is an awesome, very dedicated community of developers and users that tests things thoroughly than from following any particular method. We have a couple of tools of course (test suite, travis CI, bug tracker, etc) but I suppose that doesn't qualify as quality assurance. Also, I suppose that doing that formally costs a lot of money, which places it beyond the reach of an open-source project like ours.
I'm no software specialist, but it seems to me that "software quality" is a pretty vague concept, no? How does someone judge that?
Also, out of curiosity, why do you ask? Would there be any need for that?
You should explain a bit more what you have in mind, but if you mean: Is there an external body in charge of assessing the quality of FreeCAD, the answer is no. However, that doesn't mean FreeCAD has no quality Basically I guess the quality comes more from the fact that there is an awesome, very dedicated community of developers and users that tests things thoroughly than from following any particular method. We have a couple of tools of course (test suite, travis CI, bug tracker, etc) but I suppose that doesn't qualify as quality assurance. Also, I suppose that doing that formally costs a lot of money, which places it beyond the reach of an open-source project like ours.
I'm no software specialist, but it seems to me that "software quality" is a pretty vague concept, no? How does someone judge that?
Also, out of curiosity, why do you ask? Would there be any need for that?
-
- Posts: 32
- Joined: Fri May 06, 2016 7:43 pm
- Location: Burien, WA
Re: Testing and Quality Assurance
I think the test suite qualifies.
Not being a developer by trade, I was thinking along the lines of a series of use cases that get tested with each release. Some of the use cases would be straight forward. Some of the use cases would pathologically difficult things like a mobius strips or a torus with a toroid radius smaller than the swept profile radius.
If there was such a thing, I'd be glad to run through the use cases and make reports.
Regards,
Jason C. Wells
Not being a developer by trade, I was thinking along the lines of a series of use cases that get tested with each release. Some of the use cases would be straight forward. Some of the use cases would pathologically difficult things like a mobius strips or a torus with a toroid radius smaller than the swept profile radius.
If there was such a thing, I'd be glad to run through the use cases and make reports.
Regards,
Jason C. Wells
Re: Testing and Quality Assurance
Depending how you define "formal". Every single travis-ci build runs a set of tests. Failed test --> failed build. An example: https://travis-ci.org/FreeCAD/FreeCAD/j ... 8313#L6399
If you want to get involved adding new tests or extending existing tests would be really appreciated.
If you want to get involved adding new tests or extending existing tests would be really appreciated.
- wandererfan
- Veteran
- Posts: 6268
- Joined: Tue Nov 06, 2012 5:42 pm
- Contact:
Re: Testing and Quality Assurance
Strictly, the "Test Framework" is a regression test suite. It can only check if changes break existing functionality.Jason C. Wells wrote:I think the test suite qualifies.
Quality Assurance involves steps taken in the development process - Code standards, process steps, design reviews, etc,
Quality Control is mostly testing - developing use cases, test data, etc and performing the actual testing, including regression testing. Also inspections.
Our QA and QC are fairly informal. Some designs are reviewed in the forums, werner and yorik do some inspections of delivered code, the early adopters test & report, ...
We would need a larger and more formally organized operation to implement real QA & QC.
Sorry, past life came bubbling up.
wf
Re: Testing and Quality Assurance
OK, from that angle that also qualifies: viewtopic.php?f=18&t=12833
Also we tried to use https://en.wikipedia.org/wiki/Phabricator for code reviews (with code pre-testing using flake8, approval and so on), but it never really took off the ground. It was used mostly by bernd & me for FEM wb development.
Also we tried to use https://en.wikipedia.org/wiki/Phabricator for code reviews (with code pre-testing using flake8, approval and so on), but it never really took off the ground. It was used mostly by bernd & me for FEM wb development.
Re: Testing and Quality Assurance
It is easy to use, in FreeCAD switch to the Test framework workbench, then self-test -> RunJason C. Wells wrote:If there was such a thing, I'd be glad to run through the use cases and make reports.
This will run all the available tests one after the other. But as wandererfan says, this only tests if everything still works correctly with the latest changes in the code.
As Prezmo says, these test suites could be expanded a lot, to include more subtle cases, etc.