[Trac-dev] Trac functional testing

[figaro]

There are several pages devoted to to a description of how Trac is
functionally tested: TracDev/UnitTests, TracDev/FunctionalTests,
TracDev/WritingUnitTests.
This is about automated functional testing only and not manual testing.

1- Where do the test cases originate from? Is it only the cases where the
developer has cared to add unit tests to his/her patch? Does that mean that
those tickets marked as resolved (non-duplicate) and do not have unit tests
added are not tested?
2- Is a formal distinction being made to functional testing and unit
testing?
3- Is it fair to say that the majority of the unit tests cover testing of
the graphical user interface of Trac and less so the APIs?
4- How are generally databases initialised for testing purposes?

[Olemis Lang]

On 5/27/16, figaro <***@gmail.com> wrote:
[...]

Post by figaro
1- Where do the test cases originate from? Is it only the cases where the
developer has cared to add unit tests to his/her patch?

Patches , new features , API changes , ... in theory every new
functionality deserves a (few) test(s) , at least to track regressions
.

Post by figaro
Does that mean that
those tickets marked as resolved (non-duplicate) and do not have unit tests
added are not tested?

Might be an indicator , but not necessarily has to the case .

Post by figaro
2- Is a formal distinction being made to functional testing and unit
testing?

Yes . Unit test cases are lightweight , run fast and the scope is
usually a single software "component" , whereas functional tests
require a whole Trac instance (server et al.) up and running and are
usually aimed at checking usability/interaction aspects .

Post by figaro
3- Is it fair to say that the majority of the unit tests cover testing of
the graphical user interface of Trac and less so the APIs?

All the opposite .

Post by figaro
4- How are generally databases initialised for testing purposes?

There should be something in tests folder . Unit tests usually rely
upon mock objects or in-memory SQLite database and might be setup,
torn down every time for individual test cases , whereas functional
tests require connection to a DB engine (sqlite, postgresql, mysql,
...)

[...]
--
Regards,

Olemis - @olemislc

Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
http://blood-hound.net

Brython committer
http://brython.info
http://github.com/brython-dev/brython

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

[figaro]

Post by Olemis Lang

Post by figaro
2- Is a formal distinction being made to functional testing and unit
testing?

Yes . Unit test cases are lightweight , run fast and the scope is
usually a single software "component" , whereas functional tests
require a whole Trac instance (server et al.) up and running and are
usually aimed at checking usability/interaction aspects .
That makes sense. Is it correct then that for example our Travis CI

integration only runs the tests it finds, thereby not making a distinction
whether those are unit tests or functional tests?

Post by Olemis Lang

Post by figaro
4- How are generally databases initialised for testing purposes?

There should be something in tests folder . Unit tests usually rely
upon mock objects or in-memory SQLite database and might be setup,
torn down every time for individual test cases , whereas functional
tests require connection to a DB engine (sqlite, postgresql, mysql,
...)
Ok. Is it true then that Trac testing does not rely on an automated

regression test? In other words, all tests rely on a test-specific initial
data set that can be tied back to issues that have in the meantime been
resolved? Or put differently, there is no regression data set that has been
developed purely for regression testing core Trac functionality?

-- figaro