Fixes an aggregation bug
Run your typical ObjectScript %UnitTest tests and see which lines of your code are executed. Includes Cobertura-style reporting for use in continuous integration tools.
Note: a minimum platform version of Caché/Ensemble 2016.2 is required. This tool will also work on any version of InterSystems IRIS.
If you already have the ObjectScript Package Manager, installation is as easy as:
zpm "install testcoverage"
Download an XML file from Releases, then run:
Set releaseFile = "<path on filesystem to xml file>" Do $System.OBJ.Load(releaseFile,"ck")
First, clone or download the repository. Then run the following commands:
Set root = "<path on filesystem to which repository was cloned/downloaded>" Do $System.OBJ.ImportDir(root,"*.inc;*.cls","ck",,1)
Note: this assumes you have the Git plugin for Atelier installed.
Note that, depending on your security settings, SQL privileges may be required for access to test coverage data. The relevant permissions may be granted by running:
zw ##class(TestCoverage.Utils).GrantSQLReadPermissions("<username or role that should have read permissions>")
Generally speaking, set
^UnitTestRoot, and then call
##class(TestCoverage.Manager).RunTest() the same you would call
##class(%UnitTest.Manager).RunTest(). For more information on InterSystems' %UnitTest framework, see the tutorial and/or the class reference for %UnitTest.Manager.
The "userparam" argument can be used to pass information about code coverage data collection. For example:
Set tCoverageParams("CoverageClasses") = <$ListBuild list of class names for which code coverage data should be collected> Set tCoverageParams("CoverageRoutines") = <$ListBuild list of routine names for which code coverage data should be collected> Set tCoverageParams("CoverageDetail") = <0 to track code coverage overall; 1 to track it per test suite (the default); 2 to track it per test class; 3 to track it per test method.> Do ##class(TestCoverage.Manager).RunTest(,,.tCoverageParams)
The first two arguments to
TestCoverage.Manager:RunTest are the same as
At the selected level of granularity (before all tests or a test suite/case/method is run), there will be a search for a file named "coverage.list" within the directory for the test suite and parent directories, stopping at the first such file found. This file may contain a list of classes, packages, and routines for which code coverage will be measured. For .MAC routines only (not classes/packages), the coverage list also supports the * wildcard. It is also possible to exclude classes/packages by prefixing the line with "-". For example, to track coverage for all classes in the
MyApplication package (except those in the
MyApplication.UI subpackage), and all routines with names starting with "MyApplication":
// Include all application code MyApplication.PKG MyApplication*.MAC // Exclude Zen Pages -MyApplication.UI.PKG
As an alternative approach, with unit test classes that have already been loaded and compiled (and which will not be deleted after running tests) and a known list of classes and routines for which code coverage should be collected, use:
tPackagehas the top-level package containing all the unit test classes to run. These must already be loaded.
tLogFile(optional) may specify a file to log all output to.
tCoverageClasses(optional) has a $ListBuild list of class names within which to track code coverage. By default, none are tracked.
tCoverageRoutines(optional) has a $ListBuild list of routine names within which to track code coverage. By default, none are tracked.
tCoverageLevel(optional) is 0 to track code coverage overall; 1 to track it per test suite (the default); 2 to track it per test class; 3 to track it per test method.
tLogIndex(optional) allows for aggregation of code coverage results across unit test runs. To use this, get it back as output from the first test run, then pass it to the next.
tSourceNamespace(optional) specifies the namespace in which classes were compiled, defaulting to the current namespace. This may be required to retrieve some metadata.
tPIDList(optional) has a $ListBuild list of process IDs to monitor. If this is empty, all processes are monitored. By default, only the current process is monitored.
tTiming(optional) is 1 to capture execution time data for monitored classes/routines as well, or 0 (the default) to not capture this data.
After running the tests, a URL is shown in the output at which you can view test coverage results. If the hostname/IP address in this URL is incorrect, you can fix it by changing the "WebServerName" setting in the management portal, at System Administration > Configuration > Additional Settings > Startup.
RunTest() method reports back a log index in the "userparam" argument. This can be used to generate a report in the same format as Cobertura, a popular Java code coverage tool. For example:
Set userParams("CoverageDetail") = 0 Do ##class(TestCoverage.Manager).RunTest(,"/nodelete",.userParams) Set reportFile = "C:\Temp\Reports\"_tUserParams("LogIndex")_"\coverage.xml" Do ##class(TestCoverage.Report.Cobertura.ReportGenerator).GenerateReport(userParams("LogIndex"),reportFile)
This exports both the coverage results themselves and the associated source code (in UDL format) for correlation/display, and has been verified with the Cobertura plugin for Jenkins.
If you find a bug or would like to request an enhancement, report an issue. If you have a question, post it on the InterSystems Developer Community - consider using the "Testing" or "Continuous Integration" tags as appropriate.
Please read contributing for details on our code of conduct, and the process for submitting pull requests to us.
See also the list of contributors who participated in this project.
This project is licensed under the MIT License - see the LICENSE file for details.
Fixes an aggregation bug
Updating to avoid zpm installation issues with +snapshot version
Various quality-of-life fixes, and (most importantly) enabling for the community package manager