Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

gmattest run <run definition file>

Run everything (alternative to the command above)

gmattest runtests <run definition file>

Run the tests only, then stop

gmattest runcomparators <run definition file>

Run the truth/regression comparisons only

gmattest runreporters <run definition file>

Run the screen/file/email reporting on

Advanced Performance Configuration for Small Test Sets

When running a small set of script tests, the performance of the test system can be quite slow because by default, the test system searches ALL .tc files in the test system for cases that match your RunDef configuration.  If you need to run a few tests, you can configure the RunDef file carefully for a dramatic performance improvement.  When applying filters in And mode, the test system applies the Folders filter first.  Folders that do not match criteria are not loaded and therefore, .tc files and other data are also not loaded, saving considerable time.   For efficient execution of a small set of tests:

  1. Include the tests you want included in the Cases list

  2. Include the folders in which those cases are found in the Folders list
  3. Leave Requirements or Categories filters empty in th eRunDef file

Below is an example RunDef.m file that efficiently runs the test named STM_GMAT_PD45 which is located in the FRR-13_DynamicsModels folder.

Code Block
RunDef.Build       = 'NightlyBuild';
RunDef.GmatExe     = 'c:\PATHTOYOURBINDIRECTORY\GMAT.exe';
RunDef.Modes       = {'script'};
RunDef.Comparisons = {'truth'};
RunDef.Reporters    = {'ScreenReporter'};
RunDef.Cases        = {'STM_GMAT_PD45'};
RunDef.Folders      = {'FRR-13_DynamicsModels'};
RunDef.FilterMode   = 'and';
RunDef.Requirements = {}
RunDef.Categories = {}

Configuration for Nightly Regression Testing

Nightly regression tests require that GMAT's data configuration is consistent with the data used to create the truth data contained in the test system.   For example, some tests require additional ephemeris files, 3-D models, or other configuration data.  Configuration is performed using the function named  preparegmat.m located in the directory \test\script\bin.  When you execute the function, all data from the \test\script\gmatdata folder is copied into your local GMAT configuration.  You must provide the location of your local GMAT configuration as shown below.

Code Block
>> preparegmat('C:\Path\To\GMAT')

Writing tests

Files Used By the Test System

...

Info

All test system development should target MATLAB R2012a. This is the last version of MATLAB released for 32-bit Linux, which is used by some GMAT team members.the "current-2" version of MATLAB.

Configuring Tests That Require Ancillary Data

Some test cases have ancillary data files need to execute the test.   For example, if you are testing propagation using an SPK kernel, you must provide that kernel to the GMAT.   Ancillary files must be put in the test repo in the correct places, then everything works transparently in the nightly build.  Before the nightly test system executes, it copies the version controlled ancillary files into the GMAT folder structure.   

The test system has a copy of the directory structure the GMAT application employs.  It is located here: \trunk\test\script\gmatdata.  If your ancillary files naturally fit in the GMAT directory structure, they should be put in the appropriate folder  in that directory, and the nightly test system will copy them to the build test system to run the nightly tests.  Here is an example,  if you need a vehicle SPICE kernel to run a nightly test, it should be put here: \trunk\test\script\gmatdata\vehicle\ephem\spk.  

There are a few exceptions.  Matlab and Python functions required to run nightly tests should be placed here: "trunk\test\script\input\Functions" in the "MATLAB and Python folders in that directory.  For example, if you have a test that calls a Python file called "regexp.py", that file should be placed here: trunk\test\script\input\Functions\Python.

Automated Creation of Requirements to Test Traceability Matrix

The script test system can automatically create a Requirements to Test Traceability Matrix (RTTM).  Manual test and/or GUI test traceablity is entered into the GMAT requirements matrix by hand.  The script test system augments that information with automated test traceablity to provide a complete RTTM file.  To generate the full RTTM file, download the latest version of the requirements to your disk and follow the instructions below.  (The GMAT requirements matrix is located here).

Example

This example assumes you have the GMAT requirements downloaded in a local folder called "C:\MyFiles" and that you would like the RTTM placed in that folder.  Simply change the paths to be consistent with your local configuration.

% Define the desired target file name for the RTTM
outputFileName   = 'C:\MyFiles\RTTM.xlsx';
% Define the name and location of your GMAT requirements Excel spreadsheet
requirementsList = 'C:\MyFiles\GMAT Requirements.xlsx';
% Execute the command to read all tests cases and map tests to requirements
gmattest('computemetrics',requirementsList,outputFileName);