You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 18 Next »

Table of Contents

Things We Should Keep Doing

  • SPH: Broad regression tests
  • SPH: Process: Ticket -> Specs -> Dev -> Test
  • SPH: Group/Feature area leads

Things We Should Change

Do Better

Test System Reliability

  • SPH: Improve test system reliability.  Test systems should fail less than once a month.
    • SPH: Get two machines set up to run the tests
    • SPH Schedule maintenance work in advance, if possible
    • SPH Start system early enough to be monitored
    • SPH: Machine should be located on-center, so VPN is not needed
    • JJKP: Something keeps causing the script test system to crash when loading results. If this continues, we may need to investigate changing how the reporting code works. (This might be a good idea anyway, it takes a LONG time).
    • Bottom line: SCALABILITY. We have almost 11k tests, and both the filtering mechanism and the reporting mechanism aren't behaving well with this size.

Ticket/Schedule System

  • SPH: When submitting a bug, include scripts or detailed list of procedures, and error messages.
  • DJC: When reporting issues:
    • If you have a script that shows the issue, always include it even if it seems trivial.
    • Avoid "whack a mole" issues, where the developer will fix the presented issue only to see it reopened with a description of "this one is just like the original issue."  Reopening once is okay, but if the issue shows signs of being systemic, then change the description to clue the developer in that it is likely systemic, or open a new issue.
  • SPH: Critical path issues in ticket system put us at risk and we got lucky
    • Lots of tickets waiting for clarification or analysis until last month of 10 month release cycle.  We got lucky here for the most part.  At least one bug just coulnd't be fixed because of lack of time to solve the problem.
    • Lot's of work for SPH and JJKP tracked outside of ticket system.  The result is that P2 items in those lists get attention before P1 release items.  For example, we sent a lot of time on SBIR support, but didn't start reviewing end user docs until 3 days before scheduled app freeze even though over half the doc was ready weeks before.
  • DJC: When partially closing issues:
    • Be sure to include text explaining the current status of the issue so that when a developer returns to it, it is clear where things were left.  (See for example GMT-2686.  I should have noted where things were left, but didn't.)


JJKP: Some Visual Freeze things don't actually need to be in visual freeze.

  • Updating destination addresses in GMAT.ini
  • Updating TestComplete link tests in GMAT.ini
  • Note that if new buttons/links need to be added, these do become Viz Freeze items.


Start Doing

  • SPH: Leverage code from heritage systems
  • SPH: Consider using more third party libraries like SPICE or Asset Importer to avoid duplication of solved problems

Stop Doing

  • SPH: Spreading ourselves too thin.  Don't do more projects and improvements than we can do well.
  • No labels