Wiki needs a help page on how to run tests.

[dilijev]

See chakra-core/ChakraCore#1068

Ian Halliday (@ianwjhalliday) wrote:

Noj Vek (@nojvek) pointed out it would be useful to have information such as:

• how tests are organized by rlexedirs.xml and rlexe.xml files
• adding a new test
• clarifying that *.js files do not automatically turn into tests
• baselines vs no baselines vs exit code check only
• where rl log files go
• techniques for capturing output from and iterating quickly on failing tests

[dilijev]

High level:

• Reduce your test to a minimal repro so that it exercises a targeted bug or feature path
• Add the test to a directory under https://github.com/Microsoft/ChakraCore/tree/master/test
• Create a test which outputs PASSED or create a .baseline file.
  • Minimum: your test outputs PASSED if it was successful and otherwise produces output that is helpful to identify what went wrong
  • Otherwise add a .baseline file to test/<dir> and specify the baseline in rlexe.xml. The baseline must contain the entire stdout stream which the test generates.
• Modify the rlexe.xml file in your chosen directory to add a test case. Basically just follow the patterns.
  • If your bug repros only when running ch.exe with specific flags, add those under a <compile-flags> tag.
• You can run tests in a specific folder with jenkins\testone.cmd -dirs <DirName>

[dilijev]

For adding this content, two options;

1. Add to https://github.com/Microsoft/ChakraCore/blob/master/CONTRIBUTING.md#running-the-tests
2. Refactor the above to be partially in the Wiki, add a link to the page in the Wiki, and add the above to the wiki page.

[dilijev]

/cc Limin Zhu (@liminzhu), Derek Morris (@Penguinwizzard)

[dilijev]

See also https://github.com/Microsoft/ChakraCore/blob/master/CONTRIBUTING.md#running-the-tests -- probably want to move that into the wiki page and link out to that page rather than keeping it in the CONTRIBUTING.md file.