Client YAML output

[edit] Client test run output

Exmaple for test file:

print "1..10\n";
print "ok 1\n";
print "ok 2 - hello text\n";
# not seen 3
print "not ok 4 - i didn't seen number 3 # todo text after type - reason text\n";
print "not ok 4 - i didn't seen number 3 # todo text\n# second line\n";
print "ok 5 # skip skipping number 5\n";
print "not ok 6\n";
print "ok 7 - seven # myowntype-ok mytext after type\n # second line\n";
print "not ok 8 - eight # myowntype-not-ok mytext after type\n# second line\n";
print "not ok 9\n";
# not seen 10

# parrot-temp> perl t\harnessnew t\notseen.t --yaml

YAML client output ('results' part is Harness::Straps structure):

--- #YAML:1.0
conf:
  client_id: 2
  compressupload: 0
  email: tt-parrot@mj41.cz
  irc_nick: mj41
  repository: https://svn.perl.org/parrot/
  repository_path: trunk/
  taptinderserv: http://test.perl6.cz/somepath/upload.pl
duration: 0
harness_args: '-D40'
pconfig:
  DEVEL: '-devel'
  VERSION: 0.5.2
  archname: MSWin32-x86-multi-thread
  build_dir: C:/usr/taptinder/client-data/parrot-temp
  cc: gcc
  cpuarch: i386
  optimize: ''
  osname: MSWin32
  revision: 25533
results:
  - bonus: 0
    details:
      - actual_ok: 1
        name: ''
        ok: 1
        reason: ''
        type: ''
      - actual_ok: 1
        name: hello text
        ok: 1
        reason: ''
        type: ''
      - ~
      - actual_ok: 0
        diagnostics: "second line\n"
        name: i didn't seen number 3
        ok: 1
        reason: text
        type: todo
      - actual_ok: 1
        name: ''
        ok: 1
        reason: skipping number 5
        type: skip
      - actual_ok: 0
        name: ''
        ok: 0
        reason: ''
        type: ''
      - actual_ok: 1
        diagnostics: " # second line\n"
        name: seven
        ok: 1
        reason: mytext after type
        type: myowntype-ok
      - actual_ok: 0
        diagnostics: "second line\n"
        name: eight
        ok: 0
        reason: mytext after type
        type: myowntype-not-ok
      - actual_ok: 0
        name: ''
        ok: 0
        reason: ''
        type: ''
    exit: 0
    max: 10
    ok: 6
    passing: 0
    seen: 9
    skip: 1
    test_file: t\notseen.t
    todo: 2
    wait: 0
version: 0.01

[edit] From Harness::Starps docs

A successful test should have max == seen == ok.

• passing - true if the whole test is considered a pass (or skipped), false if its a failure
• exit - the exit code of the test run, if from a file
• wait - the wait code of the test run, if from a file
• max - total tests which should have been run
• seen - total tests actually seen
• skip_all - if the whole test was skipped, this will contain the reason.
• ok - number of tests which passed (including todo and skips)
• todo - number of todo tests seen
• bonus - number of todo tests which unexpectedly passed
• skip - number of tests skipped

The details for each subtest contains:

• ok => is the test considered ok?
• actual_ok => did it literally say 'ok'?
• name => name of the test (if any)
• diagnostics => test diagnostics (if any)
• type => 'skip' or 'todo' (if any)
• reason => reason for the above (if any)

[edit] Mapping Straps output to tresult_id

• 0 .. not seen (not ok) -- empty result info
• 1 .. failed (not ok) -- actual_ok == 0 && type == ''
• 2 .. unknown (not ok) -- actual_ok == 0 && type != 'todo' && type != 'skip'
• 3 .. todo (not ok || ok) -- actual_ok == 0 && type == 'todo'
• 4 .. bonus (ok) -- actual_ok == 1 && (type == 'todo' || type == 'skip' )
• 5 .. skip (ok) -- actual_ok == 0 && type == 'skip'
• 6 .. ok (ok) -- actual_ok == 1 && (type == || (type != 'todo' && type != 'skip') ) ( other types, reasons, ... ignored )