@apica-io/asm-pm-runner

Run a postman collection in Apica ASM and report result to CRS.

Usage no npm install needed!

<script type="module">
  import apicaIoAsmPmRunner from 'https://cdn.skypack.dev/@apica-io/asm-pm-runner';
</script>

README

Introduction

asm-pm-runner runs postman collections and export results to crs format for usage in Apica ASM. The command line tool is based on the standard Postman Newman utility. See Newman npm at.

The main enhancement to Newman is support for decryption of data files and certificates. Encryption/decryption is based on the cryptify npm module.

Apica ASM Integration

Apica ASM (Synthetics) can run Postman checks based on asm-pm-runner. This feature is currently in a beta stage. You can contact Apica about this feature/functionality.

Installation

npm install -g @apica-io/asm-pm-runner

Command line Options

Usage: asm-pm-runner [options] <collection>

Options:
  -e, --environment <path>            Environment JSON file
  -dk, --decryptKey <decryptKey>      Decrypt key
  -ev, --envVars [envVars...]         Environment variables. Format name=value format
  -v, --verbose                       Print collection information on stdout (default: false)
  -r, --resultDir <dir>               The result directory
  -dd, --dataDir <dir>                The data directory for certificates and test data
  -l, --logLevel <logLevel>           Log level in log4js (default: "info")
  --sslClientCert <path>              Specify the path to a client certificate (PEM)
  --sslClientKey <path>               Specify the path to a client certificate private key
  --sslClientPassphrase <passphrase>  Specify the client certificate passphrase (for protected key)
  -V, --version                       output the version number
  -h, --help                          display help for command

Samples

You will find some runnable examples in the installation directory under samples . See NPM documentation about where you can find the global npm directory. The package.json file contains scripts for running the samples with correct parameter settings.

Example 1 - Running some http requests on http-bin.

$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -v -ev username=foo password=bar

Example 2 - Testing certificate on https://badssl.com

This sample use encrypted certificates.

The command line

$ asm-pm-runner -l debug -dk  Encrypt.4.ssl --sslClientCert file.cert.pem --sslClientKey file.key.pem --dataDir samples/ec_badssl_certs -r results/ -v samples/BadSSL.postman_collection.json

The console output from the command

 [2021-08-18T10:08:21.470] [INFO] @apica-io/asm-pm-runner - @apica-io/asm-pm-runner (1.0.2) started
[2021-08-18T10:08:21.478] [DEBUG] @apica-io/asm-pm-runner - collection=samples/BadSSL.postman_collection.json
[2021-08-18T10:08:21.574] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem
[2021-08-18T10:08:21.580] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem
itemGroup BadSSL
item ClientBadSSL
request GET https://client.badssl.com/
response OK 200 OK 1891 { body: 687, header: 283, total: 970 }
timingPhases {
  secure_handshake_ms: 339.4973890000001,
  prepare: 73.57351800000015,
  socket_wait_ms: 59.02749899999981,
  dns_time_ms: 1147.0008660000003,
  tcp_connect_ms: 171.81406999999945,
  time_to_first_byte_ms: 152.74416699999983,
  download_time_ms: 19.97996400000011,
  process_time_ms: 1.6743750000005093,
  total_time_ms: 1965.3118480000003
}
assertion Check status code is 200 true
Collection BadSSL
run.stats {
  iterations: { total: 1, pending: 0, failed: 0 },
  items: { total: 1, pending: 0, failed: 0 },
  scripts: { total: 1, pending: 0, failed: 0 },
  prerequests: { total: 1, pending: 0, failed: 0 },
  requests: { total: 1, pending: 0, failed: 0 },
  tests: { total: 1, pending: 0, failed: 0 },
  assertions: { total: 1, pending: 0, failed: 0 },
  testScripts: { total: 1, pending: 0, failed: 0 },
  prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
[2021-08-18T10:08:24.072] [DEBUG] NewmanRunner - Clean up temporary files [
  '/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem',
  '/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem'
]
Result:  true Requests:1, tests:1, assertions:1
[2021-08-18T10:08:24.074] [DEBUG] NewmanRunner - Results written to /Users/janostgren/work/node/asm-pm-runner/results/BadSSL_res.json

Reporting of result

The json report is stored in the results directory. The format is complaint with Check Result Service in Apica ASM.

Steps and requests in the report

The json report is divided into one to many steps. A step contains requests from the postman collection.

  • Default only one step with all requests is generated.
  • Several steps will be generated if the collection contains folders. One step for each folder will be generated and step name corresponds to folder name.
  • Only one level of steps is supported. Folders in folders will be named folder/subfolder. Example root folder/sub folder 1/sub folder 2

The generated json report

The name of result file will be {collection.name}_res.json. The environment variable APICA_JOB_ID can be used to set the basename of the report file.

{
  "name": "BadSSL",
  "type": "Postman.Collection",
  "result_version": "1.0.2",
  "producer": "@apica-io/asm-pm-runner 1.0.2",
  "value": 1891,
  "unit": "ms",
  "returncode": 0,
  "success": true,
  "message": "Requests:1, tests:1, assertions:1",
  "duration_ms": 1891,
  "content_length": 687,
  "start_timestamp_ms": 1629274104025,
  "end_timestamp_ms": 1629274105916,
  "metrics": {
    "dns_time_ms": 1147,
    "download_time_ms": 20,
    "prepare": 74,
    "process_time_ms": 2,
    "secure_handshake_ms": 339,
    "socket_wait_ms": 59,
    "tcp_connect_ms": 172,
    "time_to_first_byte_ms": 153,
    "total_time_ms": 1965
  },
  "variables": [],
  "steps": [
    {
      "name": "BadSSL",
      "duration_ms": 1891,
      "content_length": 687,
      "start_timestamp_ms": 1629274104025,
      "end_timestamp_ms": 1629274105916,
      "metrics": {
        "dns_time_ms": 1147,
        "download_time_ms": 20,
        "prepare": 74,
        "process_time_ms": 2,
        "secure_handshake_ms": 339,
        "socket_wait_ms": 59,
        "tcp_connect_ms": 172,
        "time_to_first_byte_ms": 153,
        "total_time_ms": 1965
      },
      "requests": [
        {
          "name": "ClientBadSSL",
          "duration_ms": 1891,
          "content_length": 687,
          "start_timestamp_ms": 1629274104025,
          "end_timestamp_ms": 1629274105916,
          "metrics": {
            "secure_handshake_ms": 339,
            "prepare": 74,
            "socket_wait_ms": 59,
            "dns_time_ms": 1147,
            "tcp_connect_ms": 172,
            "time_to_first_byte_ms": 153,
            "download_time_ms": 20,
            "process_time_ms": 2,
            "total_time_ms": 1965
          },
          "url": "https://client.badssl.com/",
          "method": "GET",
          "status": 200,
          "status_text": "OK",
          "request_headers": {
            "User-Agent": "PostmanRuntime/7.28.2",
            "Accept": "*/*",
            "Cache-Control": "no-cache",
            "Postman-Token": "c0cfe5c6-d962-4dcc-a2bf-bef71a09f62c",
            "Host": "client.badssl.com",
            "Accept-Encoding": "gzip, deflate, br",
            "Connection": "keep-alive"
          },
          "headers": {
            "Server": "nginx/1.10.3 (Ubuntu)",
            "Date": "Wed, 18 Aug 2021 08:08:24 GMT",
            "Content-Type": "text/html",
            "Last-Modified": "Tue, 23 Feb 2021 21:28:41 GMT",
            "Transfer-Encoding": "chunked",
            "Connection": "keep-alive",
            "ETag": "W/\"60357389-2af\"",
            "Cache-Control": "no-store",
            "Content-Encoding": "gzip"
          }
        }
      ],
      "success": true,
      "assertions": [
        {
          "source": "ClientBadSSL",
          "description": "Check status code is 200",
          "status": "info",
          "expression": "Check status code is 200",
          "value": true
        }
      ]
    }
  ]
}

Encryption of secret data

Encryption/Decryption of certificates and environment files is supported. You use the NPM package cryptify for decrypting and encrypting the files.

You must supply a decryption key with -dk option. The file must been encrypted with the key using the command line version of cryptify. Install the package globally and use the tool.

Installation of cryptify

 $ npm install -g cryptify

 $ cryptify 
Usage: cryptify [options] [command]

Options:
  -v, --version                Display the current version
  -l, --list                   List available ciphers
  -h, --help                   Display help for the command

Commands:
  encrypt [options] <file...>  Encrypt files(s)
  decrypt [options] <file...>  Decrypt files(s)
  help <command>               Display help for the command

Example:
  $ cryptify encrypt file.txt -p 'Secret123!'
  $ cryptify decrypt file.txt -p 'Secret123!'

Password Requirements:
  1. Must contain at least 8 characters
  2. Must contain at least 1 special character
  3. Must contain at least 1 numeric character
  4. Must contain a combination of uppercase and lowercase

Example of encrypt a file

 $ cryptify encrypt Http_bin_environment.postman_environment.json  --password  Encrypt.4.ssl

Example with encrypted postman_environment

$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -r results  -l debug  -v -e samples/ec_env/Http_bin_environment.postman_environment.json -dk Encrypt.4.ssl