README
Introduction
asm-pm-runner runs postman collections and export results to crs format for usage in Apica ASM. The command line tool is based on the standard Postman Newman utility. See Newman npm at.
The main enhancement to Newman is support for decryption of data files and certificates. Encryption/decryption is based on the cryptify npm module.
Apica ASM Integration
Apica ASM (Synthetics) can run Postman checks based on asm-pm-runner. This feature is currently in a beta stage. You can contact Apica about this feature/functionality.
Installation
npm install -g @apica-io/asm-pm-runner
Change log summary
| Version | Summary changes |
|---|---|
| 1.3.5 | Report error/exception in the result message |
| 1.3.4 | Fixed custom return value with unit and variables in json report |
Command line Options
Usage: asm-pm-runner [options] <collection>
Options:
-e, --environment <path> Environment JSON file
-dk, --decryptKey <decryptKey> Decrypt key
-ev, --envVars [envVars...] Environment variables. Format name=value format
-v, --verbose Print collection information on stdout (default: false)
-r, --resultDir <dir> The result directory
-dd, --dataDir <dir> The data directory for certificates and test data
-l, --logLevel <logLevel> Log level in log4js (default: "info")
--sslClientCert <path> Specify the path to a client certificate (PEM)
--sslClientKey <path> Specify the path to a client certificate private key
--sslClientPassphrase <passphrase> Specify the client certificate passphrase (for protected key)
-V, --version output the version number
-h, --help display help for command
Samples
You will find some runnable examples in the installation directory under samples . See NPM documentation about where you can find the global npm directory. The package.json file contains scripts for running the samples with correct parameter settings.
Example 1 - Running some http requests on http-bin.
$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -v -ev username=foo password=bar
Example 2 - Testing certificate on https://badssl.com
This sample use encrypted certificates.
The command line
$ asm-pm-runner -l debug -dk Encrypt.4.ssl --sslClientCert file.cert.pem --sslClientKey file.key.pem --dataDir samples/ec_badssl_certs -r results/ -v samples/BadSSL.postman_collection.json
The console output from the command
[2021-08-18T10:08:21.470] [INFO] @apica-io/asm-pm-runner - @apica-io/asm-pm-runner (1.0.2) started
[2021-08-18T10:08:21.478] [DEBUG] @apica-io/asm-pm-runner - collection=samples/BadSSL.postman_collection.json
[2021-08-18T10:08:21.574] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem
[2021-08-18T10:08:21.580] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem
itemGroup BadSSL
item ClientBadSSL
request GET https://client.badssl.com/
response OK 200 OK 1891 { body: 687, header: 283, total: 970 }
timingPhases {
secure_handshake_ms: 339.4973890000001,
prepare: 73.57351800000015,
socket_wait_ms: 59.02749899999981,
dns_time_ms: 1147.0008660000003,
tcp_connect_ms: 171.81406999999945,
time_to_first_byte_ms: 152.74416699999983,
download_time_ms: 19.97996400000011,
process_time_ms: 1.6743750000005093,
total_time_ms: 1965.3118480000003
}
assertion Check status code is 200 true
Collection BadSSL
run.stats {
iterations: { total: 1, pending: 0, failed: 0 },
items: { total: 1, pending: 0, failed: 0 },
scripts: { total: 1, pending: 0, failed: 0 },
prerequests: { total: 1, pending: 0, failed: 0 },
requests: { total: 1, pending: 0, failed: 0 },
tests: { total: 1, pending: 0, failed: 0 },
assertions: { total: 1, pending: 0, failed: 0 },
testScripts: { total: 1, pending: 0, failed: 0 },
prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
[2021-08-18T10:08:24.072] [DEBUG] NewmanRunner - Clean up temporary files [
'/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem',
'/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem'
]
Result: true Requests:1, tests:1, assertions:1
[2021-08-18T10:08:24.074] [DEBUG] NewmanRunner - Results written to /Users/janostgren/work/node/asm-pm-runner/results/BadSSL_res.json
Reporting of result
The json report is stored in the results directory. The format is complaint with Check Result Service in Apica ASM.
Steps and requests in the report
The json report is divided into one to many steps. A step contains requests from the postman collection.
- Default only one step with all requests is generated.
- Several steps will be generated if the collection contains folders. One step for each folder will be generated and step name corresponds to folder name.
- Only one level of steps is supported. Folders in folders will be named folder/subfolder. Example root folder/sub folder 1/sub folder 2
Returning a custom value as the result
asm-pm-runner can return a custom value as the return value when running a collection. This functionality is often used in ASM when you want return something else than the total response time as the main result of the check run. What you need to do in the collection is use 2 special collection parameters and assign values to them in a JavaScript test. Collection Variable | Description --------------------| ----------- _Apica_ReturnValue | A numeric value used as the check value (return value) _Apica_ReturnUnit | The unit of the customized return value
pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});
let eventCount =pm.response.json().length;
pm.collectionVariables.set("_Apica_ReturnValue",eventCount)
pm.collectionVariables.set("_Apica_ReturnUnit","events")
Returning variables in the result
You can also return variable values in the result report. If you want suppress a variable from being included in the result report use a name starting with the "_" character. Collection and Environment variables not changed during the run of collection will be typed as input variables.
let json =pm.response.json();
pm.collectionVariables.set("eventName",json.name)
pm.collectionVariables.set("category",json?.category?.description|| "")
pm.collectionVariables.set("mediaLink",json?.mediaItem?.url|| "")
Reporting Variables in json result report
"variables": [
{
"key": "baseUrl",
"type": "url",
"usage": "input",
"value": "http://ticketmonster.apicasystem.com"
},
{
"key": "eventName",
"type": "string",
"usage": "inResponse",
"value": "Champions League"
},
{
"key": "category",
"type": "string",
"usage": "inResponse",
"value": "Sport"
},
{
"key": "mediaLink",
"type": "string",
"usage": "inResponse",
"value": "http://img.uefa.com/imgml/2016/ucl/social/og-default.jpg"
}
]
The generated json report
The name of result file will be {collection.name}_res.json. The environment variable APICA_JOB_ID can be used to set the basename of the report file.
{
"name": "BadSSL",
"type": "Postman.Collection",
"result_version": "1.0.2",
"producer": "@apica-io/asm-pm-runner 1.0.2",
"value": 1891,
"unit": "ms",
"returncode": 0,
"success": true,
"message": "Requests:1, tests:1, assertions:1",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"dns_time_ms": 1147,
"download_time_ms": 20,
"prepare": 74,
"process_time_ms": 2,
"secure_handshake_ms": 339,
"socket_wait_ms": 59,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"total_time_ms": 1965
},
"variables": [],
"steps": [
{
"name": "BadSSL",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"dns_time_ms": 1147,
"download_time_ms": 20,
"prepare": 74,
"process_time_ms": 2,
"secure_handshake_ms": 339,
"socket_wait_ms": 59,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"total_time_ms": 1965
},
"requests": [
{
"name": "ClientBadSSL",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"secure_handshake_ms": 339,
"prepare": 74,
"socket_wait_ms": 59,
"dns_time_ms": 1147,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"download_time_ms": 20,
"process_time_ms": 2,
"total_time_ms": 1965
},
"url": "https://client.badssl.com/",
"method": "GET",
"status": 200,
"status_text": "OK",
"request_headers": {
"User-Agent": "PostmanRuntime/7.28.2",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "c0cfe5c6-d962-4dcc-a2bf-bef71a09f62c",
"Host": "client.badssl.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive"
},
"headers": {
"Server": "nginx/1.10.3 (Ubuntu)",
"Date": "Wed, 18 Aug 2021 08:08:24 GMT",
"Content-Type": "text/html",
"Last-Modified": "Tue, 23 Feb 2021 21:28:41 GMT",
"Transfer-Encoding": "chunked",
"Connection": "keep-alive",
"ETag": "W/\"60357389-2af\"",
"Cache-Control": "no-store",
"Content-Encoding": "gzip"
}
}
],
"success": true,
"assertions": [
{
"source": "ClientBadSSL",
"description": "Check status code is 200",
"status": "info",
"expression": "Check status code is 200",
"value": true
}
]
}
]
}
Encryption of secret data
Encryption/Decryption of certificates and environment files is supported. You use the NPM package cryptify for decrypting and encrypting the files.
You must supply a decryption key with -dk option. The file must been encrypted with the key using the command line version of cryptify. Install the package globally and use the tool.
Installation of cryptify
$ npm install -g cryptify
$ cryptify
Usage: cryptify [options] [command]
Options:
-v, --version Display the current version
-l, --list List available ciphers
-h, --help Display help for the command
Commands:
encrypt [options] <file...> Encrypt files(s)
decrypt [options] <file...> Decrypt files(s)
help <command> Display help for the command
Example:
$ cryptify encrypt file.txt -p 'Secret123!'
$ cryptify decrypt file.txt -p 'Secret123!'
Password Requirements:
1. Must contain at least 8 characters
2. Must contain at least 1 special character
3. Must contain at least 1 numeric character
4. Must contain a combination of uppercase and lowercase
Example of encrypt a file
$ cryptify encrypt Http_bin_environment.postman_environment.json --password Encrypt.4.ssl
Example with encrypted postman_environment
$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -r results -l debug -v -e samples/ec_env/Http_bin_environment.postman_environment.json -dk Encrypt.4.ssl