README

@d-cat/tag-cli

@d-cat/tag-cli is a VZ CLI package that should be used to configure containers and tags when using @d-cat/tag-manager.

Install
Usage

Install

npm i -D @d-cat/tag-cli

Usage

@d-cat/tag-cli is a toolkit to deploy and develop a Tag Manager with Tags using GitLab and Google Cloud Platform.

Tag Manager commands

A D-CAT Tag Manager is a collective name for all layers that work together to generate and deploy a working JavaScript bundle of multiple GitLab repositories of a single GitLab Group in a defined Google Cloud Storage Bucket. To effectuate this, we depend on the following:

@d-cat/tag-manager;
@d-cat/tag-cli;
@d-cat/ddm-core
GitLab Group;
GitLab Group API;
GitLab Tag Manager Repository in a GitLab Group;
More than 0 GitLab Tag Repositories (generated through @d-cat/tag-cli) in a GitLab Group;

bundle-tag-manager

`create-tag-manager`

The create-tag-manager creates a Tag Manager boilerplate designed to use with @d-cat/tag-cli within GitLab CI/CD. Please make sure you update the e2e and unit tests to suite your needs and match the TMS you want to deploy.

Options:
  --version               Show version number                          [boolean]
  --help                  Show help                                    [boolean]
  --force                 Create Tag Manager without prompt            [boolean]

# creates default setup
npx tag-cli create-tag-manager --force

# prompt questions
npx tag-cli create-tag-manager --no-force

What's included

The Tag Manager boilerplate contains everything to:

unit test the build version of the tag manager
e2e test the build version of the tag manager in chrome headless browser
deploy test/prod containers in a GCP bucket
deploy testable open websites with the TMS running on it for functional testing
dockerized TMS to run the TMS locally in isolation
dockerized TMS that registers images in GitLab private registery
stable pipeline file with artifacts enabled both on success as failure
babel/ts for browser compatibility

Some parts of the generated boilerplate can look weird as the pipeline will interpolate varialbes/files/libs and build the full version when running npx tag-cli bundle-tag-manager.

The roadmap is to replace interpolated files with a configuration (json/yml) file to make it more intuitive.

Update depedencies

By default dependencies are fixed in the package-lock.json and package.json files. Updating a package manually will break the pipeline as it runs npm ci, thus you have to update both package.json as package-lock.json:

npm i yourpackage@1.0.0

Examples

Make sure docker is installed and you've updated your .env file.

# create a new tag folder
mkdir myTag

cd myTag

# creating a tag manager boilerplate
npx tag-cli create-tag-manager

# running a tag manager locally in isolation on localhost:8080
docker build -t ${img_name} .
docker run --name ${container_name} --rm -p 8080:8080 -ti ${img_name}:latest

# running a tag manager locally in isolation with docker-compose
docker-compose build bundle
docker-compose up bundle

# running the full tms pipeline (except deployment in gcp/test sites) locally in isolation
docker-compose build deploy
docker-compose up deploy

GitLab configuration

Make sure you did all necessary steps to start with GitLab.

Create a new GitLab (sub)group with name as Container ID: (i.e. 9999);
Create a new project, called Tag Manager;
Create CI/CD variables.

CI/CD variables to create in GitLab

Before you run the pipeline, make sure you create the necessary CI/CD variables in the GitLab TMS Group.

CI/CD variable	Type	Description
`PERSONAL_ACCESS_TOKEN`	variable	Your personal access token, that has owner rights of the tags group.
`CONTAINER_ID`	variable	Group ID that should be queried for tags.
`GCLOUD_SERVICE_KEY`	file	Your authentication file for Google Cloud.
`SSH_KEY`	file	Your SSH key to clone and push repo's of GitLab.
`GIT_USER_EMAIL`	variable	Your git email: `git config user.email`.
`GIT_USER_NAME`	variable	Your git user name: `git config user.name`.

`bundle-tag-manager`

The bundle-tag-manager command bundles all tag artifacts to eventually transpile and deploy a Tag Manager.

Options:
  --version                Show version number                                      [boolean]
  --help                   Show help                                                [boolean]
  --access-token, --token  The GitLab personal accesstoken with owner rights.       [string]
  --group-id, --id         The GitLab Group ID that should be analyzed.             [number]
  --dir-name, --dir        Dir where to store artifacts.                            [string]
  --branch-name, --branch  The current branch.                                      [string]
  --job-token, --j         Unique token: $CI_PIPELINE_ID.                           [string]
  --environment --e        Either `development` or `production`.                    [string]

Examples

Note that all flags are either mandatory or are set as a node environment variable.

# the environment variable should be used to trigger
# the tag managers pipeline from another pipeline or API
# more at: https://docs.gitlab.com/ee/ci/triggers/#making-use-of-trigger-variables

# gitlab variables to set as env variables
ACCESS_TOKEN=$MY_PERSONAL_ACCESS_TOKEN
GROUP_ID=$GITLAB_GROUP_ID

# build a dev bundle
# variables used are built-in gitlab env variables
# note that we use ci_pipeline_id as job_token
# this token is only used for e2e testing
npx tag-cli bundle-tag-manager --dir-name lib --branch-name $CI_COMMIT_REF_NAME --job-token $CI_PIPELINE_ID --environment $ENV_VAR_FROM_ANOTHER_PIPELINE

Test the bundle

Instead of running the pipeline from a shared runner in GitLab, you could also go for 2 other options to verify if the built is correct.

install a GitLab runner locally and execute the bundle job from a Tag Manager project
use Docker within a Tag Manager project:

docker build -t $MY_TMS_IMG .

# serve the container on localhost:8080
docker run --name $MY_CONTAINER_NAME --rm -p 8080:8080 -ti $MY_TMS_IMG:latest

# using docker-compose to only build the build job
docker-compose build build

# serve the container on localhost:8080
docker-compose up build

`deploy-test-instance`

The deploy-test-instance command deploys a build version of the Tag Manager on a private or public website at https://instances.gitlab.io/[tagmanager_id]/[branchName]. It can take up to 10 minutes before your test instance is deployed, starting from your push.

Note that his commands looks up the tdn folder in the root directory of the repo and searches for the build file of the Tag Manager, typically something like tdn9999.js. The corresponding deploy url would be https://instances.gitlab.io/9999. The command is meant to run it right after a bundle stage.

Note that you need access to the instances.gitlab.io group before you can publish to it.

Options:
  --version                Show version number                                  [boolean]
  --help                   Show help                                            [boolean]
  --access-token, --token  Personal accesstoken with enough rights.             [string]
  --group-id, --id         The GitLab Group ID that should be analyzed          [string]
  --branch-name, --n       The current branch name: $CI_COMMIT_REF_NAME.        [string]
  --job-token, --j         $CI_JOB_TOKEN.                                       [string]

Examples

# Personal access token that's needed to authenticate
ACCESS_TOKEN=$PERSONAL_ACCESS_TOKEN

# Job Token generated by GitLab so we can see in the
# TMS interface that we triggered the pipeline of the instances group
JOB_TOKEN=$CI_JOB_TOKEN

# The Group ID in which the TMS project is placed
GROUP_ID=$TMS_GROUP_ID

npx tag-cli deploy-test-instance --branch-name $CI_COMMIT_REF_NAME

`deploy-tag-manager`

The deploy tag manager commands deploys the build tag manager to a Google Cloud Bucket and optionally sends a success email to a list of receivers.

Options:
  --version            Show version number                               [boolean]
  --help               Show help                                         [boolean]
  --bucket-name, -b    Name of the Google Cloud Bucket.                  [string]
  --key-file-name, -k  Google Cloud service key.                         [string]
  --group-name, -i     $CI_PROJECT_NAMESPACE.                            [string]
  --branch-name, -n    $CI_COMMIT_REF_NAME.                              [string]
  --node-email, -gm    Gmail email transporter.                          [string]
  --node-pass, -pass   Gmail password.                                   [string]
  --receivers, -list   Receivers of mail.                                [string]
  --is-empty -e        If true deploy empty.                             [string]

Examples

Note that all flags are either mandatory or are set as an environment variable.

# The bucket name of the Google Cloud Bucket
BUCKET_NAME=$MY_GCP_BUCKET_NAME

# The Key file is your GCP IAM json.
# make sure it has write rights to the 
# correct bucket only
KEY_FILE=$MY_KEY_FILE

# The namespace of the gitlab group,
# we use this to determine wether the TMS id
# equals the namespace to prevent unexpected overrides
# of a production container
# so a GitLab Group called 9999
# must contain a TMS repo with tdn9999.ts entrypoint.
GROUP_NAME=$CI_PROJECT_NAMESPACE

# GitLab branch var
BRANCH_NAME=$CI_COMMIT_REF_NAME

# gmail where we send the mail from
NODE_EMAIL=$EMAIL_ADDRESS_OF_SENDER

# gmail pass
NODE_PASS=$EMAIL_PASS

# string with spaces of receiver mails
RECEIVERS=foo.bar@gmail.com

# You can deploy both an empty TMS (for initial release)
# or a non-empty.

# deploy empty TMS
npx tag-cli deploy-tag-manager --is-empty

# deploy non-empty (default)
npx tag-cli deploy-tag-manager --no-is-empty

Tag Template Commands

A Tag Template functions as the backbone of a tag. It's a NPM package. It contains critical logic of handling tags logic and returns a class with a set of methods that we can use, inherit and override. Typically a Tag Template contains the following:

it accepts an initializer object;
it notifies the dataLayer that a Tag Template has been initialized;
it's a class;
it returns a render method;
it's an NPM package, starting with: @d-cat/tag-template-;
it publishes according NPM semver.

tag-or-tag-template

`deploy-tag-template`

The deploy-tag-template command deploys a tag template on a NPM registery of choice. The package version is incremented based on the commits done through i.e. commitizen. Standard-version is used to publish either a major, minor or patch according to NPM's semver. A CHANGELOG.md is added or updatet within the repo. Please note that you should only use version 1.0.0 on a stable release. Using version 0.* will behave differently using NPM's semver.

Options:
  --version                  Show version number                         [boolean]
  --help                     Show help                                   [boolean]
  --access-token, --token    Personal accesstoken.                       [string]
  --branch-name, --branch    The current branch.                         [string]
  --dist-dir, --d            Dist dir. Default to `lib`.                 [string]
  --is-flat-package          NPM flatpackage publish.                    [boolean]
  --project-path, --p        $CI_PROJECT_PATH.                           [string]
  --key, --k                 SSH Key.                                    [string]
  --user, -u                 Git username.                               [string]
  --email, -e                Git email.                                  [string]
  --o-auth-token, --oauth    NPM authentication file.                    [object]

Examples

Note that all flags are either mandatory or are set as an environment variable.

# Personal access token generated within gitlab
ACCESS_TOKEN=$MY_PERSONAL_ACCESS_TOKEN

# SSH key is used to increment the version of the npm package
# based on the commits done, according to commitizen.
# standard-version is used to update a major, minor or patch and use
# NPM's semver.
KEY=$SSH_KEY

# your git user + email for authentication
GIT_USER=$GIT_USER_VAR
GIT_EMAIL=$GIT_EMAIL_VAR

# deploy a flat package to a npm registery
# dist dir is set to dist, thus the build is placed in dist
# defined in your package.json
npx tag-cli deploy-tag-template --is-flat-package --dist-dir dist --branch-name $CI_COMMIT_REF_NAME --project-path $CI_PROJECT_PATH

# deploy a standard npm package
npx tag-cli deploy-tag-template --no-is-flat-package --branch-name $CI_COMMIT_REF_NAME --project-path $CI_PROJECT_PATH

`update-dependents`

If not using npm semver, you can use update-dependents to keep track of dependencies. The update-dependents command updates and triggers the first 1000 default branches of all GitLab projects that either use the current package as dependency or as devDependency, using the GitLab API. Please note when using npm ci this will not impact the package-lock.json and thus you have to update that one manually.

# flags
Options:
  --version                Show version number                         [boolean]
  --help                   Show help                                   [boolean]
  --access-token, --token  Personal accesstoken.                       [string]
  --dir-name, --dir        The dir name.                               [string]
  --project-id, --d        $CI_PROJECT_ID.                             [string]
  --key, --k               SSH Key.                                    [string]
  --user, -u               Git username.                               [string]
  --email, -e              Git email.                                  [string]

update-dependents

Examples

Note that all flags are either mandatory or are set as an environment variable:

ACCESS_TOKEN=myAccesToken
DIR_NAME=foo
PROJECT_ID=1210
SSH_KEY=key
GIT_EMAIL=myEmail

npx tag-cli update-dependents --user myGitName

Tag commands

A Tag is used to initialize a Tag Template or to handle a Google Tag Manager's data object. A tag should never contain complex logic. Use a tag template instead. A Tag is easy to read and easy to debug. There should be no complexity in a tag. The following is applicable for a Tag:

it exports a default function without arguments;
it initializes a tag template;
it contains a publish-tag job;
it contains no complexity.

`create-new-tag`

The create-new-tag command creates a new Tag boilerplate designed to use with @d-cat/tag-cli within GitLab CI/CD.

Options:
  --version               Show version number                          [boolean]
  --help                  Show help                                    [boolean]
  --force                 Force create of tag                          [boolean]

What's included

The Tag boilerplate contains all configurations to:

Jest setup
tslint setup
prettier
husky
tag config file (tag-cli.json)
dockerized tag environment to run the tag locally in isolation
dockerized TMS that registers images in GitLab private registery
stable pipeline file with artifacts enabled both on success and failure
standard-version to created changelog + semver in the pipeline

Dependencies

npm i yourpackage@1.0.0

Examples

Make sure docker is installed and you've updated your .env file.

# creating a tag boilerplate
npx tag-cli create-new-tag --force

# creating a tag boilerplate with prompt
npx tag-cli create-new-tag --no-force

# running a tag locally in isolation on localhost:1234
docker build -t ${img_name} .
docker run --name ${tag_name} --rm -p 1234:1234 ${img_name}:latest

# running a tag locally in isolation with docker-compose
docker-compose build build
docker-compose up build

# running the full tag pipeline (except uploading artifacts) locally in isolation
docker-compose build pipeline
docker-compose up pipeline

Configuring a tag

The create-new-tag command will create a file called tag-cli.json. This file contains all configurations. The tag-cli.json file includes both pipeline as tag manager settings.

Pipeline settings

This are settings that indicates how a tag should behave inside a pipeline.

Prop	Type	Desc
`isPipelineActive`	boolean	Boolean that indicates if the tags pipeline should run.
`pipelineBranches`	string[]	String array of branches that should trigger the Tag's pipeline.
`isImportInTagManager`	boolean	Boolean to indicate if the tag should be imported in the tag manager.
`tagManagerBranches`	string[]	String array of tag branches that should be included in a tag manager bundle.
`isActive`	boolean	Boolean that indicates if the tag is active.
`isTrigger`	boolean	Boolean that indicates if the tag should trigger the Tag Managers pipeline after a publish.

Tag Manager settings

This are settings that determine how the tag should behave inside the Tag Manager.

Prop	Type	Desc
`firingAmount`	number	Amount a tag should be invoked when all business rules apply.
`priority`	number	Priority of tag. 1 is highest priority.
`id`	string	UUID.
`urls`	IUrl[]	Object Array that contains 2 props: `url` (RegExp) and `reverse` (boolean).
`triggers`	ITrigger[]	Boolean that indicates if the tag is active.

`ITrigger`

Options	Type	Required Description
`type`	string	true	Type of the trigger, this must be one of the following: `instantly`, `datalayer`, `event`, `domEvent`, `persisted`. Note that when using `instantly`, this means the other triggers automatically are ignored, thus the tag will execute immediately.
`id`	string	true	Uniquely generated ID. Using the CLI this will generate an ID using uuid/v1.
`name`	string	false	Depending on the type, the name key has a different function.
`value`	string	false	Depending on the type, the value key has a different function.
`reverse`	boolean	false	Reverse the match on either value or name.
`element`	string	false	`window`, `document` both as string or an element ID.

GitLab configuration

Make sure you did all necessary steps to start with GitLab.

CI/CD variables to create in GitLab

Before you run the pipeline, make sure you create the necessary CI/CD variables in your GitLab Tag project.

CI/CD variable	Type	Description
`PERSONAL_ACCESS_TOKEN`	variable	Your personal access token, that has owner rights of the tags group.
`TAG_MANAGER_ID`	variable	Gitlab Project ID of the Tag Manager.
`GCLOUD_SERVICE_KEY`	file	Your authentication file for Google Cloud.
`SSH_KEY`	file	Your SSH key to clone and push repo's of GitLab.
`GIT_USER_EMAIL`	variable	Your git email: `git config user.email`.
`GIT_USER_NAME`	variable	Your git user name: `git config user.name`.
`CODECOV_TOKEN`	variable	Your Codecov token to publish code coverage reports.

Troubleshooting

If the pipeline isn't running after the initial commit, trigger the pipeline manually.

`test-tag`

The test-tag command runs the test script as defined in your package.json and uploads a codecoverage report to Codecov.

Options:
  --version                 Show version number                              [boolean]
  --help                    Show help                                        [boolean]
  --codecov, -c             CodeCov token                                    [string]
  --branch-name, --n        $CI_COMMIT_REF_NAME                              [string]

Examples

Note that all flags are either mandatory or are set as an environment variable.

# secret codecov token generated in the codecov interface
CODE_COV_TOKEN=$MY_CODE_COV_TOKEN

# run tests + upload report to codecov
npx tag-cli test-tag --branch-name $CI_COMMIT_REF_NAME

`lint-tag`

The lint-tag command runs the TypeScript linter.

Options:
  --version                 Show version number                              [boolean]
  --help                    Show help                                        [boolean]
  --branch-name, --n        $CI_COMMIT_REF_NAME.                             [string]

Examples

npx tag-cli lint-tag --branch-name $CI_COMMIT_REF_NAME

`type-tag`

The type-tag command runs type checks the tag according TypeScript specs.

Options:
  --version                 Show version number                              [boolean]
  --help                    Show help                                        [boolean]
  --branch-name, --n        $CI_COMMIT_REF_NAME.                             [string]

Examples

npx tag-cli type-tag --branch-name $CI_COMMIT_REF_NAME

`publish-tag`

The publish-tag command publishes tag artifacts in GitLab, increments the tag version and additionally triggers the Tag Managers pipeline to deploy a new version. The artifacts you publised can be fetched using GitLab's API. Both incrementing the version as triggering the Tag Managers API are optional. However, triggering the Tag Managers pipeline is configured in the tag-cli.json.

Options:
  --version             Show version number                              [boolean]
  --help                Show help                                        [boolean]
  --job-token, -j       Unique token.                                    [string]
  --access-token, -t    Access token.                                    [string]
  --tag-manager-id, -i  GitLab\'s Tag Manager project ID.                [string]
  --key, -k             SSH Key.                                         [string]
  --user, -u            Git username.                                    [string]
  --email, -e           Git email.                                       [string]
  --increment-version   Update using semver.                             [boolean]

Examples

Note that all flags are either mandatory or are set as an environment variable.

# Personal access token generated within gitlab
ACCESS_TOKEN=$MY_PERSONAL_ACCESS_TOKEN

# SSH key is used to increment the version of the npm package
# based on the commits done, according to commitizen.
# standard-version is used to update a major, minor or patch and use
# NPM's semver.
KEY=$SSH_KEY

# your git user + email for authentication
GIT_USER=$GIT_USER_VAR
GIT_EMAIL=$GIT_EMAIL_VAR

# we need the job token to trigger the tag managers pipeline
# using the job token will show the tag managers pipeline
# to be triggered in GitLabs interface, and in the TMS
# project it shows our tag defined as the triggerer
JOB_TOKEN=$CI_JOB_TOKEN

# set a gitlab variable where the project id
# of the tag manager is saved
TAG_MANAGER_ID=$TAG_MANAGER_PROJECT_ID

# incrementing version
npx tag-cli publish-tag --increment-version

# do not increment version
npx tag-cli publish-tag --no-increment-version

Miscellaneous commands

Other commands.

`bundle-dependencies`

The bundle-dependencies command bundles the contents of all third party endpoints in tms.dependencies.json and writes the bundle to 3rd-party-dependencies.js or another specified output file.

Options:
  --version                Show version number                          [boolean]
  --help                   Show help                                    [boolean]
  --bucketName -b          GCP Bucket Name                              [string]
  --input -i               Name of the input file                       [string]
  --is-uglify -iu          Should uglify mangle props                   [boolean]
  --is-upload, ie          Should upload to GCP                         [boolean]
  --output -o              Name of the output file                      [string]
  --key-file-name -k       Service file from GCP                        [object]
  --tms-id -id             Associated TMS ID                            [number]

Examples

npx tag-cli bundle-dependencies --output dependencies.js

`tms.dependencies.json`

The tms.dependencies.json holds an array with objects.

Prop	Type	Desc
`prepend`	string	A JS snippet placed in front of the SDK.
`endpoint`	string	The endpoint of the SDK, same as binary.
`binary`	string	The endpoint of the SDK, same as endpoint.
`trigger`	string	Event that will be emitted into DDM's event emitter, after the SDK is loaded.
`wrapper`	string	An optional special prop is called wrapper. This prop is meant to wrap or add some code in front or behind the SDK. The wrapper prop must contain a `{{CODE}}` string, as that's the place where the SDK will be interpolated.

[
  {
    "endpoint": "https://www.googletagmanager.com/gtm.js?id=GTM-1234567",
    "prepend": "window.dataLayer=window.dataLayer||[];window.dataLayer.push({'gtm.start':new Date().getTime(),event:'gtm.js'}",
    "trigger": "3rd.gtm.loaded"
  },
  {
    "binary": "//snap.licdn.com/li.lms-analytics/insight.min.js",
    "prepend": "_linkedin_data_partner_id = '234089'",
    "trigger": "3rd.linkedin.loaded",
    "wrapper": "var handler = function() {if (_ddm.get('user.cookieConsent') === 2){ {{CODE}} }}; _ddm.listen('user.cookie.accept', handler);handler();"
  }
]

The place of {{CODE}} in the wrapper vaue will be interpolated with the SDK snippet.

`clean-up`

The clean-up command fetches all pipelines from the given project id and analyzes all it's jobs for artifacts. If an artifact found, it will be removed.

Try to always set a expire_in variable for artifacts.

Options:
  --version             Show version number                              [boolean]
  --help                Show help                                        [boolean]
  --project-id, -i      The current project ID: $CI_PROJECT_ID.          [string]
  --access-token, -t    Access token.                                    [string]
  --stage, -s           Name of the job\'s stage. Default "build".       [string]

Examples

Note that all flags are either mandatory or are set as an environment variable.

npx tag-cli clean-up --project-id $CI_PROJECT_ID --access-token 1234

Troubleshooting

Publishing tag TypeError: Cannot read property 'map' of undefined: make sure your tag-cli.json has an string array of both branchesToImportInTagManager and branchesToRunInPipeline.
Publishing tag This tag didn't triggered the pipeline.: Make sure the branch that triggered the pipeline equals a branch that is whitelisted in your tag-cli.json.
GitLab API: Fetching projects. Unexpected response received from GitLab: [object Object], or the amount of received projects meets the maximum of 1000. Received: undefined : Retry the pipeline, as the GitLab API didn't returned a correct response. It has probably something to do with a connection lost / timeout.

Usage no npm install needed!

README

@d-cat/tag-cli

Install

Usage

Tag Manager commands

create-tag-manager

What's included

Update depedencies

Examples

GitLab configuration

CI/CD variables to create in GitLab

bundle-tag-manager

Examples

Test the bundle

deploy-test-instance

Examples

deploy-tag-manager

Examples

Tag Template Commands

deploy-tag-template

Examples

update-dependents

Examples

Tag commands

create-new-tag

What's included

Dependencies

Examples

Configuring a tag

Pipeline settings

Tag Manager settings

ITrigger

GitLab configuration

CI/CD variables to create in GitLab

Troubleshooting

test-tag

Examples

lint-tag

Examples

type-tag

Examples

publish-tag

Examples

Miscellaneous commands

bundle-dependencies

Examples

tms.dependencies.json

clean-up

Examples

Troubleshooting

`create-tag-manager`

`bundle-tag-manager`

`deploy-test-instance`

`deploy-tag-manager`

`deploy-tag-template`

`update-dependents`

`create-new-tag`

`ITrigger`

`test-tag`

`lint-tag`

`type-tag`

`publish-tag`

`bundle-dependencies`

`tms.dependencies.json`

`clean-up`