@depack/depack

@depack/depack is The Source Code For Depack's JavaScript API. Depack is the compiler of Node.JS packages into a single executable, as well as the bundler for JavaScript web files using Google Closure Compiler. It scans the entry files to detect all dependencies, to passes them to GCC.

yarn add @depack/depack

Table Of Contents

• Table Of Contents
• API
• async run(args, opts=): string
  • RunConfig
• async Compile(options, runOptions=, compilerArgs=): string
  • CompileConfig
• async Bundle(options, runOptions=, compilerArgs=): string
  • BundleBase
  • BundleConfig
• async BundleChunks(options, runOptions=, compilerArgs=): string
  • ChunksConfig
  • Caching
• getOptions(options): !Array<string>
  • GetOptions
• getOutput(output, src): string
• GOOGLE_CLOSURE_COMPILER: string
• async getCompilerVersion(): string
• License & Copyright

API

The package is available by importing its named functions:

Compile, Bundle, BundleChunks, run,     getOptions, getOutput,     GOOGLE_CLOSURE_COMPILER, getCompilerVersion,

async run(
  args: !Array<string>,
  opts=: !RunConfig,
): string

Low-level API used by Compile and Bundle. Spawns Java and executes the compilation. To debug a possible bug in the GCC, the sources after each pass can be saved to the file specified with the debug command. Also, GCC does not add // # sourceMappingURL=output.map comment, therefore it's done by this method. Returns stdout of the Java process. Returns the stdout of the Java process.

• args* !Array<string>: The arguments to Java.
• opts !RunConfig (optional): General options for running of the compiler.

RunConfig: General options for running of the compiler.

async Compile(
  options: !CompileConfig,
  runOptions=: !RunConfig,
  compilerArgs=: !Array<string>,
): string

Compiles a Node.JS source file with dependencies into a single executable (with the +x addition). Performs regex-based static analysis of the whole of the dependency tree to construct the list of JS files. If any of the files use require, adds the --process_common_js_modules flag. Returns the stdout of the compiler, and prints to the console if output is not given in runOptions.

• options* !CompileConfig: Options for the Node.JS package compiler. Must have the src prop at least.
• runOptions !RunConfig (optional): General options for running of the compiler.
• compilerArgs !Array<string> (optional): The compiler args got with getOptions and/or manually extended. getOptions needs to be called first to find out the compiler's JAR at minimum.

The actual logic that makes compilation of Node.JS packages possible is:

• Scan the source code and dependency to find out what internal Node.JS modules are used, and creates the output wrapper with require calls to require those built-in modules, e.g., const path = require('path').
• Add appropriate externs for the internal modules.
• To make Closure resolve internal imports like import { join } from 'path' instead of throwing an error, mock the built-ins in node_modules folder. The mocks will reference the variable from the output wrapper generated in step 1:

  // node_modules/path/index.js
  export default path
  export * from path
  

The last argument, compilerArgs can come from the getOptions method. The output property should come from getOutput method to enable saving to directories without specifying the output filename (GCC will do it automatically, but we need to write source maps and set +x).

CompileConfig: Options for the Node.JS package compiler.

For example, given the following source:

import { constants } from 'os'
import { createWriteStream, createReadStream } from 'fs'

// ...

;(async () => {
  const result = await new Promise((r, j) => {
    const input = process.env['INPUT'] || __filename
    const output = process.env['OUTPUT']
    const rs = createReadStream(input)
    const ws = output ? createWriteStream(output) : process.stdout
    rs.pipe(ws)
    rs.on('error', (err) => {
      if (err.errno === -constants.errno.ENOENT) {
        return j(`Cannot find file ${input}`)
      }
      return j(err)
    })
    rs.on('close', () => {
      r({ input, 'output': output })
    })
  })
  const res = {
    version: process.version,
    ...result,
  }
  console.log(res)
})()

The library can be used to start the compilation:

import { getCompilerVersion, Compile, getOptions } from '@depack/depack'

(async () => {
  const compilerVersion = await getCompilerVersion()
  const options = getOptions({
    advanced: true,
    prettyPrint: true,
    languageIn: 2018,
    languageOut: 2017,
  })
  await Compile({
    src: 'example/compile-src.js',
  }, { compilerVersion }, options)
})()

The compiled output in pretty format of advanced optimisation:

#!/usr/bin/env node
'use strict';
const os = require('os');
const fs = require('fs');             
const g = os.constants;
const h = fs.createReadStream, k = fs.createWriteStream;
(async() => {
  var d = await new Promise((l, e) => {
    const a = process.env.INPUT || __filename, b = process.env.OUTPUT, c = h(a), m = b ? k(b) : process.stdout;
    c.pipe(m);
    c.on("error", f => f.errno === -g.errno.ENOENT ? e(`Cannot find file ${a}`) : e(f));
    c.on("close", () => {
      l({input:a, output:b});
    });
  });
  d = Object.assign({}, {version:process.version}, d);
  console.log(d);
})();

Stderr:

java -jar /Users/zavr/node_modules/google-closure-compiler-java/compiler.jar \
--compilation_level ADVANCED --language_in ECMASCRIPT_2018 --language_out \
ECMASCRIPT_2017 --formatting PRETTY_PRINT --package_json_entry_names module,main \
--entry_point example/compile-src.js --externs node_modules/@externs/nodejs/v8/os.js \
--externs node_modules/@externs/nodejs/v8/fs.js --externs \
node_modules/@externs/nodejs/v8/stream.js --externs \
node_modules/@externs/nodejs/v8/events.js --externs \
node_modules/@externs/nodejs/v8/url.js --externs \
node_modules/@externs/nodejs/v8/global.js --externs \
node_modules/@externs/nodejs/v8/global/buffer.js --externs \
node_modules/@externs/nodejs/v8/nodejs.js
Built-ins: os, fs
Running Google Closure Compiler 20200112

async Bundle(
  options: !BundleConfig,
  runOptions=: !RunConfig,
  compilerArgs=: !Array<string>,
): string

Bundles the browser source code into a JavaScript file. If there are any JSX dependencies, the bundler will transpile them first using ÀLaMode/JSX. Returns the stdout of the compiler, and prints to the console if output is not given in runOptions.

• options* !BundleConfig: Options for the web bundler. Must have the src prop at least.
• runOptions !RunConfig (optional): General options for running of the compiler.
• compilerArgs !Array<string> (optional): The compiler args got with getOptions and/or manually extended.

BundleBase: Options for the web bundler.

BundleConfig extends BundleBase: Options for the Bundle method.

For example, given the following single JS source:

/* eslint-env browser */
[...document.querySelectorAll('.BananaInactive')]
  .forEach((el) => {
    const parent = el.closest('.BananaCheck')
    el.onclick = () => {
      parent.classList.add('BananaActivated')
    }
  })
;[...document.querySelectorAll('.BananaActive')]
  .forEach((el) => {
    const parent = el.closest('.BananaCheck')
    el.onclick = () => {
      parent.classList.remove('BananaActivated')
    }
  })

Depack is used to make a JS file in ES2015 understood by old browsers:

import { getCompilerVersion, Bundle, getOptions } from '@depack/depack'

(async () => {
  const compilerVersion = await getCompilerVersion()
  const options = getOptions({
    advanced: true,
    prettyPrint: true,
  })
  await Bundle({
    src: 'example/bundle-src.js',
  }, { compilerVersion }, options)
})()

The bundled output:

function c(a) {
  var b = 0;
  return function() {
    return b < a.length ? {done:!1, value:a[b++]} : {done:!0};
  };
}
function e(a) {
  if (!(a instanceof Array)) {
    var b = "undefined" != typeof Symbol && Symbol.iterator && a[Symbol.iterator];
    a = b ? b.call(a) : {next:c(a)};
    for (var d = []; !(b = a.next()).done;) {
      d.push(b.value);
    }
    a = d;
  }
  return a;
}
e(document.querySelectorAll(".BananaInactive")).concat().forEach(function(a) {
  var b = a.closest(".BananaCheck");
  a.onclick = function() {
    b.classList.add("BananaActivated");
  };
});
e(document.querySelectorAll(".BananaActive")).concat().forEach(function(a) {
  var b = a.closest(".BananaCheck");
  a.onclick = function() {
    b.classList.remove("BananaActivated");
  };
});

Stderr:

java -jar /Users/zavr/node_modules/google-closure-compiler-java/compiler.jar \
--compilation_level ADVANCED --formatting PRETTY_PRINT
--js example/bundle-src.js
Running Google Closure Compiler 20200112

async BundleChunks(
  options: !ChunksConfig,
  runOptions=: !RunConfig,
  compilerArgs=: !Array<string>,
): string

Bundles the browser source code into multiple JavaScript file. Works in the same way as Bundle, generating a temp dir for JSX dependencies.

• options* !ChunksConfig: Options for the web bundler. Must have the srcs prop with paths to source files at least.
• runOptions !RunConfig (optional): General options for running of the compiler.
• compilerArgs !Array<string> (optional): The compiler args got with getOptions and/or manually extended.

ChunksConfig extends BundleBase: Options for the BundleChunks method.

For example, given the following multiple JS sources:

// chunkA.js
import test from './'
import { common } from './common'

console.log('chunk a')
test()
common()

// chunkB.js
import test from './'
import { common } from './common'

console.log('chunk b')
test()
common()

// common.js
export const common = (opts = {}) => {
  const { a } = opts
  if (window.DEBUG && a) console.log('test')
}

// index.js
export default () => {
  console.log('common')
}

Depack can generate multiple output files when a number of entries are passed:

const options = getOptions({
  chunkOutput: TEMP,
  advanced: true,
  sourceMap: false,
})
await BundleChunks({
  silent: true,
  srcs: ['test/fixture/chunks/chunkA.js',
    'test/fixture/chunks/chunkB.js'],
}, { output: TEMP, noSourceMap: true }, options)

The bundled output:

# chunkA.js

console.log("chunk a");console.log("common");c();


# chunkB.js

console.log("chunk b");console.log("common");c();


# common.js

function c(){var a=void 0===a?{}:a;a=a.a;window.b&&a&&console.log("test")};

Stderr:

java -jar /Users/zavr/node_modules/google-closure-compiler-java/compiler.jar \
--compilation_level ADVANCED --chunk_output_path_prefix test/temp/ --module_resolution \
NODE
--js test/fixture/chunks/index.js
     test/fixture/chunks/common.js
--chunk common:2
     test/fixture/chunks/chunkA.js
--chunk chunkA:1:common
     test/fixture/chunks/chunkB.js
--chunk chunkB:1:common
Running Google Closure Compiler

Caching

This method supports caching. It will shallowly analyse source files (does not go into node_modules apart from finding out their version), and run the checkCache function if it was passed. If this callback returns true, the compilation will be skipped. See an example implementation below.

import stat from 'async-stat'
import deepEqual from '@zoroaster/deep-equal'
import { BundleChunks } from '../src'

const compileOurChunks = async (srcs) => {
  let cachedMap, needsCacheUpdate

  let map = await BundleChunks({
    srcs,
    preactExtern: true,
    async checkCache(analysis) {
      // somehow get the cache object: { chunksMap, files, deps }
      const { chunksMap, ...current } = splendid.getCache('compile-comps')
      cachedMap = chunksMap
      const deps = {}
      const entries = []
      analysis.forEach(({ name, version, entry }) =>  {
        if (name) deps[name] = version
        else entries.push(entry)
      })
      const files = await entries.reduce(async (acc, file) => {
        const accRes = await acc
        /** @type {import('fs').Stats} */
        const ls = await stat(file)
        const d = new Date(ls.mtimeMs).toLocaleString()
        accRes[file] = d
        return accRes
      }, {})
      try {
        deepEqual({ files, deps }, current, ' ')
        // this is now OK, should not need to do anything else
        splendid.log2('compile-comps', 'Comps not changed.')
        return true
      } catch (err) {
        splendid.log2('compile-comps', err.message)
        needsCacheUpdate = err.actual
      }
    },
  }, { compilerVersion, output }, options)

  if (needsCacheUpdate) {
    needsCacheUpdate.chunksMap = map
    // save new cache: { chunksMap, files, deps }
    await splendid.appendCache('compile-comps', needsCacheUpdate)
  } else if (!map) {
    map = cachedMap
  }
  return map
}

getOptions(
  options: !GetOptions,
): !Array

Returns an array of options to pass to the compiler for Compile, Bundle and BundleChunks methods. Full list of supported arguments.

• options* !GetOptions: The map of options to be converted into Java arguments.

GetOptions: Parameters for getOptions.

Example:

import { getOptions } from '@depack/depack'

const opts = getOptions({
  advanced: true,
  iife: true,
  languageIn: 2019,
  languageOut: 2017,
  noWarnings: true,
  prettyPrint: true,
  output: 'bundle.js',
  argv: ['--externs', 'externs.js'],
})
console.log(opts)

[ '-jar',
  '/Users/zavr/node_modules/google-closure-compiler-java/compiler.jar',
  '--compilation_level',
  'ADVANCED',
  '--language_in',
  'ECMASCRIPT_2019',
  '--language_out',
  'ECMASCRIPT_2017',
  '--create_source_map',
  '%outname%.map',
  '--formatting',
  'PRETTY_PRINT',
  '--isolation_mode',
  'IIFE',
  '--warning_level',
  'QUIET',
  '--externs',
  'externs.js',
  '--js_output_file',
  'bundle.js' ]

getOutput(
  output: string,
  src: string,
): string

Returns the location of the output file, even when the directory is given.

• output* string: The path to the output dir or file.
• src* string: The path to the source file. Will be used when the output is a dir.

import { getOutput } from '@depack/depack'

const file = getOutput('output/example.js', 'src/example.js')
console.log('File: %s', file)
const dir = getOutput('output', 'src/index.js')
console.log('Dir: %s', dir)

File: output/example.js
Dir: output/index.js

GOOGLE_CLOSURE_COMPILER: string

If the GOOGLE_CLOSURE_COMPILER was set using the environment variable, it will be returned in this named exported.

async getCompilerVersion(): string

If GOOGLE_CLOSURE_COMPILER was set using an environment variable, returns target, otherwise reads the version from the google-closure-compiler-java package.json file.

License & Copyright

GNU Affero General Public License v3.0

	© Art Deco™ for Depack 2020