aicsfiles-js

This is a JavaScript client to the AICS File Storage Service. It provides the API to initiate, process and complete an upload of files. The client requires access to the computer's filesystem in order to copy files to the destination directory. Thus the process using it must have access to NodeJS libraries such as fs, os, path, etc. If included inside of an electron application, this should be used in the "main" process.

Eventually it will also provide API to query for and access previously uploaded files.

Level of Support

We are not currently supporting this code for external use, but simply releasing it to the community AS IS. It is used for within our organization. We are not able to provide any guarantees of support. The community is welcome to submit issues, but you should not expect an active response.

Getting started

Install the library:

yarn add @aics/aicsfiles

Create FMS instance:

const uploadClient = new FileManagementSystem(LIMS_HOST, LIMS_PORT, "debug");

Example Usage

const startUpload = async (uploads, jobName) => {
    try {
        const startUploadResponse = await uploadClient.validateMetadataAndGetUploadDirectory(uploads);
        /* result will look something like this:
        {
            jobId: "2f55ea322a654e35b98d003a74f9170e",
            uploadDirectory: "/tmp/lk/fss/incoming/asdf
        }
        **/

        const result = await uploadClient.uploadFiles(startUploadResponse, uploads, jobName);
        /*  result will look something like:
        * {
        *   a-filename.txt: {
        *       fileId: "65ebf71c8f1046dcb52931e4aa9db0f4", 
        *       fileName: "Nunito-Regular.ttf",
        *       readPath: "/allen/aics/apps/stage/file-storage-service/data/f4/65ebf71c8f1046dcb52931e4aa9db0f4"
        *   }
        *   ...
        * } 
        * */
       console.log("Upload complete", result);
    } catch (e) {
        console.log("Upload failed", e)
    }
};

const retryUpload = async (uploadJob) => {
    Logger.debug("received start upload request from renderer");
    const uploadClient = new FileManagementSystem(LIMS_HOST, LIMS_PORT, "debug");
    try {
        const result = await uploadClient.retryUpload(uploadJob);
        /*  result will look something like:
        * {
        *   a-filename.txt: {
        *       fileId: "65ebf71c8f1046dcb52931e4aa9db0f4", 
        *       fileName: "Nunito-Regular.ttf",
        *       readPath: "/allen/aics/apps/stage/file-storage-service/data/f4/65ebf71c8f1046dcb52931e4aa9db0f4"
        *   }
        *   ...
        * } 
        * */
        console.log("Upload complete", result);
    } catch (e) {
        console.log("Upload failed", e)
    }
};

startUpload({
    "/home/foo/kitties.txt": {
        file: {
            originalPath: "/home/foo/kitties.txt", // required
            fileType: "other", // required
            numberCats: 3, // arbitrary fields are allowed
        },
        anotherProperty: "foo", // arbitrary fields here are also allowed
    }
});

retryUpload(uploadJobFromJSS);

Library development

NOTE: We're using yarn for dependency management. If you don't have yarn already, set it up by following these instructions:

https://yarnpkg.com/en/docs/install#debian-stable

Gradle Tasks

Description of Gradle tasks:

script	comments
build	create CommonJS and ES module builds
bundle	run Webpack to create a UMD bundle
clean	remove generated artifacts
format	run prettier on src directory
generateTypes	generate type declarations
lint	run eslint on src directory
transpileCommonJs	run babel on src directory; transpile import/export statements for a CommonJS compatible build
transpileES	run babel on src directory; do not transpile import/export statements for an ES module compatible build (used by bundlers for tree-shaking)
test	run mocha; searches for any files matching the pattern "src/*/.test.js"
typeCheck	run tsc in type-check only mode

Simultaneously working on application and library

You can test a development copy of the library in another application using yarn-link:

yarn link creates a symlink for the library that points to your development copy.

1. Run yarn link in the root of this repository
2. Run yarn link @aics/aicsfiles in the root of your application

Publishing

After merging to master, create a release build specifying the type of release ('patch', 'minor', 'major') and checking PUBLISH. https://jenkins.corp.alleninstitute.org/job/node-packages/job/aicsfiles/job/master/build?delay=0sec