README
Minio/Amazon S3 Storage Adapter
Minio adapter for receiving upstreams. Particularly useful for streaming multipart file uploads from the Skipper body parser.
Installation
$ npm install skipper-minio --save
If you're using this module outside of Sails (e.g. Express or a vanilla Node.js server), make sure you have skipper itself installed as your body parser.
Buyer Beware I probably would not recommend using this package in production applications just yet, and I'm sorry I didn't namespace the package from the beginning. That said it seems to do what it says on the box, but streams are still black magic as far as I'm concerned.
Usage
Using Minio
In the route(s) / controller action(s) where you want to accept file uploads, do something like:
req.file('avatar')
.upload({
// ...options here...
// These *could* be better off in `sails.config.uploads`
adapter: require('skipper-minio'),
bucket: 'avatars',
endPoint: 'minio.mydomain.com',
port: 9000,
accessKey: 'ABCDEFGH123456789',
secretKey: 'ABCDEFGH123456789ABCDEFGH123456789',
useSSL: false,
maxBytes: 1024 * 1024,
maxBytesPerFile: 1024 * 1024 / 3,
allowedFileTypes: ['image/jpeg', 'image/png', 'image/gif'],
transformer: () => { return someDataModifyingPipeableToCropAvatars }
}, function whenDone(err, uploadedFiles) {
if (err) return res.negotiate(err);
else return res.ok({
files: uploadedFiles,
textParams: req.params.all()
});
});
You can also use the nifty sails-hook-uploads for async/await-able upload processing in Sails v1.
Using Amazon S3
req.file('avatar')
.upload({
adapter: require('skipper-minio'),
bucket: 'avatars',
accessKey: 'ABCDEFGH123456789',
secretKey: 'ABCDEFGH123456789ABCDEFGH123456789'
}, function whenDone(err, uploadedFiles) {
if (err) return res.negotiate(err);
else return res.ok({
files: uploadedFiles,
textParams: req.params.all()
});
});
Only allow files of type x
mmmagic is used to detect the MIME type of incoming upload streams.
You're able to restrict the types of files that will be accepted by passing an array of MIME types to the allowedFileTypes
setting.
Transforming an incoming upload
You're able to specify a transformer
method (or array of methods) to modify the incoming upload.
The following example accepts jpeg/png/gif uploads which it crops & resizes and stores as a jpeg using sharp.
const uuid = require('uuid/v4');
const sharp = require('sharp');
req.file('avatar')
.upload({
// Ensure the resulting file always has a `.jpg` extension regardless of the filename of the original upload
saveAs: (__incomingFileStream, cb) => { return cb(null, uuid() + '.jpg'); },
// Specify allowed input file types
allowedFileTypes: ['image/jpeg', 'image/png', 'image/gif'],
// Uploaded files will be piped through the `transformer`s returned value before being sent to minio
// It will be passed a single argument - the detected MIME type of the incoming stream, or `null`
transformer: (detectedMimeType) => {
return sharp()
.flatten({ background: {r: 255, g: 255, b: 255, alpha: 1} })
.resize({
height: 100,
width: 100,
})
.jpeg({ progressive: true });
}
}, function whenDone(err, uploadedFiles) {
if (err) return res.negotiate(err);
else return res.ok({
files: uploadedFiles,
textParams: req.params.all()
});
});
File extension normalisation
If you specify the runSaveAsAfterMimeDetection
boolean option then the saveAs
method will be called once again just before uploading the file to minio.
This could allow you to normalise the final filename of an upload. The use case that prompted this was
- A user uploads a jpeg file with a
.png
extension - A user uploads a jpeg file with a
.JPEG
extension, but other moving parts of the system are looking for and will only work on files with.jpg
file extension
If this option is true
then your saveAs
method will be run multiple times.
You probably shouldn't use this (it's more for my own reference when I come back in 12 months and wonder wtf is going on) but hey, check out our config/uploads.js
module.exports.uploads = {
// ...
allowedFileTypes: ['image/jpeg', 'image/png', 'image/gif'],
fileTypeMap: {
'image/jpeg': '.jpg',
'image/png': '.png',
'image/gif': '.gif',
},
runSaveAsAfterMimeDetection: true,
saveAs: function saveUploadAs(__file, cb) {
const uuid = require('uuid/v4')
const extension = (sails.config.uploads.fileTypeMap[__file.mimeType] || __file.filename.split('.').pop() || '.upload').toLowerCase()
// Because this saveAs method is gonna run probably twice, we attach the generated uuid to __file to save CPU cycles the next time we run
let filename
if (__file.__userlandFileName) {
filename = __file.__userlandFileName
} else {
filename = uuid()
__file.__userlandFileName = filename
}
return cb(null, filename + extension)
}
}
TODO
Contribute
Please! I have no idea what I'm doing here :wink:
To run the tests:
$ npm test
License
Based on work by Jarrod Linahan, Mike McNeil, Balderdash Design Co., Sails Co.
See LICENSE.md
.
This module is intended to be uesd with the Sails framework, and is free and open-source under the MIT License.