README
SOCKO! - Hierarchical file weaver.
Introduction
SOCKO! is a file builder, that takes an input directory of files and applies various features based on a hierarchy directory to produce an output.
It is typically used for configuration management.
The SOCKO! framework
This module dubbed "SOCKO!" is merely a frontend to the socko framework. It uses socko-converter-file to produce a logical hierarchy for the socko-api, which does the heavy lifting. Its output is again mapped by socko-converter-file to a file structure.
How SOCKO! works
SOCKO! reads an input directory file by file and applies various features to the encountered files based on a node in the hierarchy directory, which typically resides as "_socko" directly underneath the input directory, but can also be placed elsewhere. In the end, everything is put into an output directory.
Because of its hierarchical features, SOCKO! can overwrite basic settings with settings in higher nodes.
Usage
SOCKO! is a cli application that can be run by calling
socko generate
and specifying at least these arguments:
- --input: Path to the input directory
- --node: Node in the hierarchy directory
- --output: Path to the output directory
- --hierarchy: Path to the hierarchy directory (optional, uses the directory _socko under the input directory if not set)
Nodes are referenced as directories directly underneath the hierarchy directory. You can also use nodes beneath other nodes, by using the first node name, a : and the second node name (nodeA:nodeB).
For more arguments, see
socko generate --help
Docker
SOCKO! is also available as an official Docker image called dodevops/socko over at Docker Hub.
Simple usage
You can use the image with bind mounts for generating configurations on the fly. Let's say, you have this directory tree:
*
|
*-* socko
|
* input
|
* hierarchy
You can run socko by issuing:
docker run -v `pwd`:/socko dodevops/socko generate --input /socko/input --hierarchy /socko/hierarchy --output /socko/output
Your generated configuration will then be available in the new directory "socko/output".
Container configuration
You can also use SOCKO! to generate configurations for your container based infrastructure. We call this a "configuration container".
To do this, you'll first create a new image and put your input and hierarchy directories into it.
We have two sample Dockerfiles at hand for an environment with a separate hierarchy and one with a "_socko"-subdirectory-type hierarchy. Simply take the one that fits and put it in your socko directory. Let's assume you're standing in a directory with an "input" and a "hierarchy" subdirectory, you can run:
docker build -t my-configuration-container:0.1.0 <socko-directory>/Docker/configurationContainer
This will generate your configuration container image. You can then use the configuration container by running:
docker run --rm -v `pwd`/output:/socko/output -e NODE=nodeA my-configuration-container:0.1.0
This will run SOCKO! in your configuration container and generate the configuration for the node specified using the environment variable "NODE".
Simply mount the volume you want to fill with the generated configuration to /socko/output.
You can use this method combined with things like Kubernetes Init-Containers to populate a configuration volume with the container's configuration first and then starting the container with that volume.
SOCKO! features
Overrides
SOCKO! can override files found in the input directory with files from a hierarchy node, if their filenames match.
Example
Setup
- input
- SimpleTextFile.txt
Input content
- SimpleTextFile.txt
- hierarchy
- nodeA
- nodeB
- SimpleTextFile.txt
Content from nodeB
- nodeB
- nodeA
Generated output
socko generate --input input --hierarchy hierarchy --output output --node nodeA
- output
- SimpleTextFile.txt
Input content
- SimpleTextFile.txt
socko generate --input input --hierarchy hierarchy --output output --node nodeA:nodeB
- output
- SimpleTextFile.txt
Content from nodeB
- SimpleTextFile.txt
Sockets
SOCKO! can merge together socket- with cartridge-files (thus the name). Socket files hold different places (called _cartridge-slot_s), where cartridges should be placed.
These cartridges can either be specified by a single name or as a pattern, forming a cartridge collector.
Socket-files live in the input directory and have the prefix .socket, while cartridge-files live in the hierarchy directory and have the prefix .cartridge.
_Cartridge slot_s are defined by placing a special text inside a socket file. To support multiple text formats, SOCKO! provides multiple formats of this special text, called _flavour_s.
Please see the socko-converter-file documentation for details about the available flavours.
Example
Setup
- input
- SimpleTextFile.txt.socket
This is a socket. {{<< SOCKO: MyCartridge.txt >>}}
- SimpleTextFile.txt.socket
- hierarchy
- nodeA
- nodeB
- MyCartridge.txt.cartridge
Cartridge content
- nodeB
- nodeA
Generated output
socko generate --input input --hierarchy hierarchy --output output --node nodeA
=> Error, because no cartridge can be found
socko generate --input input --hierarchy hierarchy --output output --node nodeA:nodeB
- output
- SimpleTextFile.txt
This is a socket. Cartridge content
Buckets
SOCKO! can fill up directories in the input directory with files from the hierarchy in the same directory. These so-called bucket directories hold just one file called .socko.include. This file holds a special pattern which denotes, which files should be placed into the bucket.
Please see the socko-converter-file documentation for details.
Example
Setup
- input
- MyBucket
- .socko.include
0:G:BucketEntry*
- .socko.include
- MySecondBucket
- .socko.include
-1:G:BucketEntry*
- .socko.include
- MyBucket
- hierarchy
- nodeA
- MyBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA
- BucketEntry1.txt
- MySecondBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA
- BucketEntry1.txt
- nodeB
- MyBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA:nodeB
- BucketEntry2.txt
Bucket entry 2 in nodeB:nodeB
- MySecondBucket
- BucketEntry2.txt
Bucket entry 2 in nodeB:nodeB
- BucketEntry2.txt
- MyBucket
- MyBucket
- nodeA
Generated output
socko generate --input input --hierarchy hierarchy --output output --node nodeA
- output
- MyBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA
- BucketEntry1.txt
- MySecondBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA
- BucketEntry1.txt
- MyBucket
socko generate --input input --hierarchy hierarchy --output output --node nodeA:nodeB
- output
- MyBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA:nodeB
- BucketEntry2.txt
Bucket entry 2 in nodeA:nodeB
- BucketEntry1.txt
- MySecondBucket
- BucketEntry1.txt
Bucket entry 1 in nodeA
- BucketEntry2.txt
Bucket entry 2 in nodeB:nodeB
- BucketEntry1.txt
- MyBucket
Note, that MySecondBucket contains both the entries from nodeA and nodeA:nodeB, because the bucket was set up to search up to the root node, while it was configured to only use the first node in MyBucket.
Ignoring cartridges
If you'd like to exclude specific cartridges (and thus leaving the part of the socket file empty), you can add one or more "--ignore" parameters together with cartridge file names.
You can also specify to exclude cartridges on a specific node path by prefixing
--ignore with
socko --ignore :_root:nodeA:nodeB=dynamic_txt_content1 (...)
Renaming files in flight
If you'd like to rename the files created during a SOCKO! run, you can use the --rename argument. This argument can be specified multiple times and should be in the form
--rename source-path=destination-path
The paths should be relative to the input directory. If SOCKO! finds a path, that matches source-path during its run (in all features), it is automatically translated to destination-path.
Skipping recreation of files, that have the same content
Usually, SOCKO will simply flood your output directory with the generated files. It does not check, if the new content is the same as the old content.
If you don't want this for some reason, add the parameter
--skipIdenticalSockets
and SOCKO will check, if the files differ before it actually writes them.
Migrating from SOCKO! 1
The following things have changed between SOCKO! 1 and 2 and should be checked and adjusted in your installation:
- The --rename paramter uses a "source-path=destination-path" now instead of "source-path:destination-path" now and only supports filenames, not paths.
- The --ignore-parameter uses a node-path before the cartridge-name and not a single node name any more.
- The bucket-pattern allows RegExps now, so the RegExp-type should be given as a second part. The former version is still supported, but marked as deprecated and will be removed in a future version