Pipeliner Module

Build Status

A framework for creating and running data pipelines. Data pipelines have stages, which are made of an arbitrary number of tasks. Stages and tasks are run in serial: once a task completes, the next task in the pipeline is executed.
A task may complete synchronously, or asynchronously through use of a promise.

Pipelines take a data object as input, and each task operates on the object in some way.

For an example of the Pipeliner in action, checkout the nxus-static-site module.

Installation

> npm install nxus-pipeliner --save

Usage

Step 1: Define a pipeline

import {pipeliner} from 'nxus-pipeline'

pipeliner.pipeline('my-pipeline')

Step 2: Define tasks

A task is a javascript function that accepts any objects passed into the pipeline when it is run. Tasks are run serially in FIFO order.

let myTask = (word) => {
  word.toUpperCase();
}

app.get('pipeliner').task('my-pipeline', myTask)

Step 3: Run a pipeline

Once all the tasks for a pipeline have been defined, the last step is to run the pipeline.

app.get('pipeliner').run('my-pipeline', someData)

API


Pipeliner

src/index.js
new Pipeliner()

Extends NxusModule

Example
let myTask = (data) => {
  data.word.toUpperCase()
}
let data = {word: 'hello'}

let pipeliner = app.get('pipeliner')

pipeliner.pipeline('capitalize')
pipeliner.run('capitalize', data).then(() => {
  console.log('data') // {word: 'HELLO'}
})
Instance Members
pipeline(pipeline)
task(pipeline, job)
getPipelines()
getPipeline(pipeline)
run(pipeline, args)