Process Based Balanced Pipeline

In this example, several pipelines are created for prcoessing messages. Each pipeline consists of several threads which are connected using UNIX sockets. Each pipeline is a separate process. Additionally, the source of messages for each pipeline is a separate process. Each pipeline source is connected to a pipeline using internet sockets. Finally, a supervisor application is included to execute all of the pipelines and sources.

• Configuration
  • Sender
  • Pipeline
  • Launcher
• Group Scheduling
  • Frame Progress SDF
    • Building the SDF
    • Getting/Setting the Progress Variance Limit
    • Getting/Setting the Progress

Running the Application

A python script named run is provided to execute the program. The script provides default settings but the configuration files that it uses can be modified at the command line:

Usage: run [exec|postprocess] [options]

Options:
  -g GSH          The configuration file to use for group scheduling
  -i DSKI         The configuration file to use for dski
  -p POSTPROCESS  The configuration file to use for postprocessing.
  -h, --help      show this help message and exit

Here is an example:

./run -g fp.gsh -i bp.dski -p bp.pipes

The script invokes gschedctrl to set up the group scheduling hierarchy, starts the application using dskictrl then invokes postprocess after the experiment finishes.

The exec argument allows the program to be run without postprocessing. The postprocess argument will invoke only the postprocessing.