Welcome to Signal Pipeline DSKI with Active Filtering’s documentation!

Preface

This example is an extension of the previous one in the Initial Data Streams Tutorial Sequence which, while it gathered DSKI instrumentation points, it did so in a very basic manner. It enabled all context switch events, but did so for all processes on the machine. These events then had to be filtered by using the DSUI events to access the PIDs of the pipeline threads. However, using the Active Filtering in this example, we are able to filter the DSKI events at the time of execution. This can greatly simplify later postprocessing of the data.

Introduction

This example shows how to use the task filter in the dskictrl program. This specific task filter only logs events which are spawned by the specified process, in this case sigpipe. This greatly reduces the volume of data to postprocess, and removes the irrelevant context switch events.

Description

The sigpipe experiment spawns a number of threads representing a pipeline, and sends a number of signals down the thread pipeline. While the experiment runs, a two DSKI datastreams are opened. These datastreams collect all enabled kernel events. They only have the SCHEDULER/SWITCH_TO and SCHEDULER/SWITCH_FROM events enabled. In the first datastream, these events are unfiltered, and in the second they are filtered. These events are logged on either side of the scheduler’s context_switch() call, and during postprocessing, they are used to determine how long each thread in the pipeline actually used the CPU.

Checking Out and Building

The sigpipe experiment is in the kusp examples tree under unified_sigpipe. You will need datastreams installed on your testbed as well as the dski module inserted into your kernel. Follow the instructions for Building the Kernel [[Building the Kernel]] if you have not yet built and installed the KUSP-kernel and and Building and Installing KUSP if you have not yet built and installed the KUSP software. DSKI is packaged as a loadable module and you must insert the dski module into your kernel if you have not already done so:

FIXME.D - This should be this way, or should use tyrian's udev rule script to
accomplish the same permissions

sudo chmod 666 /dev/dski
sudo chmod 666 /dev/ccsm

If you have not built the KUSP examples before, go to Building KUSP Examples

The compiled code is loacted at:

bash> cd $KUSPROOT/examples/build/unified_sigpipe

Running the Example

To run the example, type:

bash> cd unified_sigpipe
bash> make -f unified_test.mk sigpipe-dski2

The makefile target sigpipe-dski2 both runs the file and runs the postprocessing command. You can look at the makefile unified_test.mk and see that the command to actually run the experiment is:

dskictrl -c sigpipe2.dski

Consider what the sigpipe2.dski file is specifying. First, it specifies that the data collected will be output in files in the sigpipe2.dski.bin* directory. It also tells dskictrl to spawn a child process while it logs data. By default, dskictrl will stop logging dski events after the child process has finished execution:

<dski>
output_base = "sigpipe2.dski.bin"
child = "./sigpipe --dsui-config sigpipe.dsui --threads=5 --stimuli=1000"


<channels>
chan1 = {
    buffer_size = 500000,
    buffer_count = 50
}
chan2 = {
    buffer_size = 500000,
    buffer_count = 50
}

The next section configures the channel for the datastream used in this example. Specifically, it tells DSKI to open a relay channel with the specified number of buffers of the specified size to use in logging the kernel events for each stream to a file in user space:

<datastreams>
dski_nofilter = {
  channel = chan1
  filters = []
  enabled = {
      SCHEDULER = [
          SWITCH_TO,
          SWITCH_FROM
      ]
  }
}

dski_filtered = {
  channel = chan2
  filters = [
      task(
          ccsm_name = "sigpipe"
      )
  ]
  child_alias = "sigpipe"
  enabled = {
      SCHEDULER = [
          SWITCH_TO,
          SWITCH_FROM
      ],
          FORK = [
              DO_FORK
      ]
  }
}

This section specifies the properties of each DSKI datastream. The dski_nofilter datastream logs using chan1, attaches no filters to this datastream, and enables only the SCHEDULER/SWITCH_TO and SCHEDULER/SWITCH_FROM events in the kernel. The dski_filtered datastream logs using chan2, and it also enables the SCHEDULER/SWITCH_TO and SCHEDULER/SWITCH_FROM events, but, in addition to this, it actually attaches a filter to its datastream. It enables the FORK/DO_FORK events for the purposes of the task filter.

The consequence of including the task filter is that only DSKI events that are logged by tasks that are registered under the name sigpipe will be accepted (and actually logged). However, this example does not actively name its components. Dskictrl provides a utility called child_alias which creates a set and places the main thread of the application into that set. When the main thread creates new threads they are also added to the set by the task filter.

It is instructive to read through some of the code in dskictrl to see which functions are used to apply the task filter. The dskictrl program is in kusp/trunk/subsystems/datastreams/dski/dskid.

Competing Load!

As in the previous example, we want to create competing load on the system to make the example a more realistic one. To do this, we again check out a kernel and compile it. In order to run the example in this manner, you can run this command:

make -f unified_test.mk sigpipe-dski2-compete-load

This does the same thing as in the previous example. You can note that, again, the proportion between the unfiltered and filtered events will be much higher.

Postprocessing

The postprocessing has already been run when you ran the makefile target. However, the command to run the postprocessing alone is:

postprocess f sigpipe_dski2.pipes

The actual results of this postprocessing should be very similar to those of the previous example, Sigpipe DSKI. The purpose of this example is to demonstrate the capabilities of the active filter. It completely removes the need for a custom filter to filter the data by PID, because the events are filtered as they occur.

It should be noted that the pipeline pids has been removed for this example. Its purpose has been demonstrated in the previous example, and without the custom filter custom_PIDs, we will be logging the DSKI events from the sigpipe example itself.

FIXME: However, a similar pipeline should be included in the group scheduling examples. It would demonstrate the capabilities of group scheduling to regulate the activities of processes: there would be no foreign context switch events during the gaps.

If you run the command:

postprocess n sigpipe2.dski.bin/chan2/cpu0.bin

to simply narrate the filtered datastream, you will notice something which may be strange: the SWITCH_TO events greatly outnumber the SWITCH_FROM events. FIXME: May need further examination – This is most likely because of the manner in which the active filter works. As the context switch events are occurring, those with a PID not included in the sigpipe pipeline are discarded. Thus, when the machine switches away from a sigpipe thread, it logs it. However, when the new context it switched to finishes, it does not log that SWITCH_FROM event because it is of a different process. It does, however, log the SWITCH_TO event immediately afterwards, if it is switching to a sigpipe process.

Analyzing the Results

FIXME: The results need to be analyzed!

Contents:

Indices and tables