Signal Pipeline Group Scheduling Documentation¶
This example shows how the PID based API for GS can be used to place a set of threads or processes under GS control. This is not the most expressive, best, or recommended API for GS, and we preserve it primarily for regression and sanity stesting. However, it is the simplest API, and can be adequate for simple applications being used in simple system architectures.
In this example, the main thread creates several threads. These threads wait for a signal to arrive and then forward the signal to the next thread. The threads are placed under the control of the sequential scheduling decision function with the later threads in the pipeline having better priorities.
Much of the code and postprocessing is similar to the Sigpipe DSUI/DSKI examples. It is recommended to go through these examples before looking at this example. The Sigpipe DSKI examples are located here:
Signal Pipeline Active Filtering
Additional Packages Required¶
The postprocessing for this example makes use of pygraphviz. Under fedora, to install this package, first get easy_install:
bash$ sudo yum install graphviz
bash$ sudo yum install graphviz-devel
bash$ sudo yum install python-setuptools-devel
bash$ sudo easy_install pygraphviz
The Sequential Scheduling Decision Function¶
This example utilizes the sequential scheduling decision function (SDF). The sequential SDF maintains an ascending order list of threads based on their priorities. At scheduling time, the sdf walks the list and tries to run each thread. The first thread that is runnable will be scheduled. Group scheduling refers to this SDF as “sdf_seq”.
Group Scheduling Hierarchy Structure¶
The tasks we want to schedule have to be identified to group scheduling. Tasks are identified using a hierarchy. Additionally, the hierarchy organizes the tasks and specifies which SDF should control them. In this example, a top level group is created named “pipeline_group”. All of the threads are added to this group as members named “thread-0”, “thread-1”, etc.
Examining the Source¶
The source for this example is located at:
$KUSPINSTALL/examples/sigpipe_gsched/sigpipe.c
In this example, it is important to note that any code within a ‘use_ccsm’ if-statement is not used. However, code inside “use_gs” if-statements is used.
Creating a Group¶
The first place of interest is at the beginning of the main function.:
if(use_gs) {
grp_fd = grp_open();
if (grp_fd < 0) {
perror("grp_fd");
return 1;
}
if (grp_create_group(grp_fd, "pipeline_group", "sdf_seq")) {
perror("create group");
return 1;
}
if(gsched_install_group(grp_fd, "pipeline_group", "pipeline_group_mem")) {
perror("install group");
return 1;
}
}
First, a file descriptor that refers to group scheduling is opened. You must have an open fd in order to use group scheduling. Next, a group named “pipeline_group” is created and it is assigned the “sdf_seq” sdf. If a scheduling decision occurs that involves this group then the callbacks of the sequential sdf will be called with this group as a parameter.
Finally, the “pipeline_group” is installed in the system with the member name “pipeline_group_mem”. Group scheduling internally uses a group that contains all other groups. Therefor, we are making the “pipeline_group” a member of the internal top level group. The member name given here identifies the group within group scheduling’s internal top level group. All members of any group are uniquely identified within the group by their member names.
If the group is not installed then it will never be part of the scheduling decision for the system. However, you can add members to a group before the group has been installed. It is possible to install an entire hierarchy all at once by adding groups and tasks to a top group and then installing the top group.
Adding a Member¶
The next place of interest is near the top of the thread_code function.:
tidlist[id] = gettid();
sprintf(name, "thread-%d", id);
if(use_gs) {
if(use_ccsm) {
printf("Adding T-%d to the group by name\n", id);
grp_name_join_group(grp_fd, "pipeline_group", name, 0);
} else {
printf("adding T-%d to the group by pid\n", id);
grp_pid_join_group(grp_fd, "pipeline_group", tidlist[id], name);
}
gsched_set_member_param_int(grp_fd, "pipeline_group", name, pipeline_len - id);
}
The thread_code function is executed when a thread starts up. Here, id is the stage of the pipeline which the thread represents, ie: the first stage would have an id of 0. First, the tid of the thread is retrevied. The tid uniquely identifies the thread on the system. Next, a name is created for the thread. We are not using CCSM in this example and, therefor, the grp_pid_join_group function will be executed. This function adds the thread to the “pipeline_group” using the tid and assigns its member name within the “pipeline_group” to be equal to the generated name for the thread.
Finally, the gsched_set_member_param_int function is used to set the priority of the thread. Recall that the sequential sdf considers lower priorities to be better, ie: it will try to pick threads with a lower priority first. The threads at the end of the pipeline are given lower(better) priorities by subtracting the id of the thread from the length of the pipeline.
Exclusive Control¶
Threads which are added to group scheduling are initially under the control of both group scheduling and Linux. It is necessary to explicitly tell group scheduling to place a thread exclusively under its control.:
if(use_gs) {
gsched_set_exclusive_control(grp_fd, tidlist[id]);
}
It is oot necessary to specify the tid of the thread. If an argument of ‘0’ is given to gsched_set_exclusive_control then the current thread will be used.
Cleanup¶
Group scheduling automatically removes threads from any groups they belong to when they exit. However, groups need to be removed manually. The “pipeline_group” is removed at the end of the main function.:
if (use_gs) {
gsched_uninstall_group(grp_fd, "pipeline_group_mem");
if (grp_destroy_group(grp_fd, "pipeline_group")) {
perror("destroy pipeline group");
}
close(grp_fd);
}
First, the “pipeline_group” has to be uninstalled. Uninstalling a group removes the group from the scheduling hierarchy. it will no longer be used for scheduling decisions. Recall that the member name is how a group or task is identified within a group. To uninstall the “pipeline_group”, the “pipeline_group_mem” member name, that was assigned to the group earlier, is used to uninstall it. Next, the “pipeline_group” is destroyed. Destroying a group frees any resources used by the group. Finally, the group scheduling fd is closed.
Running the Example¶
A bash script is provided to run the example. It will execute the program and perform postprocessing. Use:
./run
Optionally, the program can be run without postprocessing using:
./run exec
Also, the postprocessing can be run by itself using:
./run postprocess
GS Events¶
The DSKI configuration file(sigpipe_gsched.dski) for this example has been modified to enable the instrumentation points for group scheduling:
dski_nofilter = {
channel = chan1
filters = []
enabled = {
GSCHED = [
GRP_CREATE,
GRP_JOIN,
GRP_SET,
GRP_LEAVE,
GRP_DESTROY,
MEM_JOIN,
MEM_SET,
MEM_LEAVE,
]
}
}
These events provide the information necessary to reconstruct the group scheduling hierarchy in postprocessing.
GS Postprocessing¶
Two filters have been added to the ‘sigpipe_gsched.pipes’ postprocessing file:
dski1 = [
head.input(
file = ["/tmp/sigpipe_gsched.dski.bin/chan1/cpu*.bin"]
)
utility.timestamp()
sort.sort_time()
gschedfilter.GSFilter(
outfile = "gs.txt"
consume = false
)
gschedfilter.BuildGSHFilter(
outfile = "gs.png"
consume = true
)
]
The first filter, GSFilter, outputs a narrative of what group scheduling is doing to a text file. The purpose of this filter is to provide human readable output about group scheduling that can assist in debugging group scheduling hierarchies. The second filter, BuildGSHFilter, creates a graph of the group scheduling hierarchy and outputs the graph to a png image file.
