General Net Console Instructions

Net console is a device driver in the Linux kernel tree. The driver dumps printk’s onto the network where they can be captured and displayed similar to the serial console. Net console only dumps printks as opposed to the serial console which in general is more powerful and can even provide a login. We are normally only interested in printks, and net console provides more flexibility than the serial console because we do not need computer-computer serial cable connections. With net console a single desktop can capture debug information from multiple computers with only an ethernet connection.

We denote a computer which we are capturing output from as a SOURCE and a computer which we are collecting data on as a DEST.

  1. Configure the SOURCE Kernel by enabling the netconsole driver under device drivers. Build the driver into the kernel so we can configure it via kernel parameters and can have printks from early bootup.

  2. Gather SOURCE Info

    • Ethernet device name (e.g. eth0), SOURCE-DEV

      o use /sbin/ifconfig

    • IP address: SOURCE-IP

      o use /sbin/ifconfig o look for inet addr

    • A local port to send traffic through (e.g. 4444), SOURCE-PORT

      o use a non-privileged port(>1024) o net console uses 6666 by default

  3. Gather DEST Info

    • MAC address of receiving ethernet device, DEST-MAC

      o use /sbin/ifconfig o look for HWaddr

    • IP address where SOURCE sends packets: DEST-IP

      o use /sbin/ifconfig o look for inet addr

    • A local port to receive traffic on (e.g. 5555), DEST-PORT

      o This is used by the SOURCE computer when sending out data. A numbering scheme is helpful when capturing data from multiple sources. Each source should be able to send to a different port if they are sending to the same destination computer. DEST-PORT needs to be unique among all sources sending to the same destination. SOURCE-PORT however does not need to be unique. It only needs to be a free port on a SOURCE machine.

  4. SOURCE Configuration

This is the syntax and an example of the kernel parameters. Set this in the grub or lilo configuration file. The grub configuration file is located at: /boot/grub/menu.lst:

linux netconsole=SOURCE-PORT@SOURCE-IP/SOURCE-DEV,DEST-PORT@DEST-IP/DEST-MAC
linux netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc
  1. Capturing SOURCE Traffic on DEST

Use netcat (nc) to caputure data. Syslogd can also be used. Other methods may be available as well. A typical setup involves starting a netcat process to receive an output stream for each computer being monitored. The netcat paramaters are sometimes different between distributions. Using “&” to start netcat in the background has also caused the next instantiation of netcat to terminate the previous. Trial and error is needed here. The basic strategy is to start netcat processes dumping what they receive into named files. These files can then be streamed to a terminal or just checking periodically. For example for two computers testbed49 and testbed50 and whose data is destined for port 4449 and 4450 on the DEST computer i would run the following on the DEST computer and execute the two tail commands in separate terminals.:

$ nc -ul 4449 > path/to/testbed49.log
$ nc -ul 4450 > path/to/testbed50.log
$ tail -f path/to/testbed49.log
$ tail -f path/to/testbed50.log

Net Console on KUSP Testbeds

Sometimes, to properly set up net console on the KUSP testbeds, it is necessary to modify the network configuration. If the general steps above do not work properly then it is likely that you need to modify your network configuration.

The KUSP testbeds have two interfaces. One interface is the public interface and the other one is a private interface. Some KUSP testbeds have three network interfaces, a public interface, a private interface, and an unused interface.

It is best to use the private interface for net console. The public interface is used to mount the NFS filesystem and for SSH. So, it is best to leave it alone. It won’t affect much if something goes wrong with the private interface.

On the SOURCE machine, use ifconfig to find the private interface. Note the HWaddr field.

On the SOURCE machine, you need to change your /etc/udev/rules.d/70-persistent-net.rules file to match the names given by the kernel at boot up to the interfaces.

You can see what the kernel is naming your interfaces by looking in ‘/var/log/messages’. Look for something like:

Aug 23 10:54:44 kusp03 kernel: [    1.001945] e1000 0000:01:04.0: found PCI INT A -> IRQ 5
Aug 23 10:54:44 kusp03 kernel: [    1.261448] e1000: 0000:01:04.0:e1000_probe: (PCI:33MHz:32-bit) 00:0e:0c:b1:ac:0d
Aug 23 10:54:44 kusp03 kernel: [    1.409792] e1000: eth0: e1000_probe:Intel(R) PRO/1000 Network Connection
Aug 23 10:54:44 kusp03 kernel: [    1.410123] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
Aug 23 10:54:44 kusp03 kernel: [    1.410219] e1000e: Copyright (c) 1999-2008 Intel Corporation.
Aug 23 10:54:44 kusp03 kernel: [    1.410494] Intel(R) Gigabit Ethernet Network Driver - version 1.2.45-k2
Aug 23 10:54:44 kusp03 kernel: [    1.410589] Copyright (c) 2008 Intel Corporation.
Aug 23 10:54:44 kusp03 kernel: [    1.411393] e100: Intel(R) PRO/100 Network Driver, 3.5.23-k6-NAPI
Aug 23 10:54:44 kusp03 kernel: [    1.411491] e100: Copyright(c) 1999-2006 Intel Corporation
Aug 23 10:54:44 kusp03 kernel: [    1.411637] e100 0000:00:04.0: found PCI INT A -> IRQ 10
Aug 23 10:54:44 kusp03 kernel: [    1.434511] e100 0000:00:04.0: PME# disabled
Aug 23 10:54:44 kusp03 kernel: [    1.435463] e100: eth1: e100_probe: addr 0xfe9fe000, irq 10, MAC addr 00:e0:81:01:5a:d1
Aug 23 10:54:44 kusp03 kernel: [    1.435661] e100 0000:00:05.0: found PCI INT A -> IRQ 9
Aug 23 10:54:44 kusp03 kernel: [    1.458472] e100 0000:00:05.0: PME# disabled
Aug 23 10:54:44 kusp03 kernel: [    1.459428] e100: eth2: e100_probe: addr 0xfe9fd000, irq 9, MAC addr 00:e0:81:01:5a:d2

Match the HWAddr of the private interface you found in ifconfig to the MAC addr displayed here. You can tell which rule(in /etc/udev/rules.d/70-persistent-net.rules) is naming which interface by using the ‘HWAddr’.

Table Of Contents

Previous topic

ITTC Machines Setup on 245E Rack machines

Next topic

Subsystems Comprising KUSP

This Page