From Kerrighed

This page describes how to install Kerrighed 2.4.0 on your system.

Contents 1 Presentation 2 Getting Kerrighed 3 Pre-requisites 4 Quick Compilation and Installation 4.1 Installing Kerrighed 4.1.1 Installing Kerrighed from the tarball 4.2 Kerrighed configuration 5 Advanced configuration 5.1 Kerrighed configuration 5.1.1 Generic configure options 5.1.2 Kerrighed specific options 5.2 Build from outside of source tree 5.3 Kernel options (hints) 6 Using Kerrighed

[edit] Presentation

Kerrighed offers the view of a big SMP machine on top of a Linux cluster. Each node of the cluster must share the same filesystem. This can be done :

• either by sharing the root filesystem through the network (e.g.: NFSROOT)
• either by copying read-only files on each nodes then sharing through the network read-write files.

This manual covers the compilation and installation of Kerrighed from sources on a single node. Duplication and/or sharing of filesystem is really dependant from a distribution to another and is not covered by this manual.

Once installed, the Kerrighed OS is made of the following parts:

• a patched Linux kernel,
• a Linux module,
• some user-level tools,
• scripts to start/stop Kerrighed service,
• a library, libkerrighed.

[edit] Getting Kerrighed

• Patch, module, tools, services and library sources are available in a tarball on INRIA's GForge.
• Linux 2.6.20 vanillas sources are also needed.

[edit] Pre-requisites

You are supposed to know:

• how to compile/configure/install a Linux kernel from sources,
• a little bit (a little bit more is even better) about your distribution boot process.

The following software is required Kerrighed to compile:

• automake: >=1.10
• autoconf: >=2.59
• libtool
• rsync
• bzip2
• gcc: 3.3.x is the recommended compiler for Linux Kernel.

Kerrighed is reported to build with 4.1 series.

Optional features requires the following:

• lsb_release: to install Kerrighed startup scripts into /etc/init.d
• xsltproc: a tool to create manpages from docbook

[edit] Quick Compilation and Installation

[edit] Installing Kerrighed

[edit] Installing Kerrighed from the tarball

• Get sources on INRIA's GForge.

wget -O /usr/src/kerrighed-latest.tar.gz http://kerrighed.gforge.inria.fr/kerrighed-latest.tar.gz

• Optionally, download linux-2.6.20 sources tarball into '/usr/src', if not, `configure` script will do it for you each and every time.

wget -O /usr/src/linux-2.6.20.tar.bz2 http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.20.tar.bz2

• Decompress the Tarballs

cd /usr/src
tar zxf kerrighed-2.4.0.tar.gz
tar jxf linux-2.6.20.tar.bz2

• Configure the Sources

cd kerrighed-2.4.0
./configure

Once run, patched kernel sources are in `_kernel` dir and kernel build dir is `kernel`.

• Configure the kernel

./configure setup the kernel with a default configuration which may not suit your needs but gives you a running Kerrighed configuration.

`configure`'s --with-kernel-* options can be given to configure kernel sources, either with a given `.config` file either with one of kernel's *config targets.

Type `./configure --help` for possible options.

For some hints on kernel options, have a look at Kernel Options section below.

• Build the sources

make

• Install all, as user root

make install

• Check the installation. You should have now the following dir/files installed:

/boot/vmlinuz-2.6.20-krg	Kerrighed kernel
/boot/System.map	Kerrighed kernel symbol table
/lib/modules/2.6.20-krg	Kerrighed modules
/etc/init.d/kerrighed	Kerrighed service script *
/etc/default/kerrighed	Service configuration
/usr/local/share/man	Manpages
/usr/local/bin/krgadm	Cluster administration tool
/usr/local/bin/krgcapset	Process capabilities tool
/usr/local/bin/krgcr-run	Process checkpoint/restart helper
/usr/local/bin/migrate	Process migration tool
/usr/local/lib/libkerrighed-*	Kerrighed library
/usr/local/include/kerrighed	Kerrighed library headers

* This script prepare the node to integrate a cluster, but does not start the cluster. See krgadm (1)

[edit] Kerrighed configuration

Kerrighed needs one parameters, the session id. This id is between 1 and 254 and can be set through:

• boot parameter: session_id=XX

Each Kerrighed node is identified by its node id. This id is between 0 and 254 and can be set through:

• auto configuration: this is the default behavior if you are using dhcp and autonode-id (CONFIG_KRG_AUTONODEID=y in the kernel with network card driver built-in)
• boot parameter: node_id=XX
• /etc/kerrighed_nodes. This file must be in the initrd file if you use one. See kerrighed_nodes(5)

[edit] Advanced configuration

[edit] Kerrighed configuration

The configure script provides a lot of options to configure the build and installation process.

[edit] Generic configure options

By default, make install will install the package's files in /usr/local/bin, /usr/local/man, etc. You can specify an installation prefix other than /usr/local by giving configure the option --prefix=PATH.

configure recognizes the following options to control how it operates.

--cache-file=FILE
    Use and save the results of the tests in FILE instead of
    ./config.cache.  Set FILE to /dev/null to disable caching, for
    debugging configure.

--help
    Print a summary of the options to configure, and exit.

--quiet
--silent
-q
    Do not print messages saying which checks are being made.

--srcdir=DIR
    Look for the package's source code in directory DIR.  Usually
    configure can determine that directory automatically.

--version
    Print the version of Autoconf used to generate the configure
    script, and exit.

[edit] Kerrighed specific options

Following options can be passed to the toplevel configure script:

 --disable-kernel
     Disable automatic kernel configuration/build/install [default=enable]

 --disable-libkerrighed
     Disable libkerrighed [default=enable]

  --disable-tools
      Disable tools [default=enable]

  --disable-service
      Do not install Kerrighed service [default=enable if lsb found]

  --enable-tests
      Globally enable tests [default=disable]

  --disable-tests-ktp
      Disable 'ktp' tests
  
  --disable-tests-apps
      Disable 'apps' tests
  
  --disable-tests-proc
      Disable 'proc' tests

  --disable-tests-benchmark
      Disable 'benchmark' tests

  --with-kernel-mirror
     kernel.org mirror used to get vanilla kernel [default=ftp.eu.kernel.org]

  --with-kernel-config
      config|menuconfig|xconfig|gconfig|defconfig|allmodconfig|allyesconfig|allnoconfig
      If you use automatic kernel config/build/install, use one of these target to configure the kernel. [default=defconfig]

  --with-ltp-base
      Path to ltp base dir [default=LIBDIR/debian-test/tests/linux]

[edit] Build from outside of source tree

Compilation and installation can be done from outside of source dir. Just create a new directory where you want to build and cd into it. Call configure script from this dir. All make targets can be used as usual.

[edit] Kernel options (hints)

If you want to edit kernel options, note that the following are currently broken with Kerrighed:

• General setup -> IPC Namespaces (CONFIG_IPC_NS=n)
• Processor type and features -> Preemption Model -> Voluntary Kernel Preemption
• Processor type and features -> Preemption Model -> Preemptible Kernel (CONFIG_PREEMPT_NONE=y, CONFIG_PREEMPT_VOLUNTARY=y, CONFIG_PREEMPT=n)
• Processor type and features -> Non Uniform Memory Access (NUMA) Support (CONFIG_NUMA=n)
• Security options -> Enable access key retention support (CONFIG_KEYS=n)

Note: don't forget to add your NIC driver into the kernel, not as a module.

[edit] Using Kerrighed

Now that Kerrighed is installed and configured on your system, have a look at the user manual.