StarPU Handbook

MIC Xeon Phi Support

# Compilation

MIC Xeon Phi support actually needs two compilations of StarPU, one for the host and one for the device. The `PATH` environment variable has to include the path to the cross-compilation toolchain, for instance `/usr/linux-k1om-4.7/bin` . The `SINK_PKG_CONFIG_PATH` environment variable should include the path to the cross-compiled `hwloc.pc`. The script `mic-configure` can then be used to achieve the two compilations: it basically calls `configure` as appropriate from two new directories: `build_mic` and `build_host`. `make` and `make install` can then be used as usual and will recurse into both directories. If different `configure` options are needed for the host and for the mic, one can use `–with-host-param=–with-fxt` for instance to specify the `–with-fxt` option for the host only, or `–with-mic-param=–with-fxt` for the mic only.

One can also run StarPU just natively on the Xeon Phi, i.e. it will only run directly on the Phi without any exchange with the host CPU. The binaries in `build_mic` can be run that way.

For MPI support, you will probably have to specify different MPI compiler path or option for the host and the device builds, for instance:

    ./mic-configure --with-mic-param=--with-mpicc="/.../mpiicc -mmic" \
        --with-host-param=--with-mpicc=/.../mpiicc

In case you have troubles with the `coi` or `scif` libraries (the Intel paths are really not standard, it seems...), you can still make a build in native mode only, by using `mic-configure –enable-native-mic` (and notably without `–enable-mic` since in that case we don't need `mic` offloading support).

# Porting Applications To MIC Xeon Phi

The simplest way to port an application to MIC Xeon Phi is to set the field starpu_codelet::cpu_funcs_name, to provide StarPU with the function name of the CPU implementation, so for instance:

    struct starpu_codelet cl =
    {
        .cpu_funcs = {myfunc},
        .cpu_funcs_name = {"myfunc"},
        .nbuffers = 1,
    }

StarPU will thus simply use the existing CPU implementation (cross-rebuilt in the MIC Xeon Phi case). The functions have to be globally-visible (i.e. not `static`) for StarPU to be able to look them up, and `-rdynamic` must be passed to `gcc` (or `-export-dynamic` to `ld`) so that symbols of the main program are visible.

If you have used the starpu_codelet::where field, you additionally need to add in it STARPU_MIC for the Xeon Phi.

For non-native MIC Xeon Phi execution, the 'main' function of the application, on the sink, should call starpu_init() immediately upon start-up; the starpu_init() function never returns. On the host, the 'main' function may freely perform application related initialization calls as usual, before calling starpu_init().

For MIC Xeon Phi, the application may programmatically detect whether executing on the sink or on the host, by checking whether the STARPU_SINK environment variable is defined (on the sink) or not (on the host).

# Launching Programs

MIC programs are started from the host. StarPU automatically starts the same program on MIC devices. It however needs to get the MIC-cross-built binary. It will look for the file given by the environment variable STARPU_MIC_SINK_PROGRAM_NAME or in the directory given by the environment variable STARPU_MIC_SINK_PROGRAM_PATH, or in the field starpu_conf::mic_sink_program_path. It will also look in the current directory for the same binary name plus the suffix `-mic` or `_mic`.

The testsuite can be started by simply running `make check` from the top directory. It will recurse into both `build_host` to run tests with only the host, and into `build_mic` to run tests with both the host and the MIC devices. Single tests with the host and the MIC can be run by starting `./loader-cross.sh ./the_test` from `build_mic/tests`.