Testing XALT to produce \*.json output
--------------------------------------

The generation of \*.json files part of XALT can be tested in a user
account without any making any changes to the system level.  These
techniques can be use when trying to debug how XALT is working (or not
working).  Setting the environment variable **XALT_TRACING** will give
verbose output.


**DO NOT USE THIS INSTALLED VERSION OF XALT** for your system. The
**Config/rtm_config.py** file is designed for testing but not
production. It allows too many executables to be tracked.

Step 1: Configure and build XALT
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Lets assume that **~/xalt** is your XALT source directory and you want to
install it under the **~/pkg/**.  Then you can do the following::

   $ cd ~/xalt;
   $ unset XALT_EXECUTABLE_TRACKING
   $ ./configure --prefix=$HOME/pkg --with-syshostConfig=hardcode:local --with-transmission=file --with-MySQL=no --with-config=Config/rtm_config.py
   $ make install
   $ export PATH=$HOME/pkg/xalt/xalt/bin:$PATH
   $ export COMPILER_PATH=$HOME/pkg/xalt/xalt/bin
   

Step 2: Set XALT_EXECUTABLE_TRACKING to yes.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

XALT will not track anything unless this environment variable is set to
**yes**::

   $ export XALT_EXECUTABLE_TRACKING=yes
   
Step 3: $HOME and $USER must have values
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

XALT will not produce anything \*.json without $HOME and $USER having
a value. This might be true when run under a container.

Step 4: Test xalt_configuration_report
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You should run **xalt_configuration_report** to see how xalt is
currently configured.  Make sure that the transmission style is
**file** and that you are happy with the path patterns::

   $ xalt_configuration_report

Step 5: Test linking a program
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Create a **try.c** that has::

   #include <stdio.h>
   int main()
   { printf("Hello World!\n"); return 0;}

Then compile and link this with::

   $ XALT_TRACING=link gcc -o try try.c

You should see something like the following::

   16:37:15 XALT Tracing Activated for version: 2.6.11

   16:37:15 Starting ld

   16:37:15 find_real_command: Searching for the real: ld

   16:37:15 find_real_command: found /opt/apps/gcc/6.3.0/bin/ld

   16:37:15 COMP_T: {"compiler":"gcc","compilerPath":"/opt/apps/gcc/6.3.0/bin/gcc","link_line":["gcc","-o","try","try.c"]}

   16:37:15 Using ld to find functions:  /opt/apps/gcc/6.3.0/bin/ld -plugin /opt/apps/gcc/6.3.0/libexec/gcc/x86_64-pc-linux-gnu/6.3.0/liblto_plugin.so -plugin-opt=/opt/apps/gcc/6.3.0/libexec/gcc/x86_64-pc-linux-gnu/6.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccF6X3sh.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o try /lib/../lib64/crt1.o /lib/../lib64/crti.o /opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/crtbegin.o -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0 -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/intel/compilers_and_libraries_2018.2.199/linux/daal/../tbb/lib/intel64_lin/gcc4.4 -L/opt/intel/compilers_and_libraries_2018.2.199/linux/daal/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2018.2.199/linux/tbb/lib/intel64/gcc4.7 -L/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2018.2.199/linux/ipp/lib/intel64 -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../../../x86_64-pc-linux-gnu/lib -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../.. /tmp/ccMlm4OD.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/crtend.o /lib/../lib64/crtn.o --unresolved-symbols=report-all -o /dev/null

   16:37:15 Not adding XALT initialize routines to user code

   16:37:15 /opt/apps/gcc/6.3.0/bin/ld -plugin /opt/apps/gcc/6.3.0/libexec/gcc/x86_64-pc-linux-gnu/6.3.0/liblto_plugin.so -plugin-opt=/opt/apps/gcc/6.3.0/libexec/gcc/x86_64-pc-linux-gnu/6.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccF6X3sh.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o try /lib/../lib64/crt1.o /lib/../lib64/crti.o /opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/crtbegin.o -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0 -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/intel/compilers_and_libraries_2018.2.199/linux/daal/../tbb/lib/intel64_lin/gcc4.4 -L/opt/intel/compilers_and_libraries_2018.2.199/linux/daal/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2018.2.199/linux/tbb/lib/intel64/gcc4.7 -L/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin -L/opt/intel/compilers_and_libraries_2018.2.199/linux/ipp/lib/intel64 -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../../../x86_64-pc-linux-gnu/lib -L/opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../.. /tmp/ccMlm4OD.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /opt/apps/gcc/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0/crtend.o /lib/../lib64/crtn.o /tmp/mclay_08c8ba01-689e-456e-b6e9-e618c594c327_u0yMaa/xalt.o    -t > /tmp/mclay_08c8ba01-689e-456e-b6e9-e618c594c327_u0yMaa/link.txt  2>&1

   16:37:15 XALT_TRANSMISSION_STYLE file
     Wrote json link file : /home/user/.xalt.d/link.stampede2.2019_05_08_16_37_15_3155.08c8ba01-689e-456e-b6e9-e618c594c327.json


If you want to see this output written to a file instead try this::

   $ XALT_TRACING=link:$HOME/linker.log gcc -o try try.c

and the tracing output will be written to $HOME/linker.log instead.


You should also try running the link without XALT tracing::

   $ gcc -o try try.c

And check that there is a new \*.json file in ~/.xalt.d


Step 6: run a program with XALT tracing turned on 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Normally XALT will produce no output but for testing you can run a program under XALT by doing the following::

    $ export LD_PRELOAD=$HOME/pkg/xalt/xalt/lib64/libxalt_init.so
    $ XALT_TRACING=yes XALT_SAMPLING=no ./try

You should see something that looks like::

    myinit(LD_PRELOAD,/home/user/t/xalt/try){
      Test for __XALT_INITIAL_STATE__: "LD_PRELOAD", STATE: "LD_PRELOAD"
      Test for XALT_EXECUTABLE_TRACKING: yes
      Test for rank == 0, rank: 0
        -> XALT is build to track all programs, Current program is a scalar program -> Not producing a start record
    }

    Hello World!

    myfini(LD_PRELOAD){
      Recording State at end of scalar user program:
        LD_LIBRARY_PATH= PATH=/usr/bin:/bin /home/user/l/pkg/xalt/xalt//libexec/xalt_run_submission --interfaceV 4 --pid 10 --ppid 1--syshost "..." --start "..." --end "..." --exec "..." --ntasks 1 --kind "scalar" --uuid "..." --prob 1 --ngpus 0 --watermark "..." --path "..." --ld_libpath "..." -- ["./try"]
    }


    xalt_run_submission(zzz) {
      Built envT
      Extracted recordT from executable
      Built userT, userDT, scheduler: SLURM
      Filter envT
      Parsed ProcMaps
      Using XALT_TRANSMISSION_STYLE: file
      cmdlineA: ["./try"]
      Built json string
      Wrote json run file : /home/user/.xalt.d/run.stampede2.2019_05_08_16_39_12_3657.zzz.cbaec70f-d9e8-41e1-a76b-4db0729ddb06.json
    }   

Also please try running the program again to see if there is a new
\*.json file generated in the ~/.xalt.d directory::

    $ ./try