Mistral DKRZ¶

Table of contents
Mistral DKRZ

This page provides information for Mistral at DKRZ where ICON-HAMMOZ has been ported.

Contact persons (for icon-hammoz-related questions only)¶

Anne Kubin (input files, general help for ICON-HAMMOZ)

Sylvaine Ferrachat (configuration, general help for ICON-HAMMOZ)

DKRZ help desk (any general technical issue)

Full platform documentation¶

https://www.dkrz.de/pdfs/docs/docu-mistral

Environment settings¶

Nothing specific: icon configuration integrates all proper environment settings. It will purge your currently loaded modules and will reload the necessary ones.

How to download, configure, build, run icon-hammoz on Mistral¶

#-- Preliminary step: create an experiment directory on /scratch
#   Since the model output can become very large, while the setup expects a folder "experiments" in the base directory, it is recommended to redirect it to /scratch.

mkdir /scratch/[group]/[your account]/experiments

#-- Get a copy of the icon-hammoz code

git clone git@git.iac.ethz.ch:hammoz/icon-hammoz-distrib.git [optional: name of the directory to create to store this local repo.]
cd [icon-hammoz-distrib] # or to the custom directory you asked to create
./scripts/deploy.sh  # this loads the actual code of HAMMOZ and ICON which are embedded as submodules in icon-hammoz-distrib

#-- Configure

cd icon
./configure --enable-jsbach --with-hammoz --with-fortran=intel18

#-- Compile
#   !! Important! Do not use a simple make command here but 

./build_command

#-- Prepare runscript for running icon with HAM
#   Currently three examples runscripts for ACCMIP-style experiments are provided in the directory run/: 
#   accmip: exp.atm_amip_hammoz; 
#   accmip with GFAS fires: exp.atm_amip_hammoz_GFAS; 
#   pre-industrial: exp.atm_amip_hammoz_PIA 

export use_account_no=[your project account]
./make_runscripts atm_amip_hammoz

#-- Link experiment directory
ln -sf /scratch/[group]/[your account]/experiments .

#-- Adjust your runscript

cd run
# --> then edit run/exp.atm_amip_hammoz.run, in particular, watch the #SBATCH header for the number of nodes, etc.,
#     Check the input data pathes:
#     On mistral there is a central directory where input files are maintained
#     /work/bb1191/ICON-HAMMOZ
#     Set the icon_data_poolFolder="/work/bb1191/ICON-HAMMOZ/v0002/icon_input" 
#     Set indata_hammoz_root="/work/bb1191/ICON-HAMMOZ"   # the v0002 or other version is specified later in the script

#-- Launch your run

sbatch exp.atm_amip_hammoz_marc.run

#-- Inspect your job log
#-- > vim or tail -f LOG.exp.atm_amip_hammoz.run.*.o

#-- Inspect your output

cd ${SCRATCH}/atm_amip_hammoz

#-- Warning: A crashed run produces numerous and large core files which have the potential to occupy all your granted disk space 
    on mistral and, thus, could prevent you from submitting new jobs. Make sure to look for core* files in your output directory
    and delete them. If you want to prevent core files from being written at all, see the related DKRZ-page: 
    https://www.dkrz.de/up/de-help/de-faq_folder/de-mistral/how-can-i-avoid-core-files-if-my-program-crashes

Alternate customizations¶

if you want to have a custom experiment name (as opposed to producing a sole 'atm_amip_hammoz' experiment output), there's several ways, but the easiest and cleanest is to simply copy the experiment template run/exp.atm_amip_hammoz e.g. run/exp.my_icon_ham_exp. Then, you create the final runscript with:

cd [root of your icon-hammoz distrib, e.g. '..' if you were in run/]
./make_runscripts my_icon_ham_exp

--> It is also possible to specify directly the number of nodes or the time limit to use with:

sbatch --nodes=[some number as you wish] --time=[HH:MM:SS as you wish] exp.[your exp].run

Important recommendation: If you start doing some modifications in files that are meant to be versioned, it is strongly recommended that you create your own branch to do whatever you want and version that.
```
git checkout -b [your new branch name, e.g. hammoz/myname_someintention]
# hack, hack, hack...
git status # to see your modified files
git add [the files you want to commit]
git commit
```

Files (0)