# useful to autoreload the module without restarting the kernel
%load_ext autoreload
%autoreload 2

from mppi import InputFiles as I, Calculators as C, Utilities as U

omp = 1
mpi = 4

Tutorial for the YamboCalculator class

This tutorial describes the usage of the YamboCalculator class, that manages the run of (many) calculations in parallel with the Yambo package.

Perform an Hartree-Fock computation for Silicon

Follow the tutorial for the QeCalculator to produce the .save folder needed to generate the yambo SAVE.

The first step needed to perform a Yambo computation is to generate the SAVE folder from a QuantumESPRESSO computation.

The mppi.Utilities module has a function that perform this task

source_dir = 'QeCalculator_test/bands_12.save'
run_dir = 'YamboCalculator_test'

U.build_SAVE(source_dir,run_dir)

SAVE folder already present in YamboCalculator_test

Now the YamboInput class can create the yambo input object

from copy import deepcopy
inp = I.YamboInput(args='yambo -x -V rl',folder=run_dir)
inp.set_kRange(1,2) #restrict the analysis to the first two kpoints

inputs = []
names = []
jobnames = []

for e in [3.0,4.0]:
    inp['variables']['EXXRLvcs'] = [e*1e3,'mHa']
    inputs.append(deepcopy(inp))
    names.append('hf_exx'+str(e))
    jobnames.append('hf_job_exx'+str(e))

To run the computation we create an istance of YamboCalculator. This object behaves almost exactly as the QeCalculator for what concerns the user interface

C.YamboCalculator?

Init signature: C.YamboCalculator(omp=1, mpi=2, mpi_run='mpirun -np', executable='yambo', scheduler='direct', multiTask=True, skip=True, verbose=True, IO_time=5, clean_restart=True, **kwargs)
Docstring:
Manage (multiple) Yambo calculations performed in parallel. Computations
are managed by a scheduler that, in the actual implementation of the class, can
be `direct` or `slurm`.

Parameters:
   omp (:py:class:`int`) : value of the OMP_NUM_THREADS variable
   mpi (:py:class:`int`) : number of mpi processes
   mpi_run (:py:class:`string`) : command for the execution of mpirun, e.g. 'mpirun -np' or 'mpiexec -np'
   executable (:py:class:`string`) : set the executable (yambo, ypp, yambo_rt, ...) of the Yambo package
   scheduler (:py:class:`string`) : choose the scheduler used to submit the job, actually the choices implemented are
        'direct' that runs the computation using the python multiprocessing package and 'slurm' that creates a slurm script
   multiTask  (:py:class:`bool`) : if true a single run_script is built and all the computations are performed in parallel,
        otherwise an independent script is built for each elements of inputs and the computations are performed sequentially
   skip (:py:class:`bool`) : if True evaluate if one (or many) computations can be skipped.
       This is done by checking that the folder where yambo write the results contains at least
       one file 'o-*', for each name in names
   verbose (:py:class:`bool`) : set the amount of information provided on terminal
   IO_time (int) : time step (in second) used by the wait method to check that the job is completed
   kwargs : other parameters that are stored in the _global_options dictionary
   clean_restart (:py:class:`bool`) : if True the delete the folder(s) with the output files and database before
        running the computation

Example:
 >>> code = YamboCalculator(omp=1,mpi=4,mpi_run='mpirun -np',executable='yambo',skip=True,verbose=True,scheduler='direct')
 >>> code.run(inputs = ..., run_dir = ...,names = ...,jobnames = ...)

 where the arguments of the run method are:

Args:
    run_dir (:py:class:`string`) : the folder in which the simulation is performed
    inputs (:py:class:`list`) : list with the instances of the :class:`YamboInput` class
        that define the input objects
    names (:py:class:`list`) : list with the names associated to the input files (without extension),
        given in the same order of the inputs list. These strings are used also as the radicals of the
        folders in which results are written as well as a part of the name of the output files.
    jobnames (:py:class:`list`) : list with the values of the jobname. If this variable is not specified
        the value of name is attributed to jobname by process_run.
    kwargs : other parameters that are stored in the run_options dictionary

When the run method is called the class runs the command:
            executable_name -F name.in -J jobname -C name
File:           ~/Applications/MPPI/mppi/Calculators/YamboCalculator.py
Type:           type

code = C.YamboCalculator(omp=omp,mpi=mpi)
code.global_options()

Initialize a parallel Yambo calculator with scheduler direct

{'omp': 1,
 'mpi': 4,
 'mpi_run': 'mpirun -np',
 'executable': 'yambo',
 'scheduler': 'direct',
 'multiTask': True,
 'skip': True,
 'verbose': True,
 'IO_time': 5,
 'clean_restart': True}

The structure of the folder in which yambo writes its results is governed by the names and jobnames variables. It is possible to provide only the names variable.

The effect of this choice can be seen in the command string executed by the calculator

result = code.run(run_dir = run_dir, inputs = inputs, names= names)
result

run 0 command: cd YamboCalculator_test; mpirun -np 4 yambo -F hf_exx3.0.in -J hf_exx3.0 -C hf_exx3.0
run 1 command: cd YamboCalculator_test; mpirun -np 4 yambo -F hf_exx4.0.in -J hf_exx4.0 -C hf_exx4.0
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: False
Job completed

{'output': [['YamboCalculator_test/hf_exx3.0/o-hf_exx3.0.hf'],
  ['YamboCalculator_test/hf_exx4.0/o-hf_exx4.0.hf']],
 'dbs': ['YamboCalculator_test/hf_exx3.0', 'YamboCalculator_test/hf_exx4.0']}

In this case yambo create the hf_exx3.0 and hf_exx4.0 folders and write all the files in these folders. Result is a dictionary with the name of the o- file and the folders that contains the databases.

Instead, if we set also set a jobname we have

result = code.run( run_dir=run_dir, inputs=inputs ,names=names,jobnames=jobnames,skip=False)
result

run 0 command: cd YamboCalculator_test; mpirun -np 4 yambo -F hf_exx3.0.in -J hf_job_exx3.0 -C hf_exx3.0
run 1 command: cd YamboCalculator_test; mpirun -np 4 yambo -F hf_exx4.0.in -J hf_job_exx4.0 -C hf_exx4.0
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: True
run0_is_running: True run1_is_running: False
Job completed

{'output': [['YamboCalculator_test/hf_exx3.0/o-hf_job_exx3.0.hf'],
  ['YamboCalculator_test/hf_exx4.0/o-hf_job_exx4.0.hf']],
 'dbs': ['YamboCalculator_test/hf_job_exx3.0',
  'YamboCalculator_test/hf_job_exx4.0']}

In this case yambo creates also the folders hf_job_exx and write the databases in these folders.

Usage of the skip parameter

If we repeat a calculation that has been already performed and skip = True the class skip its computation, for instance

result = code.run(run_dir = run_dir, inputs = inputs, names= names, skip = True)
result

Skip the computation for input hf_exx3.0
Skip the computation for input hf_exx4.0
Job completed

{'output': [['YamboCalculator_test/hf_exx3.0/o-hf_exx3.0.hf'],
  ['YamboCalculator_test/hf_exx4.0/o-hf_exx4.0.hf']],
 'dbs': ['YamboCalculator_test/hf_exx3.0', 'YamboCalculator_test/hf_exx4.0']}

Usage of the multiTask feature

By default the calculator runs in parallel all the computations. However if the multiTask = False option is used the the computations are performed in sequence.

result = code.run(run_dir = run_dir, inputs = inputs, names= names, multiTask = False, skip = False)
result

run 0 command: cd YamboCalculator_test; mpirun -np 4 yambo -F hf_exx3.0.in -J hf_exx3.0 -C hf_exx3.0
run 1 command: cd YamboCalculator_test; mpirun -np 4 yambo -F hf_exx4.0.in -J hf_exx4.0 -C hf_exx4.0
run0_is_running: True
run0_is_running: True
run0_is_running: True
Job completed
run1_is_running: True
run1_is_running: True
run1_is_running: True
run1_is_running: True
Job completed

{'output': [['YamboCalculator_test/hf_exx3.0/o-hf_exx3.0.hf'],
  ['YamboCalculator_test/hf_exx4.0/o-hf_exx4.0.hf']],
 'dbs': ['YamboCalculator_test/hf_exx3.0', 'YamboCalculator_test/hf_exx4.0']}

Test of the slurm scheduler

If the slurm scheduler is chosen the calculator prepare the slurm script and submit it. The effects of skip and multiTask parameters can be tested

results = code.run(run_dir=run_dir,inputs=inputs,names=names, scheduler = 'slurm', skip = False, multiTask = True)
results

The slurm script is written in the run_dir. The execution of the run requires that the slurm scheduler is installed.

Perform a GW computation for Silicon

We make usage of the YamboCalculator to perform a different yambo computation. In this way we control how this class manage the output files and the ndb database in various cases.

inp = I.YamboInput(args='yambo -d -k hartee -g n -p p -V qp',folder=run_dir)
inp.set_kRange(1,2)
#inp

result = code.run(inputs = [inp], run_dir = run_dir,names=['qp_test1'],skip=False)
result

delete folder: YamboCalculator_test/qp_test1
run 0 command: cd YamboCalculator_test; mpirun -np 4 yambo -F qp_test1.in -J qp_test1 -C qp_test1
run0_is_running: True
run0_is_running: True
run0_is_running: True
run0_is_running: True
run0_is_running: True
Job completed

{'output': [[]], 'dbs': ['YamboCalculator_test/qp_test1']}

Perform the same computation but specify also a jobname

result = code.run(inputs = [inp], run_dir = run_dir,names=['qp_test2'],jobnames = ['qp_job_test2'])
result

Executing command: cd YamboCalculator_test; mpirun -np 4 yambo -F qp_test2.in -J qp_job_test2 -C qp_test2
run0_is_running:True
run0_is_running:True
Job completed

{'output': [['YamboCalculator_test/qp_test2/o-qp_job_test2.hf']],
 'dbs': ['YamboCalculator_test/qp_job_test2']}

Test of the ExtendOut option in the input file

The ExtendOut option enables the writing of all the variables in the output files of Yambo. This feature has no effect for an HF computation and we test for a QP one:

inp = I.YamboInput(args='yambo -d -k hartee -g n -p p -V qp',folder=run_dir)
inp.set_kRange(1,2)
inp.set_extendOut()
#inp

result = code.run(inputs = [inp], run_dir = run_dir,names=['qp_test_ExtendOut'])
result

Executing command: cd YamboCalculator_test; mpirun -np 4 yambo -F qp_test_ExtendOut.in -J qp_test_ExtendOut -C qp_test_ExtendOut
run0_is_running:True
run0_is_running:True
run0_is_running:True
run0_is_running:True
run0_is_running:True
Job completed

{'output': [['YamboCalculator_test/qp_test_ExtendOut/o-qp_test_ExtendOut.qp']],
 'dbs': ['YamboCalculator_test/qp_test_ExtendOut']}

Perform a ypp computation

The YamboCalculator class can manage also ypp computation.

Let’s see an example by performing a band calculation along a path

inp = I.YamboInput(args='ypp -s b',folder=run_dir,filename='ypp.in')
inp

{'args': 'ypp -s b',
 'folder': 'YamboCalculator_test',
 'filename': 'ypp.in',
 'arguments': [],
 'variables': {'OutputAlat': [0.0, ''],
  'INTERP_Shell_Fac': [20.0, ''],
  'BANDS_steps': [10.0, ''],
  'INTERP_mode': 'NN',
  'cooIn': 'rlu',
  'cooOut': 'rlu',
  'CIRCUIT_E_DB_path': 'none',
  'BANDS_bands': [[1, 12], ''],
  'INTERP_Grid': [['-1', '-1', '-1'], '']}}

We define a calculator for ypp. This calculation requires 1 mpirun (see yambo for further information)

code = C.YamboCalculator(mpi_run='mpirun -np 1',executable='ypp',skip=False)
code.global_options()

Initialize a parallel Yambo calculator with scheduler direct

{'omp': 1,
 'executable': 'ypp',
 'multiTask': True,
 'scheduler': 'direct',
 'mpi_run': 'mpirun -np 1',
 'cpus_per_task': 4,
 'ntasks': 3,
 'skip': False,
 'verbose': True,
 'IO_time': 5}

Set the input parameter to perform the band computation along a path

# in alat
G = [0.,0.,0.]
X = [1.,0.,0.]
L = [0.5,0.5,0.5]
K = [1.0,0.5,0.]
path = [L,G,X,K,G]

band_range = [2,5]
bands_step = 30

# scissor
# inp['variables']['GfnQP_E'] = [1.0,1.0,1.0]

# band structure
# Some methods that perform these operation can be added in the YamboInput class
inp['variables']['BANDS_steps'] = [bands_step,'']
inp['variables']['BANDS_bands'] = [band_range,'']
inp['variables']['BANDS_kpts'] = [path,'']
inp['variables']['cooIn'] = 'alat'
inp['variables']['cooOut'] = 'alat'
inp

{'args': 'ypp -s b',
 'folder': 'YamboCalculator_test',
 'filename': 'ypp.in',
 'arguments': [],
 'variables': {'OutputAlat': [0.0, ''],
  'INTERP_Shell_Fac': [20.0, ''],
  'BANDS_steps': [30, ''],
  'INTERP_mode': 'NN',
  'cooIn': 'alat',
  'cooOut': 'alat',
  'CIRCUIT_E_DB_path': 'none',
  'BANDS_bands': [[2, 5], ''],
  'INTERP_Grid': [['-1', '-1', '-1'], ''],
  'BANDS_kpts': [[[0.5, 0.5, 0.5],
    [0.0, 0.0, 0.0],
    [1.0, 0.0, 0.0],
    [1.0, 0.5, 0.0],
    [0.0, 0.0, 0.0]],
   '']}}

result = code.run(run_dir=run_dir,inputs=[inp],names=['bands_test1'])
result

Executing command: cd YamboCalculator_test; mpirun -np 1 ypp -F bands_test1.in -J bands_test1 -C bands_test1
run0_is_running:True
Job completed

{'output': [['YamboCalculator_test/bands_test1/o-bands_test1.bands_interpolated']],
 'dbs': ['YamboCalculator_test/bands_test1']}

Result can be parsed using the YamboParser class of this package or other external tools

Test the jobname specification for this type of computation

result = code.run(run_dir=run_dir,inputs=[inp],names=['bands_test2'],jobnames=['bands_job_test2'])
result

Executing command: cd YamboCalculator_test; mpirun -np 1 ypp -F bands_test2.in -J bands_job_test2 -C bands_test2
run0_is_running:True
Job completed

{'output': [['YamboCalculator_test/bands_test2/o-bands_job_test2.bands_interpolated']],
 'dbs': ['YamboCalculator_test/bands_job_test2']}