After you've completed the configuration steps, you can use several methods to start Distributed ANSYS: We recommend that you use the Mechanical APDL Product Launcher to ensure the correct settings. All methods are explained here.
Notes on Running Distributed ANSYS:
You can use an NFS mount to the ANSYS 18.2 installation; however, we do not recommend NFS-mounting the working directories. Doing so can result in significant declines in performance.
Only the master process reads the config.ans file. Distributed ANSYS ignores the /CONFIG,NOELDB and /CONFIG,FSPLIT commands.
The program limits the number of processes used to be less than or equal to the number of physical cores on the machine. This is done to avoid running the program on virtual cores (for example, by means of hyperthreading), which typically results in poor per-core performance. For optimal performance, consider closing down all other applications before launching Mechanical APDL.
Use the following procedure to start Distributed ANSYS via the launcher.
This procedure is also used for running ANSYS LS-DYNA MPP. See LS-DYNA Parallel Processing Capabilities in the ANSYS LS-DYNA User's Guide for a detailed explanation on using ANSYS LS-DYNA's parallel processing capabilities.
Open the Mechanical APDL Product Launcher:
launcher182
Select the correct environment and license.
Go to the High Performance Computing Setup tab. Select .
Specify the MPI type to be used for this distributed run. MPI types include:
Intel MPI
IBM MPI
MS MPI (Windows only)
See the MPI table in the beginning of this chapter for the specific MPI version for each platform. If you choose MS MPI, you cannot specify multiple hosts or an MPI file.
Choose whether you want to run on a local machine, specify multiple hosts, or specify an existing MPI file (such as a host.list or an IBM MPI appfile):
If local machine, specify the number of cores you want to use on that machine.
If multiple hosts, select the machines you want to use from the list of available hosts. The list of available hosts is populated from the hosts182.ans file. Click on the machines you want to use and click to move them to the Selected Hosts list to use them for this run. If you click more than once for a machine, the number of cores to be used on that machine will increment each time, up to the maximum allowed in the hosts182.ans file. (Note that the option is not available when running the LS-DYNA MPP version on a Windows system.)
You can also add or remove a host, but be aware that adding or removing a host from here will modify only this run; the hosts182.ans file will not be updated with any new information from this dialog box.
If specifying an MPI file, type in the full path to the file, or browse to the file. If typing in the path, you must use the absolute path.
Additional Options for Linux Systems On Linux systems, you can choose to use the remote shell (rsh) protocol instead of the secure shell (ssh) protocol; ssh is the default.
Also in the launcher, you can select the option on the High Performance Computing Setup tab. This option uses the working directory as specified on the File Management tab as the directory structure on the master and all nodes. If you select this option, all machines will require the identical directory structure matching the working directory specified on the launcher.
Click to launch ANSYS.
You can also start Distributed ANSYS via the command line using the following procedures.
These procedures are also used for running ANSYS LS-DYNA MPP. To run the LS-DYNA MPP
version, substitute lsdyna182 for
ansys182 in the examples below. Only the command line
options specific to Distributed ANSYS apply to LS-DYNA MPP (i.e., -dis,
-np, -machines, -mpifile).
See LS-DYNA Parallel
Processing Capabilities in the ANSYS LS-DYNA User's
Guide for a detailed explanation on using ANSYS LS-DYNA's parallel
processing capabilities.
Running on a Local Host If you are running Distributed ANSYS locally (that is, running across multiple cores on a single machine), you need to specify the number of cores you want to use:
ansys182 -dis -np nIf you are using Intel MPI, you do not need to specify the
MPI software via the command line option. To specify IBM MPI, use the -mpi
ibmmpi command line option as shown below:
ansys182 -dis -mpi ibmmpi -np n For example, if you run a job in batch mode on a local host using four cores with an input file named input1 and an output file named output1, the launch commands for Linux and Windows would be as shown below.
On Linux:
ansys182 -dis -np 4 -b < input1 > output1 (for default Intel MPI)or
ansys182 -dis -mpi ibmmpi -np 4 -b < input1 > output1 (for IBM MPI)
On Windows:
ansys182 -dis -np 4 -b -i input1 -o output1 (for default Intel MPI)or
ansys182 -dis -mpi ibmmpi -np 4 -b -i input1 -o output1 (for IBM MPI)
Running on Multiple Hosts If you are running Distributed ANSYS across multiple hosts, you need to specify the number of cores you want to use on each machine:
ansys182 -dis -machinesmachine1:np:machine2:np:machine3:np
To specify IBM MPI (instead of the default Intel MPI), use the -mpi command line option as shown below:
ansys182 -dis -mpi ibmmpi -machinesmachine1:np:machine2:np:machine3:np
On Linux, you may also need to specify the shell protocol
used by the MPI software. Distributed ANSYS uses the secure shell protocol by default, but in
some cluster environments it may be necessary to force the use of the remote shell
protocol. This can be done via the -usersh command line
option.
Consider the following examples which assume a batch mode run using two machines (using four cores on one machine and using two cores on the other machine), with an input file named input1 and an output file named output1. The launch commands for Linux and Windows would be as shown below.
On Linux:
ansys182 -dis -b -machines machine1:4:machine2:2 < input1 > output1 (for default Intel MPI)or
ansys182 -dis -mpi ibmmpi -b -machines machine1:4:machine2:2 < input1 > output1 (for IBM MPI)
or
ansys182 -dis -b -machines machine1:4:machine2:2 -usersh < input1 > output1 (for remote shell protocol between compute nodes)On Windows:
ansys182 -dis -b -machines machine1:4:machine2:2 -i input1 -o output1 (for default Intel MPI)or
ansys182 -dis -mpi ibmmpi -b -machines machine1:4:machine2:2 -i input1 -o output1 (for IBM MPI)
The first machine specified with -machines in a Distributed ANSYS run must
be the host machine and must contain any files necessary for the initiation of your
job (i.e., input file, config.ans file,
database file, etc.).
If both the -np and -machines options are used
on a single command line, the -np will be ignored.
Note that the -machines option is not available when running the
LS-DYNA MPP version on a Windows system.
Specifying a Preferred Parallel Feature License If you have more than one HPC license feature, you can use the
-ppf command line option to specify which HPC license to
use for the parallel run. See HPC Licensing for more
information.
If you are running on Windows x64 systems using Microsoft HPC Pack (MS MPI), you need to use the HPC Job Manager to start Distributed ANSYS. For more information, refer to the following README files:
Program Files\ANSYS Inc\V182\commonfiles\MPI\WindowsHPC\README.mht
or
Program Files\ANSYS Inc\V182\commonfiles\MPI\WindowsHPC\README.docx
If you are running ANSYS Workbench, you can start a Distributed ANSYS job in the Mechanical
application. Go to ; select the
remote solve process you want to use and click the
button. To enable Distributed ANSYS, ensure that is selected (this is the default). If necessary, enter any
additional command line arguments to be submitted; the options are described in
Starting Distributed ANSYS via Command Line. If is selected, you do not need to specify the
-dis flag on the command line.
If you are running a remote solution on multiple machines, use the
-machines option to specify the machines and cores on which to
run the job. If you are running a remote solution on one machine with multiple
cores, specify the number of cores in the field; no command line arguments are
needed.
For more information on running a distributed solution in ANSYS Workbench, see Using Solve Process Settings.
You can specify an existing MPI file (such as an Intel MPI appfile) on the command line, rather than typing out multiple hosts or a complicated command line:
ansys182 -dis -mpifile appfile_nameFor an IBM MPI appfile, include the -mpi command line option:
ansys182 -dis -mpi ibmmpi -mpifile appfile_nameThe format of the appfile is system-dependent.
If the file is not in the current working directory, you will need to include the full path to the file. The file must reside on the local machine.
You cannot use the -mpifile option in conjunction with the
-np (local host) or -machines (multiple hosts)
options.
If the launcher option or the
-machines command line option was used, ANSYS generates a
default appfile named host.list. You can rename this file, move
it, or modify it for future runs if necessary. See the documentation for your
vendor/MPI type for details on working with the appfile.
Using the Intel MPI appfile Intel MPI uses an appfile to define the machines in the array (or the local host). A typical Intel MPI appfile might look like this:
-host mach1 -env env_vars env_var_settings -np 2 /ansys_inc/v182/ansys/bin/ansysdis182 -dis -host mach2 -env env_vars env_var_settings -np 2 /ansys_inc/v182/ansys/bin/ansysdis182 -dis
Using the IBM MPI appfile IBM MPI uses an appfile to define the machines in the array (or the local host). A typical IBM MPI appfile might look like this:
-h mach1 -np 2 /ansys_inc/v182/ansys/bin/ansysdis182 -dis -mpi ibmmpi -h mach2 -np 2 /ansys_inc/v182/ansys/bin/ansysdis182 -dis -mpi ibmmpi
See the IBM MPI user documentation for details on working with the IBM MPI appfile.
Using Other appfiles A typical appfile for other machines might look like this:
mach1 mach1 mach2 mach2
Distributed ANSYS writes files to the master and slave machines (or compute nodes) as the analysis progresses. The working directory for each machine can be on a local disk drive or on a network shared drive. For optimal performance, local disk storage is recommended instead of network storage. (For more information see the Performance Guide.) You must set up the same working directory path structure on the master and slave machines.
Keep the following points in mind as you set up your cluster environment:
If you don't have identical working directory structures set up on the master and all slave machines, Distributed ANSYS will fail to launch.
Distributed ANSYS will always use the current working directory on the master machine and will expect identical directory structures to exist on all slave nodes. If you are using the launcher, the working directory specified on the File Management tab is the directory that Distributed ANSYS will expect.