To start the parallel version of ANSYS Fluent using command line options, you can use the following syntax in a command prompt window:
fluent
version
-t
nprocs
[-gpgpu=
ngpgpus
]
[-p
interconnect
]
[-mpi=
mpi_type
]
[-cnf=
hosts
]
where
version must be replaced by the
version of ANSYS Fluent you want to run (2d, 3d, 2ddp, or 3ddp).
-t
nprocs specifies the number of processes to use. When the
-cnf option is present, the hosts
argument is used to determine which machines to use for the parallel job. For example,
if there are 10 machines listed in the hosts file and you want to run a job with 5
processes, set nprocs to 5 (that is,
-t5) and ANSYS Fluent will use the first 5 machines listed in the
hosts file. Note that if the -gpgpu option is used,
nprocs must be chosen such that the number of solver processes
per machine is equal on all machines.
-p
interconnect (optional) specifies the
type of interconnect. The auto-select interconnect is used by
default so that the best available interconnect is used if the option is not explicitly
specified. See Table 34.5: Supported Interconnects for Linux Platforms (Per Platform), Table 34.6: Available MPIs for Linux Platforms, and Table 34.7: Supported MPIs for Linux Architectures (Per Interconnect) for more information.
-mpi=
mpi_type (optional) specifies the
type of MPI. If the option is not specified, the default MPI for the given interconnect
will be used (the use of the default MPI is recommended). The available MPIs for Linux
are shown in Table 34.6: Available MPIs for Linux Platforms.
-cnf=
hosts (optional) specifies the hosts
file, which contains a list of the machines on which you want to run the parallel job;
if this option is not used, then the session will run on the local machine. If the hosts
file is not located in the directory where you are typing the startup command, you must
supply the full pathname to the file.
You can
use a plain text editor to create the hosts file. The only restriction
on the filename is that there should be no spaces in it. For example, hosts.txt is an acceptable hosts file name, but my hosts.txt is not.
Your hosts file (for
example, hosts.txt) might contain the following
entries:
machine1 machine2
If a machine in the network is a multiprocessor, you can list it more than once. For example,
if machine1 has 2 CPUs, then, to take advantage of both CPUs,
the hosts.txt file should list
machine1 twice:
machine1 machine1 machine2
As an alternative to a hosts file, you can simply type the
names of the machines in a list separated by commas, as shown in the following example:
-cnf=machine1,machine1,machine2.
By default, Fluent allocates ranks to machines in contiguous
blocks, where the block sizes are as equal as possible. You can control the per machine
block allocation size using the
machine
X
:
Y
convention in the hosts specification, where Y is the process block
count for
machine
X. The
process assignment will cycle through the machine list until all
processes are allocated in specified blocks. A fully round-robin
assignment of processes can be achieved by setting the machine block
allocation sizes to 1 (for example, machine1:1,
machine2:1, and so on).
-gpgpu=
ngpgpus specifies the number of
GPGPUs per machine to use for AMG execution. Note that when this option is used, the
number of solver process per machine must be equal on all machines and
ngpgpus must be chosen such that the number of solver processes
per machine is an integer multiple of ngpgpus. That is, for
nprocs solver processes running on M
machines using ngpgpus GPGPUS per machine:
See Using General Purpose Graphics Processing Units (GPGPUs) With the Algebraic Multigrid (AMG) Solver for more information about using GPGPU acceleration.
For example, to use the Infiniband interconnect, and to start
the 3D solver with 4 compute nodes on the machines defined in the
text file called fluent.hosts, you can enter
the following in the command prompt:
fluent 3d -t4 -pinfiniband -cnf=fluent.hosts
Note that if the optional
-cnf=
hosts is specified, a compute node will
be spawned on each machine listed in the file hosts.
Also, ANSYS Fluent provides a fault-tolerance feature on Infiniband Linux clusters running OFED.
To invoke this feature, use the command line flag
-pinfiniband.ofedft (or -pib.ofedft),
which enables transparent port fail-over and high-availability features using the IBM MPI.
Note that while the simulations proceed more robustly with this option, there may be some
degradation in performance.
During startup, ANSYS Fluent will create very small text files in the
/tmp area. If you would like to specify a different location for
these files, set the following environment variable: export
FL_TMPDIR=
directory.
The supported interconnects for parallel Linux machines are listed below (Table 34.5: Supported Interconnects for Linux Platforms (Per Platform), Table 34.6: Available MPIs for Linux Platforms, and Table 34.7: Supported MPIs for Linux Architectures (Per Interconnect)), along with their associated communication libraries, the corresponding syntax,
Table 34.5: Supported Interconnects for Linux Platforms (Per Platform)
| Platform | Processor | Architecture | Interconnects/Systems* |
|---|---|---|---|
| Linux | 64-bit |
lnamd64
|
ethernet, infiniband, myrinet
|
(*) Node processes on the same machine communicate by shared memory. ANSYS Fluent lets the MPI autoselect the best interconnect available on the system. Users can specify an interconnect to override that selection. Ethernet is the fallback choice.
Table 34.6: Available MPIs for Linux Platforms
| MPI | Syntax (flag) | Communication Library | Notes |
|---|---|---|---|
ibmmpi
|
-mpi=ibmmpi
| IBM MPI | General purpose
for SMPs and clusters (1) |
intel | -mpi=intel | Intel MPI | General purpose for SMPs and clusters |
openmpi | -mpi=openmpi | Open MPI | Open source MPI-2 implementation. For both SMPs and clusters. |
cray
|
-mpi=cray
| Cray MPI (MPT) | Only available on Cray systems. |
(1) IBM Platform MPI Community Edition is installed with Fluent, which is limited to 4096 processes without an additional license. Refer to IBM Platform MPI with High (>4096) Process Count for details.
Table 34.7: Supported MPIs for Linux Architectures (Per Interconnect)
| Architecture | Ethernet | Myrinet* | Infiniband | Proprietary Systems | Omni-Path**** |
|---|---|---|---|---|---|
lnamd64
|
ibmmpi (default), intel,
and openmpi
|
ibmmpi (default), openmpi** |
ibmmpi (default), intel, and
openmpi
| Cray*** |
intel
|
(*) Both MX and GM Myrinet protocols are supported. ANSYS Fluent will
automatically detect which type is running on the system and will
use that particular protocol. You only have to supply the -pmyrinet option. If the hardware supports it, the installation
and usage of Myrinet MX is recommended (consult your Myrinet vendor
for applicability).
(**) The OpenMPI version packaged with ANSYS Fluent does not support Myrinet. However, you can use a local version of OpenMPI with Myrinet support.
(***) Using the Cray platform, the Extreme Scalability Mode (ESM)
option is used by default, employing the native Cray MPI (MPT) library for parallel
communication. Optionally, you can use the Cluster Compatibility Mode (CCM) mode, in which
the standard IBM MPI library is used for parallel communication. For simulations with high
core counts (for example, above 1000 cores), the ESM mode is recommended for better
performance. Note that when using MPT version 5.0 and higher (up to, but not including 7.0),
you must set the following environment variable: export
FLUENT_USE_CRAY_MPT5=1.
(****) Omni-Path requires OFED driver version 10.2 or higher.
(Linux Only) To enable a job scheduler supported native remote node access mechanism,
you can use the -scheduler_rsh command line flag or set the
environment variable SCHEDULER_RSH=1. This tight integration is
only supported for the MPI/Job scheduler combinations listed below:
IBM MPI + LSF
Intel MPI + LSF
Intel MPI + UGE
Intel MPI + PBS Pro