7.2.1. Managing RSM Cluster Configurations and Queues (rsm.exe | rsmutils config)

Using the RSM Utilities application you can manually create, delete and list cluster configurations and queues.

A cluster configuration contains information about the cluster to which jobs will be submitted, and how RSM will work with the cluster.

RSM queues are the queues that users will see in client applications when submitting jobs to RSM. Each RSM queue maps to a cluster queue and cluster configuration.

7.2.1.1. Creating a Cluster Configuration

The RSM Utilities application provides a way of manually creating a cluster configuration. For information on creating a cluster configuration using the RSM Cluster Configuration application, and the settings specified in a cluster configuration, see Defining a Cluster Configuration.

Cluster configurations are saved to .rsmcc files in the cluster configuration directory. To determine the location of this directory, refer to Specifying a Directory for Cluster Configuration Files.

To manually create a cluster configuration, run the appropriate command below, appending options from the accompanying table to specify cluster configuration settings:

Windows: Run the following command in the [RSMInstall]\bin directory:

rsm.exe config create [cluster type]

Linux: Run the following command in the [RSMInstall]/Config/tools/linux directory:

rsmutils config create [cluster type]

For the [cluster type], specify one of the following to create a configuration for that cluster type: default (see Example 7.1: Default ANSYS RSM Cluster (ARC) Configuration), ARC | LSF | PBS | TORQUE | SGE | UGE | MSHPC.

Table 7.1:  Options for Creating a Cluster Configuration

Option (Windows | Linux)Usage
-name | -n nameThe name of the cluster configuration as it appears in the list of configurations. Defaults to the specified cluster type.
-rsmQueue | -rq nameThe name of the RSM queue with which this configuration and the cluster queue will be associated. Defaults to the name of the cluster queue.
-clusterQueue | -cq nameThe name of the cluster queue to which the RSM queue and cluster configuration will map. Required except for ANSYS RSM Cluster (ARC) and Microsoft HPC (MSHPC) cluster configurations.
-submitHost | -sh machineThe machine name of the cluster submit host. Defaults to 'localhost'.
-sshAccount|-ssh accountIf SSH will be used for communication between a Windows RSM client and a Linux cluster submit host, this specifies the account to use on the remote SSH submit host. Password-less SSH is required.
-platform | -p win | linThe platform of the cluster submit host (Windows or Linux). This is always required.
-transferType | -tt NO | RSM | OS | SCPSpecify how files will get to the cluster staging directory.

NO = No file transfer needed. Client files will already be in a cluster staging directory.

RSM = RSM uses TCP sockets to stream files from the client machine to the submit host. Use when the cluster staging directory is in a remote location that is not visible to client machines.

OS = RSM finds the cluster staging directory via a Windows network share or Linux mount point, and copies files to it using the built-in operating system copy commands. Use when the cluster staging directory is a shared location that client machines can access.

SCP = SSH/SCP will be used to transfer files from the client machine to the submit host.

-stagingDir | -sd pathThe cluster staging directory as the RSM client sees it. A Windows client will see the shared file system as a UNC path (for example, \\machine\shareName). A Linux client may mount the cluster staging directory such that the path appears different than it does on the cluster (for example, /mounts/cluster1/staging). Leave empty if using the no-file-transfer method.
-stagingMapDirs | -sdmap path;path;...

The path to the shared file system as the cluster sees it (for example, /staging on a Linux machine). This maps the client-visible path to the cluster-visible path. For example, the Windows client sees \\machine\STAGING which is a Linux Samba share of /staging.

Multiple paths are only supported when all of the following are true:

  • The submit host is not 'localhost'

  • The submit host platform is Linux

  • SSH is not being used

  • You have specified that no file transfer is needed

-localScratch | -ls pathLocal scratch path if jobs will run in a scratch directory local to the execution node. Leave empty to run jobs in the cluster staging directory.
-scratchUnc | -su path(Windows clusters only): UNC share path of -localScratch path not including the '\\machine\' portion.
-peSmp | -ps name(UGE/SGE only): Parallel Environment (PE) names for Shared Memory Parallel. If not specified, default will be 'pe_smp'.
-peMpi | -pm name(UGE/SGE only): Parallel Environment (PE) names for Distributed Parallel. If not specified, default will be 'pe_mpi'.
-noCleanup | -ncKeep job files in the cluster staging directory after the job has run.

Use the examples below as a guide when generating configuration files.

Example 7.1:  Default ANSYS RSM Cluster (ARC) Configuration

The default ANSYS RSM Cluster (ARC) configuration is used to submit jobs to the local machine (on which the cluster configuration resides). Every RSM installation has a basic ARC cluster already configured.

Running the command rsm.exe | rsmutils config create default is the equivalent of running rsm.exe | rsmutils config create ARC -name localhost -rq Local, where the name of the cluster configuration is localhost, and the RSM queue that is associated with this configuration is named Local.

A file named LOCALHOST.rsmcc is generated that contains the following settings:

<?xml version="1.0" encoding="utf-8"?>
<ClusterConfiguration version="2">
    <name>localhost</name>
    <type>ARC</type>
    <machine>localhost</machine>
    <submitHostPlatform>AllWindows</submitHostPlatform>
    <stagingDirectory />
    <networkStagingDirectory />
    <localScratchDirectory />
    <fileCopyOption>None</fileCopyOption>
    <nativeSubmitOptions />
    <useSsh>False</useSsh>
    <sshAccount />
    <useSshForLinuxMpi>True</useSshForLinuxMpi>
    <deleteStagingDirectory>True</deleteStagingDirectory>
    <readonly>False</readonly>
</ClusterConfiguration>

Example 7.2:  LSF on Linux (Jobs Use Local Scratch Directory)

To configure RSM to use an LSF queue, and run jobs in the local scratch directory, you would run the following command:

rsmutils config create LSF -name LSFSCRATCH -submitHost lsfheadnode -platform lin -localScratch /rsmtmp -rsmQueue LSF-SCRATCH -clusterQueue normal

The following arguments are used in this example:

  • LSF = cluster type is LSF

  • -name LSFSCRATCH = cluster configuration name will be LSFSCRATCH

  • -submitHost lsfheadnode = machine name of LSF cluster head node is lsfheadnode

  • -platform lin = platform of cluster submit host is Linux

  • -localScratch /rsmtmp = jobs will run in a local scratch directory /rsmtmp

  • -rsmQueue LSF-SCRATCH = RSM queue name will be LSF-SCRATCH

  • -clusterQueue normal = LSF cluster queue name is normal

An LSFSCRATCH.rsmcc file is created which contains the following settings:

<?xml version="1.0" encoding="utf-8"?>
<ClusterConfiguration version="1">
    <name>LSFSCRATCH</name>
    <type>LSF</type>
    <machine>lsfheadnode</machine>
    <submitHostPlatform>AllLinux</submitHostPlatform>
    <stagingDirectory />
    <networkStagingDirectory />
    <localScratchDirectory>/rsmtemp</localScratchDirectory>
    <fileCopyOption>None</fileCopyOption>
    <nativeSubmitOptions />
    <useSsh>False</useSsh>
    <sshAccount />
    <useSshForLinuxMpi>True</useSshForLinuxMpi>
    <deleteStagingDirectory>True</deleteStagingDirectory>
    <readonly>False</readonly>
</ClusterConfiguration>

In this example, an RSM queue name (LSF-SCRATCH) is specified. This will be the queue name displayed in client applications. If an RSM queue name is not included in the command line (for example, -rsmQueue LSF-SCRATCH), the actual cluster queue name will be displayed instead.

If you were to open the queues.rsmq file, you would see the LSF-SCRATCH queue added there:

<?xml version="1.0" encoding="utf-8"?>
<Queues>
    <Queue version="1">
        <name>LSF-SCRATCH</name>
        <clusterConfigurationName>LSFSCRATCH</clusterConfigurationName>
        <clusterQueueName>normal</clusterQueueName>
        <enabled>True</enabled>
    </Queue>
</Queues>

The clusterConfigurationName value, LSFSCRATCH in this example, is what links the queue to the actual cluster configuration.


Example 7.3:  SGE on Linux (Jobs Use Cluster Staging Directory)

2 cluster queues: all.q and ekmshare1

In the RSM client, the SGE queue ekmshare1 is referred to as SGE_SHARE. all.q is not aliased and is referred to as all.q.

Local scratch setup: Jobs submitted to RSM all.q queue will run in a local scratch folder /rsmtmp.

rsmutils config create UGE -name SGELOCAL -localScratch /rsmtmp -peMpi myPE -clusterQueue all.q

No local scratch. Jobs submitted to RSM’s SGE_SHARE queue will run in shared cluster staging directory.

rsmutils config create SGE -name SGESHARE -peSmp myPE -clusterQueue ekmshare1 -rsmQueue SGE_SHARE

Note that you can specify UGE or SGE for config create. They are the same.


Example 7.4:  Microsoft Windows HPC on Windows Server 2012

HPC does not define named queues.

We define 2 queues in RSM: HPC-LOCAL and HPC-SHARE.

Local scratch setup. Jobs submitted to RSM’s HPC-SCRATCH queue will run in a local scratch folder C:\RSMTemp. Note that the cluster nodes will all share this folder as \\[ExecutionNode]\RSMTemp.

rsm.exe config create MSHPC -name HPCSCRATCH -localscratch C:\RSMTemp -scratchUnc RSMTemp -rsmQueue HPC-SCRATCH

No local scratch. Jobs submitted to RSM’s HPC-SHARE queue will run in shared cluster staging directory.

rsm.exe config create MSHPC -name HPCSHARE -rsmQueue HPC-SHARE


7.2.1.2. Deleting a Cluster Configuration

To delete a cluster configuration:

Windows: Run the following command in the [RSMInstall]\bin directory:

rsm.exe config delete -clusterconfig |- cc clusterConfigurationName

Linux: Run the following command in the [RSMInstall]/Config/tools/linux directory:

rsmutils config delete -clusterconfig |- cc clusterConfigurationName

7.2.1.3. Creating an RSM Queue

When creating an RSM queue you must associate the queue with a cluster configuration and cluster queue.

To create an RSM queue:

Windows: Run the following command in the [RSMInstall]\bin directory:

rsm.exe config create queue -name queueName -clusterconfig clusterConfigurationName -clusterQueue clusterQueueName

Linux: Run the following command in the [RSMInstall]/Config/tools/linux directory:

rsmutils config create queue -n queueName -cc clusterConfigurationName -cq clusterQueueName

7.2.1.4. Deleting an RSM Queue

To delete an RSM queue:

Windows: Run the following command in the [RSMInstall]\bin directory:

rsm.exe config delete -rsmqueue | -rq rsmQueueName

Linux: Run the following command in the [RSMInstall]/Config/tools/linux directory:

rsmutils config delete -rsmqueue | -rq rsmQueueName

7.2.1.5. Listing Cluster Configurations and RSM Queues

The cluster configuration directory contains cluster configurations and RSM queue definitions.

To list cluster configurations and queues:

Windows: Run the following commands in the [RSMInstall]\bin directory:

All configurations: rsm.exe config list

The following is a sample listing:

C:\Program Files\ANSYS Inc\v182\RSM\bin>rsm.exe config list
Configuration location: C:\Users\atester\AppData\Roaming\Ansys\v182\RSM

Queues:
    Default [WinHPC Cluster, Default]
    LSF-SCRATCH [LSFSCRATCH, normal]
    Local [localhost, local]

Configurations:
    ARC Cluster
    localhost
    LSFSCRATCH
    WinHPC Cluster

Specific configuration: rsm.exe config list -cc clusterConfigurationName

The following is a sample listing:

C:\Program Files\ANSYS Inc\v182\RSM\bin>rsm.exe config list -cc LSFSCRATCH
Configuration location: C:\Users\atester\AppData\Roaming\Ansys\v182\RSM

Showing single configuration LSFSCRATCH

Queues:
    LSF-SCRATCH [LSFSCRATCH, normal]

<ClusterConfiguration version="2">
    <name>LSFSCRATCH</name>
    <type>LSF</type>
    <machine>lsfheadnode</machine>
    <submitHostPlatform>allLinux</submitHostPlatform>
    <stagingDirectory />
    <networkStagingDirectory />
    <localScratchDirectory>/rsmtmp</localScratchDirectory>
    <fileCopyOption>None</fileCopyOption>
    <nativeSubmitOptions />
    <useSsh>False</useSsh>
    <sshAccount />
    <useSshForLinuxMpi>True</useSshForLinuxMpi>
    <deleteStagingDirectory>True</deleteStagingDirectory>
    <readonly>False</readonly>
</ClusterConfiguration>

Linux: Run the following commands in the [RSMInstall]/Config/tools/linux directory:

All configurations: rsmutils config list

Specific configuration: rsmutils config list -cc clusterConfigurationName


Release 18.2 - © ANSYS, Inc. All rights reserved.