3.3.2. Defining a Cluster Configuration

RSM provides an intelligent and responsive interface for defining cluster configurations.

As you define a cluster configuration, RSM validates paths and machine names that you enter, and presents settings that are specific to the cluster type and options that you have chosen.

The process consists of the following main steps:

  1. On the Cluster tab, specifying cluster information such as the cluster type and machine name of the cluster submit host

  2. On the File Management tab, specifying file management properties that determine how job input files get transferred to the cluster staging directory

  3. On the Queues tab, importing or adding cluster queues, and mapping them to RSM queues

When you finish specifying information on a tab, click Apply to validate the information and apply it to the configuration. Colored icons indicate the status of information on each tab:

If information is missing or invalid, a warning icon   is displayed on the tab.

An overview of each step is provided below.

For detailed instructions and examples of specific cluster configuration types, refer to ANSYS RSM Cluster (ARC) Configuration and RSM Integration with a Cluster.

3.3.2.1. Specifying Cluster Information

The first step in defining a cluster configuration is specifying information about the cluster, and how the client communicates with the cluster. This information is specified on the Cluster tab in the editing pane:

General settings include:

Cluster Configuration
Name

The name of the cluster configuration as it appears in the Clusters list in the left pane. Do not use the name of an existing cluster configuration.

Submit host

Identify the machine that will serve as the cluster submit host. This is the machine that will handle job scheduling. In other words, it is the machine on which scheduling software is installed, or, in the case of an ANSYS RSM Cluster (ARC), the machine on which the ARC Master service has been installed.

  • If jobs will be submitted to the cluster submit host from any other machine, enter the submit host's full domain name (for example, machineName.company.com), even if the machine on which you are currently working (the local machine) is the submit host.

  • If the machine on which you are currently working (the local machine) is the cluster submit host, and jobs will not be submitted to it from any other machine, you can enter localhost in this field.


Important:
  • Correctly identifying the submit host is a crucial step, as this is the key piece of information that enables RSM to communicate with the cluster.

  • If the current (local) machine is the submit host, do not enter localhost in the Submit host field if jobs will be submitted to this machine from other machines. You must use the full domain name in this case.


Cluster type

The scheduling system used to submit jobs to the cluster. If you are integrating with an established third-party cluster, select either Windows HPC, LSF, PBS Pro, TORQUE with Moab, UGE (SGE), or Custom. If you are not using a third-party job scheduler, select ARC (ANSYS RSM Cluster) as your scheduling system.

If UGE (SGE) is selected as the cluster type, settings become available to specify Parallel Environment (PE) names:

Shared memory parallel processing enables you to distribute solve power over multiple processors on the same machine. Distributed parallel processing enables you to distribute solve power across multiple cores on a single node, or across multiple nodes. For information on configuring parallel environments, consult the documentation of the simulation product you are using.

RSM integrates with Windows HPC, LSF, PBS Pro, TORQUE with Moab and UGE (SGE) without requiring job script customization. For custom cluster types, customization will likely be necessary to make job submission work. Refer to Customizing RSM.

Job submission arguments

Scheduler-specific arguments that will be added to the job submission command line of the job scheduler. For example, you can enter job submission arguments to specify the queue (LSF, PBS, SGE) or the nodegroup (MS HPC) name. For valid entries, see the documentation for your job scheduler.

Use SSH protocol for inter and intra-node communication (Linux only)

This setting is used for distributed computing with multiple nodes involved. It specifies that RSM and solvers use SSH for communications between Linux execution nodes, and within the nodes themselves. If left deselected, this indicates that RSH is used.

This setting will be applied to all Linux cluster nodes, allowing for solvers to run in distributed parallel mode.

When ANSYS Fluent, ANSYS CFX, ANSYS Mechanical, and ANSYS Mechanical APDL are configured to send solves to RSM, their solvers will use the same RSH/SSH settings as RSM.

How does the client communicate with the cluster?
Able to directly submit and monitor cluster jobs

Specifies that the RSM client can use the RSM internal communication mechanism to directly submit jobs to the cluster, and monitor cluster jobs. This requires that an IT administrator open ports and adjust firewall settings on the cluster submit host to allow communication from the RSM client.

When the cluster submit host is a remote machine, the RSM launcher service launches a user proxy process on the submit host which performs operations such as job submission, monitoring, and file transfer on the user's behalf. The RSM launcher service will use one port, while each user proxy process will use a separate port chosen by RSM. Ports for user proxy processes are chosen from a port range if one has been specified in the RSM application settings (see Specifying a Port Range for User Proxy Processes). Otherwise, RSM will randomly select a port that is free.

If jobs will be submitted to RSM from EKM, the launcher service will automatically reserve ports for RSM XmlRpc proxies used by EKM. If you encounter an issue you can add or delete ports to be reserved for RSM XmlRpc proxies. For details refer to Managing XmlRpc Ports for RSM Proxies (rsm.exe xmlrpc).

Uses non-RSM communication to a remote cluster node (e.g. SSH)

This option is only available when the submit host is a Linux machine and the RSM client is a Windows machine.

When a job from Windows client is submitted to a remote Linux cluster, this specifies that an external communication mechanism (for example, SSH) will be used to communicate with the cluster submit host instead of RSM's internal communication mechanism. Use this option if your IT administrator does not want to open ports and adjust firewall settings to allow communication from the RSM client, in adherence with your organization's IT policy.

In the Account name field, specify the account name that the Windows RSM client will use to access the remote Linux submission host.


Note:
  • This account must be set up before this mode can be used. For information on configuring SSH to allow access from a Windows machine, see Configuring PuTTY SSH.

  • This is not an account that is specified in the Credentials section of the RSM Configuration application. The accounts listed there are RSM client accounts, not user proxy accounts.

  • The SSH protocol is not needed (or supported) if jobs will be submitted to a cluster from EKM, or from Workbench to an EKM Portal.


3.3.2.2. Specifying File Management Properties

For jobs to be successfully executed in the cluster, client job files need to be staged in a location that the cluster can access. Also, job output files may need to be transferred back to the client after a job has been executed.

When you submit a job to RSM in a client application, a client working directory is created to which all necessary job files are written. The location of this directory is configured in the client application. For more information, refer to Setting Up Client Working Directories to Eliminate the Need for File Transfers.

If the client working directory is created under a shared directory that is visible to all cluster nodes (in other words, it is already inside the shared cluster staging directory), then it is possible for the job to be run directly in the working directory. Otherwise, if files need to be transferred from the client working directory to a cluster staging directory, you will need to specify this in your cluster configuration.

You will also need to specify where jobs will run on the cluster side.

This information is specified on the File Management tab in the editing pane:


Tip:  Use the Tell me more options to view detailed information about each file transfer method so that you can select the method that best suits your IT environment, file storage strategy, and simulation requirements.


Client-to-Cluster File Management

Specify how files will get to the cluster staging directory.


Important:
  • The cluster staging directory must a shared directory that is visible to all cluster nodes.

  • If jobs will be submitted to a cluster from EKM, or from Workbench to an EKM Portal, you must select the No file transfer needed option.


RSM internal file transfer mechanism

Use this option when the cluster staging directory is in a remote location that is not visible to client machines.

RSM uses TCP sockets to stream files from the client machine to the submit host machine. In this case you must specify the path to the directory where job files will be staged (as the cluster sees it):

When transferring files to a single-node cluster, it may not be necessary for the staging directory to be a shared path (e.g. UNC on windows).

Use an external mechanism for file transfer (e.g. SSH/SCP)

Use this option when the cluster staging directory is in a remote location that is not visible to client machines, and you are using an external protocol such as SSH for file transfers.

This option is available if you have selected Uses non-RSM communication to a remote cluster node (e.g. SSH) on the Cluster tab. RSM has built-in support for SCP transfer. In using the SSH/SCP protocol for communication and file transfer, it is not necessary to have any RSM components running on the remote submit host.

In this scenario, the cluster staging directory is not visible to the client.

You will need to specify the path to the Cluster staging directory as the cluster sees it (for example, /staging on a Linux machine):

Operating system file transfer to existing network share (Samba, CIFS, NFS)

Use this option when the cluster staging directory is a shared location that client machines can access.

The RSM client finds the cluster staging directory via a Windows network share or Linux mount point that has been set up on the client machine, and copies files to it using the built-in operating system copy commands.

In the Cluster staging network share field, specify the path to the shared file system as the RSM client sees it. A Windows client will see the shared file system as a UNC path (for example, \\machine\shareName), while a Linux client will see a mounted directory (for example, /mounts/cluster1/staging).

In the Cluster staging directory field, specify a path to the cluster staging directory which all execution nodes can see. This is a path on the cluster side (for example, /staging on a Linux machine).

If jobs will be running directly on the client machine or a single-node cluster (for example, ARC operating in basic mode), the staging area may just be a preferred local scratch area, and may not need to be a shared path.

When using this option, you must ensure that the cluster staging directory is both visible to and writable by the client machine. For more information, see Enabling OS Copy to the Cluster Staging Directory.

No file transfer needed. Client files will already be in a cluster staging directory.

Use this option if the client files are already located in a shared file system that is visible to all cluster nodes. You must select this option if jobs will be submitted to a cluster from EKM, or from Workbench to an EKM Portal.

When the client and cluster are running on the same platform, or the submit host is localhost, further action is not required in most cases.

When the client and cluster platforms differ, it is necessary to map the client-visible path to a cluster-visible path. The most common scenario is a user working on a Windows client, but their work files are located in a network shared 'home' directory. For example, they work with their files using \\homeServer\myhome, but on the Linux cluster side, this can be referred to as $HOME.

The Network share paths on cluster table is displayed if the submit host is a Linux machine that is not localhost, and SSH is not being used. Use the table to specify network paths that map to cluster paths:

Client working directories in Windows UNC format (\\machine\shareName) are mapped to Linux format using these mapped directories. On the cluster side, each network share path is the root of the cluster staging directory, meaning that the Linux mapping directory is substituted for only the \\machine\shareName portion of the client UNC path. For example,

Client directory: \\homeServer\myHome\projects\project1\model1

Mapping directory: $HOME (expands to /nfs/homes/joed)

Resulting Linux directory: /nfs/homes/joed/projects/project1/model1


Note:
  • The client directory must be visible to all execution nodes.

  • If jobs will be submitted from Linux clients to a Linux submit host, you may not need to enter a path in the Network share paths on cluster table if the client working directory can be used as a cluster staging directory.


In all other cases (for example, SSH is being used), you will be prompted to specify the Cluster Staging Directory (or nothing at all):

For information on creating client working directories under a shared cluster directory, see Setting Up Client Working Directories to Eliminate the Need for File Transfers.

Cluster Side File Management

Specify the working directory in the cluster where job (or solver) commands will start running.

In the cluster staging directory

This option is recommended if one or both of the following is true:

  • There is a fast network connection between the execution nodes and the cluster staging directory.

  • You are using a solver that produces fewer, relatively small files as part of the solution and does not make heavy use of local scratch space (for example, the CFX or the Fluent solver).

In a scratch directory local to the execution node

This option is recommended to optimize performance when one or both of the following is true:

  • There is a slower network connection between the execution nodes and the cluster staging directory.

  • You are using a solver that produces numerous, relatively large files as part of the solution and makes heavy use of local scratch space (for example, Mechanical solvers).


Note:  For jobs submitted from EKM, the use of a local scratch directory is supported for MAPDL/Mechanical solutions only. If using other solvers, you must use a shared cluster staging directory.

For jobs submitted from Workbench (to RSM or an EKM Portal), the use of a local scratch directory is supported for all solution types.


All input files will be copied from the cluster staging directory into that local scratch directory. Then, when the job finishes running, the requested output files generated by the job will be copied back to the cluster staging directory.

In the Local cluster scratch directory field, enter the local path of a scratch directory on the execution node (for example, C:\Shares\Local_Share\ScratchDir on Windows). You can enter the path of the scratch directory manually, or use an environment variable in the format %VAR%.

If the cluster is running on Windows, you must create a network share path for the local scratch directory on each node. In the Share path for local scratch field, enter the network share path of the local scratch directory. This path starts with a non-editable [ExecutionNode] variable. When a job is submitted, the [ExecutionNode] variable will be replaced with the actual machine name of each execution node assigned to the job.

By default, job files will be deleted from the cluster staging directory after the job has run. Choosing Keep job files in staging directory when job is complete may be useful for troubleshooting failed jobs. However, retained job files will consume disk space, and require manual removal.

3.3.2.3. Defining and Testing RSM Queues

When you choose to submit a job to RSM, you must choose an RSM queue for the job. An RSM queue maps to an actual cluster queue, and provides a way to link to the cluster configuration.

RSM queues are defined on the Queues tab in the editing pane:

Defining an RSM Queue

RSM queues are the queues that users will see in client applications when they choose to submit jobs to RSM. RSM queue names can match cluster queue names if desired, but each RSM queue name must be unique.

RSM provides two ways of defining RSM queues: you can either import a list of cluster queues and define an RSM queue for each cluster queue, or you can manually add an RSM queue and assign a cluster queue to it.

  • To import a list of cluster queues, or refresh the list if you have imported cluster queues previously, click  . Then, for each cluster queue, double-click in the RSM Queue field and specify a unique RSM queue name.

  • To add an RSM queue to the list, click  , then specify a unique name for the queue in the RSM Queue field.

    Double-click in the Cluster Queue field and enter the name of an existing cluster queue. RSM will check to see if the cluster queue is valid.

Enabling/Disabling an RSM Queue

When you create an RSM queue it is enabled by default. This means that it will be available for selection in client applications.

To control whether or not an RSM queue is available for use, check or uncheck the queue's Enabled check box.

Testing an RSM Queue

When you test an RSM queue, RSM sends a test job to the cluster via the associated cluster queue.

To test an RSM queue, click   in the queue's Test column, or right-click the queue in the tree in the left pane and select Submit Test.


Note:
  • You may need to click Apply on the Queues tab before being able to submit test jobs.

  • Only enabled queues can be tested.


The status of the test is displayed in the Status column:

 Job is being submitted
 Job is queued
 Job is in progress
 Job completed successfully
 Job completed successfully and released
 Job aborted
 Job aborted and released
 Job failed
 Job failed and released

When a job is running, the   button is replaced by an   button, enabling you to abort the test job if desired.

Performing an Advanced Queue Test

In an advanced test of an RSM queue you can select a client working directory in which to run the test. This is a good way of testing whether or not files are being transferred to the cluster staging directory (if, for example, the client working directory is a network share of cluster staging directory).

To perform an advanced RSM queue test:

  1. Right-click the queue in the tree in the left pane, then select Advanced Test.

  2. In the Advanced Test dialog box, select or specify the client directory that you want to use for the test job. You can leave it set to the default %TEMP% environment variable, or enter a path or environment variable manually. Manually entered items will be added as drop-down options.

  3. If you want to clean up the client directory after the test job is done, enable the Cleanup Client Directory check box.

  4. Click Submit.

The status of the test is displayed in the Status column of the queue table, as described in Testing an RSM Queue.

Viewing a Test Job Report

If you have submitted a test job to an RSM queue, you can view a detailed test report by clicking   in the queue's Report column.

Saving a Test Job Report

You can save a job report to an HTML file that can be shared with others.

To save the job report:

  1. Click   in the job report window.

  2. Accept or specify the save location, filename, and content to include.

  3. Click Save.

Deleting an RSM Queue

You can delete an RSM queue that appears on the Queues tab in one of three ways:

  • Select the queue(s) in the queue list, then click   on the queues toolbar.

  • Right-click the queue in the queue list, then select Delete Selected RSM Queue(s).

  • Right-click the queue in the tree in the left pane, then right-click and select Delete Queue. Note that only enabled queues appear in the tree.


Release 18.2 - © ANSYS, Inc. All rights reserved.