## 4.1. Decomposing Structured Calculations

In the following example, you will create a domain decomposition set for a Lagrange impact benchmark calculation. Load in the model B3D2P2. Though this example specifically applies to a Lagrange process, the method shown is appropriate to decompose any grid calculation, including Euler processes.

The problem consists of a single Lagrange part (49x100x10 cells) filled with two materials. The steel has an initial z-velocity of 0.06 cm/microsecond, while the aluminum is initially at rest.

If you run this problem for 100 cycles and then view a velocity vector plot, you will see the screen above, showing the impact of the steel on the aluminum.

This model already contains Decomposition Set information. The current decomposition is shown in the Decomposition Set text box.

Presently the current decomposition set is set to None. The current decomposition set can be changed by using the drop-down list activated by pressing the list button on the right of the text box. The image below shows a close up of the Parallel panel.

Choose the decomposition set denoted by TWO. Further buttons allowing the modification of the decomposition set will appear.

Select the check box Plot parallel decomposition. The decomposition of the model will now be displayed (you may need to manually refresh the screen if automatic refresh is not activated):

This image shows the decomposition of the model into its constitutive sub-domains. The scale illustrates which sub-domain is assigned to which task. Here the bottom sub-domain is assigned to task 1 and the top sub-domain assigned to task 2. This is the optimum decomposition for efficient parallel computing over two tasks. The model has been decomposed in the J direction. As the J direction contains the most elements, this decomposition will result in the smallest amount of information being passed between tasks.

There are 490 elements on each sub-domain boundary; 49 in the I direction and 10 in the K direction. If we had decomposed the part in the I direction, there would have been 1000 elements on each sub-domain boundary; 100 in the J direction and 10 in the K direction. Obviously there is a computational overhead associated with the amount of information passed between tasks and therefore each element on the sub-domain boundaries increases the amount of communicated information and therefore computational expense.

Before proceeding click Delete to discard the TWO decomposition. The instructions to enable you to replace it are detailed below.

Click Add in the Decomposition Set section of the Parallel panel.

Additional buttons and information will then appear in the Parallel panel:

Type `TWO` in the Selected decomposition text box. This will be the name of the decomposition set that will be defined. This name is also displayed in the Current text box:

Click Subdomains to activate a popup window in which the decomposition information can be entered. The following window will appear:

As we wish to decompose the part into two sub-domains in the J direction, enter `2` in the Number of J-sections text box. Autodyn will automatically evaluate the optimum dividing plane for the decomposition and enter it in the Dividing plane #1 text box:

You may alter the J value of the dividing plane by changing the number within the text box. There may be situations where this would be advantageous, but this would involve more complicated models.

Click OK to store this information.

In this case the divisions can be chosen such that the resulting two sub-domains are identical in size. This is the optimum way to set up a parallel processing calculation for excellent load-balancing and most efficient execution.

This may not always be possible. You may have multiple parts, each of different dimensions, making it more difficult to decompose each part so that one sub-domain of equal size is created for each processor.

- For the current host configuration, if you have two parts of differing size, it may be possible to subdivide each part into two equal size sub-domains, so that you have a total of four sub-domains. These sub-domains could then be allocated one to each processor (one from each part) to allow perfect load balancing. There would be a small amount of overhead, due to processing two sub-domains instead of one on each processor, but this is likely to be a small price to pay for perfect load-balancing.

- Taking this principal a step further, for multiple parts that do not offer very uniform decomposition, you can decompose the parts into many more sub-domains than there are processors and let Autodyn automatically load-balance the sub-domains over all processors (a discussion of automatic load-balancing follows). The more sub-domains Autodyn has to distribute, the easier it will be to effectively load-balance a problem. However, the more sub-domains created, the more overhead will be involved to exchange data at sub-domain boundaries. You may have to experiment to find a good balance, but our experience so far indicates that parallel calculation are generally computation intensive, so it is likely that good load-balancing is more important than minimizing data communication between sub-domains. Most of our benchmarking has been on well load-balanced problems with minimal communication overhead, so we cannot offer more detailed guidelines at present.

The part has now been decomposed into sub-domains. Though, if you view the parallel decomposition you will see that both of the sub-domains have been automatically allocated to one task. Therefore, we need to assign these sub-domains to the available tasks in order to produce an efficient calculation. This process is initiated be clicking Task Assignments. The following popup window will appear:

The sub-domains can be assigned manually or automatically. If the Automatic option button is selected then the sub-domains will be assigned at run time. If the model is a very simple one then this course of action could be taken. Here we will manually assign the sub-domains to the available tasks in order to become familiar with the process.

We wish to assign the two J-sections of the model to two different tasks. To do this, enter `1` into the to J-Section text box and click Assign Now:

These actions have assigned the first J-section sub-domain to the first task.

To assign the second J-section sub-domain to the second task enter `2` into the From J-section, to J-section, and Assign to task text boxes and click Assign Now.

If you have automatic refresh activated the image of the model on screen should have been updated to show the two sub-domains on two different tasks:

We could have balanced the task assignments automatically and viewed the results instantly by selecting the Balance automatically option button in the Task Assignment window. This results in a change to the configuration of the window:

By clicking Balance Now, the task assignments will be immediately processed automatically. If you view the domain decomposition image then you will see that the decomposition is identical to that which we defined manually.

The model is now ready to be run in parallel. Click Run to execute the calculation.

Release 18.2 - © ANSYS, Inc. All rights reserved.