Chapter 9 Parallel Processing

Parallel processing increases the speed of running the MSE in DLMtool significantly. The use of parallel processing in DLMtool has changed slightly from previous versions of the package.

By default the MSE runs without using parallel processing. We recommend running a few test runs of your MSE with a low number of simulations and without parallel processing. Once you are satisfied the model is running correctly for your operating model, you can increase the number of simulations and use parallel processing.

9.1 Setting up Parallel Processing

The setup function is used to set up parallel processing.

setup()

## 
## Stopping cluster

## snowfall 1.84-6.1 initialized (using snow 0.4-3): parallel execution on 10 CPUs.

## Library DLMtool loaded.

## Library DLMtool loaded in cluster.

By default the setup function initializes half of the available processors as we have found this to be the most efficient for most systems. You can change the number of processors by specifying the cpu argument, e.g., setup(cpu=6).

See Determining Optimal Number of Processors for more details on calculating the optimal number of processors to use on your system.

9.2 Running MSE with Parallel Processing

Use the parallel=TRUE argument in runMSE to use parallel processing. Note that you must run setup() first.

You will notice that the usual update messages are not printed to the console when parallel processing is used. This is why it is important to initially test your MSE with a small number of simulations without parallel processing.

myMSE_200P <- runMSE(myOM, parallel = TRUE)

## Running MSE in parallel on 10 processors

## MSE completed

Parallel processing can increase the speed of running the MSE considerably. For example, although in this demonstration we are only running a low number of simulations, run time decreased from 4 to 23 seconds when using parallel processing on 10 processors.

9.3 Determining Optimal Number of Processors

The optCPU function can be used to evaluate the relationship between number of processors and run time:

optCPU()

##    ncpu  time
## 1     1 77.84
## 2     2 37.43
## 3     3 36.77
## 4     4 21.49
## 5     5 21.02
## 6     6 20.31
## 7     7 20.16
## 8     8 20.41
## 9     9 21.19
## 10   10 21.50
## 11   11 20.41
## 12   12 20.40