12.5.2 Communications

Figure 12.2a gives an example of multi-tasking with $num\_processors=3$ processors and jmt=14 latitude rows for a second order memory window. Note that latitude rows on disk (or ramdrive) for each processor in Figure 12.2b look like a miniture version of Figure 11.4 where jstask=1 and jetask=jmt. For example, on the disk (or ramdrive) of processor #1, the global latitude index runs from jrow=1 to jrow=6. On this processor, the task limits are jstask=1 and jetask=6 and the limits for integrating prognostic equations are from jscomp=2 through jecomp=5. On processor #2, the task limits are jstask=5 and jetask=10 and the limits for integrating prognostic equations are from jscomp=6 through jecomp=9.

Apart from dividing up the domain into pieces, the new aspect in Figure 12.2 is communication between processors indicated by short arrows pointing to the boundary rows. In the second order memory window, there is one boundary row at the borders of each task. Look at latitude row jrow=6 on the updated disk of processor #1. This row cannot be updated to $\tau+1$ by processor #1's MW because data from jrow=7 is needed. Instead, data at $\tau+1$ from jrow=6 on processor #2 is copied into the jrow=6 slot of processor #1 by a call to the communication routine after all processors have finished working on their tasks. Similarly, jrow=5 on processor #2 is updated with $\tau+1$data from jrow=5 on processor #1 and so forth.

The situation with fourth order numerics is similar except that more communication is required at the end of the timestep. This is illustrated in Figure 12.3a. Note that since jbuf=2for forth order windows, the task for processor #1 ranges from jstask=1 to jetask=7. For processor #2, the task ranges from jstask=4 to jetask=11, and so forth. However, the rows on which prognostic equations are solved within each task are the same as in the case with a 2nd order memory window. Additional communication is indicated by the extra arrows which are needed because there are now two buffer rows on the borders of each task (i.e. jbuf=2).

For all processor numbers from $pn=2,num\_processors$ the following prescribes the required communication for second order numerics:

• copy all data from latitude row ``jstask+1'' on processor ``pn'' to latitude row ``jetask'' on processor ``pn-1''.

• copy all data from latitude row ``jetask-1'' on processor ``pn-1'' to latitude row ``jstask'' on processor ``pn''.

When a fourth order memory window is involved, the following communication is required:

• copy all data from latitude row ``jstask+3'' on processor ``pn'' to latitude row ``jetask'' on processor ``pn-1''.

• copy all data from latitude row ``jstask+2'' on processor ``pn'' to latitude row ``jetask-1'' on processor ``pn-1''.

• copy all data from latitude row ``jetask-3'' on processor ``pn-1'' to latitude row ``jstask'' on processor ``pn''.

• copy all data from latitude row ``jetask-2'' on processor ``pn-1'' to latitude row ``jstask+1'' on processor ``pn''.