Using Associativity

Using Associativity

• Can allow reordering of communication
• May change numerical properties
• Example of Gramm-Schmidt orthogonalization (dot(w1,w2)=0):v = v - dot(v,w1) w1v = v - dot(v,w2) w2(two Allreduces) replaced witha = dot(v,w1); b = dot(v,w2) (Allreduce of 2)v = v - a w1 - b w2
• This is NOT THE SAME in floating-point arithmetic