Developing applications that scale well on massively parallel architectures is quite difficult, due to problems with synchronization and communication time as well as fault tolerance. This book reviews current algorithms and presents new mathematical versions that avoid such problems. It details the implementation and validates the proposed versions on real examples. The methods are built on top of asynchronous algorithms, which do not need synchronization and thus have low sensitivity to communication delay. Moreover, they have huge potential as fault tolerant algorithms.