Parallel and distributed computation: numerical methods

A comprehensive and theoretically sound treatment of parallel and distributed numerical methods. It focuses on algorithms that are naturally suited for massive parallelization, and it explores the fundamental convergence, rate of convergence, communication, and synchronization issues associated with such algorithms. This is an extensive book, which aside from its focus on parallel and distributed algorithms, contains a wealth of material on a broad variety of computation and optimization topics. Among its special features, the book: 1) Quantifies the performance of parallel algorithms, including the limitations imposed by the communication and synchronization penalties. 2) Describes communication algorithms for a variety of system architectures including tree, mesh, and hypercube. 3) Provides a comprehensive convergence analysis of asynchronous methods and a comparison with their asynchronous counterparts. 4) Covers direct and iterative algorithms for linear and nonlinear systems of equations and variational inequalities. 5) Describes optimization methods for nonlinear programming, shortest paths, dynamic programming, network flows, and large-scale decomposition. 6) Includes extensive research material on optimization methods, asynchronous algorithm convergence, rollback synchronization, asynchronous communication network protocols, and others. 7) Supplements the text material with many exercises, whose complete solutions are posted on the internet. 8) Contains a lot of material not found in any other book.

Introduction -- Algorithms for systems of linear equations and matrix inversion -- Iterative methods for nonlinear problems -- Shortest paths and dynamic programming -- Network flow problems -- Totally asynchronous iterative algorithms -- Partially asynchronous iterative algorithms -- Organizing an asynchronous network of processors for distributed computation.