Engineering Scalable Distributed List Ranking
Improves the sparse ruling-set algorithm for list ranking to scale to large machines and input sizes.
Provides a novel improvement to the sparse ruling-set algorithm for list ranking, which has not been significantly updated in 20 years.
Before reading this…
Applications
- →Solving other problems in parallel computing
- →Large-scale data processing
To understand this paper, make sure you know these concepts first:
- Parallel computingfind papers →
- Algorithm engineeringfind papers →
Abstract
More Like ThisThe list ranking problem is one of the classical problems of parallel computing, with nontrivial algorithms and many applications as a subroutine for solving other problems. While it has been intensively studied in the early days of parallel computing, few things happened in the last 20 years. In particular, there is little work on scaling list ranking to large machines and input sizes. We reconsider list ranking starting from the ground-breaking results of Sibeyn a quarter century ago. We employ algorithm and performance engineering to improve his sparse ruling-set algorithm, making it capable of scaling to many processors, and provide a more detailed analysis of the impact of the algorithm's parameters, further guiding our practical implementation. We perform an extensive experimental study across a variety of input instances with different structural properties. We demonstrate that indirect communication, exploiting input locality, and message coalescing allows scaling to billions of elements on up to 24,576 cores.