Numerical Linear Algebra Group

The Society for Industrial and Applied Mathematics (SIAM) has chosen Nick Higham, Royal Society Research Professor and Richardson Professor of Applied Mathematics, as the 2021 recipient of the George Pólya Prize for Mathematical Exposition.

The prize, which is awarded for expository work that communicates mathematics effectively, is named after George Pólya, the Hungarian mathematician who wrote the million-selling book How to Solve It, first published in 1945.

“Pólya was a brilliant researcher, teacher, and expositor of mathematics,” said Higham. “It is an honor to receive this SIAM prize named after him, especially as most of my research and expository writing has been published with SIAM.”

Higham was cited for “the crisp clarity, elegance and accessibility of his mathematical and popular exposition on a broad range of topics in applied mathematics.”

The award ceremony will be held at the 2021 SIAM Annual Meeting, scheduled to be held in a virtual format during July 19-23, 2021. See also the SIAM announcement of the award.

Professor Nicholas J. Higham, University of Manchester. Photo: Rob Whitrow

By Sven Hammarling.

An article in a recent issue of the journal Nature has the title “Ten Codes that Transformed Science”¹ .

Two of the codes are purely numerical, namely the fast Fourier transform (FFT) and the Basic Linear Algebra Subprograms (BLAS). The BLAS evolved in three stages, the Level 1 BLAS² for scalar and vector operations, the Level 2 BLAS³ for matrix-vector operations and the Level 3 BLAS⁴ for matrix-matrix operations. The first draft proposal for the Level 1 BLAS appeared in the ACM Signum Newsletter in 1973, so the project as a whole spanned about seventeen years.

Two members of the Numerical Linear Algebra Group were directly involved in the development of the BLAS. Jack Dongarra wrote a number of the Level 1 BLAS routines, was involved in testing the routines on an IBM 370/195 and provided several efficient implementations of the routines, all of which was acknowledged in the published paper. Both Jack Dongarra and Sven Hammarling were authors of the Level 2 and 3 BLAS, together with Jeremy Du Croz and Richard Hanson for the Level 2 BLAS, and Jeremy Du Croz and Iain Duff for the Level 3 BLAS.

It should be said that the BLAS were very much a community project, with input from many people. The BLAS, especially the Level 3 BLAS, provided the basic high performance operations, particularly for the block partitioned methods, used by the LAPACK⁵ project, which started in 1987, before the actual publication of the Level 3 BLAS.

More recently, driven by modern parallel machines and the desire to solve larger and larger problems, a standard for Batched BLAS (BBLAS) operations has been proposed and has been accepted for publication by ACM ToMS⁶. The development of the BBLAS was done by the Numerical Linear Algebra Group as part of an EU project, NLAFET⁷, in collaboration with Jack Dongarra’s Innovative Computing Laboratory at the University of Tennessee.

¹J. M. Perkel. Ten Codes that Transformed Science. Nature, 589:344–349, January 2021.
²C. L. Lawson, R. J. Hanson, D. Kincaid, and F. T. Krogh. Basic Linear Algebra Subprograms for FORTRAN usage. ACM Trans. Math. Software, 5:308–323, 1979.
³J. J. Dongarra, J. Du Croz, S. Hammarling, and R. J. Hanson. An extended set of FORTRAN Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 14:1–32, 399, 1988.
⁴J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling. A set of Level 3 Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 16:1–28, 1990.
⁵E. Anderson, Z. Bai, C. H. Bischof, S. Blackford, J. Demmel, J. J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. C. Sorensen. LAPACK Users’ Guide. SIAM, Philadelphia, PA, USA, 3rd edition, 1999. ISBN 0-89871-447-8. (http://www.netlib.org/lapack/lug/).
⁶A Set of Batched Basic Linear Algebra Suprograms, ACM Trans. Math. Software. To appear.
⁷Deliverable D7.6, (https://www.nlafet.eu/public-deliverables/).

The biannual SIAM Conference on Computational Science and Engineering (CSE) was conducted virtually between March 1 to 5, 2021. Theo Mary and I organised a two-part minisymposium on recent algorithmic and software advances of mixed precision methods in scientific computing. Below are the links to the slides of the talk.

Minisymposium description: The increasing support of lower precision arithmetics in hardware, such as fp16 and bfloat16, provides new opportunities for high performance scientific computing. However, even though low precision arithmetics can provide significant speed, communication, and energy benefits, their use in scientific computing poses the challenge of preserving the accuracy and stability of the computation. To address this issue, a variety of mixed precision algorithms that combine low and high precisions have emerged.

• Part 1
  • Mixed Precision Low Rank Compression and its Application to BLR Matrix Factorization. Patrick R. Amestoy, Mumps Technologies, France; Olivier Boiteau, EDF, France; Alfredo Buttari, CNRS-IRIT, France; Matthieu Gerest, EDF R&D, France; Fabienne Jezequel, Sorbonne Universités, France; Jean-Yves L’Excellent, Mumps Technologies, France; Theo Mary, Sorbonne Universités and CNRS, France.
  • Tile-Centric Mixed Precision Computations for HPC Applications. Sameh Abdulah, Kadir Akbudak, and Rabab Alomairy, King Abdullah University of Science & Technology (KAUST), Saudi Arabia; George Bosilca and Qinglei Cao, University of Tennessee, Knoxville, U.S.; Jack J. Dongarra, University of Tennessee and Oak Ridge National Laboratory, U.S.; David E. Keyes and Hatem Ltaief, King Abdullah University of Science & Technology (KAUST), Saudi Arabia; Yu Pei, University of Tennessee, U.S.
  • Mixed Precision LU Factorization using GPU Tensor Cores: Improving Performance, Accuracy, and Memory Footprint. Florent Lopez, University of Tennessee, Knoxville, U.S.; Theo Mary, Sorbonne Universités and CNRS, France.
  • Mixed Precision Cholesky-QR Algorithm with Applications. Francoise Tisseur and Srikara Pranesh, University of Manchester, United Kingdom
  • TSQR on Tensor Cores with Error Correction. Hiroyuki Ootomo and Rio Yokota, Tokyo Institute of Technology, Japan.
• Part 2
  • Three-Precision GMRES-Based Iterative Refinement for Least Squares Problems. Srikara Pranesh, University of Manchester, United Kingdom; Erin C. Carson, Charles University, Czech Republic; Nicholas J. Higham, University of Manchester, United Kingdom.
  • How NVIDIA Tensor Cores can Help HPC Scientific Application Unleash the Power of GPUs using Mixed Precision Solvers. Azzam Haidar and Harun Bayraktar, NVIDIA, U.S.; Abeynaya Gnanasekaran, Stanford University, U.S.
  • Iterative Refinement in up to Five Precisions for the Solution of Large Sparse Linear Systems. Patrick R. Amestoy, Mumps Technologies, France; Alfredo Buttari, CNRS-IRIT, France; Nicholas J. Higham, University of Manchester, United Kingdom; Jean-Yves L’Excellent, Mumps Technologies, France; Theo Mary, Sorbonne Universités and CNRS, France; Bastien Vieublé, Universite de Toulouse, France.
  • Compressed Basis GMRES on High Performance GPUs. José I. Aliaga, Universitat Jaume I, Spain; Hartwig Anzt, University of Tennessee, U.S.;  Thomas Gruetzmacher, Karlsruhe Institute of Technology, Germany; Enrique S. Quintana-Orti, Universidad Jaume I, Spain; Andres Tomas, Universidad Politecnica de Valencia, Spain.
  • DGEMM using Tensor Cores. Daichi Mukunoki, RIKEN, Japan; Katsuhisa Ozaki, Shibaura Institute of Technology, Japan; Takeshi Ogita, Tokyo Woman’s Christian University, Japan; Toshiyuki Imamura, RIKEN, Japan.

Professor Nick Higham has been named among the 2020 Association for Computing Machinery (ACM) Fellows, who are recognised for work underpinning contemporary computing.

The accomplishments of the 2020 ACM Fellows have driven innovations that have ushered in significant improvements across many areas of technology, industry, and personal life.

Nick has been recognised for his contributions to numerical linear algebra, numerical stability analysis, and communication of mathematics.

He is among 95 ACM Fellows, representing universities, corporations and research centres around the world, who are celebrated for their wide-ranging and fundamental contributions in areas including artificial intelligence, cloud computing, computer graphics, virtual reality, and more.

The ACM Fellows programme recognises the top 1% of ACM members for their outstanding accomplishments in computing and information technology and/or outstanding service to ACM and the larger computing community. Fellows are nominated by their peers, with nominations reviewed by a distinguished selection committee.

ACM President Gabriele Kotsis said: “The 2020 ACM Fellows have demonstrated excellence across many disciplines of computing. These men and women have made pivotal contributions to technologies that are transforming whole industries, as well as our personal lives. We fully expect that these new ACM Fellows will continue in the vanguard in their respective fields.”

Dr Theo Mary, a CNRS researcher at Sorbonne University (Paris) and a former postdoctoral researcher in the Numerical Linear Algebra Group (2018-2019), has been awarded the SIAG/LA Early Career Prize by the SIAM Activity Group on Linear Algebra.

The SIAM Activity Group on Linear Algebra (SIAG/LA) awards the SIAG/LA Early Career Prize every three years to one post-PhD early career researcher in the field of applicable linear algebra for outstanding contributions to the field within six years of receiving the PhD or equivalent degree as of January 1 of the award year. The selection committee wishes to recognize Theo for his “significant contributions to linear algebra topics, including block low rank methods, software development, probabilistic rounding error analysis, mixed precision arithmetic, and backward error analysis.”

The prize will be awarded at the 2021 SIAM Conference on Applied Linear Algebra (LA21), to be held on a virtual platform May 17-21, 2021.

Professor Nick Higham has been awarded the IMA Gold Medal 2020 by the Institute of Mathematics and its Applications. Institute Gold Medals are awarded every two years in recognition of outstanding contributions to mathematics and its applications.

This is the second such success for a member of the Department of Mathematics: Professor Fritz Ursell won the IMA Gold Medal 1994.

The full prize citation is available here.

Professor Nicholas J. Higham, University of Manchester

Read more

Sixteen publications by members of the NLA group feature among the top 40 most read articles in two SIAM journals, both leading venues for publications in numerical linear algebra. 

The following articles are amongst the 20 most read in the SIAM Journal on Matrix Analysis and Applications:

• Matrix Inverse Trigonometric and Inverse Hyperbolic Functions: Theory and Algorithms by Mary Aprahamian and Nicholas J. Higham
• An Arbitrary Precision Scaling and Squaring Algorithm for the Matrix Exponential by Massimiliano Fasi and Nicholas J. Higham
• Higher Order Fréchet Derivatives of Matrix Functions and the Level-2 Condition Number by Nicholas J. Higham and Samuel D. Relton
• Generalized Rational Krylov Decompositions with an Application to Rational Approximation by Mario Berljafa and Stefan Güttel
• The Structured Condition Number of a Differentiable Map between Matrix Manifolds, with Applications by Bahar Arslan, Vanni Noferini, and Françoise Tisseur
• An Algorithm for the Matrix Lambert $W$ Function by Massimiliano Fasi, Nicholas J. Higham, and Bruno Iannazzo
• The Block Rational Arnoldi Method by Steven Elsworth and Stefan Güttel
• Multiprecision Algorithms for Computing the Matrix Logarithm by Massimiliano Fasi and Nicholas J. Higham

The following articles are amongst the 20 most read in the SIAM Journal on Scientific Computing:

• Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions by Erin Carson and Nicholas J. Higham
• A New Approach to Probabilistic Rounding Error Analysis by Nicholas J. Higham and Theo Mary
• Simulating Low Precision Floating-Point Arithmetic by Nicholas J. Higham and Srikara Pranesh
• A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems by Erin Carson and Nicholas J. Higham
• Shifted Cholesky QR for Computing the QR Factorization of Ill-Conditioned Matrices by Takeshi Fukaya, Ramaseshan Kannan, Yuji Nakatsukasa, Yusaku Yamamoto, and Yuka Yanagisawa
• Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems by Nicholas J. Higham, Srikara Pranesh, and Mawussi Zounon
• A Class of Fast and Accurate Summation Algorithms by Pierre Blanchard, Nicholas J. Higham, and Theo Mary
• Estimating the Largest Elements of a Matrix by Nicholas J. Higham and Samuel D. Relton

The full lists (updated daily) are available here and here. 

Professor Françoise Tisseur has been awarded the London Mathematics Society’s Fröhlich Prize for “her important and highly innovative contributions to the analysis, perturbation theory, and numerical solution of nonlinear eigenvalue problems”.

The Fröhlich Prize is awarded even two years in memory of Albrecht Fröhlich, FRS. The prize is awarded for original and extremely innovative work in any branch of mathematics.

The full prize citation is available here.