Numerical Linear Algebra Group

Professor Nick Higham has been awarded the 2022 Hans Schneider Prize by the International Linear Algebra Society (ILAS). The Hans Schneider Prize in Linear Algebra is awarded every three years by ILAS for research, contributions, and achievements at the highest level of Linear Algebra.

Nick is cited for his fundamental contributions in the analysis of a wide range of numerical linear algebra problems and matrix functions. He will present his lecture at the 25th ILAS Conference in Madrid, Spain, June 5-9, 2023.

Professor Nicholas J. Higham, The University of Manchester

Read more

The SIAM Annual Meeting 2021 was held virtually, July 19 – 23, 2021. Nick Higham and I organised a two-part minisymposium “Computational Frontiers in Numerical Linear Algebra” that addressed recent algorithmic and software advances in numerical linear algebra. Links to slides from some of the talks are given below.

Minisymposium description: Numerical linear algebra (NLA) is fundamental to many applications in scientific computing. Therefore developing fast algorithms for various NLA problems is crucial to enhance our ability to tackle bigger scientific challenges. Furthermore, NLA software is used as a black box in various applications and hence theoretical guarantees on the computed results are important. Algorithmic development in NLA needs to work in tandem with the ongoing advances in computer hardware. This minisymposium will give a broad overview of various theoretical, algorithmic and software ideas that are being pursued to accelerate NLA problems.

• Part 1
  • When Floating-Point Error Matters: the Hazards and Challenges of Low-Precision Computation. Erin C. Carson, Charles University, Czech Republic. Abstract. Slides.
  • Randomization for Solving Large Systems of Linear Equations. Laura Grigori, Oleg Balabanov, and Matthias Beaupere, Inria Paris, France. Abstract.
  • Mixed Precision Algorithms for Pushing the Performance Limits of Modern HPC Architectures. Hartwig Anzt, University of Tennessee, U.S. Fritz Goebel, Thomas Gruetzmacher, and Terry Cojean, Karlsruhe Institute of Technology, Germany. Andres Tomas and Enrique S. Quintana-Orti, Universitat Politècnica de València, Spain. Abstract. Slides.
  • HeFFTe: FFT Computations Towards Exascale. Alan F. Ayala, University of Tennessee, U.S. Miroslav Stoyanov, Oak Ridge National Laboratory, U.S. Stanimire Tomov and Sebastien Cayrols, University of Tennessee, Knoxville, U.S. Jack J. Dongarra, University of Tennessee and Oak Ridge National Laboratory, U.S. Abstract. Slides.
• Part 2
  • Replacing Pivoting in Distributed Gaussian Elimination with Randomized Transforms. Neil Lindquist and Piotr Luszczek, University of Tennessee, U.S. Jack J. Dongarra, University of Tennessee and Oak Ridge National Laboratory, U.S. Abstract. Slides.
  • Data-Aware Mixed Precision Algorithms. Theo Mary, Sorbonne Universités and CNRS, France. Abstract. Slides.
  • Random Matrices Generating Large Growth in LU Factorization with Pivoting. Srikara Pranesh and Nicholas J. Higham, The University of Manchester, United Kingdom; Desmond John Higham, University of Edinburgh, United Kingdom. Abstract. Slides.
  • Mixed Precision Randomized SVD. Michael P. Connolly, Nicholas J. Higham, and Srikara Pranesh, The University of Manchester, United Kingdom. Abstract.

Members of the Numerical Linear Algebra Group will be giving six presentations at the upcoming SIAM Annual Meeting (AN21). They are also organizing the two-part minisymposiums Computational Frontiers in Numerical Linear Algebra (Part 1, Part 2) and Bohemian Matrices and Applications (Part 1, Part 2).

The conference will be held virtually, from 19th July to 23rd July, 2021.

Tuesday 20 July
18:30 – 20:30 Massimiliano Fasi
Cpfloat: A C Library for Emulating Low-Precision Arithmetic (poster)

Wednesday 21 July
14:30 – 15:30 Nick Higham
Past President’s Address – Mixed Precision Numerical Linear Algebra with More Realistic Error Bounds

Thursday 22 July
16:45 – 17:10 Massimiliano Fasi
Determinants of Normalized Bohemian Upper Hessenberg Matrices

Friday 23 July
10:15 – 10:40 Theo Mary
Data-Aware Mixed Precision Algorithms
10:45 – 11:10 Srikara Pranesh 
Random Matrices Generating Large Growth in LU Factorization with Pivoting
11:15 – 11:40 Michael Connolly
Mixed Precision Randomized SVD

More information on AN21 is available here.

The Society for Industrial and Applied Mathematics (SIAM) has chosen Nick Higham, Royal Society Research Professor and Richardson Professor of Applied Mathematics, as the 2021 recipient of the George Pólya Prize for Mathematical Exposition.

The prize, which is awarded for expository work that communicates mathematics effectively, is named after George Pólya, the Hungarian mathematician who wrote the million-selling book How to Solve It, first published in 1945.

“Pólya was a brilliant researcher, teacher, and expositor of mathematics,” said Higham. “It is an honor to receive this SIAM prize named after him, especially as most of my research and expository writing has been published with SIAM.”

Higham was cited for “the crisp clarity, elegance and accessibility of his mathematical and popular exposition on a broad range of topics in applied mathematics.”

The award ceremony will be held at the 2021 SIAM Annual Meeting, scheduled to be held in a virtual format during July 19-23, 2021. See also the SIAM announcement of the award.

Professor Nicholas J. Higham, University of Manchester. Photo: Rob Whitrow

By Sven Hammarling.

An article in a recent issue of the journal Nature has the title “Ten Codes that Transformed Science”¹ .

Two of the codes are purely numerical, namely the fast Fourier transform (FFT) and the Basic Linear Algebra Subprograms (BLAS). The BLAS evolved in three stages, the Level 1 BLAS² for scalar and vector operations, the Level 2 BLAS³ for matrix-vector operations and the Level 3 BLAS⁴ for matrix-matrix operations. The first draft proposal for the Level 1 BLAS appeared in the ACM Signum Newsletter in 1973, so the project as a whole spanned about seventeen years.

Two members of the Numerical Linear Algebra Group were directly involved in the development of the BLAS. Jack Dongarra wrote a number of the Level 1 BLAS routines, was involved in testing the routines on an IBM 370/195 and provided several efficient implementations of the routines, all of which was acknowledged in the published paper. Both Jack Dongarra and Sven Hammarling were authors of the Level 2 and 3 BLAS, together with Jeremy Du Croz and Richard Hanson for the Level 2 BLAS, and Jeremy Du Croz and Iain Duff for the Level 3 BLAS.

It should be said that the BLAS were very much a community project, with input from many people. The BLAS, especially the Level 3 BLAS, provided the basic high performance operations, particularly for the block partitioned methods, used by the LAPACK⁵ project, which started in 1987, before the actual publication of the Level 3 BLAS.

More recently, driven by modern parallel machines and the desire to solve larger and larger problems, a standard for Batched BLAS (BBLAS) operations has been proposed and has been accepted for publication by ACM ToMS⁶. The development of the BBLAS was done by the Numerical Linear Algebra Group as part of an EU project, NLAFET⁷, in collaboration with Jack Dongarra’s Innovative Computing Laboratory at the University of Tennessee.

¹J. M. Perkel. Ten Codes that Transformed Science. Nature, 589:344–349, January 2021.
²C. L. Lawson, R. J. Hanson, D. Kincaid, and F. T. Krogh. Basic Linear Algebra Subprograms for FORTRAN usage. ACM Trans. Math. Software, 5:308–323, 1979.
³J. J. Dongarra, J. Du Croz, S. Hammarling, and R. J. Hanson. An extended set of FORTRAN Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 14:1–32, 399, 1988.
⁴J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling. A set of Level 3 Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 16:1–28, 1990.
⁵E. Anderson, Z. Bai, C. H. Bischof, S. Blackford, J. Demmel, J. J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. C. Sorensen. LAPACK Users’ Guide. SIAM, Philadelphia, PA, USA, 3rd edition, 1999. ISBN 0-89871-447-8. (http://www.netlib.org/lapack/lug/).
⁶A Set of Batched Basic Linear Algebra Suprograms, ACM Trans. Math. Software. To appear.
⁷Deliverable D7.6, (https://www.nlafet.eu/public-deliverables/).

The biannual SIAM Conference on Computational Science and Engineering (CSE) was conducted virtually between March 1 to 5, 2021. Theo Mary and I organised a two-part minisymposium on recent algorithmic and software advances of mixed precision methods in scientific computing. Below are the links to the slides of the talk.

Minisymposium description: The increasing support of lower precision arithmetics in hardware, such as fp16 and bfloat16, provides new opportunities for high performance scientific computing. However, even though low precision arithmetics can provide significant speed, communication, and energy benefits, their use in scientific computing poses the challenge of preserving the accuracy and stability of the computation. To address this issue, a variety of mixed precision algorithms that combine low and high precisions have emerged.

• Part 1
  • Mixed Precision Low Rank Compression and its Application to BLR Matrix Factorization. Patrick R. Amestoy, Mumps Technologies, France; Olivier Boiteau, EDF, France; Alfredo Buttari, CNRS-IRIT, France; Matthieu Gerest, EDF R&D, France; Fabienne Jezequel, Sorbonne Universités, France; Jean-Yves L’Excellent, Mumps Technologies, France; Theo Mary, Sorbonne Universités and CNRS, France.
  • Tile-Centric Mixed Precision Computations for HPC Applications. Sameh Abdulah, Kadir Akbudak, and Rabab Alomairy, King Abdullah University of Science & Technology (KAUST), Saudi Arabia; George Bosilca and Qinglei Cao, University of Tennessee, Knoxville, U.S.; Jack J. Dongarra, University of Tennessee and Oak Ridge National Laboratory, U.S.; David E. Keyes and Hatem Ltaief, King Abdullah University of Science & Technology (KAUST), Saudi Arabia; Yu Pei, University of Tennessee, U.S.
  • Mixed Precision LU Factorization using GPU Tensor Cores: Improving Performance, Accuracy, and Memory Footprint. Florent Lopez, University of Tennessee, Knoxville, U.S.; Theo Mary, Sorbonne Universités and CNRS, France.
  • Mixed Precision Cholesky-QR Algorithm with Applications. Francoise Tisseur and Srikara Pranesh, University of Manchester, United Kingdom
  • TSQR on Tensor Cores with Error Correction. Hiroyuki Ootomo and Rio Yokota, Tokyo Institute of Technology, Japan.
• Part 2
  • Three-Precision GMRES-Based Iterative Refinement for Least Squares Problems. Srikara Pranesh, University of Manchester, United Kingdom; Erin C. Carson, Charles University, Czech Republic; Nicholas J. Higham, University of Manchester, United Kingdom.
  • How NVIDIA Tensor Cores can Help HPC Scientific Application Unleash the Power of GPUs using Mixed Precision Solvers. Azzam Haidar and Harun Bayraktar, NVIDIA, U.S.; Abeynaya Gnanasekaran, Stanford University, U.S.
  • Iterative Refinement in up to Five Precisions for the Solution of Large Sparse Linear Systems. Patrick R. Amestoy, Mumps Technologies, France; Alfredo Buttari, CNRS-IRIT, France; Nicholas J. Higham, University of Manchester, United Kingdom; Jean-Yves L’Excellent, Mumps Technologies, France; Theo Mary, Sorbonne Universités and CNRS, France; Bastien Vieublé, Universite de Toulouse, France.
  • Compressed Basis GMRES on High Performance GPUs. José I. Aliaga, Universitat Jaume I, Spain; Hartwig Anzt, University of Tennessee, U.S.;  Thomas Gruetzmacher, Karlsruhe Institute of Technology, Germany; Enrique S. Quintana-Orti, Universidad Jaume I, Spain; Andres Tomas, Universidad Politecnica de Valencia, Spain.
  • DGEMM using Tensor Cores. Daichi Mukunoki, RIKEN, Japan; Katsuhisa Ozaki, Shibaura Institute of Technology, Japan; Takeshi Ogita, Tokyo Woman’s Christian University, Japan; Toshiyuki Imamura, RIKEN, Japan.

Professor Nick Higham has been named among the 2020 Association for Computing Machinery (ACM) Fellows, who are recognised for work underpinning contemporary computing.

The accomplishments of the 2020 ACM Fellows have driven innovations that have ushered in significant improvements across many areas of technology, industry, and personal life.

Nick has been recognised for his contributions to numerical linear algebra, numerical stability analysis, and communication of mathematics.

He is among 95 ACM Fellows, representing universities, corporations and research centres around the world, who are celebrated for their wide-ranging and fundamental contributions in areas including artificial intelligence, cloud computing, computer graphics, virtual reality, and more.

The ACM Fellows programme recognises the top 1% of ACM members for their outstanding accomplishments in computing and information technology and/or outstanding service to ACM and the larger computing community. Fellows are nominated by their peers, with nominations reviewed by a distinguished selection committee.

ACM President Gabriele Kotsis said: “The 2020 ACM Fellows have demonstrated excellence across many disciplines of computing. These men and women have made pivotal contributions to technologies that are transforming whole industries, as well as our personal lives. We fully expect that these new ACM Fellows will continue in the vanguard in their respective fields.”

Dr Theo Mary, a CNRS researcher at Sorbonne University (Paris) and a former postdoctoral researcher in the Numerical Linear Algebra Group (2018-2019), has been awarded the SIAG/LA Early Career Prize by the SIAM Activity Group on Linear Algebra.

The SIAM Activity Group on Linear Algebra (SIAG/LA) awards the SIAG/LA Early Career Prize every three years to one post-PhD early career researcher in the field of applicable linear algebra for outstanding contributions to the field within six years of receiving the PhD or equivalent degree as of January 1 of the award year. The selection committee wishes to recognize Theo for his “significant contributions to linear algebra topics, including block low rank methods, software development, probabilistic rounding error analysis, mixed precision arithmetic, and backward error analysis.”

The prize will be awarded at the 2021 SIAM Conference on Applied Linear Algebra (LA21), to be held on a virtual platform May 17-21, 2021.

Professor Nick Higham has been awarded the IMA Gold Medal 2020 by the Institute of Mathematics and its Applications. Institute Gold Medals are awarded every two years in recognition of outstanding contributions to mathematics and its applications.

This is the second such success for a member of the Department of Mathematics: Professor Fritz Ursell won the IMA Gold Medal 1994.

The full prize citation is available here.

Professor Nicholas J. Higham, University of Manchester

Read more