Effects of round-to-nearest and stochastic rounding in the numerical
solution of the heat equation in low precision

Motivated by the advent of machine learning, the last few years have seen the
return of hardware-supported low-precision computing. Computations with fewer
digits are faster and more memory and energy efficient, but can be extremely
susceptible to rounding errors. As shown by recent studies into
reduced-precision climate simulations, an application that can largely benefit
from the advantages of low-precision computing is the numerical solution of
partial differential equations (PDEs). However, a careful implementation and
rounding error analysis are required to ensure that sensible results can still
be obtained.
In this paper we study the accumulation of rounding errors in the solution of
the heat equation, a proxy for parabolic PDEs, via Runge-Kutta finite
difference methods using round-to-nearest (RtN) and stochastic rounding (SR).
We demonstrate how to implement the scheme to reduce rounding errors and we
derive \emph{a priori} estimates for local and global rounding errors. Let $u$
be the unit roundoff. While the worst-case local errors are $O(u)$ with respect
to the discretization parameters (mesh size and timestep), the RtN and SR error
behavior is substantially different. In fact, the RtN solution always stagnates
for small enough $\Delta t$, and until stagnation the global error grows like
$O(u\Delta t^{-1})$. In contrast, we show that the leading-order errors
introduced by SR are zero-mean, independent in space and mean-independent in
time, making SR resilient to stagnation and rounding error accumulation. In
fact, we prove that for SR the global rounding errors are only $O(u\Delta
t^{-1/4})$ in 1D and are essentially bounded (up to logarithmic factors) in
higher dimensions.