To numerical analysts and other applied mathematicians Jacobians and Hessians
are matrices, i.e. rectangular arrays of numbers or algebraic expressions.
Possibly taking account of their sparsity such arrays are frequently passed
into library routines for performing various computational tasks.
\\
\\
A central goal of an activity called automatic differentiation has been the
accumulation of all nonzero entries from elementary partial derivatives
according to some variant of the chainrule. The elementary partials arise
in the user-supplied procedure for evaluating the underlying vector- or
scalar-valued function at a given argument.
\\
\\
We observe here that in this process a certain kind of structure that we
call "Jacobian scarcity" might be lost. This loss will make the subsequent
calculation of Jacobian vector-products unnecessarily expensive.
Instead we advocate the representation of the Jacobian as a linear computational
graph of minimal complexity. Many theoretical and practical questions remain unresolved.