The first edition of this handbook appeared in 2003, which is a bit more than twenty years ago. That first edition contained a collection of monographs written by experts of that time on subjects of parameter estimation and inverse problem analysis that were heavily used over the prior 20 years. This second edition retains many of the classic contributions from that first edition and builds upon them with contributions from a cadre of practitioners who have developed prominent techniques and approaches for solving inverse problems over the last 20 years. Additionally, important emerging topics regarding application of machine learning and artificial intelligence approaches are also included in this second edition.
Chapter 1
Inverse Problems and Parameter Estimation:
Integration of Measurements and Analysis
James V. Beck, Michigan State University, East Lansing, MI
Keith A. Woodbury, The University of Alabama, Tuscaloosa, AL
Inverse Techniques are a suite of methods which, when fully embraced, promise to provide for better experiments and improved understanding of physical processes. This chapter provides an overview of the general procedure and concepts related to identification of parameters or functions by inverse techniques. A discussion of errors and their implication for an appropriate function for minimization in inverse procedures is presented, and two methods for achieving this minimization are discussed. Some sequential concepts for parameter estimation are presented, along with a discussion of residuals and confidence intervals. Experiment design and optimization are reviewed, and a discussion of residuals and their relation to model-building is presented.
Chapter 2
Matrix Analysis for Parameter Estimation
James V. Beck, Michigan State University
Kenneth J. Arnold, Michigan State University
This chapter extends parameter estimation techniques to cases involving more than two parameters by employing matrix formulations. Matrix notation provides compact representation, simplifies manipulations, and promotes broader generalization of estimation methods. The primary focus is on linear parameter estimation, though nonlinear estimation approaches are also addressed in a concluding section. Several estimation techniques are presented, including ordinary least squares, weighted least squares, maximum likelihood, and maximum a posteriori, with the latter incorporating prior information to enable sequential procedures. Modeling considerations are emphasized through the use of sensitivity coefficients, residual analysis, and statistical tools such as the F-test for model development. Additional topics include the calculation of confidence regions, the treatment of correlated errors, and the analysis of biased errors. The assumptions underpinning linear estimation are clarified, with particular attention to requirements for linear maximum likelihood methods, including errorless independent variables and consistent covariance structures of measurement errors.
Chapter 3
Sequential Function Specification Method
Forooza Samadi, The University of Alabama
This chapter outlines the methodology for solving function estimation problems using the Sequential Function Specification Method (SFSM). The discussion focuses on heat transfer problems, with the primary application being estimation of boundary disturbances in the parabolic heat conduction equation. Parabolic problems in other fields can be treated similarly.
An historical evolution of the method for linear problems originating with the work of Stolz is included. Two possible alternative approaches for SFSM based on assumption of piecewise constant surface disturbance are given based on different boundary conditions choices. Illustration of other polynomial assumptions for the assumed function are given. A discussion of nonlinear problems, including estimation of unknown heat transfer coefficients, is included.
Chapter 4
Tikhonov Regularization and Optimal Regularization
Keith A. Woodbury, The University of Alabama
This chapter presents Tikhonov regularization as a stabilization technique for solving the inverse heat conduction problem (IHCP). Beginning with its historical development by A. N. Tikhonov, the chapter introduces the whole-domain approach while formulating the IHCP in discrete linear form with piecewise constant heat flux parameterization. The ill-posedness by the diffusive nature of heat conduction, which smooths boundary effects, motivates the introduction of zeroth-, first-, and second-order Tikhonov regularization. Extensions to nonlinear problems are also described. The choice of the regularization parameter is addressed through optimization strategies such as error minimization, the Morozov discrepancy principle, generalized cross-validation, and the L-curve method.
Tikhonov regularization provides a practical means of controlling instability in inverse heat conduction problems. Although the approach inevitably introduces bias, careful selection of the regularization parameter allows a reasonable balance between stability and accuracy. The techniques outlined here show how such a balance can be achieved.
Chapter 5
Filter Coefficients Approach for Solving Inverse Heat Conduction Problems
Hamidreza Najafi, Florida Institute of Technology
Accurate, real-time heat flux estimation is critical in a wide range of applications, from aerospace engineering to manufacturing. Direct measurement of surface conditions is often impractical; instead, sub-surface temperature measurements can be used to estimate surface heat flux. This estimation process is mathematically ill-posed and is referred to as the inverse heat conduction problem (IHCP).
This chapter focuses on solving IHCPs using the filter-coefficient formulation of the Tikhonov regularization (TR) method. The filter-coefficient approach provides fast, stable, and accurate solutions for IHCPs across varying levels of complexity. The chapter begins by outlining the development of filter matrices and coefficients for the basic TR method in a one-dimensional domain. It then extends the discussion to more complex cases, including IHCPs in multi-layer domains, problems with temperature-dependent material properties, and scenarios involving moving boundaries.
Practical examples such as heat flux sensors and the thermal protection systems of space vehicles are presented to illustrate the applicability of the filter-coefficient method in diverse IHCP scenarios. Finally, the chapter explores artificial neural network (ANN) approaches inspired by the filter-coefficient method, discussing two network architectures and their use in representing filter coefficients.
Chapter 6
Adjoint Method Primer
Yvon Jarny, Ecole polytechnique de l’université de Nantes
This chapter presents a tutorial primer on the “adjoint method”. This method consists of two distinct parts: minimization of a function through gradient-based methods (primarily the conjugate gradient method) and computation of the gradient needed for this minimization through application of Lagrange multiplier techniques. The tutorial considers a progression of target problems for application of the adjoint method beginning with simple algebraic problems and progressing through ordinary differential equation problems to partial differential equation problems. As well, a range of parameter estimation and inverse problems associated with these applications are considered.
Chapter 7
The Iterative Regularization Technique based on the
Conjugate Gradient Method with Adjoint Problem Formulation
Yvon Jarny, Ecole Polytechnique, Université de Nantes
Helcio R. B. Orlande, Federal University of Rio de Janeiro, Brazil
This chapter presents basic concepts and applications of the conjugate gradient method for the solution of inverse problems based on the minimization of an objective functional. This method was developed in 1952 by Hestenes and Stiefel for the iterative solution of linear systems of algebraic equations. Later, the method was extended for the minimization of non-quadratic functionals, in finite and infinite dimensional spaces. Starting in the 1970’s, the conjugate gradient method with adjoint problem formulation for the solution of inverse problems was advanced by Dean Oleg Alifanov and his group of the Moscow Aviation Institute, notably Eugene Artyukhin and Aleksey Nenarokomov. Especial attention is given in this chapter to inverse problems of function estimation, where the gradient of the objective functional is obtained through the solution of an adjoint problem. As opposed to Tikhonov’s regularization, the method presented in this chapter does not require a penalization of the objective functional to regularize the inverse problem. The conjugate gradient method with adjoint problem formulation belongs to the class of iterative regularization methods, where stable solutions are obtained for the inverse problem by suitably selecting a stopping criterion based on Morozov’s Discrepancy Principle for the iterative procedure. As applied to function estimation problems, the method does not require a projection of the unknown function onto a space of finite dimension, spanned by a set of linearly independent basis functions. Mathematical derivations required for the implementation of the method are usually performed with the assumption that the unknown function belongs to the Hilbert space of square integrable functions. The method is robust and provides accurate and stable solutions for inverse problems of practical interest, as evidenced by the vast literature on the subject.
Chapter 8
The Effect of Correlations and Uncertain Parameters on the Efficiency of Estimating and the Precision of Estimated Parameters
Ashley F. Emery, University of Washington
This chapter examines the effects of correlated data and parameters with uncertainty on estimation of parameters in the context of a one-dimensional heat conduction problem, emphasizing both the determination of parameter values and quantification of their precision. Criteria for effective estimators, including unbiasedness and efficiency, are introduced. The influence of uncertainties in temperature measurements, sensor locations, convective coefficients, and thermophysical properties is analyzed through error propagation methods. Expressions for the variance of estimated parameters are derived, accounting for correlated and uncorrelated errors. The treatment highlights the role of measurement correlations arising from shared dependencies, illustrating their impact on the precision and reliability of parameter estimates.
Chapter 9
Estimation of Parameters or Functions After Analysis of Experimental Uncertainties
Philippe Le Masson, IRDL, UMR CNRS 6027, Université Bretagne Sud
Morgan Dal, PIMM, UMR CNRS 8006, ENSAM
Thomas Pierre, IRDL, UMR CNRS 6027
In this chapter, we first recall the sources of uncertainties found in the inverse problems by focusing on the uncertainties connected to measurements by thermocouples. This approach allows underlining the phenomena involved as well as the analytical models in permanent and in transitory regimes developed in the literature. Although fundamental, this analysis remains limited to complex cases. Numerical models are therefore implemented to highlight the errors made during measurements with different types of thermocouples and the different implementation methods for applications linked to highly transient phenomena.
Chapter 10
Statistical Inference and Bayesian Analysis
Kyle Daun, University of Waterloo
Inverse problems are inherently ill-posed, often lacking unique or stable solutions due to insufficient information in the data. Deterministic approaches mitigate this deficit through regularization, but typically yield a single solution and risk bias when prior information is overemphasized. A statistical inference framework, grounded in Bayes’ theorem, provides a more comprehensive characterization of solution spaces by incorporating prior knowledge, quantifying uncertainty, and visualizing the range of plausible solutions. This approach also enables principled treatment of uncertain model parameters, guides experimental design to enhance data informativeness, and supports statistically robust model selection.
Chapter 11
Machine Learning and AI for Inverse Problems
Shima Hajimirza, Stevens Institute of Technology
Inverse problems arise across science and engineering when hidden parameters or structures are inferred from indirect, noisy data, with applications ranging from medical imaging and geophysics to structural health monitoring. These problems are typically ill-posed, lacking uniqueness or stability, and require regularization or Bayesian methods to produce meaningful solutions. Classical approaches impose assumptions such as smoothness or piecewise constancy, but these priors may not capture real-world complexity.
Recent advances in machine learning, particularly deep learning, have enabled data-driven solutions that approximate inverse mappings, learn surrogate forward models, and embed implicit priors from training data. In medical imaging, for example, learning-based reconstructions can outperform analytic and variational methods by reducing noise and artifacts. The integration of rigorous inverse theory with flexible data-driven models defines a new frontier, offering both theoretical insights and practical methodologies for solving ill-posed problems.
Chapter 12
Mollification and space marching
Diego A. Murio, University of Cincinnati
The mollification method is a regularization technique that restores stability to ill-posed problems by re-establishing continuity with respect to data. Beyond its role in inverse analysis, it functions as an efficient smoothing algorithm for noisy data and as a stabilizer for unstable numerical schemes in well-posed parabolic and hyperbolic problems. Originating in the work of Manselli and Miller (1980) and extended through Murio’s studies of the inverse heat conduction problem and related space-marching algorithms, the method has since been refined through advances such as generalized cross-validation for automatic parameter selection. This chapter develops the fundamental properties of mollification and illustrates applications ranging from one- to two-dimensional problems, with further extensions to stable numerical differentiation and phase-change heat conduction problems.
Chapter 13
Inverse Heat Conduction Using Monte Carlo Method
A. Haji-Sheikh, The University of Texas at Arlington
Monte Carlo techniques are rooted in early twentieth-century developments of statistical approaches to differential and integral equations. These methods represent powerful tools in science and engineering for both direct and inverse problem solving. By simulating large ensembles of random walks, the method enables statistical estimation of solutions to physical problems such as heat conduction with specified boundary and initial conditions. This article provides a brief overview of the classical Monte Carlo technique before focusing on its application to inverse heat conduction problems. Emphasis is placed on how analyzing the trajectories of random walks offers valuable insight into hidden details of inverse methodologies, highlighting the method’s utility as a viable and intuitive framework for addressing complex inverse problems.
Chapter 14
Optimal Experiment Design to Solve Inverse Heat Transfer Problems
Aleksey V. Nenarokomov, Moscow State Aviation Institute
This chapter addresses the role of optimal experiment design in solving inverse heat transfer problems for the estimation of thermophysical properties and boundary conditions from transient temperature data. While inverse methods allow experimentation under realistic operating conditions, their ill-posedness requires careful mathematical treatment and appropriate experimental planning to ensure accurate solutions. Numerical simulations demonstrate that estimation errors depend strongly on experimental models, particularly sensor placement and boundary conditions, motivating the application of optimal design theory. The methodology combines statistical analysis and constrained optimization, with the Fisher information matrix serving as a basis for sensitivity analysis and the D-optimum criterion guiding sensor configuration. Algorithms for optimal measurement placement in one- and two-dimensional heat transfer problems are presented and validated, along with extensions to lumped-parameter thermal systems.