Plain-language explanation.
Numerical methods are algorithms for solving mathematical problems that cannot be solved exactly in closed form — finding roots of equations, integrating functions, solving differential equations, and fitting curves to data. They are the computational backbone of science, engineering, and data science.
Core concepts and standard treatment.
Core numerical methods covers sources of error (round-off error vs truncation error, floating-point arithmetic — IEEE 754, machine epsilon), root finding (bisection method, Newton-Raphson method — quadratic convergence, secant method), interpolation and polynomial approximation (Lagrange interpolation, Newton's divided differences, cubic splines), numerical integration / quadrature (Trapezoidal rule, Simpson's rule, Gaussian quadrature, adaptive integration), numerical linear algebra (Gaussian elimination with partial pivoting, LU factorisation, iterative methods — Jacobi, Gauss-Seidel), and numerical ODEs (Euler method, Runge-Kutta methods — RK4; stability and stiffness).
Deeper theory, debates and edge cases.
Advanced numerical methods covers numerical PDEs (finite difference — Crank-Nicolson for heat equation; finite element method — Galerkin formulation, basis functions, assembly; finite volume method — conservation laws, Godunov schemes for hyperbolic PDEs), iterative solvers for large sparse systems (conjugate gradient, GMRES, multigrid methods — algebraic multigrid — amg), numerical optimisation (gradient descent and variants — Adam, L-BFGS; quasi-Newton methods; interior point methods), and Monte Carlo methods (importance sampling, variance reduction techniques — antithetic variables, control variates; Markov chain Monte Carlo — Metropolis-Hastings, Gibbs sampling).
How it is applied in practice.
At the computational scientist, engineer, and ML practitioner level, numerical methods are the foundation of high-performance computing: scientific simulation (CFD — Navier-Stokes solvers; structural FEA; plasma physics — particle-in-cell codes); GPU-accelerated scientific computing (CUDA — NVIDIA; JAX — automatic differentiation + JIT; PyTorch tensor operations); machine learning training (automatic differentiation — backpropagation as numerical reverse-mode AD; optimiser design — adaptive learning rate methods; numerical stability — gradient clipping, mixed precision training FP16/BF16); and quantitative finance (Monte Carlo option pricing — barrier options, American options with Longstaff-Schwartz LSM).