Lesson 04-Numerical Computation and Simulation

Numerical Computation Methods

Numerical computation methods are core tools for solving problems in engineering, science, and mathematics, covering areas such as linear algebra, differential equations, optimization, signal and image processing. This document systematically analyzes the core methods of numerical computation, combining theoretical derivations with practical MATLAB examples, to help readers master complete skills from basic calculations to complex applications.

Linear Algebra Computation

Linear algebra is the foundation of numerical computation, with the core being matrix operations and decompositions, widely applied in solving systems of equations, data dimensionality reduction, system modeling, and other scenarios.

Matrix Operation Core Functions:

Matrix operations include addition, subtraction, multiplication, transpose, inverse, determinant, etc., which are the core operations of linear algebra.

Key operations and MATLAB implementations:

  • Matrix Multiplication: C = A * B (matrix multiplication rule: m×n multiplied by n×p yields m×p).
  • Matrix Transpose: A' (conjugate transpose) or transpose(A) (non-conjugate transpose).
  • Determinant: det(A) (only for square matrices; accurate for small matrices, limited by precision for large matrices).
  • Inverse Matrix: inv(A) (requires non-singular matrix; large errors for ill-conditioned matrices).

Example: Solving linear system Ax=b

A = [3 2 -1; 2 -2 4; -1 0.5 -1];  % Coefficient matrix
b = [1; -2; 0];                   % Right-hand side vector
x = A \ b;                        % Direct solution (LU decomposition)
disp(x);  % Output: [1; -2; -2] (Verify: A*x ≈ b)
% Matrix inversion (recommended to use backslash operator)
A = rand(3); b = rand(3,1);
x1 = inv(A)*b;          % Traditional method (not recommended)
x2 = A\b;               % Optimal method (numerically stable)

% Eigenvalue decomposition
[V,D] = eig(A);         % V: eigenvectors, D: diagonal matrix of eigenvalues
eigvals = eig(A);       % Return only eigenvalues

% Linear system solving
x = A\b;                % Most commonly used method
x = linsolve(A,b);      % Specify matrix properties for faster computation

Matrix Decomposition Techniques:

Matrix decomposition breaks down complex matrices into products of simpler factors, reducing computational complexity, applied in solving linear systems, eigenvalue analysis, etc.

Common Decomposition Methods:

Decomposition TypeFormulaApplication ScenariosMATLAB Function
LU DecompositionA=LU (L lower triangular, U upper triangular)Solving linear systems, computing determinantslu(A)
QR DecompositionA=QR (Q orthogonal, R upper triangular)Least squares, matrix orthogonalizationqr(A)
SVD DecompositionA=UΣV^T (U,V orthogonal, Σ diagonal of singular values)Data dimensionality reduction, matrix approximationsvd(A)
CholeskyA=LL^T (A symmetric positive definite)Solving symmetric positive definite systemschol(A)

SVD decomposition for image compression

% Read image and convert to grayscale
img = imread('cameraman.tif');
img_gray = rgb2gray(img);
img_double = im2double(img_gray);

% SVD decomposition
[U, S, V] = svd(img_double);

% Retain top 50 singular values (compression)
k = 50;
img_compressed = U(:, 1:k) * S(1:k, 1:k) * V(:, 1:k)';

% Display original and compressed images
figure;
subplot(1,2,1); imshow(img); title('Original Image');
subplot(1,2,2); imshow(img_compressed); title('SVD Compression (k=50)');
% LU decomposition (suitable for solving linear systems)
[L,U] = lu(A);          % Basic decomposition
[L,U,P] = lu(A);        % With permutation matrix (more stable)

% QR decomposition (least squares problems)
[Q,R] = qr(A);          % Column-pivoted QR decomposition
[Q,R] = qr(A,0);        % Economy-size QR decomposition

% SVD decomposition (dimensionality reduction/pseudo-inverse)
[U,S,V] = svd(A);       % Full SVD
A_pinv = V*pinv(S)*U';  % Compute pseudo-inverse

Matrix Condition Number and Stability Analysis:

% Compute matrix condition number
A = [1e-10 0; 0 1];
cond_A = cond(A);  % Condition number is extremely large, matrix is ill-conditioned

% Ill-conditioned matrix solving example
b = [1; 1];
x_naive = A \ b;          % Direct solution
x_stable = inv(A)' * b;   % Even less stable method

% Regularization method (Tikhonov regularization)
lambda = 1e-8;            % Regularization parameter
I = eye(size(A));
x_reg = (A' * A + lambda * I) \ (A' * b);  % Regularized solution

Sparse Matrix Optimization:

% Create sparse matrix
n = 10000;
A_sparse = sprandn(n, n, 0.001);  % 0.1% non-zero elements
b_sparse = randn(n,1);

% Compare storage space
whos A_sparse b_sparse

% Sparse matrix solving
x_sparse = A_sparse \ b_sparse;   % Automatically uses sparse solver

% Convert to full matrix (use cautiously)
A_full = full(A_sparse);          % Extremely memory-intensive

Nonlinear Equation Solving

Nonlinear equations f(x)=0 are widely present in engineering (e.g., circuit analysis) and science (e.g., dynamical systems), requiring iterative or direct methods for solution.

Key Methods and MATLAB Implementations:

  • Bisection Method: Halves the interval to approximate the root (stable but slow convergence).

Condition: f(a)f(b)<0 (root exists in interval).

function root = bisection(f, a, b, tol)
	if f(a)*f(b) >= 0
		error('Function values at interval endpoints have same sign, no root');
	end
	while (b - a) > tol
		c = (a + b)/2;
		if f(c) == 0
			root = c;
			return;
		elseif f(a)*f(c) < 0
			b = c;
		else
			a = c;
		end
	end
	root = (a + b)/2;
end

% Example: Solve x^3 - 2x - 5 = 0 in [2,3]
f = @(x) x.^3 - 2*x - 5;
root = bisection(f, 2, 3, 1e-6);
disp(root);  % Output: 2.0946 (true root approx. 2.09455)
  • Newton’s Method: Uses derivative to accelerate convergence (depends on initial guess, may diverge).

Formula: x_{n+1} = x_n – f(x_n)/f'(x_n).

function root = newton(f, df, x0, tol, maxit)
	x = x0;
	for i = 1:maxit
		fx = f(x);
		if abs(fx) < tol
			root = x;
			return;
		end
		dfx = df(x);
		if dfx == 0
			error('Derivative is zero, Newton's method fails');
		end
		x = x - fx/dfx;
	end
	root = x;
end

% Example: Solve x^3 - 2x - 5 = 0 (initial guess x0=2)
f = @(x) x.^3 - 2*x - 5;
df = @(x) 3*x.^2 - 2;
root = newton(f, df, 2, 1e-6, 100);
disp(root);  % Output: 2.0946 (fast convergence)

Multivariable Nonlinear System Solving

Multivariable system F(x)=0 (x∈R^n) requires extended Newton’s method or quasi-Newton methods (e.g., Broyden’s method).

MATLAB Implementation (Extended Newton’s Method):
function x = newton_multivar(F, J, x0, tol, maxit)
    x = x0;
    for i = 1:maxit
        Fx = F(x);
        if norm(Fx) < tol
            return;
        end
        Jx = J(x);
        delta = -Jx \ Fx;  % Solve linear system J*delta = -F
        x = x + delta;
    end
    error('Maximum iterations reached, no convergence');
end

% Example: Solve system x^2 + y^2 = 1, x + y = 0.5
F = @(x) [x(1)^2 + x(2)^2 - 1; x(1) + x(2) - 0.5];
J = @(x) [2*x(1), 2*x(2); 1, 1];  % Jacobian matrix
x0 = [0; 0];  % Initial guess
x = newton_multivar(F, J, x0, 1e-6, 100);
disp(x);  % Output: [0.8090; -0.3090] (Verify: 0.809²+(-0.309)²≈1, sum=0.5)

Scalar Equation Solving:

% fzero for single-variable equation
f = @(x) x^3 - 2*x - 5;
x0 = 2;                 % Initial guess
root = fzero(f, x0);    % Solve

% Interval search method
root = fzero(f, [1,3]); % Solve in interval [1,3]

% Handling complex roots
f_complex = @(x) x^2 + 1;
root_complex = fzero(f_complex, 1i);  % Start from imaginary initial guess

System Solving:

% fsolve for nonlinear systems
fun = @(x) [x(1)^2 + x(2)^2 - 1;  % x² + y² = 1
            x(1) - x(2)^2];       % x = y²
x0 = [0.5; 0.5];              % Initial guess
solution = fsolve(fun, x0);   % Solve

% Set options (higher precision)
options = optimoptions('fsolve','Display','iter','TolFun',1e-10);
solution = fsolve(fun, x0, options);

Multivariable Nonlinear Systems:

% Complex nonlinear system
fun = @(x) [x(1)*cos(x(2)) - x(3)^2; 
            x(1)^2 + x(2)^2 - 4; 
            exp(x(1)) + x(2) - x(3)];

% Initial guess
x0 = [1; 1; 1];

% Use fsolve
options = optimoptions('fsolve','Display','iter','Algorithm','trust-region-dogleg');
solution = fsolve(fun, x0, options);

% Provide Jacobian (accelerates convergence)
jac = @(x) [cos(x(2)), -x(1)*sin(x(2)), -2*x(3);
            2*x(1),     2*x(2),      0;
            exp(x(1)),  1,           -1];
solution = fsolve(fun, x0, options, jac);

Parametric Equation Solving:

% Nonlinear equation with parameter
param = 2;  % Adjustable parameter
fun_param = @(x) x^3 - param*x - 2;

% Solve for different parameter values
solutions = arrayfun(@(p) fzero(@(x) x^3 - p*x - 2, 1), param_values);

Numerical Differentiation and Integration

Numerical differentiation and integration are core methods for handling experimental data (discrete points), used to compute derivatives, integrals, and other continuous features.

Numerical Differentiation: Approximating Derivatives

Numerical differentiation approximates continuous function derivatives via finite differences, commonly using forward difference, central difference.

Formulas and MATLAB Implementations:

  • Forward Difference: f′(xi) ≈ (f(x{i+1}) − f(x_i))/h (first-order accuracy).
  • Central Difference: f′(xi) ≈ (f(x{i+1}) − f(x_{i-1}))/(2h) (second-order accuracy).
% Generate test function (f(x) = sin(x), true derivative cos(x))
x = linspace(0, pi, 100);
h = x(2) - x(1);  % Step size
f = sin(x);

% Forward difference
df_forward = (f(2:end) - f(1:end-1))/h;

% Central difference (endpoints use forward/backward)
df_center = zeros(size(f));
df_center(2:end-1) = (f(3:end) - f(1:end-2))/(2*h);
df_center(1) = (f(2) - f(1))/h;       % Forward difference
df_center(end) = (f(end) - f(end-1))/h;  % Backward difference

% Plot comparison
figure;
plot(x, cos(x), 'r--', 'LineWidth', 2);  % True derivative
hold on;
plot(x(1:end-1), df_forward, 'bo', 'MarkerSize', 4);  % Forward difference
plot(x, df_center, 'g*', 'MarkerSize', 4);  % Central difference
legend('True Derivative', 'Forward Difference', 'Central Difference');
xlabel('x'); ylabel('f''(x)');
title('Numerical Differentiation Comparison');

Numerical Integration Methods:

Numerical integration approximates definite integrals ∫_a^b f(x)dx via summation, common methods include rectangle, trapezoidal, Simpson’s.

Formulas and MATLAB Implementations:

  • Rectangle Method: ∫a^b f(x)dx ≈ h ∑{i=1}^n f(x_i) (first-order accuracy).
  • Trapezoidal Rule: ∫a^b f(x)dx ≈ h/2 [f(x_1) + 2∑{i=2}^{n-1} f(x_i) + f(x_n)] (second-order accuracy).
  • Simpson’s Rule: ∫a^b f(x)dx ≈ h/3 [f(x_1) + 4∑{i=2,4,…}^{n-2} f(xi) + 2∑{i=3,5,…}^{n-1} f(x_i) + f(x_n)] (fourth-order accuracy).
% Trapezoidal rule (trapz)
x = linspace(0, pi, 100);
y = sin(x);
area_trapz = trapz(x, y);   % Trapezoidal integration

% Adaptive integration (integral)
f = @(x) sin(x);
area_integral = integral(f, 0, pi);  % Adaptive Simpson's method

% Highly oscillatory function integration
f_osc = @(x) sin(100*x);
area_osc = integral(f_osc, 0, pi, 'AbsTol',1e-12,'RelTol',1e-12);

Numerical Differentiation:

% Finite difference (gradient)
x = linspace(0, 2*pi, 100);
y = sin(x);
dy_dx = gradient(y, x);     % First derivative

% Higher-order derivative (central difference)
h = x(2)-x(1);
d2y_dx2 = (y(3:end) - 2*y(2:end-1) + y(1:end-2))/h^2;

Higher-Order Numerical Differentiation:

% Second-order derivative central difference
h = 1e-5;
f = @(x) sin(x);
x0 = 1;
d2f_dx2 = (f(x0+h) - 2*f(x0) + f(x0-h))/h^2;

% Fourth-order difference formula
d4f_dx4 = (-f(x0+2*h) + 4*f(x0+h) - 6*f(x0) + 4*f(x0-h) - f(x0-2*h))/h^4;

Adaptive Integration Strategies:

% Complex integrand
f_complex = @(x) sin(1./x) .* exp(-x.^2);

% Compare different tolerances
q1 = integral(f_complex, 0.01, 1, 'AbsTol',1e-6);
q2 = integral(f_complex, 0.01, 1, 'AbsTol',1e-10);

% Handling singularity in interval
f_singular = @(x) 1./sqrt(x);
q_singular = integral(f_singular, 0, 1, 'Waypoints',[0.01]);  % Avoid singularity

Ordinary Differential Equation Solving

Ordinary Differential Equation Types and Solvers

  • Explicit ODE: y′=f(t,y) (e.g., y′=y, solution y=C e^t).
  • Stiff ODE: Jacobian eigenvalues differ greatly (e.g., y′=−1000y+3000−2000e^{-t}), requires implicit methods.

Common Numerical Methods:

MethodFormulaApplicable ScenariosMATLAB Function
Euler Methody_{n+1} = y_n + h f(t_n, y_n)Simple problems (low accuracy)ode1
Runge-Kuttay_{n+1} = y_n + h/6 (k1 + 2k2 + 2k3 + k4) (RK4)General non-stiff problems (high accuracy)ode45 (adaptive RK4)
Implicit Eulery{n+1} = y_n + h f(t{n+1}, y_{n+1})Stiff problemsode15i

ODE Solver Selection Guide:

SolverApplicable Problem TypeFeatures
ode45Non-stiff medium-scale problemsDefault choice, Dormand-Prince method
ode23Non-stiff simple problemsBogacki-Shampine method
ode113Non-stiff high-precision problemsAdams-Bashforth-Moulton method
ode15sStiff/semi-stiff problemsVariable-order multistep method
ode23sStiff problemsModified Rosenbrock method

Example: Solve y′=y−t²+1 using RK4 (initial condition y(0)=0.5)

% Define ODE function
f = @(t, y) y - t.^2 + 1;

% Initial conditions and time span
t0 = 0;
y0 = 0.5;
tspan = [0, 2];

% Solve using ode45 (adaptive step size)
[t, y] = ode45(f, tspan, y0);

% Analytical solution (y(t) = t² + 2t + 1 - 0.5 e^t)
analytic = t.^2 + 2*t + 1 - 0.5*exp(t);

% Plot comparison
figure;
plot(t, y, 'b-', 'LineWidth', 2); hold on;
plot(t, analytic, 'r--', 'LineWidth', 2);
legend('Numerical Solution (ode45)', 'Analytical Solution');
xlabel('t'); ylabel('y(t)');
title('ODE Numerical vs Analytical Solution Comparison');

ODE Solving Examples:

% Define ODE function
f = @(t,y) -2*y + sin(t);   % dy/dt = -2y + sin(t)

% Initial conditions and time span
tspan = [0 5];
y0 = 1;

% Solve using ode45
[t,y] = ode45(f, tspan, y0);

% Plot results
plot(t,y);
xlabel('Time'); ylabel('Solution y');

Stiff Equation Solving:

% Stiff ODE example (Van der Pol equation)
mu = 1000;
f = @(t,y) [y(2); mu*(1-y(1)^2)*y(2)-y(1)];

% Solve using ode15s
options = odeset('RelTol',1e-6,'AbsTol',[1e-8 1e-8]);
[t,y] = ode15s(f, [0 3000], [2;0], options);

Stiff System Solving:

% Stiff chemical reaction model
function dydt = chemical_ode(t,y)
    k1 = 1e7; k2 = 1e4; k3 = 2e7;
    dydt = [-k1*y(1)+k3*y(2)*y(3);
            k1*y(1)-k2*y(2)^2-k3*y(2)*y(3);
            k2*y(2)^2];
end

% Initial conditions
y0 = [1;0;0];
tspan = [0 1e-3];  % Extremely short time span

% Solve using ode15s
options = odeset('RelTol',1e-6,'Stats','on');
[t,y] = ode15s(@chemical_ode, tspan, y0, options);

Event Detection Application:

% Spring-mass system (detect zero crossing)
function [value,isterminal,direction] = event_func(t,y)
    value = y(1);          % Trigger when displacement is zero
    isterminal = 1;        % Terminate simulation
    direction = 0;         % Any direction
end

% Main ODE function
function dydt = spring_mass(t,y)
    m = 1; c = 0.2; k = 1;
    dydt = [y(2); (-c*y(2)-k*y(1))/m];
end

% Solve settings
options = odeset('Events',@event_func);
[t,y,te,ye,ie] = ode45(@spring_mass, [0 10], [1;0], options);

Optimization Algorithms

Optimization algorithms are used to find minimum or maximum values of objective functions, widely applied in machine learning, engineering design, etc.

Unconstrained Optimization

Unconstrained optimization problem min f(x) (x∈R^n), common methods include gradient descent, Newton’s method.

Key methods and MATLAB implementations:

  • Gradient Descent: Iterate along negative gradient direction (x_{k+1} = x_k − α ∇f(x_k)), requires choosing learning rate α.
  • Newton’s Method: Uses Hessian matrix (second derivatives) to accelerate convergence (x_{k+1} = x_k – H^{-1} ∇f(x_k)).
% Test function (Rosenbrock function, minimum at (1,1))
rosenbrock = @(x) (1 - x(1))^2 + 100*(x(2) - x(1)^2)^2;

% Gradient descent (learning rate 0.001, max iterations 10000)
x0 = [-1.2; 1];
alpha = 0.001;
maxit = 10000;
x = x0;
for i = 1:maxit
    grad = gradient(rosenbrock, x);  % Compute gradient
    x = x - alpha*grad;
end
disp(['Gradient Descent Result: ', num2str(x)]);  % Output: [1; 1] (approximate)

% Newton's method (requires Hessian matrix)
function [x, iter] = newton_opt(f, grad_f, hess_f, x0, tol, maxit)
    x = x0;
    for iter = 1:maxit
        g = grad_f(x);
        if norm(g) < tol
            return;
        end
        H = hess_f(x);
        delta = -H \ g;
        x = x + delta;
    end
end

% Define gradient and Hessian
grad_rosenbrock = @(x) [-2*(1 - x(1)) - 400*x(1)*(x(2) - x(1)^2); 200*(x(2) - x(1)^2)];
hess_rosenbrock = @(x) [2 - 400*x(2) + 1200*x(1)^2, -400*x(1); -400*x(1), 200];

% Solve using Newton's method
x_newton = newton_opt(rosenbrock, grad_rosenbrock, hess_rosenbrock, x0, 1e-6, 100);
disp(['Newton Method Result: ', num2str(x_newton)]);  % Output: [1; 1] (fast convergence)

Constrained Optimization (fmincon):

Constrained optimization problem min f(x) subject to g(x)≤0, h(x)=0, common methods include Lagrange multipliers, penalty functions.

% Objective function
fun = @(x) exp(x(1))*(4*x(1)^2 + 2*x(2)^2 + 4*x(1)*x(2) + 2*x(2) + 1);

% Initial point
x0 = [-1;1];

% Linear constraints A*x ≤ b
A = [1 1];
b = 0;

% Nonlinear constraints
nonlcon = @(x) deal([], x(1)*x(2) - x(1) - x(2) + 1.5);

% Optimization options
options = optimoptions('fmincon','Display','iter');

% Solve
[x,fval] = fmincon(fun,x0,A,b,[],[],[],[],nonlcon,options);

Global Optimization:

% Genetic algorithm (ga)
fitnessfcn = @(x) 100*(x(1)^2 - x(2))^2 + (1-x(1))^2;
nvars = 2;
lb = [-2 -2];
ub = [2 2];
[x_ga,fval_ga] = ga(fitnessfcn,nvars,[],[],[],[],lb,ub);

% Particle swarm optimization (particleswarm)
options = optimoptions('particleswarm','SwarmSize',100);
[x_pso,fval_pso] = particleswarm(fitnessfcn,nvars,lb,ub,options);

Mixed-Integer Programming:

% Mixed-integer linear programming example
f = [-3; -2];  % Objective coefficients
intcon = [1 2]; % Variables 1 and 2 are integers
A = [1 1; 4 2];
b = [5; 10];
lb = [0; 0];
ub = [inf; inf];

% Solve
[x,fval] = intlinprog(f,intcon,A,b,[],[],lb,ub);

Multi-Objective Optimization:

% Multi-objective Pareto front
f1 = @(x) x(1)^2 + x(2)^2;
f2 = @(x) (x(1)-1)^2 + x(2)^2;
fun = @(x) [f1(x); f2(x)];

% Solve using gamultiobj
options = optimoptions('gamultiobj','PopulationSize',100);
[x_pareto,fval_pareto] = gamultiobj(fun,2,[],[],[],[],[0 0],[1 1],[],options);

% Plot Pareto front
plot(fval_pareto(:,1),fval_pareto(:,2),'o');
xlabel('f1'); ylabel('f2');

Membership Required

You must be a member to access this content.

View Membership Levels

Already a member? Log in here
Share your love