Lesson 02-Matrices and Linear Algebra

Matrix Creation Methods

Matrix creation is the first step in matrix operations. MATLAB provides a variety of functions to generate basic matrices, special matrices, and matrices with custom structures.

Basic Matrix Generation Functions

Common Functions and Examples:

FunctionDescriptionExample CodeOutput Description
zeros(m,n)Create an m×n all-zero matrixA = zeros(3,4)3 rows and 4 columns, all elements 0
ones(m,n)Create an m×n all-one matrixB = ones(2,2)2 rows and 2 columns, all elements 1
rand(m,n)Create an m×n uniformly distributed random matrix ([0,1))C = rand(3,3)3×3 matrix with random floating-point elements
randn(m,n)Create an m×n standard normal distributed random matrixD = randn(2,5)2×5 matrix with elements following N(0,1)
eye(n)Create an n×n identity matrixI = eye(4)4th-order identity matrix (diagonal 1, others 0)
diag(v)Create a diagonal matrix with vector v on the diagonalv = [1,2,3]; D = diag(v)3rd-order diagonal matrix with diagonal [1,2,3]

Advanced Techniques:

  • Specify data type: Specify matrix type with suffix (e.g., zeros(3,3,'single') generates single-precision matrix).
  • Sparse matrix initialization: sparse(m,n) generates an m×n sparse matrix (initially all zeros, saves memory).

All-zero/all-one matrices:

% Create 3x4 all-zero matrix
A = zeros(3,4);  % 3 rows, 4 columns

% Create 2x2 all-one matrix
B = ones(2);     % Default square matrix

% Specify data type
C = zeros(3,3,'uint8');  % Unsigned 8-bit integer matrix

Identity matrix:

% Generate 4x4 identity matrix
I = eye(4);

% Sparse identity matrix
sparse_I = speye(1000);  % Saves memory

Random matrices:

% Uniformly distributed random matrix [0,1]
R_uniform = rand(3,2);

% Standard normal distributed random matrix
R_normal = randn(2,3);

% Random integers in specified range
R_int = randi([1,10],3,3);  % Random integers from 1 to 10

Special Matrix Construction

In engineering and scientific computing, matrices with specific structures (such as Toeplitz, Hankel, Vandermonde, etc.) are often needed. MATLAB provides dedicated functions for these.

Common Special Matrices and Examples:

FunctionDescriptionExample CodeOutput Description
toeplitz(c,r)Create Toeplitz matrix (constant elements on each anti-diagonal)c = [1,2,3]; r = [1,4]; T = toeplitz(c,r)3×2 matrix, first column [1;2;3], first row [1,4]
hankel(c,r)Create Hankel matrix (constant elements on each diagonal)c = [1,2,3]; r = [3,4]; H = hankel(c,r)3×2 matrix, first column [1;2;3], last row [3,4]
vander(v)Create Vandermonde matrix (row powers increasing)v = [1,2,3]; V = vander(v)3×3 matrix, row i is [v(i)^2, v(i), 1]
hilb(n)Create n-order Hilbert matrix (ill-conditioned)H = hilb(3)3rd-order Hilbert matrix (elements 1/(i+j-1))
pascal(n)Create n-order Pascal matrix (symmetric positive definite)P = pascal(3)3rd-order Pascal matrix (elements are binomial coefficients)

Application Scenarios:

  • Toeplitz matrix: Commonly used in signal processing (e.g., autocorrelation matrix).
  • Hankel matrix: Suitable for system identification (e.g., state-space models).
  • Hilbert matrix: Used to test numerical algorithm stability (ill-conditioned matrix).

Diagonal matrices:

% Create diagonal matrix from vector
v = [1 2 3];
D = diag(v);  % Main diagonal

% Specify diagonal position
D_sup = diag(v,1);  % Super-diagonal

% Extract diagonal elements
diag_D = diag(D);

Tridiagonal matrices:

% Create tridiagonal matrix
e = ones(4,1);
T = spdiags([e 2*e e], -1:1, 4,4);  % Sparse storage
full_T = full(T);  % Convert to full matrix

Matrix Operations and Analysis

Matrix operations and analysis are the core of linear algebra. MATLAB provides efficient functions for computing determinants, inverse matrices, eigenvalues, and eigenvectors.

Determinant and Inverse Matrix

The determinant is used to determine if a matrix is invertible. The inverse matrix is used to solve linear systems Ax=b (x=A⁻¹b).

Key Functions and Notes:

  • det(A): Compute the determinant of matrix A.
  • Note: Determinant values are affected by floating-point precision; suitable only for small matrices (n≤10). Large matrices may lose accuracy.
  • inv(A): Compute the inverse of matrix A (requires A to be nonsingular).
  • Note: Inverse computation for ill-conditioned matrices (e.g., Hilbert) has large errors; use with caution.

Determinant computation:

A = [1 2; 3 4];
det_A = det(A);  % Compute determinant

% Singularity detection
if abs(det_A) < eps
    warning('Matrix is nearly singular!');
end

Matrix inversion:

% Direct inversion (note numerical stability)
A_inv = inv(A);

% More stable approach (recommended)
x = A \ b;  % Equivalent to inv(A)*b but more numerically stable

Condition number analysis:

cond_A = cond(A);  % Condition number
if cond_A > 1e10
    warning('Ill-conditioned matrix; inversion may be inaccurate');
end

Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors describe the intrinsic properties of a matrix, widely used in vibration analysis, data dimensionality reduction (e.g., PCA), etc.

Key Functions and Output:

  • eig(A): Returns eigenvalues (diagonal matrix) and eigenvectors (columns of matrix).
  • Format: [V, D] = eig(A), where D is the diagonal eigenvalue matrix, and columns of V are corresponding eigenvectors.

Eigenvalue computation:

% Compute eigenvalues
eig_vals = eig(A);  % Returns eigenvalue vector

% Eigenvalue decomposition
[V,D] = eig(A);     % V: eigenvector matrix, D: diagonal eigenvalue matrix

% Generalized eigenvalue problem
[A,B] = deal(rand(3),rand(3));
[V,D] = eig(A,B);   % Solve Ax=λBx

Symmetric matrix handling:

% Ensure symmetry
S = (A + A')/2;     % Force symmetrization

% More efficient eigenvalue computation
eig_sym = eig(S,'vector');  % Return only eigenvalues

Solving Linear Systems

Linear systems Ax=b are among the most common problems in science and engineering. MATLAB provides direct methods (e.g., LU decomposition) and iterative methods (e.g., Gauss-Seidel).

Direct Methods

Direct methods solve exactly via matrix factorizations (e.g., LU, QR), suitable for small-to-medium dense matrices (n≤10⁴).

Key Methods and Functions:

  • Backslash operator (\): MATLAB’s default direct solver, automatically selects optimal factorization (LU, Cholesky, etc.).
    • Format: x = A \ b.
  • LU decomposition solving: [L, U, P] = lu(A), then PA=LU, solve x=U∖(L∖(Pb)).

Example: Solving linear system with backslash

A = [3 2 -1; 2 -2 4; -1 0.5 -1];  % Coefficient matrix
b = [1; -2; 0];                   % Right-hand side
x = A \ b;                        % Direct solution
disp(x);  % Output: [1; -2; -2] (verify: A*x ≈ b)

Matrix left division:

A = rand(3,3);
b = rand(3,1);
x = A \ b;  % Most common method

% Performance comparison
tic; x1 = A \ b; t1 = toc;
tic; x2 = inv(A)*b; t2 = toc;
fprintf('Left division is %.2fx faster than inversion\n', t2/t1);

linsolve function:

% Specify matrix properties for speed
opts.POSDEF = true;  % Positive definite matrix
x = linsolve(A,b,opts);

% Sparse matrix dedicated
S = sparse(A);
x_sparse = linsolve(S,b);

Iterative Methods (Large-Scale Problems)

Iterative methods converge gradually, suitable for large-scale sparse matrices (e.g., finite element systems), with low memory usage.

Common Iterative Algorithms and MATLAB Implementations:

AlgorithmSuitable ScenariosMATLAB Function/ImplementationConvergence Conditions
Jacobi iterationDiagonally dominant matricesManual implementation (component update)Diagonally dominant or symmetric positive definite
Gauss-Seidel iterationDiagonally dominant matricesManual implementation (uses latest components)Diagonally dominant or symmetric positive definite
Conjugate Gradient (CG)Symmetric positive definite sparse matricespcg(A, b, tol, maxit)Symmetric positive definite
Generalized Minimal Residual (GMRES)Nonsymmetric sparse matricesgmres(A, b, restart, tol, maxit)Generally nonsingular

Example: Gauss-Seidel iteration

n = 100;  % Matrix size
A = gallery('poisson', sqrt(n));  % Generate sparse Poisson matrix (symmetric positive definite)
b = ones(n, 1);
x0 = zeros(n, 1);  % Initial guess
tol = 1e-6;        % Convergence tolerance
maxit = 1000;      % Maximum iterations

% Gauss-Seidel iteration implementation
x = x0;
for k = 1:maxit
    x_old = x;
    for i = 1:n
        sigma = A(i, [1:i-1, i+1:n]) * x([1:i-1, i+1:n]);
        x(i) = (b(i) - sigma) / A(i, i);
    end
    if norm(x - x_old) < tol
        break;
    end
end
disp(['Iterations: ', num2str(k)]);
disp(['Error: ', num2str(norm(A*x - b))]);

Preconditioned Conjugate Gradient (PCG):

% Create sparse matrix
n = 1000;
A = gallery('poisson',sqrt(n));  % Poisson matrix

% Preconditioner
M = ichol(A);  % Incomplete Cholesky factorization

% Solve
b = rand(n,1);
x = pcg(A,b,1e-6,1000,M);

Matrix Decomposition Techniques

Matrix decomposition factors a complex matrix into products of simpler matrices, core to solving systems, data compression, and dimensionality reduction.

LU Decomposition

LU decomposition factors A into lower triangular L and upper triangular U (possibly with permutation P), i.e., PA=LU.

Application Scenarios:

  • Solving linear systems (Ax=b → LUx=Pb, solved by forward/backward substitution).
  • Determinant computation (det(A)=det(P)det(L)det(U)=±∏diag(U)).

Basic LU decomposition:

[L,U] = lu(A);      % Basic decomposition
[L,U,P] = lu(A);    % With permutation matrix

% Solve system
b = rand(3,1);
y = L \ (P*b);      % Forward substitution
x = U \ y;          % Backward substitution

Sparse LU decomposition:

S = sparse(A);
[L,U,P,Q] = lu(S);  % Sparse matrix dedicated

QR Decomposition

QR decomposition factors A into orthogonal Q and upper triangular R, i.e., A=QR.

Application Scenarios:

  • Least squares (Ax≈b → Rx=Qᵀb).
  • Matrix orthogonalization (e.g., Gram-Schmidt process).

Column-pivoted QR decomposition:

[Q,R,P] = qr(A);    % P is permutation matrix

% Least squares solution
b = rand(3,1);
x = R \ (Q'*b);     % Solve Rx=Q'b

Economy QR decomposition:

[Q,R] = qr(A,0);    % Compute only necessary parts

SVD Decomposition

SVD decomposes A into A=UΣVᵀ, where U and V are orthogonal, and Σ is diagonal (singular values in descending order).

Application Scenarios:

  • Data dimensionality reduction (e.g., PCA, keep top k singular values).
  • Matrix approximation (truncated SVD, keep top k terms).
  • Singular value analysis (e.g., image denoising, recommendation systems).

Full SVD:

[U,S,V] = svd(A);   % Full decomposition

% Pseudoinverse computation
A_pinv = V * pinv(S) * U';

% Dimensionality reduction application
k = 2;              % Keep top 2 singular values
Uk = U(:,1:k);
Sk = S(1:k,1:k);
Vk = V(:,1:k);
A_approx = Uk * Sk * Vk';

Truncated SVD (large data):

% Randomized SVD (for large-scale matrices)
[U,S,V] = svds(A,2);  % Compute only top 2 singular values

Sparse Matrix Techniques

Sparse matrices have mostly zero elements (e.g., finite element stiffness matrices, social network adjacency matrices). MATLAB provides dedicated storage and optimized operations.

Functions and Examples:

  • sparse(i,j,v): Create sparse matrix, i and j are row/column indices of nonzero elements, v are values.
  • full(S): Convert sparse matrix to dense (only for small matrices).

Sparse Matrix Creation

Direct construction:

% Coordinate format (COO)
i = [1 2 3];        % Row indices
j = [2 3 1];        % Column indices
s = [4 5 6];        % Values
S = sparse(i,j,s,3,3);

% Convert to full matrix
full_S = full(S);

Diagonal sparse matrix:

% Create banded sparse matrix
diags = [1 2 3];    % Diagonal values
offsets = [-1 0 1]; % Diagonal offsets
S = spdiags(diags,offsets,4,4);

Sparse Matrix Operations

Efficient operation rules:

% Sparse matrix multiplication
S1 = sparse(rand(1000));
S2 = sparse(rand(1000));
S_product = S1 * S2;  % Automatically maintains sparsity

% Element-wise operations
S_abs = abs(S);       % Maintains sparse structure

Sparse matrix storage formats:

% Convert storage format
CSR_S = full(S);      % Convert back to full (demo only)
CSC_S = sparse(CSR_S,'Column');  % Column-compressed storage

% View storage info
whos S

Sparse Matrix Application Cases

Sparse matrix operations (e.g., multiplication, inversion, eigenvalues) are optimized in MATLAB, avoiding storage of zero elements and improving efficiency.

Key Operations:

  • Sparse matrix multiplication: C = A*B (computes only nonzero products).
  • Sparse matrix inversion: inv(S) (only for sparse diagonal or triangular matrices).
  • Sparse eigenvalues: eigs(S, k) (computes top k largest eigenvalues, suitable for large sparse matrices).

Large-scale linear system solving:

% Create 10000x10000 sparse matrix
n = 10000;
A = gallery('poisson',sqrt(n));  % Poisson problem matrix

% Sparse solving
b = rand(n,1);
x = A \ b;  % Automatically uses sparse solver

% Compare memory usage
full_A = full(A);  % Warning: consumes ~800MB memory

Graph theory application:

% Adjacency matrix representing graph
n_nodes = 100;
adj_matrix = sprand(n_nodes,n_nodes,0.1);  % 10% nonzero elements
adj_matrix = triu(adj_matrix) + triu(adj_matrix)';  % Symmetrize

% Compute connected components
graph_obj = graph(adj_matrix);
components = conncomp(graph_obj);

Performance Optimization Techniques

Memory Management Strategies

  • Preallocate arrays: Avoid dynamic array growth (e.g., x = []; x(end+1) = 5), use zeros or ones to preallocate.
  • Avoid unnecessary copying: MATLAB uses copy-on-write; function arguments pass references (unless modified).
  • Use appropriate data types: Single precision (single) saves 50% memory vs. double (double), suitable for lower precision needs.

Preallocate memory:

% Incorrect way (dynamic growth)
A = [];
for i = 1:1000
    A(i,:) = rand(1,1000);  % Reallocates memory each iteration
end

% Correct way (preallocation)
A = zeros(1000,1000);
for i = 1:1000
    A(i,:) = rand(1,1000);  % Directly fills preallocated memory
end

Data type selection:

% Default double precision (8 bytes)
A_double = rand(1000);

% Single precision (4 bytes)
A_single = single(rand(1000));

% Logical type (1 byte)
A_logical = rand(1000) > 0.5;

Computation Acceleration Techniques

  • Vectorization: Avoid loops, use matrix operations instead of element-wise (e.g., A.*B instead of for i=1:n, C(i)=A(i)*B(i); end).
  • Parallel computing: Use Parallel Computing Toolbox (PCT) with parfor for parallel loops or gpuArray for multicore CPU/GPU.
  • Built-in function optimization: Prefer MATLAB built-ins (e.g., sum, mean), highly optimized in C/Fortran.

Vectorization replacing loops:

% Inefficient loop
result = zeros(1000);
for i = 1:1000
    result(i) = sum(A(i,:));
end

% Efficient vectorized
result = sum(A,2);  % Sum along rows

Parallel computing:

% Start parallel pool
parpool('local',4);  % Use 4 workers

% Parallel computation
parfor i = 1:100
    results(i) = computefun(i);
end

% Close parallel pool
delete(gcp);

GPU Acceleration

MATLAB’s gpuArray supports data migration to NVIDIA GPUs, leveraging CUDA cores for accelerated matrix operations (e.g., multiplication, SVD).

Key Steps:

  1. Detect GPU: gpuDevice to view available GPU devices.
  2. Data migration: A_gpu = gpuArray(A) moves CPU matrix to GPU.
  3. GPU computation: Call GPU versions (e.g., gpuArray.rand, svd(A_gpu)).
  4. Result retrieval: A_cpu = gather(A_gpu) moves results back to CPU.

GPU array conversion:

% Check GPU availability
if gpuDeviceCount > 0
    % Transfer matrix to GPU
    A_gpu = gpuArray(A);
    
    % Perform computation on GPU
    b_gpu = A_gpu * rand(1000,1,'gpuArray');
    
    % Transfer result back to CPU
    b = gather(b_gpu);
else
    warning('No GPU device detected');
end

GPU suitable scenarios:

  • Large-scale matrix operations (>1000×1000)
  • Parallelizable element-wise operations
  • Repeated identical computations
Share your love