SVD Calculator

What is Singular Value Decomposition (SVD)?

The Singular Value Decomposition is the most general and arguably the most important matrix decomposition in linear algebra. Every matrix — square or rectangular, real or complex — has an SVD. It factorizes any m×n matrix A into the product of three special matrices:

where:

• U is an m×m orthogonal matrix whose columns are the left singular vectors (eigenvectors of AAᵀ)
• Σ (Sigma) is an m×n diagonal matrix containing the singular values σ₁ ≥ σ₂ ≥ … ≥ σ_min(m,n) ≥ 0 along its diagonal
• Vᵀ is the transpose of an n×n orthogonal matrix whose columns are the right singular vectors (eigenvectors of AᵀA)

Because U and V are orthogonal (UᵀU = I, VᵀV = I), the SVD reveals the fundamental geometric action of a matrix: any linear transformation can be decomposed into a rotation (Vᵀ), a scaling along coordinate axes (Σ), and a second rotation (U).

Geometric Interpretation

The SVD gives the clearest geometric picture of what a matrix does to space. When you multiply a vector by A = UΣVᵀ, three things happen in sequence:

1. Rotation (or reflection) by Vᵀ — the input vector is rotated into a coordinate system aligned with the right singular vectors.
2. Scaling by Σ — each coordinate axis is independently stretched (or shrunk) by the corresponding singular value σ_i. The unit sphere is mapped to an axis-aligned ellipsoid.
3. Rotation (or reflection) by U — the scaled vector is rotated into the final output coordinate system defined by the left singular vectors.

This means every matrix maps the unit sphere to an ellipsoid. The singular values are the semi-axis lengths of that ellipsoid, and the singular vectors give the axis directions. This interpretation is why SVD is the natural tool for understanding the geometry of linear maps.

How SVD is Computed

Computing the SVD is closely related to eigendecomposition. The standard approach works as follows:

1. Form AᵀA — this is an n×n symmetric positive semi-definite matrix.
2. Compute eigenvalues and eigenvectors of AᵀA — the eigenvalues λ_i are the squares of the singular values (σ_i = √λ_i), and the eigenvectors form the columns of V.
3. Compute U — for each nonzero singular value, the corresponding left singular vector is u_i = Av_i / σ_i.
4. Assemble the decomposition — arrange singular values in descending order in Σ, and order the columns of U and V accordingly.

In practice, production implementations (LAPACK, NumPy, MATLAB) use more numerically stable algorithms. The typical approach first reduces A to bidiagonal form via Householder reflections, then applies an iterative algorithm (the Golub-Kahan bidiagonalization or divide-and-conquer method) to extract the singular values and vectors from the bidiagonal matrix.

The Eckart-Young Theorem

One of the most powerful results about SVD is the Eckart-Young-Mirsky theorem: the best rank-k approximation of a matrix (in both the Frobenius norm and the spectral norm) is obtained by keeping only the k largest singular values and zeroing out the rest.

Formally, if A = UΣVᵀ and we define A_k = U_kΣ_kV_kᵀ using only the top k singular values, then for any matrix B with rank(B) ≤ k:

This theorem is the mathematical foundation for data compression, noise reduction, and dimensionality reduction. It guarantees that truncated SVD is optimal — no other rank-k matrix is closer to the original.

Applications of SVD

SVD is ubiquitous across science and engineering. Here are the major application areas:

• Principal Component Analysis (PCA) — PCA is equivalent to computing the SVD of the centered data matrix. The right singular vectors are the principal components, the singular values encode the variance explained by each component, and the left singular vectors give the scores.
• Image Compression — an image can be stored as a matrix of pixel intensities. A rank-k SVD approximation keeps only k singular values, dramatically reducing storage while preserving the dominant visual features. A 1000×1000 image normally requires 1 million values; a rank-50 approximation needs only 100,500.
• Recommendation Systems — collaborative filtering (as used by Netflix, Spotify, and Amazon) relies on low-rank matrix factorization. SVD of the user-item interaction matrix reveals latent factors that capture user preferences and item characteristics.
• Pseudoinverse and Least Squares — the Moore-Penrose pseudoinverse A⁺ is computed directly from the SVD as VΣ⁺Uᵀ, where Σ⁺ inverts only the nonzero singular values. This provides the minimum-norm least squares solution to overdetermined or underdetermined systems.
• Noise Reduction — in noisy data, the large singular values correspond to signal and the small ones to noise. Truncating the small singular values (a technique called spectral filtering) removes noise while preserving the underlying structure.
• Latent Semantic Analysis (LSA) — in natural language processing, SVD of the term-document matrix reveals latent semantic dimensions. Documents and terms are mapped to a shared low-dimensional space where semantic similarity is captured by geometric proximity.
• Numerical Rank Determination — the number of singular values above a threshold reveals the numerical rank of a matrix. This is more robust than row reduction because singular values degrade gracefully in the presence of rounding errors.
• Matrix Condition Number — the ratio σ_max / σ_min is the condition number of the matrix, which measures how sensitive solutions are to perturbations in the input data.

Computational Complexity

Computing the full SVD of an m×n matrix (with m ≥ n) requires O(mn²) floating-point operations. The bidiagonalization step costs O(mn²) operations, and the iterative phase for extracting singular values from the bidiagonal form costs O(n²) per iteration with rapid convergence. For large sparse matrices where only the top k singular values are needed, iterative methods like Lanczos bidiagonalization or randomized SVD reduce the cost to approximately O(mnk), making it feasible to decompose matrices with millions of rows and columns.

SVD vs Other Decompositions

When should you choose SVD over another decomposition?

• SVD vs LU — LU is faster for solving square linear systems (O(n³/3) vs O(mn²)), but SVD works for any matrix shape and provides rank information. Use LU when you just need to solve Ax = b for a square invertible A; use SVD when the matrix may be rectangular, singular, or ill-conditioned.
• SVD vs QR — QR is cheaper and sufficient for least squares problems on well-conditioned matrices. SVD is more robust for rank-deficient or ill-conditioned problems because it reveals the exact rank and provides the minimum-norm solution.
• SVD vs Eigendecomposition — eigendecomposition only applies to square matrices and may not exist (defective matrices). SVD always exists for any matrix. For symmetric positive definite matrices, the two are essentially equivalent (singular values equal eigenvalues). For general matrices, SVD is the preferred choice.
• SVD vs Cholesky — Cholesky is restricted to symmetric positive definite matrices and is the fastest decomposition for that class. SVD applies universally but at higher computational cost.

In summary, SVD is the most versatile decomposition. When in doubt, SVD is the safe and informative choice. The trade-off is computational cost, so prefer specialized decompositions when their assumptions are met and you do not need the extra information SVD provides.