# Singular Value Decomposition

A singular value decomposition (SVD) is powerful for analyzing matrices and for analyzing linear mappings. Applying the SVD transforms an *M x N* real matrix *A *into the following form:

where *U* is a *M x N* matrix of which column vectors are unit vectors and are orthogonal to each other and *V* is an *N x N* orthonormal matrix such that

Here, *u _{i} (i =* 1,2, ...,M) is a unit M-vector and

*ujuj =*0 if

*i ф j,*and

*v*

_{i}(i =1,2 , , M) is a unit *N*-vector and v *j Vj = 0* if *i ф j.* It should be noted that

{v_{i}|i = 1,2 , ..., *N}* is an orthonormal basis of the N-dimensional space. E in (2.52) is an *N* x *N* diagonal matrix,

where the scalars, *o _{i} (i =* 1,2 , ..., N), are called

*singular values.*In the following equations, it is assumed that the singular values are in decreasing order: ст

_{1}> o

_{2}> •••> on .

Let *y* denote an M-vector generated from an N-vector by a linear mapping such that

where A is a *M x N* matrix. Applying the SVD to A, the range of the linear mapping and of its zero-space can be derived, as will be described below. Substituting A = *U*E *V ^{T}* produces the following equation:

The last two factors, *V ^{T}x,* are transforming the coordinates of

*x*using the orthonormal basis {v

_{i}|i = 1,2 , ..., N}, as described in the Sect.2.2.2.1. Let the new coordinates be denoted by a N-vector,

*z =*[zi, z

_{2}, ...,

*z*]

_{N}^{T}= V

^{T}x. Now, the linear mapping shown in (2.56) is represented as

Substituting E = diag(o1,o_{2}, ..., on) and *U =* [u_{1}|u_{2}|... |u_{N}], (2.57) can be rewritten as follows:

Let a set of indexes that indicates the singular values of zeros be denoted by Z *= *{ j|j e {1,2 ,..., *N}*, *=* {jj e {1,2 , ..., *N},* oj- *ф* 0}, where Z U N = {1,2 , ..., *N} *and *Z* П *N **=* ф. Then, the M-vectors, *uj,* that correspond to the nonzero singular values, *j e* N, determine the domain of the linear map shown in (2.58): for any input vector, x, the mapped vector,y, that is in the subspace spanned by {uj|j e N}. The N-vectors, *v _{k}*, that correspond to the zero-singular values,

*k e*

*Z*, determine the zero-space of A: the mapped vector, y, is always zero if the input vector, x, is in a subspace that is spanned by the N-vectors that correspond to the nonzero singular values, that is,

*x e*span(V

_{Z}) where

*V*{vl e Z}. This is because, in (2.58),

_{Z}=*a*= 0 for

_{;}*i e*

*Z*and

*z*0 for

_{;}= vfx =*i e*

*N*.

*
*