1 - Electrical and Computer Engineering



Condition number

From Wikipedia, the free encyclopedia

Jump to: navigation, search

In the numerical analysis, the condition number associated with a problem is a measure of that problem's amenability to digital computation, that is, how numerically well-conditioned the problem is. A problem with a low condition number is said to be well-conditioned, while a problem with a high condition number is said to be ill-conditioned.

[edit] Matrices

For example, the condition number associated with the linear equation Ax = b gives a bound on how inaccurate the solution x will be after approximate solution. Note that this is before the effects of round-off error are taken into account; conditioning is a property of the matrix, not the algorithm or floating point accuracy of the computer used to solve the corresponding system. In particular, one should think of the condition number as being (very roughly) the rate at which the solution, x, will change with respect to a change in b. Thus, if the condition number is large, even a small error in b may cause a large error in x. On the other hand, if the condition number is small then the error in x will not be much bigger than the error in b.

The condition number is defined more precisely to be the maximum ratio of the relative error in x divided by the relative error in b.

Let e be the error in b. Assuming that A is a square matrix, the error in the solution A−1b is A−1e. The ratio of the relative error in the solution to the relative error in b is

[pic]

This is easily transformed to

[pic]

The maximum value (for nonzero b and e) is easily seen to be the product of the two operator norms:

[pic]

The same definition is used for any consistent norm. This number arises so often in numerical linear algebra that it is given a name, the condition number of a matrix.

Of course, this definition depends on the choice of norm.

• If [pic]is the ℓ2 norm then

[pic]where σmax(A) and σmin(A) are maximal and minimal singular values of A respectively. Hence

• If A is normal then

[pic]where λmax(A) and λmin(A) are maximal and minimal (by moduli) eigenvalues of A respectively

• If A is unitary then

[pic]

• If [pic]is [pic]norm and A is lower triangular non-singular (i.e., [pic]) then

[pic]

[edit] Other contexts

Condition numbers for singular-value decompositions, polynomial root finding, eigenvalue and many other problems may be defined.

Generally, if a numerical problem is well-posed, it can be expressed as a function ƒ mapping its data, which is an m-tuple of real numbers x, into its solution, an n-tuple of real numbers ƒ(x).

Its condition number is then defined to be the maximum value of the ratio of the relative errors in the solution to the relative error in the data, over the problem domain:

[pic]

where ε is some reasonably small value in the variation of data for the problem.

If ƒ is also differentiable, this is approximately

[pic]

And the condition number of the inverse of ƒ at ƒ(x) is approximately

[pic]

[edit] External links

Matrix norm

From Wikipedia, the free encyclopedia

Jump to: navigation, search

In mathematics, a matrix norm is a natural extension of the notion of a vector norm to matrices.

|Contents |

|[hide] |

|1 Definition |

|2 Induced norm |

|3 "Entrywise" norms |

|3.1 Frobenius norm |

|3.2 Max norm |

|4 Schatten norms |

|5 Consistent norms |

|6 Equivalence of norms |

|6.1 Examples of norm equivalence |

|7 References |

[pic][edit] Definition

In what follows, K will denote the field of real or complex numbers. Let [pic]denote the vector space containing all matrices with m rows and n columns with entries in K.

A matrix norm is a vector norm on [pic]. That is, if [pic]denotes the norm of the matrix A, then,

• [pic]if [pic]and [pic]iff A = 0

• [pic]for all α in K and all matrices A in [pic]

• [pic]for all matrices A and B in [pic]

Additionally, in the case of square matrices (thus, m = n), some (but not all) matrix norms satisfy the following condition, which is related to the fact that matrices are more than just vectors:

• [pic]for all matrices A and B in [pic]

A matrix norm that satisfies this additional property is called a sub-multiplicative norm (in some books, the terminology matrix norm is used only for those norms which are sub-multiplicative). The set of all n-by-n matrices, together with such a sub-multiplicative norm, is an example of a Banach algebra.

[edit] Induced norm

If vector norms on Km and Kn are given (K is field of real or complex numbers), then one defines the corresponding induced norm or operator norm on the space of m-by-n matrices as the following maxima:

[pic]

These are different from the entrywise p-norms and the Schatten p-norms for matrices treated below, which are also usually denoted by [pic]

If m = n and one uses the same norm on the domain and the range, then the induced operator norm is a sub-multiplicative matrix norm.

The operator norm corresponding to the p-norm for vectors is:

[pic]

In the case of p = 1 and [pic], the norms can be computed as:

[pic]which is simply the maximum absolute column sum of the matrix

[pic]which is simply the maximum absolute row sum of the matrix

For example, if the matrix A is defined by

[pic]

then we have ||A||1 = 7+4+8 = 19. and ||A||∞ = 3+5+7 = 15

In the special case of p = 2 (the Euclidean norm) and m = n (square matrices), the induced matrix norm is the spectral norm. The spectral norm of a matrix A is the largest singular value of A or the square root of the largest eigenvalue of the positive-semidefinite matrix A*A:

[pic]

where A* denotes the conjugate transpose of A.

Any induced norm satisfies the inequality

[pic]

where ρ(A) is the spectral radius of A. In fact, it turns out that ρ(A) is the infimum of all induced norms of A.

Furthermore, we have the spectral radius formula:

[pic]

[edit] "Entrywise" norms

These vector norms treat an [pic]matrix as a vector of size mn, and use one of the familiar vector norms.

For example, using the p-norm for vectors, we get:

[pic]

This is a different norm from the induced p-norm (see above) and the Schatten p-norm (see below), but the notation is the same.

The special case p = 2 is the Frobenius norm, and p = ∞ yields the maximum norm.

[edit] Frobenius norm

For p = 2, this is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is often reserved for operators on Hilbert space. This norm can be defined in various ways:

[pic]

where A* denotes the conjugate transpose of A, σi are the singular values of A, and the trace function is used. The Frobenius norm is very similar to the Euclidean norm on Kn and comes from an inner product on the space of all matrices.

The Frobenius norm is submultiplicative and is very useful for numerical linear algebra. This norm is often easier to compute than induced norms.

[edit] Max norm

The max norm is the elementwise norm with p = ∞:

[pic]

This norm is not submultiplicative.

[edit] Schatten norms

For more details on this topic, see Schatten norm.

The Schatten p-norms arise when applying the p-norm to the vector of singular values of a matrix. If the singular values are denoted by σi, then the Schatten p-norm is defined by

[pic]

These norms again share the notation with the induced and entrywise p-norms, but they are different.

All Schatten norms are submultiplicative. They are also unitarily invariant, which means that ||A|| = ||UAV|| for all matrices A and all unitary matrices U and V.

The most familiar cases are p = 1, 2, ∞. The case p = 2 yields the Frobenius norm, introduced before. The case p = ∞ yields the spectral norm, which is the matrix norm induced by the vector 2-norm (see above). Finally, p = 1 yields the nuclear norm (also known as the trace norm, or the Ky Fan 'n'-norm), defined as

[pic]

(Here [pic]denotes a matrix B such that BB = A * A. More precisely, since A * A is a positive semidefinite matrix, its square root is well-defined.)

[edit] Consistent norms

A matrix norm [pic]on [pic]is called consistent with a vector norm [pic]on Kn and a vector norm [pic]on Km if:

[pic]

for all [pic]. All induced norms are consistent by definition.

[edit] Equivalence of norms

For any two vector norms ||·||α and ||·||β, we have

[pic]

for some positive numbers r and s, for all matrices A in [pic]. In other words, they are equivalent norms; they induce the same topology on [pic].

Moreover, when [pic], then for any vector norm [pic], there exists a unique positive number k such that [pic]is a (submultiplicative) matrix norm for every [pic].[clarification needed]

A matrix norm ||·||α is said to be minimal if there exists no other matrix norm ||·||β satisfying ||·||β ≤ ||·||α.

[edit] Examples of norm equivalence

For matrix [pic]the following inequalities hold 1,2:

• [pic], where r is the rank of A

• [pic]

• [pic]

• [pic]

Here, ||·||p refers to the matrix norm induced by the vector p-norm.

Another useful inequality between matrix norms is

[pic]

[edit] References

1. Golub, Gene; Charles F. Van Loan (1996). Matrix Computations - Third Edition. Baltimore: The Johns Hopkins University Press, 56-57. ISBN 0-8018-5413-X.

2. Roger Horn and Charles Johnson. Matrix Analysis, Chapter 5, Cambridge University Press, 1985. ISBN 0-521-38632-2.

3. Douglas W. Harder, Matrix Norms and Condition Numbers [1]

4. James W. Demmel, Applied Numerical Linear Algebra, section 1.7, published by SIAM, 1997.

5. Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, published by SIAM, 2000. [2]

6. John Watrous, Theory of Quantum Information, 2.4 Norms of operators, lecture notes, University of Waterloo, 2008.

Retrieved from ""

Categories: Norms (mathematics) | Linear algebra

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download