The Wikipedia of Numerical Methods!

Overview and Conventions

  • These are reference pages. They are intended to be like Wikipedia, providing the factual basis of each topic. They are not written as introdutory material (that’s what lectures and lecture notes are for).
  • You are expected to know everything in these pages, but not limited to these pages.
  • Each section starts with a set of learning objectives. These are designed to help guide your learning, but they do not limit what you are supposed to know.

Big Picture

  • Numerical algorithms are distinguished by their cost and error, and the tradeoff between them.
  • Every algorithm or method in these reference pages indicates its error and cost whenever possible. These might be exact expressions or asymptotic bounds like as or as . For asymptotics we always indicate the limit.

Style conventions

  • When we are defining a word or phrase we write it in bold italics, like this.
  • Every algorithm contains a precise name like algorithm name. Wherever possible we write an algorithm both in mathematical notation and in code.
  • All equations are numbered for easy reference (THIS DOESN’T WORK YET).
  • We normally use for the integer size of something, such as -th order Taylor expansion or matrix. For asymptotics we are normally interested in the case .
  • We normally use for a small real-valued scalar, such as a step size or approximation increment. For asymptotics we are normally interested in the case .
  • In some cases and may be related, such as when we put evenly spaced points in an interval and let be the spacing between them, so . In other cases and may be unrelated, such as for Taylor expansions where is the order and is the distance from the expansion point. We will clearly indicate when and are related.
  • Matrices are unbolded, capital letters.
    • Example:
  • Matrices are indexed by 2 subscripts.
    • The subscripts are often separated by commas, but the commas may be excluded whenever unambiguous.
    • The first subscript is the row index and the second subscript is the column index
    • Example: and both indicate the 1st row, 3rd column (using 1-based indexing).
    • Example: When indexing the 2nd row, 14th column in a , the comma between the indices is explicitly included to differentiate between from .
  • Vectors are bolded, lower case letters.
    • Example:
  • Vectors are indexed by a single subscript.
    • Example: indicates the 2nd entry of (using 1-based indexing)
  • Scalars are lower case.
    • Example:
  • Use
        import numpy as np
        import numpy.linalg as la
        import scipy as sp
        import scipy.linalg as sla
        import scipy.sparse as sparse
        import scipy.sparse.linalg as spla
        import matplotlib.pyplot as plt
        import seaborn as sns