Kinds of Errors in Numerical Procedures

Errors can occur in doing numerical procedures

• Error in original Data
• Blunders: Sometimes a test run with known results is worthwhile, but is no guarantee of freedom from foolish error.
• Truncation Error: i.e., approximate $e^x$ by the cubic power
  
  $$P_3(x)=1+\frac{x}{1!}+\frac{x^2}{2!}+\frac{x^3}{3!};~~~~~~e^x=P_3(x)+\sum_{n=4}^\infty \frac{x^n}{n!}$$
  
  
  approximating $e^x$ with the cubic gives an inexact answer. The error is due to truncating the series.
  When to cut series expansion $\Longrightarrow$ be satisfied with an approximation to the exact analytical answer
• Propagated Error:
  • more subtle
  • by propagated we mean an error in the succeeding steps of a process due to an Occurrence of an earlier error
  • of critical importance
  • stable numerical methods; errors made at early points die out as the method continues
  • unstable numerical method; does not die out
• Round-off Error:
  • All computing devices represents numbers, except for integers and some fractions, with some imprecision
  • floating-point numbers of fixed word length; the true values are usually not expressed exactly by such representations
  • if the number are rounded when stored as floating-point numbers, the round-off error is less than if the trailing digits were simply chopped off
• Absolute vs Relative Error:
  • accuracy $\longrightarrow$ great importance
  • $absolute~error=\vert true~value-approximate~error\vert$
    A given size of error is usually more serious when the magnitude of the true value is small
  • $relative~error=\frac{absolute~error}{\vert true~value\vert}$
• Floating-Point Arithmetic: Performing an arithmetic operation $\Rightarrow$ no exact answers unless only integers or exact powers of 2 are involved
  • floating-point (real numbers)$\rightarrow$ not integers
  • resembles scientific notation
    

    Table 1: Floating$\rightarrow$ Normalized.

    floating	normalized (shifting the decimal point)
    13.524	$.13524*10^2~(.13524E2)$
    -0.0442	$-.442E-1$

    
    
  • IEEE standard $\rightarrow$ storing floating-point numbers (see the Table 1.4)
    • the sign $\pm$
    • the fraction part (called the mantissa)
    • the exponent part
  • There are three level of precision (see the Fig. 3)

    Figure 3: Level of precision.

    
  • Rather than use one of the bits for the sign of the exponent, exponents are biased. For single precision:
    $2^8$=256, 0 (255) $\Longrightarrow$ -127 (128). An exponent of -127 (128) stored as 0 (255).
    0 $\longrightarrow$00000000 = 0
    255 $\longrightarrow$11111111=255
    So biased $\longrightarrow 2^{128}=3.40282E+38$, mantissa gets 1 as maximum
    Largest: 3.40282E+38; Smallest: 2.93873E-39
    For double and extended precision the bias values are 1023 and 16383, respectively.
  • Certain mathematical operations are undefined,
    $\frac{0}{0},0*\infty,\sqrt{-1}\Longrightarrow NaN$
• EPS: short for epsilon $\longrightarrow$used for represent the smallest machine value that can be added to 1.0 that gives a result distinguishable from 1.0 In MATLAB:

  >> eps
  ans=2.2204E=016
  

  $eps\longrightarrow\varepsilon\longrightarrow(1+\varepsilon)+\varepsilon=1~but~1+(\varepsilon+\varepsilon)>1$
• Round-off Error vs Truncation Error: Round-off occurs, even when the procedure is exact, due to the imperfect precision of the computer
  Analytically $\frac{df}{dx}\Rightarrow f^{'}(x)=lim_{h\rightarrow0}\frac{f(x+h)-f(x)}{(x+h)-x}$: procedure
  approximate value for $f^{'}(x)$ wit a small value for h.
  $h\longrightarrow smaller$, the result is closer to the true value $\longrightarrow$truncation error is reduced
  but at some point (depending of the precision of the computer) round-off errors will dominate $\longrightarrow$less exact $\Longrightarrow$There is a point where the computational error is least
• Well-posed and well-conditioned problems: The accuracy depends not only on the computer's accuracy
  • A problem is well-posed if a solution; exists, unique, depends on varying parameters
    • A nonlinear problem $\longrightarrow$linear problem
    • infinite $\longrightarrow$large but finite
    • complicated $\longrightarrow$simplified
  • A well-conditioned problem is not sensitive to changes in the values of the parameters (small changes to input do not cause to large changes in the output)
  • Modelling and simulation; the model may be not a really good one
  • if the problem is well-conditioned, the model still gives useful results in spite of small inaccuracies in the parameters
• Forward and Backward Error Analysis:
  $y=f(x)$
  $y_{calc}=f(x_{calc}):~computed~value$
  $E_{fwd}=y_{calc}-y_{exact}$ where $y_{exact}$ is the value we would get if the computational error were absent
  $E_{backwd}=x_{calc}-x,~y_{calc}=f(x_{calc})$
  Example:
  $y=x^2,~x=2.37$ used only two digits
  $y_{calc}=5.6~while~y_{exact}=5.6169$
  $E_{fwd}=-0.0169$, relative error $\rightarrow$0.3$\%$
  $\sqrt{5.6}=2.3664 \Rightarrow E_{backfwd}=-0.0036$, relative error $\rightarrow$0.15$\%$
• Examples of Computer Numbers:
  Say we have six bit representation (not single, double) (see the Fig. 4)
  • $1~bit\rightarrow sign$
  • $3(+1)~bits\rightarrow mantissa$
  • $2~bits\rightarrow exponent$

  Figure 4: Computer numbers with six bit representation.

  
  For positive range $\frac{9}{32}\longleftrightarrow \frac{15}{4}$
  For negative range $\frac{-15}{4}\longleftrightarrow \frac{-9}{32}$; even discontinuity at point zero since it is not in the ranges.
  

  Figure 5: Upper: number line in the hypothetical system, Lower: IEEE standard.

  

  
  Very simple computer arithmetic system $\Rightarrow$ the gaps between stored values are very apparent. Many values can not be stored exactly. i.e., 0.601, it will be stored as if it were 0.6250 because it is closer to $\frac{10}{16}$, an error of $4\%$ In IEEE system, gaps are much smaller but they are still present. (see the Fig. 5)
• Anomalies with Floating-Point Arithmetic:
  For some combinations of values, these statements are not true
  • $(x+y)+z=x+(y+z)$
    $(x*y)*z=x*(y*z)$
    $x*(y+z)=(x*y)+(x*z)$
    
  • adding 0.0001 one thousand times should equal 1.0 exactly but this is not true with single precision
  • $z=\frac{(x+y)^2-2xy-y^2}{x^2}$, problem with single precision