Skip to main content
Logo image

Mathematics in Trades and Life

Section 1.2 Accuracy and Precision

Subsection 1.2.1 Explanation

Figure 1.2.1. Introduction to Precision and Accuracy
When working with measurements, we care about the reasonableness of the results. This leads us to two concepts.

Definition 1.2.2. Accuracy.

The accuracy of a measurement is how close the measurement is to the actual value.
Note, if we are measuring something, it is because we don’t know the actual value. Thus we can’t determine the exact value of many kinds of data. Instead we will settle for repeatability. If we get the same result often enough, we can convince ourselves that it is reasonably accurate.

Definition 1.2.3. Precision.

The precision of a measurement is the size of the smallest unit in it.
Note we can have high precision with low accuracy. That is, just because we write a lot of decimal places does not mean they are close to the actual value.

Subsection 1.2.2 Significant Figures

When writing down measurements we need a way to indicate how precise the measurement is. Significant digits also called significant figures or simply “sig figs” are a way to do this.
The rules for writing numbers with significant digits have two parts: non-zero digits, and zero digits.
  1. All non-zero digits are signficant.
  2. Zeros between non-zeros are significant.
  3. Any zeros written to the right of the decimal point are significant.
  4. If zeros between non-zero digits (on left) and the decimal point (on right) are supposed to be significant, a line is drawn over top of the last significant digit.
  5. For numbers less than 1, zeros between the decimal point (on left) and non-zero digits (on right) are not significant.
We can summarize these rules as: write only digits that you mean, and if it is ambiguous clarify.

Example 1.2.4. Writing Significant Digits.

Each of these numbers is written with five (5) significant digits.
  • \(\displaystyle 10267\)
  • \(\displaystyle 1.2400\)
  • \(\displaystyle 7201\bar{0}\)
  • \(\displaystyle 2834100\)
  • \(\displaystyle 0.0010527\)

Checkpoint 1.2.5.

How many significant digits does \(203\) have?
How many significant digits does \(20\bar{0}0\) have?
Answer 1.
\(3\)
Answer 2.
\(3\)
Solution.
Because 203 ends with a non-zero digit all three digits are significant.
Because the 0 in the tens position is marked as significant (the bar) there are 3 significant digits
We also need rules for arithmetic with significant digits. These are based on two principles
  • A result of arithmetic cannot be more precise than the least precise measurement.
  • The number of significant digits cannot increase.
For addition and subtraction the result (sum or difference) has the same precision as the least precise number added or subtracted. After adding or subtracting we round to the farthest left, last significant digit.

Example 1.2.6. Subtraction with Significant Digits.

\(11050-723=10330\text{.}\) This is because the last significant digit of \(11050\) is the 10’s position (with the 5 in it) whereas the last significant digit of \(723\) is the 1’s position (with the 3 in it). We do not know the 1’s position of 11050, so we cannot know the 1’s position in the result.

Example 1.2.7. Addition with Significant Digits.

\(311+8310+202200=210800\text{.}\) This is because the farthest left, last significant digit is in the 100’s position in 202200. The extra precision of the other two numbers is not useful.
The significant digits addition/subtraction rule basically says that adding precise data to imprecise data does not increase the precision of the imprecise data. A detailed explanation is in this video.

Checkpoint 1.2.8.

Calculate \(646+21.12+120\text{:}\)
Calculate \(63.97-21\text{:}\)
Answer 1.
\(790\)
Answer 2.
\(43\)
Solution.
\(646+21.12+120=787.12 \approx 790\) because \(120\) is significant to only the 10’s position (the others are more precise).
\(63.97-21=42.97 \approx 43\) because \(21\) is significant to only the 1’s position (the other is more precise).
For multiplication and division the result (product or quotient) has the same number of significant digits as the least number of the input numbers.

Example 1.2.9. Division with Significant Digits.

\(11050/722=15.3\text{.}\) This is because \(722\) has only 3 significant digits.

Example 1.2.10. Multiplication with Significant Digits.

\(17 \times 14\bar{0} \times 3.178= 7600\text{.}\) This is because \(17\) has only two significant digits.
The significant digit multiplication/division rule basically says that digits that were multiplied by imprecise data cannot be precise. An explanation of why this rule works is in this video.

Checkpoint 1.2.11.

Calculate \(646 \times 21.12\text{:}\)
Calculate \(63.97/21\text{:}\)
Answer 1.
\(13600\)
Answer 2.
\(3\)
Solution.
\(646 \times 21.12=13643.52 \approx 13600\) because \(646\) has only 3 significant digits (the other has more).
\(63.97/21=3.046190476 \approx 3.0\) because \(21\) has only 2 significant digits (the other has more).
Significant digit rules must be applied at each step. That is if we have a mix of addition, subtraction, multiplication, and division then we do one operation at a time and apply the appropriate significant digits rule before performing the next arithmetic step.

Example 1.2.12. Multi-Step Arithmetic with Significant Digits.

Consider
\begin{equation*} 11,728+39(17.9+1.23). \end{equation*}
By order of operations we first calculate \(17.9+1.23 \approx 19.1.\) Second by order of operations we calculate \(39 \times 19.1 = 740.\) Finally we calculate \(11,728+740=12,470.\)

Subsection 1.2.3 Rounding

For a variety of reasons in applications we need to round a number, that is ignore some level of precision.
Table 1.2.13. Reasons for Rounding
Reality Constraints For example we cannot buy partial packages or have fractional people
Remove Detail For example when describing the population of a nation
Control Error When used in significant digits
The reason for rounding determines how we do it. Consider the following reality constraints requiring rounding. For example if we need 21 eggs and eggs are sold in cartons of one dozen (12) eggs, we need \(21/12=1.75\) cartons. Since we cannot purchase part of a carton, we must round 1.75 to 2, and purchase 2 cartons.
Note in this example reality requires us to round up to the nearest integer. We round to an integer because we cannot purchase fractional cartons of eggs. We had to round up, because rounding down would leave us with insufficient eggs (and we are hungry).
Suppose you have a bank account containing $11410 that accrues 1.65% interest. The bank calculates the payment should be \(\$11410 \cdot 0.0165 = \$188.265\text{.}\) The bank will pay you $188.26. They round to the nearest one hundredth because cents is a unit which can be paid. They round down, because they like paying less.
For removal of detail consider reporting the population of a country. We might report the population as 9 million rather than 9,904,607. We do this because we don’t need the exact number which likely is changing every day. When reporting on salary ranges we might report a range between $60,000 and $80,000. That the range is actually $61,233.57 and $80,290.11 is unlikely to change a decision. A common usage of removing detail is when we care about the scale of things rather than the count.
Rounding to control error is the use of significant digits.
We can round to any digit. We can round up, down, or simply “round”. Context or instructions will specify which digit and which type of rounding.

Example 1.2.14. Rounding Up/Down.

(a)
Round 72481 down to the nearest hundred.
Solution.
72400 is rounding down: we leave the 4 (hundred position) alone and “truncate” (turn to 0) all digits to the right. Note \(72400 \le 72481\text{.}\)
(b)
Round 72481 up to the nearest hundred.
Solution.
72500 is rounding up: we increase the 4 to a 5 and “truncate” (turn to 0) all digits to the right. Note \(72500 \ge 72481\text{.}\)
(c)
Round 72481 the nearest hundred.
Solution.
Because 72481 is closer to 72500 than it is to 72400, we round to 72500. We can recognize that we should round up because the tens position is \(8 \ge 5\) which makes it closer to go up. We could also recognize the need to round up by calculating \(500-481=19\) and \(481-400=81\) and noticing that \(81 \ge 19\text{.}\)

Example 1.2.15. Rounding Up/Down.

(a)
Round 72481 down to the nearest thousand.
Solution.
72000 is rounding down: we leave the 2 (thousands position) alone and “truncate” (turn to 0) all digits to the right. Note \(72000 \le 72481\text{.}\)
(b)
Round 72481 up to the nearest thousand.
Solution.
73000 is rounding up: we increase the 2 to a 3 and “truncate” (turn to 0) all digits to the right. Note \(73000 \ge 72481\text{.}\)
(c)
Round 72481 to the nearest thousand.
Solution.
Because 72481 is closer to 72000 than it is to 73000, we round to 72000. We can recognize that we should round up because the hundreds position is \(4 < 5\) which makes it closer to go down. We could also recognize the need to round down by calculating \(3000-2481=519\) and \(2481-2000=481\) and noticing that \(481 < 519\text{.}\)

Example 1.2.16. Rounding to Different Precisions.

Round 72321.83 to the specified precision.
  • Thousands: 72000
  • Ones:72322
  • Tenths: 72321.8

Checkpoint 1.2.17.

Round 812,247 to the nearest ten:
Round 812,247 to the nearest hundred:
Answer 1.
\(812250\)
Answer 2.
\(812200\)

Subsection 1.2.4 Greatest Possible Error

When we write a number with significant digits notation or we round a number we write a number that has some error in it. For example if it rained 0.86 inches in a day and we write 0.9 inches, there is an error of 0.4. The important question is “how big can the error be?”.
Because our rule for rounding is digits 0-4 round down round up and digits 5-9, rounding will always have a greatest possible error of 5. Consider Example 1.2.18.

Example 1.2.18.

What is the greatest possible error if 130 was rounded to the nearest 10?
Solution.
One possibility is that 130 was rounded down. Then the original number was one of 130, 131, 132, 133, or 134. 134 is the farthest away from 130 at \(134-130=4\text{.}\)
The other possibility is that 130 was rounded up. Then the original number was one of 125, 126, 127, 128, or 129. 125 is the farthest away at \(130-125=5\text{.}\)
Thus the greatest possible error was 5 from the case that 125 was rounded up.
Note in this solution we assumed the number rounded was an integer. However, if we allowed for 134.927 and 125.01 the result would be the same. the extra digits don’t change the rounding.

Example 1.2.19.

What is the greatest possible error if 9.31 was rounded to the nearest hundredth?
Solution.
The largest possible error is if 9.31 was rounded up from 9.305. Thus the greatest possible error is 5 one thousdandths.

Example 1.2.20.

What is the greatest possible error if 223 was rounded up to the nearest one?
Solution.
223 could have been rounded up from 222.1. But it could also have been rounded up from 222.01 or anything else. Thus the greatest possible error is less than 1 (\(223-222=1\)).
Notice we have to know what type of rounding was used. In most measurements (i.e., significant digits) standard rounding will be used. For example think about measuring on a ruler: if the object isn’t exactly on one of the lines, you will choose the closest one. The closest one requires rounding.

Checkpoint 1.2.21.

What is the greatest possible error if 8120 was rounded to the nearest ten?
Answer.
\(5\)
Solution.
This could be any number from 8115 to 8124. Thus the greatest possible error is 5.

Checkpoint 1.2.22.

What is the greatest possible error in the result \(934\bar{0}00\text{?}\)
Answer.
\(50\)
Solution.
The smallest accurate digit is the hundreds position as indicated by the bar. Thus the number could be anything from 933950-934049. Thus the greatest possible error is 50.

Subsection 1.2.5 Purposes of Rounding

Ask the instructor to complete this section.

Exercises 1.2.6 Exercises

1. Significant Digits.

2. Count Significant Digits.

3. Count Significant Digits.

4. Count Significant Digits.

5. Count Significant Digits.

6. Count Significant Digits.

7. Significant Digits Arithmetic.

8. Significant Digits Arithmetic.

9. Significant Digits Arithmetic.

10. Significant Digits Rounding.

11. Greatest Error.

12. Greatest Error.

13. Greatest Error.

14. Greatest Error.