Precision Loss Caused by Rounding
When resources are distributed to a large number of tasks, because rounding happens to two places after the decimal point, precision can be lost. When this happens Cartegraph implements an alternative rounding behavior of rounding down instead of up.
Example
This example shows how Cartegraph checks rounding behavior results to keep data precision. For this example 500 Tasks are selected, 8 Labor hours entered, resulting in 0.01 hours per task.
- The system first uses rounding up behavior:
- 8 hours/500 tasks = 0.016 and is rounded up to 0.02
- 0.02 is multiplied by 499 = 9.98
499 is used because 1 task is reserved to receive the adjusted amount to keep precision.
- 9.98 is subtracted from 8 hours creating a negative number and an inaccurate results caused by rounding up.
- The system detects this situation and recalculates using rounding down behavior called flooring.
- 8 hours/500 = 0.016 and is rounded down to 0.01.
- 0.01 is multiplied by 499 = 4.99
- 4.99 is subtracted from 8 hours = 3.01
- 3.01 is applied to the reserved task to keep data precision.
