Have you performed simple arithmetic operations like 0.1 + 0.2? You might have gotten something strange: 0.1 + 0.2 = 0.30000000000000004.

  • Zagorath@aussie.zone
    link
    fedilink
    arrow-up
    5
    arrow-down
    1
    ·
    1 month ago

    I thought it was a rather simple analogue, but I guess it was too complicated for some?

    I said nothing about JavaScript or Python or any other language with my 1/3 example. I wasn’t even talking about binary. It was an example of something that might be problematic if you added numbers in an imprecise way in decimal, the same way binary floating point fails to accurately represent 1/10 + 1/5 from the OP.

    • ericjmorey@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      1 month ago

      Perhaps the following rewording of your last sentence would be easier for readers to follow along:

      With a lack of precision, 1/3 might become 0.33333333. When evaluating the expression 1/3 + 1/3 + 1/3, using 0.33333333 as an approximate representation of 1/3 will return a result of 0.99999999, instead of the correct answer of 1.