2

I have an array of numbers with 16 decimal places. When I copy them into a Libre Office Calc spreadsheet, the numbers are rounded off to the 15th decimal place. I assume this is a result of the double/real datatype being limited in what it can store.

To be clear, this isn't a formatting issue. I've adjusted the cell formatting to display 16 decimal places, and it does. Digits past the 15th are rounded to zero, however, and any manual corrections I attempt to make to the extra zeros are immediately discarded.

Is there any solution for this? For example, can I change the data type from double/real to something like Libre Office Base's decimal?

  • It maybe that spread sheets are not the correct tool, for what you are doing. If you are doing high precision physics simulations (I assume that you are, if you need this level of precision), then you should be looking at python, with the arbitrary precision library. If you are doing stuff with money (also needs precision), then I thing that there is a currency type. – ctrl-alt-delor Oct 1 '18 at 17:50
  • What are you modelling, that need this level of precision? – ctrl-alt-delor Oct 1 '18 at 17:51
  • 1
    According to en.wikipedia.org/wiki/IEEE_754 binary double 64bit have a precision of 15.95 digit.if leading digit are not null, you have hit representation limit. – Archemar Oct 1 '18 at 19:50
0

You have to do two things:

1- Round off the numbers: To calculate with the rounded off numbers instead of the internal exact values:

Choose Tools menu >> Options >> LibreOffice Calc >> Calculate page. Mark the "Precision" as shown field and exit the dialog with OK. 

enter image description here

2- Keep in mind that the decimal place for the first round after decimal is zero, so if you would like to adjust for 16 decimals, then your decimal place must be 15

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.