-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
partly wrong rounding when using printf with "%f" format #6
Comments
@jghub I have been researching this issue for several days now. At present, it seems that if the float is less than zero and has a succeeding zero or zeroes after the fraction separator rounding fails unless enough additional precision is given.
As a workaround, you can apply precision to a float via
As an example of supplying printf the float data as compared to its expansion:
I believe the problem is even more complex than I have described at present. I am now creating some scripts to help provide more insights into this rounding with floats and precision issue. |
@hyenias: thanks for this information. I agree that the situation is rather confusing. it would be good if you can manage to get a better grasp of what is going on here :). |
Interesting:
off by one error? |
why/how could that be an off-by-one error? I am too slow... I mean in your example every number in the semi-open interval [0.0150, 0.0250) should round to 0.02, but actually the numbers in the open interval (0.0195, 0.0295) are "rounded" to 0.02 ... so the range is shifted by 0.045 in this example and, additionally, the lower boundary is not included (open rather than semi-open interval). |
I mean that while 0.019 are incorrect if we supply another digit things start to get right. Maybe just "n+1" digit factor is wrong somewhere. As @hyenias noticed all is fine with 1.019, so looks likes only the first non-zero digit does not get rounded up or down properly. |
I am still working on some testing script(s) to help identify the extent of the problem as well as provide workaround(s). In particular, I thought I had a good working ksh rounding function but when I tried it out on a 32bit system instead of my normal 64bit it failed. Here are some of my current observations:
More to come later... |
I now have my test script running correctly on an ARMv7 box with an adjustment in my formula. The 32bitbox numbers supplied previously were run using text as input. Here are the corrected results. Corrected:
Next up for me is to see if my test script holds up on 64bit CPU using 32bit OS. |
Hey @hyenias, since ksh-community is not going anywhere fast (see discussion in #11), I'd like to invite you to submit a pull request to the 93u+m branch at https://github.com/ksh93/ksh when you've figured out a fix. Floating point math is well over my head, so your contribution and expertise would be an asset. |
I was able to get my test rounding script to work on my macOS, various Linux, and FreeBSD versions back in late April. Interestingly, ksh93's floating point behavior differed the most on my 32-bit FreeBSD box with its floating point responses being more inline with what I expected. Up until this issue, I did not remember the floating point math material from long ago. During my research, I found 0.30000000000000004.com as the most enlightening.
@McDutchie Thank you for the invite and I will be pleased to submit a pull request to your ksh93 branch. When all of my other ksh93 installations produced different results than the above along with the continuing kickoff efforts for ksh-community, I started learning other things. With all of your ksh93 branch activity, I am energized to continue working on this issue. |
Well, I have stared at the numbers long enough to determine the 3 following problems occurring with ksh's rounding using printf and float precision expansion.
|
I will now work on some code fixes for the above. Primary culprit is src/lib/libast/sfio/sfcvt.c. |
Update: I have made successful progress on problem 1 and 2 above. Need to validate against the other float formats, |
https://members.loria.fr/PZimmermann/papers/accuracy.pdf - interesting as well ... |
I have submitted corrections for sfcvt.c that fixes the original problem identified which is my 2nd bullet above about subnormal numbers. Still working on my items 1 and 3. I have also found another problem with rounding to whole numbers via a precision of 0. |
I accidentally noted the following behaviour:
so
%f
format seems partly broken...The text was updated successfully, but these errors were encountered: