← index #1450PR #1180
Related · high · value 0.220
QUERY · ISSUE

Single precision floating point formatting issues

openby dhylandsopened 2015-08-31updated 2024-08-28
docs

While investigating #1435 I ran into some other formatting issues. The output isn't necessarily wrong, but it doesn't match CPython either (although I am concerned about the outputting more data than requested potentially causing a buffer overflow).

1 - Outputting more data than requested.
On pyboard:

>>> '{:.4e}'.format(9.99999e-5)
'1.00000e-04'

On CPython:

>>> '{:.4e}'.format(9.99999e-5)
'1.0000e-04'

2 - making the wrong choice about when to use e notation:
pyboard:

>>> '%5.g' % 9.99999
'   10'

CPython:

>>> '%5.g' % 9.99999
'1e+01'

3 - Failing to normalize when rounding:
pyboard:

>>> '%5.e' % 9.99999
'10e+00'

CPython:

>>> '%5.e' % 9.99999
'1e+01'
CANDIDATE · PULL REQUEST

Fix floating point number near 1.0

closedby dhylandsopened 2015-04-12updated 2015-04-12

In particular, numbers which are less than 1.0 but which
round up to 1.0.

This also makes those numbers which round up to 1.0 to
print with e+00 rather than e-00 for those formats which
print exponents.

Fixes #1178

Keyboard

j / / n
next pair
k / / p
previous pair
1 / / h
show query pane
2 / / l
show candidate pane
c
copy suggested comment
r
toggle reasoning
g i
go to index
?
show this help
esc
close overlays

press ? or esc to close

copied