Need to address inconsistencies between mktime and localtime.
Currently, if I do:
>>> time.localtime(-1072915200)
(1966, 1, 1, 0, 0, 0, 5, 1)
>>> time.mktime((1966, 1, 1, 0, 0, 0, 5, 1))
3222052096
>>> time.localtime(3222052096)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: overflow converting long int to machine word
>>> hex(3222052096)
'0xc00c9d00'
>>> hex(0x80000000 - 1072915200)
'0x400c9d00'
I think that time.mktime((1966, 1, 1, 0, 0, 0, 5, 1)) should return the negative representation. However, I noticed that timeutils_seconds_since_2000 returns an mp_uint_t rather than an mp_int_t
So a couple of questions came to mind:
1 - Do we want to support times from 1966 to 2000 (which is the -ve small int range)?
2 - Should we support mpz ints?
3 - Adding full support for mpz ints seems like overkill, but we could get 1 extra bit by allowing mpz ints in the full 32-bit range to be supported. Then we'd be able to support dates from 2000 +/- 68 years. This would only require a slightly different conversion into and out of the timeutils functions. The actual code itself wouldn't need to change.
Thoughts?
shared/timeutils: Standardize supported date range on all platforms.
This is pull request is to ensure that time functions (mktime, localtime and gmtime) work properly on a reasonable date range, on all platforms, regardless of the epoch.
It does fix issue #17340
Summary
The most important point is to have a test case that checks conversion from timestamp to tuple and back for every day in the supported time range. This was currently missing. The new test code starts from 2001, first iterates backward on tuple to find the earliest supported date and then goes forward by adding 86400 to the timestamp, to verify that the date computations are correct, including leap years, and do not raise an overflow.
The second part of the pull request is to ensure that this test does actually pass on all platforms.
The most severely affected platforms were 32 bit platforms using a 1970 epoch, as
- there was an assumption that timestamps can be stored in a
mp_int_t - conversion from the 1970 epoch to the 2000 epoch used internally by timeutils breaks computations for dates before 2000
On some of these platforms, the resulting supported date range was only 2001 to 2037. As 2038 is now only 13 years away, this clearly has to be fixed quickly.
Small code footprint beeing a key value for MicroPython, the suggested minimal supported time range for all platforms is 1970 to 2099. This range can be covered using 32 bit timestamps on ressource-limited platforms, fixes the 1970 epoch problems and requires less code for handling of leap years.
Support for years 2100 and beyond is only enabled for platforms using 64 bit pointers, which are typically less ressource-constrained.
Testing
The PR includes new test cases that verify the supported time range using architecture-independent Python code.
Trade-offs and Alternatives
The new timeutils code has a smaller footprint than the original for 32 bit machines, as it does not have to handle signed inputs and does not have to handle century-based leap years.
We could have opted to use signed timestamps centered on 2000, as was probably the original intent of the timeutils code. We felt however that it was more useful to fully support the 21st century rather going supporting dates back to 1940's but breaking in 2068. Handling timestamps as unsigned also simplifies the code by removing some checks for negative values.
We could have opted to upgrade all platforms to use 64-bit signed integers for time functions, which would have offered the broadest possible range for abstract date operations, spanning from the year 1600 to the year 3000 and beyond. However, this would have had a code size impact on 32-bit platforms, with little added value. We guessed that, if a specific application on a 32-bit platform required handling exotic dates, a Python-based implementation would remain a good option.