time coding refactor #9906
Open
time coding refactor #9906
+108
−40
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 tasks
kmuehlbauer
commented
Dec 18, 2024
kmuehlbauer
commented
Dec 18, 2024
There was a problem hiding this comment.
Thanks for splitting this off @kmuehlbauer—these changes look good. Just a few very minor questions.
| delta = date * np.timedelta64(1, unit) | ||
| if not np.isnan(delta): | ||
| # this will raise on dtype overflow for integer dtypes | ||
| if date.dtype.kind in "iu" and not np.int64(delta) == date: |
There was a problem hiding this comment.
Is this not only a risk for uint64 values?
| # this should be safe, as errors would have been raised above | ||
| ns_time_unit = _NS_PER_TIME_DELTA[time_unit] | ||
| ns_ref_date_unit = _NS_PER_TIME_DELTA[ref_date.unit] | ||
| if flat_num_dates.dtype.kind in "iuf" and (ns_time_unit > ns_ref_date_unit): |
There was a problem hiding this comment.
Sorry I could be missing something—is flat_num_dates ever anything other than an integer, unsigned integer, or float dtype?
Comment on lines
+382
to
+387
| # retrieve cftype | ||
| cftype = type(dates[np.nanargmin(num_dates)]) | ||
| # "ns" borders | ||
| # between ['1677-09-21T00:12:43.145224193', '2262-04-11T23:47:16.854775807'] | ||
| lower = cftype(1677, 9, 21, 0, 12, 43, 145224) | ||
| upper = cftype(2262, 4, 11, 23, 47, 16, 854775) |
There was a problem hiding this comment.
Indeed, good to make this more exact 👍.
Comment on lines
+281
to
+283
| # if date is exactly NaT (np.iinfo("int64").min) return refdate | ||
| # to make follow-up checks work | ||
| return ref_date |
There was a problem hiding this comment.
Would it make more sense to return pd.Timestamp("NaT")?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This introduces some changes in the encoding/decoding pipeline to prepare for other units beside
ns. It also fixes some inconsistencies wrt the exact ns.resolution datetime64 bounds.Finally it will reduce the "noise" over in #9618 quite substantial.
whats-new.rst