a Of@sddlmZddlmZddlmZddlmZddlm Z ddl m Z m Z m Z mZmZmZddlZddlZddlmZmZdd lmZmZmZmZmZmZdd lm Z dd l!m"Z"m#Z#dd l$m%Z%dd l&m'Z'm(Z(m)Z)ddl*m+Z+ddl,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2ddl3m4Z4m5Z5ddl6m7Z7m8Z8ddl9m:Z:m;Z;mZ>ddl?m@Z@ddlAmBZBddlCmDZDmEZEmFZFddlGmHZHddlImJZJddlKmLZLe rddlMmNZNddlOmPZPddlQmRZRddlSmTZTmUZUeeVeWe'fZXeeYeZfZ[ee[eej\fZ]ee]eXfZ^eeVe[eWe[dfe'fZ_Gdd d e d!d"Z`Gd#d$d$e`d%d"Zaeead&fZbd'Zcdqd(d)d*d+d,Zddrd.d/d0d1d2d3d4Zed.d)d1d5d6d7d8d9Zfdsd:d1d;dd?Zgdtd@d6d;d|St||dddkrjtjdttddS)NrPzCould not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format. stacklevel) rfirst_non_nulltypestrrwarningswarn UserWarningr)arrrPrWZfirst_non_nan_elementZguessed_formatrBrBrC _guess_datetime_format_for_array~sr^ffffff?ArrayConvertiblefloatz int | Nonebool)arg unique_share check_countrQcCsd}|dur` section for details. .. warning:: In a future version of pandas, parsing datetimes with mixed time zones will raise an error unless `utc=True`. Please specify `utc=True` to opt in to the new behaviour and silence this warning. To create a `Series` with mixed offsets and `object` dtype, please use `apply` and `datetime.datetime.strptime`. See also: pandas general documentation about `timezone conversion and localization `_. format : str, default None The strftime to parse time, e.g. :const:`"%d/%m/%Y"`. See `strftime documentation `_ for more information on choices, though note that :const:`"%f"` will parse all the way up to nanoseconds. You can also pass: - "ISO8601", to parse any `ISO8601 `_ time string (not necessarily in exactly the same format); - "mixed", to infer the format for each element individually. This is risky, and you should probably use it along with `dayfirst`. .. note:: If a :class:`DataFrame` is passed, then `format` has no effect. exact : bool, default True Control how `format` is used: - If :const:`True`, require an exact `format` match. - If :const:`False`, allow the `format` to match anywhere in the target string. Cannot be used alongside ``format='ISO8601'`` or ``format='mixed'``. unit : str, default 'ns' The unit of the arg (D,s,ms,us,ns) denote the unit, which is an integer or float number. This will be based off the origin. Example, with ``unit='ms'`` and ``origin='unix'``, this would calculate the number of milliseconds to the unix epoch start. infer_datetime_format : bool, default False If :const:`True` and no `format` is given, attempt to infer the format of the datetime strings based on the first non-NaN element, and if it can be inferred, switch to a faster method of parsing them. In some cases this can increase the parsing speed by ~5-10x. .. deprecated:: 2.0.0 A strict version of this argument is now the default, passing it has no effect. origin : scalar, default 'unix' Define the reference date. The numeric values would be parsed as number of units (defined by `unit`) since this reference date. - If :const:`'unix'` (or POSIX) time; origin is set to 1970-01-01. - If :const:`'julian'`, unit must be :const:`'D'`, and origin is set to beginning of Julian Calendar. Julian day number :const:`0` is assigned to the day starting at noon on January 1, 4713 BC. - If Timestamp convertible (Timestamp, dt.datetime, np.datetimt64 or date string), origin is set to Timestamp identified by origin. - If a float or integer, origin is the difference (in units determined by the ``unit`` argument) relative to 1970-01-01. cache : bool, default True If :const:`True`, use a cache of unique, converted dates to apply the datetime conversion. May produce significant speed-up when parsing duplicate date strings, especially ones with timezone offsets. The cache is only used when there are at least 50 values. The presence of out-of-bounds values will render the cache unusable and may slow down parsing. Returns ------- datetime If parsing succeeded. Return type depends on input (types in parenthesis correspond to fallback in case of unsuccessful timezone or out-of-range timestamp parsing): - scalar: :class:`Timestamp` (or :class:`datetime.datetime`) - array-like: :class:`DatetimeIndex` (or :class:`Series` with :class:`object` dtype containing :class:`datetime.datetime`) - Series: :class:`Series` of :class:`datetime64` dtype (or :class:`Series` of :class:`object` dtype containing :class:`datetime.datetime`) - DataFrame: :class:`Series` of :class:`datetime64` dtype (or :class:`Series` of :class:`object` dtype containing :class:`datetime.datetime`) Raises ------ ParserError When parsing a date from string fails. ValueError When another datetime conversion error happens. For example when one of 'year', 'month', day' columns is missing in a :class:`DataFrame`, or when a Timezone-aware :class:`datetime.datetime` is found in an array-like of mixed time offsets, and ``utc=False``. See Also -------- DataFrame.astype : Cast argument to a specified dtype. to_timedelta : Convert argument to timedelta. convert_dtypes : Convert dtypes. Notes ----- Many input types are supported, and lead to different output types: - **scalars** can be int, float, str, datetime object (from stdlib :mod:`datetime` module or :mod:`numpy`). They are converted to :class:`Timestamp` when possible, otherwise they are converted to :class:`datetime.datetime`. None/NaN/null scalars are converted to :const:`NaT`. - **array-like** can contain int, float, str, datetime objects. They are converted to :class:`DatetimeIndex` when possible, otherwise they are converted to :class:`Index` with :class:`object` dtype, containing :class:`datetime.datetime`. None/NaN/null entries are converted to :const:`NaT` in both cases. - **Series** are converted to :class:`Series` with :class:`datetime64` dtype when possible, otherwise they are converted to :class:`Series` with :class:`object` dtype, containing :class:`datetime.datetime`. None/NaN/null entries are converted to :const:`NaT` in both cases. - **DataFrame/dict-like** are converted to :class:`Series` with :class:`datetime64` dtype. For each row a datetime is created from assembling the various dataframe columns. Column keys can be common abbreviations like ['year', 'month', 'day', 'minute', 'second', 'ms', 'us', 'ns']) or plurals of the same. The following causes are responsible for :class:`datetime.datetime` objects being returned (possibly inside an :class:`Index` or a :class:`Series` with :class:`object` dtype) instead of a proper pandas designated type (:class:`Timestamp`, :class:`DatetimeIndex` or :class:`Series` with :class:`datetime64` dtype): - when any input element is before :const:`Timestamp.min` or after :const:`Timestamp.max`, see `timestamp limitations `_. - when ``utc=False`` (default) and the input is an array-like or :class:`Series` containing mixed naive/aware datetime, or aware with mixed time offsets. Note that this happens in the (quite frequent) situation when the timezone has a daylight savings policy. In that case you may wish to use ``utc=True``. Examples -------- **Handling various input formats** Assembling a datetime from multiple columns of a :class:`DataFrame`. The keys can be common abbreviations like ['year', 'month', 'day', 'minute', 'second', 'ms', 'us', 'ns']) or plurals of the same >>> df = pd.DataFrame({'year': [2015, 2016], ... 'month': [2, 3], ... 'day': [4, 5]}) >>> pd.to_datetime(df) 0 2015-02-04 1 2016-03-05 dtype: datetime64[ns] Using a unix epoch time >>> pd.to_datetime(1490195805, unit='s') Timestamp('2017-03-22 15:16:45') >>> pd.to_datetime(1490195805433502912, unit='ns') Timestamp('2017-03-22 15:16:45.433502912') .. warning:: For float arg, precision rounding might happen. To prevent unexpected behavior use a fixed-width exact type. Using a non-unix epoch origin >>> pd.to_datetime([1, 2, 3], unit='D', ... origin=pd.Timestamp('1960-01-01')) DatetimeIndex(['1960-01-02', '1960-01-03', '1960-01-04'], dtype='datetime64[ns]', freq=None) **Differences with strptime behavior** :const:`"%f"` will parse all the way up to nanoseconds. >>> pd.to_datetime('2018-10-26 12:00:00.0000000011', ... format='%Y-%m-%d %H:%M:%S.%f') Timestamp('2018-10-26 12:00:00.000000001') **Non-convertible date/times** Passing ``errors='coerce'`` will force an out-of-bounds date to :const:`NaT`, in addition to forcing non-dates (or non-parseable dates) to :const:`NaT`. >>> pd.to_datetime('13000101', format='%Y%m%d', errors='coerce') NaT .. _to_datetime_tz_examples: **Timezones and time offsets** The default behaviour (``utc=False``) is as follows: - Timezone-naive inputs are converted to timezone-naive :class:`DatetimeIndex`: >>> pd.to_datetime(['2018-10-26 12:00:00', '2018-10-26 13:00:15']) DatetimeIndex(['2018-10-26 12:00:00', '2018-10-26 13:00:15'], dtype='datetime64[ns]', freq=None) - Timezone-aware inputs *with constant time offset* are converted to timezone-aware :class:`DatetimeIndex`: >>> pd.to_datetime(['2018-10-26 12:00 -0500', '2018-10-26 13:00 -0500']) DatetimeIndex(['2018-10-26 12:00:00-05:00', '2018-10-26 13:00:00-05:00'], dtype='datetime64[ns, UTC-05:00]', freq=None) - However, timezone-aware inputs *with mixed time offsets* (for example issued from a timezone with daylight savings, such as Europe/Paris) are **not successfully converted** to a :class:`DatetimeIndex`. Parsing datetimes with mixed time zones will show a warning unless `utc=True`. If you specify `utc=False` the warning below will be shown and a simple :class:`Index` containing :class:`datetime.datetime` objects will be returned: >>> pd.to_datetime(['2020-10-25 02:00 +0200', ... '2020-10-25 04:00 +0100']) # doctest: +SKIP FutureWarning: In a future version of pandas, parsing datetimes with mixed time zones will raise an error unless `utc=True`. Please specify `utc=True` to opt in to the new behaviour and silence this warning. To create a `Series` with mixed offsets and `object` dtype, please use `apply` and `datetime.datetime.strptime`. Index([2020-10-25 02:00:00+02:00, 2020-10-25 04:00:00+01:00], dtype='object') - A mix of timezone-aware and timezone-naive inputs is also converted to a simple :class:`Index` containing :class:`datetime.datetime` objects: >>> from datetime import datetime >>> pd.to_datetime(["2020-01-01 01:00:00-01:00", ... datetime(2020, 1, 1, 3, 0)]) # doctest: +SKIP FutureWarning: In a future version of pandas, parsing datetimes with mixed time zones will raise an error unless `utc=True`. Please specify `utc=True` to opt in to the new behaviour and silence this warning. To create a `Series` with mixed offsets and `object` dtype, please use `apply` and `datetime.datetime.strptime`. Index([2020-01-01 01:00:00-01:00, 2020-01-01 03:00:00], dtype='object') | Setting ``utc=True`` solves most of the above issues: - Timezone-naive inputs are *localized* as UTC >>> pd.to_datetime(['2018-10-26 12:00', '2018-10-26 13:00'], utc=True) DatetimeIndex(['2018-10-26 12:00:00+00:00', '2018-10-26 13:00:00+00:00'], dtype='datetime64[ns, UTC]', freq=None) - Timezone-aware inputs are *converted* to UTC (the output represents the exact same datetime, but viewed from the UTC time offset `+00:00`). >>> pd.to_datetime(['2018-10-26 12:00 -0530', '2018-10-26 12:00 -0500'], ... utc=True) DatetimeIndex(['2018-10-26 17:30:00+00:00', '2018-10-26 17:00:00+00:00'], dtype='datetime64[ns, UTC]', freq=None) - Inputs can contain both string or datetime, the above rules still apply >>> pd.to_datetime(['2018-10-26 12:00', datetime(2020, 1, 1, 18)], utc=True) DatetimeIndex(['2018-10-26 12:00:00+00:00', '2020-01-01 18:00:00+00:00'], dtype='datetime64[ns, UTC]', freq=None) >rZISO8601z8Cannot use 'exact' when 'format' is 'mixed' or 'ISO8601'zThe argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.rUrzerrors='ignore' is deprecated and will raise in a future version. Use to_datetime without passing `errors` and catch exceptions explicitly insteadNr)r~rrPrrrr~)rrrrr7rrrorp),r no_defaultrrZr[r FutureWarningrrrrvrrrrr&r|emptyrrZ _constructorrrrr%rMutableMapping_assemble_from_unit_mappingsr1rr!r r rrr,rwrxrrtr7ruryrbZbool_)rcrrPrr~rlrrrrrmrnrr{valuesZargcr7rBrBrCrsY              r:r;r<hmsrLrMrN)r:Zyearsr;monthsr<daysrFrGrHrIrJrKrLZ millisecondZ millisecondsrM microsecond microsecondsrNZ nanosecond nanoseconds)rr~c sddlm}mm}||}|jjs,tdddfdd|D}dd|D}gd }t t |t |}t |rd |} td | d t t |t t } t | rd | } td | dfdd} | ||dd| ||dd| ||d} zt| d|d} Wn:ttfyp}ztd||WYd}~n d}~00gd}|D]}||}|dur~||vr~z| || |||d7} Wn@ttfy}z"td|d||WYd}~n d}~00q~| S)a. assemble the unit specified fields from the arg (DataFrame) Return a Series for actual parsing Parameters ---------- arg : DataFrame errors : {'ignore', 'raise', 'coerce'}, default 'raise' - If :const:`'raise'`, then invalid parsing will raise an exception - If :const:`'coerce'`, then invalid parsing will be set as :const:`NaT` - If :const:`'ignore'`, then invalid parsing will return the input utc : bool Whether to convert/localize timestamps to UTC. Returns ------- Series r)r6 to_numeric to_timedeltaz#cannot assemble with duplicate keyscSs,|tvrt|S|tvr(t|S|Sr) _unit_maplower)valuerBrBrCrs   z'_assemble_from_unit_mappings..fcsi|]}||qSrBrB).0k)rrBrC z0_assemble_from_unit_mappings..cSsi|]\}}||qSrBrB)rrvrBrBrCrr)r:r;r<,zNto assemble mappings requires at least that [year, month, day] be specified: [z ] is missingz9extra keys have been passed to the datetime assemblage: [rcs(|d}t|jr$|jddd}|S)NrZint64Fr)r rqr)r)rrrBrCrs  z,_assemble_from_unit_mappings..coercer:i'r;dr<z%Y%m%d)rlrr~zcannot assemble the datetimes: N)rrrrLrMrN)rrzcannot assemble the datetimes [z]: )rtr6rrcolumnsrzrkeysitemssortedrirgjoinrrrrjget)rcrr~r6rrZunit_revrequiredreqZ _requiredexcessZ_excessrrrZunitsurrB)rrrrCrqs\     & r)rrkr)F)r_N)FN)N)NFNrNNT) ..........) ..........) ..........)r __future__r collectionsrdatetimer functoolsr itertoolsrtypingrrr r r r rZnumpyrwZ pandas._libsr rZpandas._libs.tslibsrrrrrrrZpandas._libs.tslibs.conversionrZpandas._libs.tslibs.parsingrrZpandas._libs.tslibs.strptimerZpandas._typingrrrZpandas.util._exceptionsrZpandas.core.dtypes.commonrrrr r!r"Zpandas.core.dtypes.dtypesr#r$Zpandas.core.dtypes.genericr%r&Z pandas.arraysr'r(r)Zpandas.core.algorithmsr*Zpandas.core.arraysr+Zpandas.core.arrays.baser,Zpandas.core.arrays.datetimesr-r.r/Zpandas.core.constructionr0Zpandas.core.indexes.baser1Zpandas.core.indexes.datetimesr2collections.abcr3Zpandas._libs.tslibs.nattyper4Zpandas._libs.tslibs.timedeltasr5rtr6r7rrr`rarYZScalarZ datetime64rrr9r8rEZDictConvertiblerhr^rkr|rrrrrrrrrr__all__rBrBrBrCs                        <3 DH&&&&;^