a OfW@sRUdZddlmZddlmZddlZddlmZmZddl Z ddl Z ddl Z ddl m Z ddlmZmZmZmZmZmZmZddlZddlZddlmZmZmZmZdd lmZm Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'dd l(m)Z)ddl*m+Z+m,Z,m-Z-m.Z.m/Z/ddl0m1Z1ddl2m3Z3ddl4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:ddl;mZ>m?Z?ddl@mAZAddlBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZLddlMmNZNmOZOmPZPddlQmRmSZTddlUmVZVmWZWddlXmYZYddlZm[Z[ddl\m]Z]m^Z^ddl_m`Z`ddlambZbmcZcervddldmeZemfZfmgZgddlhmiZiddljmkZkmlZlmmZmddlnmoZompZpmqZqmrZrmsZsmtZtmuZumvZvdd l\mwZwd!Zxd"Zyd#d$Zzd%d&d'd(d)Z{d*d+Z|eVZ}d,d-d.d/Z~d0Zd1ed2<d3Zd1ed4<d5Zd1ed6<d7d7d8d8d9ZeCdgiZd:Zd1ed;<d@ejd?d@eejdAejdBdeegdCdAWdn1sV0Ydad@adDdEZddId&dJd&dKd%dLd%dLdMdNdOd&d&dPdQdRdSZddId&d&dUdKdKdVdLdKdW dXdYZdZdZdLd[d\d]ZGd^d_d_ZGd`dadaZGdbdcdcZGdddedeeZGdfdgdgeZGdhdidieZGdjdkdkeZGdldmdmZGdndodoeZGdpdqdqeZGdrdsdseZGdtdudueZGdvdwdweZGdxdydyeZGdzd{d{eZGd|d}d}eZGd~ddeZGdddeZGdddeZGdddeZddddddddZdddddZeddddLddddZedddPdLddddZddddLddddZd&dd&d&dcdddZd&d&d&ddddZd&dddddZdd&d&ddddZdd&d&ddddZdd&d&d&dddZd&d&d&dddZd&dLdddZd&dd&dddZd&d&dddZddddZGdddZdS)zY High level interface to PyTables for reading and writing pandas data structures to disk ) annotations)suppressN)datetzinfo)dedent) TYPE_CHECKINGAnyCallableFinalLiteralcastoverload)config get_optionusing_copy_on_writeusing_pyarrow_string_dtype)libwriters)is_string_array) timezones)import_optional_dependency) patch_pickle)AttributeConflictWarningClosedFileErrorIncompatibilityWarningPerformanceWarningPossibleDataLossError)cache_readonly)find_stack_level) ensure_object is_bool_dtypeis_complex_dtype is_list_likeis_string_dtypeneeds_i8_conversion)CategoricalDtypeDatetimeTZDtypeExtensionDtype PeriodDtype)array_equivalent) DataFrame DatetimeIndexIndex MultiIndex PeriodIndex RangeIndexSeriesTimedeltaIndexconcatisna) Categorical DatetimeArray PeriodArray) PyTablesExprmaybe_expression) extract_array) ensure_index) ArrayManager BlockManager)stringify_path)adjoin pprint_thing)HashableIteratorSequence) TracebackType)ColFileNode) AnyArrayLike ArrayLikeAxisIntDtypeArgFilePathSelfShapenpt)Blockz0.15.2UTF-8cCst|tjr|d}|S)z(if we have bytes, decode them to unicoderP) isinstancenpbytes_decode)srVK/var/www/ai-form-bot/venv/lib/python3.9/site-packages/pandas/io/pytables.py_ensure_decodeds  rX str | Nonestr)encodingreturncCs|dur t}|SN)_default_encodingr[rVrVrW_ensure_encodingsr`cCst|trt|}|S)z Ensure that an index / column name is a str (python 3); otherwise they may be np.string dtype. Non-string dtypes are passed through unchanged. https://github.com/pandas-dev/pandas/issues/13492 )rQrZnamerVrVrW _ensure_strs rcint scope_levelcsV|dt|ttfr*fdd|D}nt|r>t|d}|dusNt|rR|SdS)z Ensure that the where is a Term or a list of Term. This makes sure that we are capturing the scope of variables that are passed create the terms here with a frame_level=2 (we are 2 levels down) cs0g|](}|durt|r(t|ddn|qS)Nrgre)r8Term).0termlevelrVrW sz _ensure_term..reN)rQlisttupler8rhlen)whererfrVrkrW _ensure_terms   rrz where criteria is being ignored as this version [%s] is too old (or not-defined), read the file in and write it out to a new file to upgrade (with the copy_to method) r incompatibility_doczu the [%s] attribute of the existing index is [%s] which conflicts with the new [%s], resetting the attribute to None attribute_conflict_docz your performance may suffer as PyTables will pickle object types that it cannot map directly to c-types [inferred_type->%s,key->%s] [items->%s] performance_docfixedtable)frvtrwz; : boolean drop ALL nan rows when appending to a table dropna_docz~ : format default format writing format, if None, then put will default to 'fixed' and append will default to 'table' format_doczio.hdfZ dropna_tableF) validatordefault_format)rvrwNcCsLtdurHddl}|att|jjdkaWdn1s>0YtS)Nrstrict) _table_modtablesrAttributeErrorfileZ_FILE_OPEN_POLICY!_table_file_open_policy_is_strict)rrVrVrW_tabless  raTr~zFilePath | HDFStoreDataFrame | Series int | Noneboolint | dict[str, int] | None bool | None Literal[True] | list[str] | NoneNone) path_or_bufkeyvaluemode complevelcomplibappendformatindex min_itemsizedropna data_columnserrorsr[r\c s|r$ f dd}n f dd}t|}t|trt||||d}||Wdq1s0Yn||dS)z+store this object, close it if we opened itc s|j d S)N)rrrnan_reprrrr[)rstore rrr[rrrrrrrrVrWszto_hdf..c s|j d S)N)rrrrrrr[rputrrrVrWr%s)rrrN)r=rQrZHDFStore)rrrrrrrrrrrrrrr[rxrrVrrWto_hdfs   (rrzstr | list | Nonezlist[str] | None) rrrrqstartstopcolumnsiterator chunksizec  Ks|dvrtd|d|dur,t|dd}t|trN|jsDtd|} d} ntt|}t|tshtd zt j |} Wnt tfyd} Yn0| st d |d t|f||d | } d } zt|dur| }t|dkrtd|d}|ddD]}t||stdq|j}| j||||||| | dWStt tfyt|tstt| Wdn1s0YYn0dS)a" Read from the store, close it if we opened it. Retrieve pandas object stored in file, optionally based on where criteria. .. warning:: Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the "fixed" format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more. Parameters ---------- path_or_buf : str, path object, pandas.HDFStore Any valid string path is acceptable. Only supports the local file system, remote URLs and file-like objects are not supported. If you want to pass in a path object, pandas accepts any ``os.PathLike``. Alternatively, pandas accepts an open :class:`pandas.HDFStore` object. key : object, optional The group identifier in the store. Can be omitted if the HDF file contains a single pandas object. mode : {'r', 'r+', 'a'}, default 'r' Mode to use when opening the file. Ignored if path_or_buf is a :class:`pandas.HDFStore`. Default is 'r'. errors : str, default 'strict' Specifies how encoding and decoding errors are to be handled. See the errors argument for :func:`open` for a full list of options. where : list, optional A list of Term (or convertible) objects. start : int, optional Row number to start selection. stop : int, optional Row number to stop selection. columns : list, optional A list of columns names to return. iterator : bool, optional Return an iterator object. chunksize : int, optional Number of rows to include in an iteration when using an iterator. **kwargs Additional keyword arguments passed to HDFStore. Returns ------- object The selected object. Return type depends on the object stored. See Also -------- DataFrame.to_hdf : Write a HDF file from a DataFrame. HDFStore : Low-level access to HDF files. Examples -------- >>> df = pd.DataFrame([[1, 1.0, 'a']], columns=['x', 'y', 'z']) # doctest: +SKIP >>> df.to_hdf('./store.h5', 'data') # doctest: +SKIP >>> reread = pd.read_hdf('./store.h5') # doctest: +SKIP )rr+rzmode zG is not allowed while performing a read. Allowed modes are r, r+ and a.Nrgrez&The HDFStore must be open for reading.Fz5Support for generic buffers has not been implemented.zFile z does not exist)rrTrz]Dataset(s) incompatible with Pandas data types, not table, or no datasets found in HDF5 file.z?key must be provided when HDF5 file contains multiple datasets.)rqrrrrr auto_close) ValueErrorrrrQris_openOSErrorr=rZNotImplementedErrorospathexists TypeErrorFileNotFoundErrorgroupsrp_is_metadata_of _v_pathnameselect LookupErrorrrclose)rrrrrqrrrrrkwargsrrrrZcandidate_only_groupZgroup_to_checkrVrVrWread_hdf<sjO           (rrF)group parent_groupr\cCsF|j|jkrdS|}|jdkrB|j}||kr:|jdkr:dS|j}qdS)zDCheck if a given group is a metadata group for a given parent_group.FrgmetaT)Z_v_depthZ _v_parent_v_name)rrcurrentparentrVrVrWrs  rc@s6eZdZUdZded<ded<ddd d d d d dZddddZeddZeddddZ ddddZ dd dddZ dd dddZ dddd Z dd dd!d"Zd#dd$d%Zddd&d'Zd(dd)d*Zd+d,d-d d.d/d0Zddd2d3d4d5Zd6dd7d8Zd9dd:d;Zddd d<d=d>Zd dd?d@Zed ddAdBZdd d dCdDdEZdddFdGZddd d d dHdIdJZddd d dKdLdMZdddd d dNdOdPZdd d d dQdRdSZdddVd d d dWdXdd d d dY dZd[Zddd dd\d]Z dddVd^d d dWd d_dXdd d` dadbZ!ddcd d dddedfZ"ddd dgd dhdidjZ#dkddldmZ$dddodpdqdrZ%ddsddtduZ&ddvddwdxZ'ddd d d d ddzd{d|Z(ddd}d~Z)d dddZ*dddddZ+dddddvdddZ,dddVd^d d dWd d dd d d ddZ-ddddZ.dd ddddZ/dddddZ0dS)raS Dict-like IO interface for storing pandas objects in PyTables. Either Fixed or Table format. .. warning:: Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the "fixed" format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more. Parameters ---------- path : str File path to HDF5 file. mode : {'a', 'w', 'r', 'r+'}, default 'a' ``'r'`` Read-only; no data can be modified. ``'w'`` Write; a new file is created (an existing file with the same name would be deleted). ``'a'`` Append; an existing file is opened for reading and writing, and if the file does not exist it is created. ``'r+'`` It is similar to ``'a'``, but the file must already exist. complevel : int, 0-9, default None Specifies a compression level for data. A value of 0 or None disables compression. complib : {'zlib', 'lzo', 'bzip2', 'blosc'}, default 'zlib' Specifies the compression library to be used. These additional compressors for Blosc are supported (default if no compressor specified: 'blosc:blosclz'): {'blosc:blosclz', 'blosc:lz4', 'blosc:lz4hc', 'blosc:snappy', 'blosc:zlib', 'blosc:zstd'}. Specifying a compression library which is not available issues a ValueError. fletcher32 : bool, default False If applying compression use the fletcher32 checksum. **kwargs These parameters will be passed to the PyTables open_file method. Examples -------- >>> bar = pd.DataFrame(np.random.randn(10, 4)) >>> store = pd.HDFStore('test.h5') >>> store['foo'] = bar # write to HDF5 >>> bar = store['foo'] # retrieve >>> store.close() **Create or load HDF5 file in-memory** When passing the `driver` option to the PyTables open_file method through **kwargs, the HDF5 file is loaded or created in-memory and will only be written when closed: >>> bar = pd.DataFrame(np.random.randn(10, 4)) >>> store = pd.HDFStore('test.h5', driver='H5FD_CORE') >>> store['foo'] = bar >>> store.close() # only now, data is written to disk z File | None_handlerZ_moderNFrrr)rr fletcher32r\cKsd|vrtdtd}|dur@||jjvr@td|jjd|durX|durX|jj}t||_|durnd}||_d|_|r|nd|_ ||_ ||_ d|_ |j fd|i|dS) Nrz-format is not a defined argument for HDFStorerzcomplib only supports z compression.rrr)rrfiltersZ all_complibsZdefault_complibr=_pathrr _complevel_complib _fletcher32_filtersopen)selfrrrrrrrrVrVrW__init__*s&  zHDFStore.__init__r\cCs|jSr]rrrVrVrW __fspath__KszHDFStore.__fspath__cCs||jdusJ|jjS)zreturn the root nodeN)_check_if_openrrootrrVrVrWrNsz HDFStore.rootcCs|jSr]rrrVrVrWfilenameUszHDFStore.filenamercCs ||Sr])getrrrVrVrW __getitem__YszHDFStore.__getitem__rr\cCs|||dSr]r)rrrrVrVrW __setitem__\szHDFStore.__setitem__cCs ||Sr])removerrVrVrW __delitem___szHDFStore.__delitem__rac CsDz ||WSttfy"Yn0tdt|jd|ddS)z$allow attribute access to get stores'z' object has no attribute 'N)rKeyErrorrrtype__name__)rrbrVrVrW __getattr__bs zHDFStore.__getattr__cCs4||}|dur0|j}|||ddfvr0dSdS)zx check for existence of this key can match the exact pathname or the pathnm w/o the leading '/' NrgTF)get_noder)rrnoderbrVrVrW __contains__ls  zHDFStore.__contains__rdcCs t|Sr])rprrrVrVrW__len__xszHDFStore.__len__cCst|j}t|d|dS)N File path:  )r?rr)rZpstrrVrVrW__repr__{s zHDFStore.__repr__rLcCs|Sr]rVrrVrVrW __enter__szHDFStore.__enter__ztype[BaseException] | NonezBaseException | NonezTracebackType | None)exc_type exc_value tracebackr\cCs |dSr])r)rrrrrVrVrW__exit__szHDFStore.__exit__pandas list[str])includer\cCs^|dkrdd|DS|dkrJ|jdus0Jdd|jjddd DStd |d dS) a Return a list of keys corresponding to objects stored in HDFStore. Parameters ---------- include : str, default 'pandas' When kind equals 'pandas' return pandas objects. When kind equals 'native' return native HDF5 Table objects. Returns ------- list List of ABSOLUTE path-names (e.g. have the leading '/'). Raises ------ raises ValueError if kind has an illegal value Examples -------- >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df) # doctest: +SKIP >>> store.get('data') # doctest: +SKIP >>> print(store.keys()) # doctest: +SKIP ['/data1', '/data2'] >>> store.close() # doctest: +SKIP rcSsg|] }|jqSrVrrinrVrVrWrmz!HDFStore.keys..nativeNcSsg|] }|jqSrVrrrVrVrWrms/Table) classnamez8`include` should be either 'pandas' or 'native' but is 'r)rrZ walk_nodesr)rrrVrVrWkeyss z HDFStore.keysz Iterator[str]cCs t|Sr])iterrrrVrVrW__iter__szHDFStore.__iter__zIterator[tuple[str, list]]ccs|D]}|j|fVqdS)z' iterate on key->group N)rr)rgrVrVrWitemss zHDFStore.items)rr\cKst}|j|krR|jdvr$|dvr$n(|dvrL|jrLtd|jd|jd||_|jr`||jr|jdkrtj|j|j|j d|_ t r|jrd }t ||j |j|jfi||_d S) a9 Open the file in the specified mode Parameters ---------- mode : {'a', 'w', 'r', 'r+'}, default 'a' See HDFStore docstring or tables.open_file for info about modes **kwargs These parameters will be passed to the PyTables open_file method. )rw)rr)rzRe-opening the file [z ] with mode [z] will delete the current file!r)rzGCannot open HDF5 file, which is already opened, even in read-only mode.N)rrrrrrrFiltersrrrrr open_filer)rrrrmsgrVrVrWrs*    z HDFStore.opencCs|jdur|jd|_dS)z0 Close the PyTables file handle N)rrrrVrVrWrs  zHDFStore.closecCs|jdurdSt|jjS)zF return a boolean indicating whether the file is open NF)rrZisopenrrVrVrWrs zHDFStore.is_open)fsyncr\cCsT|jdurP|j|rPtt t|jWdn1sF0YdS)a Force all buffered modifications to be written to disk. Parameters ---------- fsync : bool (default False) call ``os.fsync()`` on the file handle to force writing to disk. Notes ----- Without ``fsync=True``, flushing may not guarantee that the OS writes to disk. With fsync, the operation will block until the OS claims the file has been written; however, other caching layers may still interfere. N)rflushrrrrfileno)rrrVrVrWrs    zHDFStore.flushcCsVt<||}|dur*td|d||WdS1sH0YdS)a  Retrieve pandas object stored in file. Parameters ---------- key : str Returns ------- object Same type as object stored in file. Examples -------- >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df) # doctest: +SKIP >>> store.get('data') # doctest: +SKIP >>> store.close() # doctest: +SKIP NNo object named in the file)rrr _read_grouprrrrVrVrWrs  z HDFStore.get)rrrrc st||} | dur"td|dt|dd}|| fdd} t|| |j|||||d } | S) a6 Retrieve pandas object stored in file, optionally based on where criteria. .. warning:: Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the "fixed" format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more. Parameters ---------- key : str Object being retrieved from file. where : list or None List of Term (or convertible) objects, optional. start : int or None Row number to start selection. stop : int, default None Row number to stop selection. columns : list or None A list of columns that if not None, will limit the return columns. iterator : bool or False Returns an iterator. chunksize : int or None Number or rows to include in iteration, return an iterator. auto_close : bool or False Should automatically close the store when finished. Returns ------- object Retrieved object from file. Examples -------- >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df) # doctest: +SKIP >>> store.get('data') # doctest: +SKIP >>> print(store.keys()) # doctest: +SKIP ['/data1', '/data2'] >>> store.select('/data1') # doctest: +SKIP A B 0 1 2 1 3 4 >>> store.select('/data1', where='columns == A') # doctest: +SKIP A 0 1 1 3 >>> store.close() # doctest: +SKIP Nrrrgrecsj|||dS)N)rrrqrread_start_stop_whererrUrVrWfuncyszHDFStore.select..funcrqnrowsrrrrr)rrrr_create_storer infer_axes TableIteratorr get_result) rrrqrrrrrrrritrVrrWr/s(@    zHDFStore.selectrrrcCs8t|dd}||}t|ts(td|j|||dS)a return the selection as an Index .. warning:: Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the "fixed" format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more. Parameters ---------- key : str where : list of Term (or convertible) objects, optional start : integer (defaults to None), row number to start selection stop : integer (defaults to None), row number to stop selection rgrez&can only read_coordinates with a tablerqrr)rr get_storerrQrrread_coordinates)rrrqrrtblrVrVrWselect_as_coordinatess    zHDFStore.select_as_coordinates)rcolumnrrcCs,||}t|tstd|j|||dS)a~ return a single column from the table. This is generally only useful to select an indexable .. warning:: Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the "fixed" format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more. Parameters ---------- key : str column : str The column of interest. start : int or None, default None stop : int or None, default None Raises ------ raises KeyError if the column is not found (or key is not a valid store) raises ValueError if the column can not be extracted individually (it is part of a data block) z!can only read_column with a tablerrr)rrQrr read_column)rrrrrrrVrVrW select_columns#  zHDFStore.select_column)rrrc  svt|dd}t|ttfr.t|dkr.|d}t|trRj||||||| dSt|ttfshtdt|sxtd|dur|d}fdd |D |} d} t | |fgt |D]\\} } | durt d | d | jstd | jd | dur | j} q| j| krtdqdd D}dd|Dfdd}t| ||| ||||| d }|jddS)a Retrieve pandas objects from multiple tables. .. warning:: Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the "fixed" format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more. Parameters ---------- keys : a list of the tables selector : the table to apply the where criteria (defaults to keys[0] if not supplied) columns : the columns I want back start : integer (defaults to None), row number to start selection stop : integer (defaults to None), row number to stop selection iterator : bool, return an iterator, default False chunksize : nrows to include in iteration, return an iterator auto_close : bool, default False Should automatically close the store when finished. Raises ------ raises KeyError if keys or selector is not found or keys is empty raises TypeError if keys is not a list or tuple raises ValueError if the tables are not ALL THE SAME DIMENSIONS rgrer)rrqrrrrrrzkeys must be a list/tuplez keys must have a non-zero lengthNcsg|]}|qSrV)rrikrrVrWrmrz/HDFStore.select_as_multiple..zInvalid table []zobject [z>] is not a table, and cannot be used in all select as multiplez,all tables must have exactly the same nrows!cSsg|]}t|tr|qSrV)rQrrixrVrVrWrm-rcSsh|]}|jddqSr)non_index_axesriryrVrVrW 0rz.HDFStore.select_as_multiple..cs*fddD}t|ddS)Ncsg|]}|jdqS)rqrrrr r))r r rrrVrWrm5sz=HDFStore.select_as_multiple..func..F)axisverify_integrity)r2 _consolidate)r r rZobjs)r,rtblsr rWr2sz)HDFStore.select_as_multiple..funcrT coordinates)rrrQrnrorprZrrrr itertoolschainzipris_tablepathnamerpoprr)rrrqselectorrrrrrrrUrryr#Z_tblsrrrV)r,rrr/rWselect_as_multiplesd+           zHDFStore.select_as_multipleTr~rrr) rrrrrrrr track_timesrr\cCsH|durtdpd}||}|j||||||||| | | | | |ddS)a Store object in HDFStore. Parameters ---------- key : str value : {Series, DataFrame} format : 'fixed(f)|table(t)', default is 'fixed' Format to use when storing object in HDFStore. Value can be one of: ``'fixed'`` Fixed format. Fast writing/reading. Not-appendable, nor searchable. ``'table'`` Table format. Write as a PyTables Table structure which may perform worse but allow more flexible operations like searching / selecting subsets of the data. index : bool, default True Write DataFrame index as a column. append : bool, default False This will force Table format, append the input data to the existing. data_columns : list of columns or True, default None List of columns to create as data columns, or True to use all columns. See `here `__. encoding : str, default None Provide an encoding for strings. track_times : bool, default True Parameter is propagated to 'create_table' method of 'PyTables'. If set to False it enables to have the same h5 files (same hashes) independent on creation time. dropna : bool, default False, optional Remove missing values. Examples -------- >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df) # doctest: +SKIP Nio.hdf.default_formatrv) rrrrrrrrr[rr:r)r_validate_format_write_to_group)rrrrrrrrrrrr[rr:rrVrVrWrMs&8  z HDFStore.putc Cst|dd}z||}Wnty.Yntty@Ynbty}zJ|dur`td|||}|dur|jddWYd}~dSWYd}~n d}~00t |||r|j jddn|j std|j |||dSdS) a: Remove pandas object partially by specifying the where condition Parameters ---------- key : str Node to remove or delete rows from where : list of Term (or convertible) objects, optional start : integer (defaults to None), row number to start selection stop : integer (defaults to None), row number to stop selection Returns ------- number of rows removed (or None if not a Table) Raises ------ raises KeyError if key is not a valid store rgreNz5trying to remove a node with a non-None where clause!T recursivez7can only remove with where on objects written as tablesr) rrrrAssertionError ExceptionrrZ _f_removecomall_nonerr5delete)rrrqrrrUerrrrVrVrWrs2     $zHDFStore.removezbool | list[str]r) rrrrrrrrrrr\cCsl| durtd|dur td}|dur4tdp2d}||}|j||||||||| | | | ||||ddS)a| Append to Table in file. Node must already exist and be Table format. Parameters ---------- key : str value : {Series, DataFrame} format : 'table' is the default Format to use when storing object in HDFStore. Value can be one of: ``'table'`` Table format. Write as a PyTables Table structure which may perform worse but allow more flexible operations like searching / selecting subsets of the data. index : bool, default True Write DataFrame index as a column. append : bool, default True Append the input data to the existing. data_columns : list of columns, or True, default None List of columns to create as indexed data columns for on-disk queries, or True to use all columns. By default only the axes of the object are indexed. See `here `__. min_itemsize : dict of columns that specify minimum str sizes nan_rep : str to use as str nan representation chunksize : size to chunk the writing expectedrows : expected TOTAL row size of this table encoding : default None, provide an encoding for str dropna : bool, default False, optional Do not write an ALL nan row to the store settable by the option 'io.hdf.dropna_table'. Notes ----- Does *not* check if data being appended overlaps with existing data in the table, so be careful Examples -------- >>> df1 = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df1, format='table') # doctest: +SKIP >>> df2 = pd.DataFrame([[5, 6], [7, 8]], columns=['A', 'B']) >>> store.append('data', df2) # doctest: +SKIP >>> store.close() # doctest: +SKIP A B 0 1 2 1 3 4 0 5 6 1 7 8 Nz>columns is not a supported keyword in append, try data_columnszio.hdf.dropna_tabler;rw)raxesrrrrrrr expectedrowsrrr[r)rrr<r=)rrrrrFrrrrrrrrrGrrr[rrVrVrWrs6I  zHDFStore.appenddict)drr\c s|durtdt|ts"td||vr2tdttttjtt t }d} g} | D]0\} dur| durtd| } qh| qh| durֈj |} | t| } t| | } | | || <|dur||}|r*fdd|D}t|}|D]}||}qj||dd}| D]h\} | |krT|nd}j|d }|durfd d | Dnd}|j| |f||d |q>dS) a Append to multiple tables Parameters ---------- d : a dict of table_name to table_columns, None is acceptable as the values of one node (this will get all the remaining columns) value : a pandas object selector : a string that designates the indexable table; all of its columns will be designed as data_columns, unless data_columns is passed, in which case these are used data_columns : list of columns to create as data columns, or True to use all columns dropna : if evaluates to True, drop rows from all tables if any single row in each table has all NaN. Default False. Notes ----- axes parameter is currently not accepted Nztaxes is currently not accepted as a parameter to append_to_multiple; you can create the tables independently insteadzQappend_to_multiple must have a dictionary specified as the way to split the valuez=append_to_multiple requires a selector that is in passed dictzrz.HDFStore.append_to_multiple..rr,csi|]\}}|vr||qSrVrVrirr)vrVrW rz/HDFStore.append_to_multiple..)rr)rrQrHrnextrsetrangendim _AXES_MAPrrextendrF differencer,sorted get_indexertakevalues intersectionlocr7reindexr)rrIrr8rrFrrr,Z remain_keyZ remain_valuesr#orderedZorddZidxsZ valid_indexrrdcvalfilteredrV)rPrrWappend_to_multiple8sZ &    zHDFStore.append_to_multiplerY)roptlevelkindr\cCsBt||}|durdSt|ts.td|j|||ddS)a Create a pytables index on the table. Parameters ---------- key : str columns : None, bool, or listlike[str] Indicate which columns to create an index on. * False : Do not create any indexes. * True : Create indexes on all columns. * None : Create indexes on all columns. * listlike : Create indexes on the given columns. optlevel : int or None, default None Optimization level, if None, pytables defaults to 6. kind : str or None, default None Kind of index, if None, pytables defaults to "medium". Raises ------ TypeError: raises if the node is not a table Nz1cannot create table index on a Fixed format store)rrerf)rrrQrr create_index)rrrrerfrUrVrVrWcreate_table_indexs  zHDFStore.create_table_indexrncCs<t||jdusJtdus(Jdd|jDS)a Return a list of all the top-level nodes. Each node returned is not a pandas storage object. Returns ------- list List of objects. Examples -------- >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df) # doctest: +SKIP >>> print(store.groups()) # doctest: +SKIP >>> store.close() # doctest: +SKIP [/data (Group) '' children := ['axis0' (Array), 'axis1' (Array), 'block0_values' (Array), 'block0_items' (Array)]] NcSsPg|]H}t|tjjst|jddsHt|ddsHt|tjjr|jdkr|qS) pandas_typeNrw) rQrlinkLinkgetattr_v_attrsrwrr)rirrVrVrWrms  z#HDFStore.groups..)rrrr walk_groupsrrVrVrWrs zHDFStore.groupsrz*Iterator[tuple[str, list[str], list[str]]])rqr\ccst||jdusJtdus(J|j|D]}t|jdddurLq4g}g}|jD]B}t|jdd}|durt |tj j r| |j q^| |j q^|jd||fVq4dS)a Walk the pytables group hierarchy for pandas objects. This generator will yield the group path, subgroups and pandas object names for each group. Any non-pandas PyTables objects that are not a group will be ignored. The `where` group itself is listed first (preorder), then each of its child groups (following an alphanumerical order) is also traversed, following the same procedure. Parameters ---------- where : str, default "/" Group where to start walking. Yields ------ path : str Full path to a group (without trailing '/'). groups : list Names (strings) of the groups contained in `path`. leaves : list Names (strings) of the pandas objects contained in `path`. Examples -------- >>> df1 = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df1, format='table') # doctest: +SKIP >>> df2 = pd.DataFrame([[5, 6], [7, 8]], columns=['A', 'B']) >>> store.append('data', df2) # doctest: +SKIP >>> store.close() # doctest: +SKIP >>> for group in store.walk(): # doctest: +SKIP ... print(group) # doctest: +SKIP >>> store.close() # doctest: +SKIP Nrir)rrrrrnrlrmZ _v_childrenr\rQrGrouprrrrstrip)rrqrrleaveschildrirVrVrWwalks ' z HDFStore.walkz Node | NonecCs~||dsd|}|jdus(Jtdus4Jz|j|j|}Wntjjy`YdS0t|tj szJt ||S)z9return the node with the key or None if it does not existrN) r startswithrrrr exceptionsZNoSuchNodeErrorrQrFr)rrrrVrVrWr$s  zHDFStore.get_nodeGenericFixed | TablecCs8||}|dur"td|d||}||S)z| | vrj|rj| | || } t| trd} |rdd| j D} | j | | | t | dd| j dq>| j | | | j dq>| S) a; Copy the existing store to a new file, updating in place. Parameters ---------- propindexes : bool, default True Restore indexes in copied file. keys : list, optional List of keys to include in the copy (defaults to all). overwrite : bool, default True Whether to overwrite (remove and replace) existing nodes in the new store. mode, complib, complevel, fletcher32 same as in HDFStore.__init__ Returns ------- open file handle of the new store )rrrrNFcSsg|]}|jr|jqSrV) is_indexedrbrirrVrVrWrmlrz!HDFStore.copy..r)rrr[r_)rrnrrQrorrrrrFrrlr[r)rrrrwrrrrrxZ new_storer#rUdatarrVrVrWcopy>s6       z HDFStore.copyc Cs t|j}t|d|d}|jrt|}t|rg}g}|D]}z<||}|dur|t|j pj||t|p|dWqDt yYqDt y}z0||t|} |d| dWYd}~qDd}~00qD|t d||7}n|d7}n|d 7}|S) a Print detailed information on the store. Returns ------- str Examples -------- >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']) >>> store = pd.HDFStore("store.h5", 'w') # doctest: +SKIP >>> store.put('data', df) # doctest: +SKIP >>> print(store.info()) # doctest: +SKIP >>> store.close() # doctest: +SKIP File path: store.h5 /data frame (shape->[2,2]) rrNzinvalid_HDFStore nodez[invalid_HDFStore node: r$ EmptyzFile is CLOSED) r?rrrrYrrprrr6r@rAr>) rroutputZlkeysrr\r#rUdetailZdstrrVrVrWinfoys.     * z HDFStore.infocCs|jst|jddS)Nz file is not open!)rrrrrVrVrWrszHDFStore._check_if_open)rr\c CsLzt|}Wn6tyF}ztd|d|WYd}~n d}~00|S)zvalidate / deprecate formatsz#invalid HDFStore format specified [r$N) _FORMAT_MAPlowerrr)rrrErVrVrWr<s (zHDFStore._validate_formatrPzDataFrame | Series | None)rr[rr\c Cs>|durt|ttfstdtt|jdd}tt|jdd}|dur|durttdusdJt|dds~t|tj j rd}d}qtdn$t|trd }nd }|dkr|d 7}d|vr.t t d }z ||} WnFt y} z,td |dt|d|| WYd} ~ n d} ~ 00| ||||dS|dur|dur|dkrt|dd} | dur| jdkrtd}n| jdkrd}nB|dkrt|dd} | dur| jdkrd}n| jdkrd}ttttttd} z | |} WnFt y,} z,td|dt|d|| WYd} ~ n d} ~ 00| ||||dS)z"return a suitable class to operateNz(value must be None, Series, or DataFrameri table_typerw frame_table generic_tablezKcannot create a storer if the object is not existing nor a value are passedseriesframe_table)rrz=cannot properly create the storer for: [_STORER_MAP] [group->,value->z ,format->r[r series_tablerrgappendable_seriesappendable_multiseriesappendable_frameappendable_multiframe)rrrrrwormz)rQr0r*rrXrlrmrrrwr SeriesFixed FrameFixedrrnlevels GenericTableAppendableSeriesTableAppendableMultiSeriesTableAppendableFrameTableAppendableMultiFrameTable WORMTable) rrrrr[rptttZ _STORER_MAPclsrErZ _TABLE_MAPrVrVrWrs                   zHDFStore._create_storer) rrrrrrrrrr:r\cCst|ddr|dks|rdS|||}|j|||||d}|rr|jrZ|jrb|dkrb|jrbtd|jsz|n||js|rtd|j|||||| | | | | |||d t|t r|r|j |ddS) NemptyrwrrvzCan only append to Tablesz0Compression not supported on Fixed format stores) objrFrrrrrrrGrrrr:)r) rl_identify_grouprr5 is_existsrset_object_infowriterQrrg)rrrrrFrrrrrrrrGrrrr[rr:rrUrVrVrWr=s:   zHDFStore._write_to_grouprFrcCs||}||Sr])rrr )rrrUrVrVrWrUs zHDFStore._read_group)rrr\cCsN||}|jdusJ|dur8|s8|jj|ddd}|durJ||}|S)z@Identify HDF5 group based on key, delete/create group if needed.NTr>)rr remove_node_create_nodes_and_group)rrrrrVrVrWrZs   zHDFStore._identify_groupcCsv|jdusJ|d}d}|D]P}t|s.q |}|dsD|d7}||7}||}|durl|j||}|}q |S)z,Create nodes from key and return group name.Nr)rsplitrpendswithrZ create_group)rrpathsrpnew_pathrrVrVrWrls   z HDFStore._create_nodes_and_group)rNNF)r)r)F)NNNNFNF)NNN)NN)NNNNNFNF) NTFNNNNNNr~TF)NNN)NNTTNNNNNNNNNNr~)NNF)NNN)r)rTNNNFT)NNrPr~)NTFNNNNNNFNNNr~T)1r __module__ __qualname____doc____annotations__rrpropertyrrrrrrrrrrrrrrrrrrrrrr!r9rrrrdrhrrsrrr|rrr<rr=rrrrVrVrVrWrs A!   *- `$+}$L=$kd('< ;5 `$>rc@speZdZUdZded<ded<ded<dddd dd d d d dZddddZd dddZdd dddZdS)raa Define the iteration interface on a table Parameters ---------- store : HDFStore s : the referred storer func : the function to execute the query where : the where of the query nrows : the rows to iterate on start : the passed start value (default is None) stop : the passed stop value (default is None) iterator : bool, default False Whether to use the default iterator. chunksize : the passed chunking value (default is 100000) auto_close : bool, default False Whether to automatically close the store at the end of iteration. rrrrrvrUNFrr)rrUrrrr\c Cs||_||_||_||_|jjrN|dur,d}|dur8d}|durD|}t||}||_||_||_d|_ |sr| dur| dur~d} t | |_ nd|_ | |_ dS)Nr順) rrUrrqr5minrrrr1rdrr) rrrUrrqrrrrrrrVrVrWrs,    zTableIterator.__init__rArccsv|j}|jdurtd||jkrjt||j|j}|dd|j||}|}|dust|sbq|Vq|dS)Nz*Cannot iterate until get_result is called.) rr1rrrrrrpr)rrrrrVrVrWrs  zTableIterator.__iter__cCs|jr|jdSr])rrrrrVrVrWrszTableIterator.closer0cCs|jdur4t|jtstd|jj|jd|_|S|rft|jtsLtd|jj|j|j|j d}n|j}| |j|j |}| |S)Nz0can only use an iterator or chunksize on a table)rqz$can only read_coordinates on a tabler) rrQrUrrrrqr1rrrr)rr1rqresultsrVrVrWrs    zTableIterator.get_result)NNFNF)F) rrrrrrrrrrVrVrVrWrs  *rc @seZdZUdZdZded<dZded<gdZdKdd d d d d Ze ddddZ e ddddZ dd dddZ ddddZ dddddZddddZe dddd Zd!ddd"d#d$d%Zd&d'Ze d(d)Ze d*d+Ze d,d-Ze d.d/Zd0dd1d2ZdLd dd3d4Zd dd5d6Zd7dd d8d9d:ZdMd;d<Zdd d=d>d?Zd dd@dAZd ddBdCZd ddDdEZd7d dFdGdHZ d7d dFdIdJZ!dS)NIndexCola an index column description class Parameters ---------- axis : axis which I reference values : the ndarray like converted values kind : a string description of this type typ : the pytables type pos : the position in the pytables Tris_an_indexableis_data_indexable)freqtz index_nameNrZrYr)rbcnamer\cCst|tstd||_||_||_||_|p0||_||_||_ ||_ | |_ | |_ | |_ | |_| |_||_|dur|||t|jtsJt|jtsJdS)Nz`name` must be a str.)rQrZrr\rftyprbrr,posrrrr`rwrmetadataset_pos)rrbr\rfrrr,rrrrr`rwrrrVrVrWrs(   zIndexCol.__init__rdrcCs|jjSr])ritemsizerrVrVrWr/szIndexCol.itemsizecCs |jdS)N_kindrarrVrVrW kind_attr4szIndexCol.kind_attr)rr\cCs$||_|dur |jdur ||j_dS)z,set the position of this column in the TableN)rrZ_v_pos)rrrVrVrWr8szIndexCol.set_poscCs@ttt|j|j|j|j|jf}dddt gd|DS)N,cSsg|]\}}|d|qSz->rVrOrVrVrWrmCsz%IndexCol.__repr__..)rbrr,rrf) romapr?rbrr,rrfjoinr4rtemprVrVrWr>s zIndexCol.__repr__objectotherr\cstfdddDS)compare 2 col itemsc3s&|]}t|dt|dkVqdSr]rlrzrrrVrWrMKsz"IndexCol.__eq__..)rbrr,rrJrrrVrrW__eq__IszIndexCol.__eq__cCs || Sr])rrrVrVrW__ne__PszIndexCol.__ne__cCs"t|jdsdSt|jj|jjS)z%return whether I am an indexed columnrLF)hasattrrwrlrLrryrrVrVrWrySs zIndexCol.is_indexed np.ndarrayz3tuple[np.ndarray, np.ndarray] | tuple[Index, Index]r\r[rr\c Cst|tjsJt||jjdur2||j}t|j }t ||||}i}t|j |d<|j durtt|j |d<t }t|jdst|jtrt}n|jdkrd|vrdd}z||fi|}Wn2tyd|vrd|d<||fi|}Yn0t||j} | | fS)zV Convert the data from this selection to the appropriate pandas type. NrbrMi8c[s tj||ddd|dS)Nr)rrb)r.Z from_ordinalsrZ_rename)r&kwdsrVrVrWr{s z"IndexCol.convert..)rQrRndarrayrdtypefieldsrr|rXrf_maybe_convertrrr,r is_np_dtyper&r+r_set_tzr) rr\rr[rval_kindrfactoryZ new_pd_indexZfinal_pd_indexrVrVrWconvert[s0     zIndexCol.convertcCs|jS)zreturn the valuesr\rrVrVrW take_dataszIndexCol.take_datacCs|jjSr])rwrmrrVrVrWattrsszIndexCol.attrscCs|jjSr]rw descriptionrrVrVrWrszIndexCol.descriptioncCst|j|jdS)z!return my current col descriptionN)rlrrrrVrVrWcolsz IndexCol.colcCs|jSzreturn my cython valuesrrrVrVrWcvaluesszIndexCol.cvaluesrAcCs t|jSr])rr\rrVrVrWrszIndexCol.__iter__cCsPt|jdkrLt|tr$||j}|durL|jj|krLtj ||j d|_dS)z maybe set a string col itemsize: min_itemsize can be an integer or a dict with this columns name with an integer size stringN)rr) rXrfrQrHrrbrrr StringColr)rrrVrVrWmaybe_set_sizes   zIndexCol.maybe_set_sizecCsdSr]rVrrVrVrWvalidate_namesszIndexCol.validate_namesAppendableTable)handlerrr\cCs:|j|_||||||||dSr])rw validate_col validate_attrvalidate_metadatawrite_metadataset_attr)rrrrVrVrWvalidate_and_sets    zIndexCol.validate_and_setcCs^t|jdkrZ|j}|durZ|dur*|j}|j|krTtd|d|jd|jd|jSdS)z:validate this column: return the compared against itemsizerNz#Trying to store a string with len [z] in [z)] column but this column has a limit of [zC]! Consider using min_itemsize to preset the sizes on these columns)rXrfrrrr)rrcrVrVrWrs zIndexCol.validate_col)rr\cCsB|r>t|j|jd}|dur>||jkr>td|d|jddS)Nzincompatible kind in col [ - r$)rlrrrfr)rrZ existing_kindrVrVrWrs zIndexCol.validate_attrc Cs|jD]}t||d}||ji}||}||vr|dur||kr|dvrt|||f}tj|tt dd||<t ||dqt d|jd|d|d|d q|dus|dur|||<qdS) z set/update the info for this indexable with the key/value if there is a conflict raise/warn as needed N)rr stacklevelzinvalid info for [z] for [z], existing_value [z] conflicts with new value [r$) _info_fieldsrl setdefaultrbrrtwarningswarnrrsetattrr)rrrridxZexisting_valuewsrVrVrW update_infos*   zIndexCol.update_infocCs$||j}|dur |j|dS)z!set my state from the passed infoN)rrb__dict__update)rrrrVrVrWset_infos zIndexCol.set_infocCst|j|j|jdS)zset the kind for this columnN)rrrrfrrVrVrWrszIndexCol.set_attr)rr\cCsH|jdkrD|j}||j}|durD|durDt||dddsDtddS)z:validate that kind=category does not change the categoriescategoryNTZ strict_nanZ dtype_equalzEcannot append a categorical with different categories to the existing)rr read_metadatarr)r)rrZ new_metadataZ cur_metadatarVrVrWr s  zIndexCol.validate_metadatacCs|jdur||j|jdS)zset the meta dataN)rrr)rrrVrVrWr s zIndexCol.write_metadata) NNNNNNNNNNNNN)N)N)"rrrrrrrrrrrrrrrrryrrrrrrrrrrrrrrrrrrVrVrVrWrsb   + 2       rc@sDeZdZdZeddddZddddd d d Zd dd dZdS)GenericIndexColz:an index which is not represented in the data of the tablerrcCsdSNFrVrrVrVrWry szGenericIndexCol.is_indexedrrZztuple[Index, Index]rcCs,t|tjsJt|tt|}||fS)z Convert the data from this selection to the appropriate pandas type. Parameters ---------- values : np.ndarray nan_rep : str encoding : str errors : str )rQrRrrr/rp)rr\rr[rrrVrVrWr! s  zGenericIndexCol.convertrcCsdSr]rVrrVrVrWr3 szGenericIndexCol.set_attrN)rrrrrryrrrVrVrVrWr s rc s>eZdZdZdZdZddgZd;dddd d fd d Zedd ddZ edd ddZ dd ddZ dddddZ dd dddZ ddZeddd d!d"Zed#d$Zedd%d&d'd(Zeddd&d)d*Zed+d,Zed-d.Zed/d0Zed1d2Zd d d3d4Zd5ddd6d7d8Zd d d9d:ZZS)<DataCola3 a data holding column, by definition this is not indexable Parameters ---------- data : the actual data cname : the column name in the table to hold the data (typically values) meta : a string description of the metadata metadata : the actual metadata Frr`NrZrYzDtypeArg | Noner)rbrrr\c s2tj||||||||| | | d | |_| |_dS)N) rbr\rfrrrrr`rwrr)superrrr{)rrbr\rfrrrrr`rwrrrr{ __class__rVrWrH s zDataCol.__init__rcCs |jdS)N_dtyperarrVrVrW dtype_attrh szDataCol.dtype_attrcCs |jdS)N_metararrVrVrW meta_attrl szDataCol.meta_attrcCs@ttt|j|j|j|j|jf}dddt gd|DS)NrcSsg|]\}}|d|qSrrVrOrVrVrWrmw sz$DataCol.__repr__..)rbrrrfshape) rorr?rbrrrfrrr4rrVrVrWrp s zDataCol.__repr__rrrcstfdddDS)rc3s&|]}t|dt|dkVqdSr]rrzrrVrWrM sz!DataCol.__eq__..)rbrrrrrrVrrWr} szDataCol.__eq__rH)r{r\cCs@|dus J|jdusJt|\}}||_||_t||_dSr])r_get_data_and_dtype_namer{_dtype_to_kindrf)rr{ dtype_namerVrVrWset_data s   zDataCol.set_datacCs|jS)zreturn the datar{rrVrVrWr szDataCol.take_datarD)r\r\cCs|j}|j}|j}|jdkr&d|jf}t|trJ|j}|j||jj d}n~t |ds`t|t rl| |}n\t |dr||}nDt|rtj||dd}n&t|r|||}n|j||j d}|S)zW Get an appropriately typed and shaped pytables.Col object for values. rgrfrmrrr)rrrrUsizerQr4codes get_atom_datarbrrr&get_atom_datetime64get_atom_timedelta64r!rZ ComplexColr#get_atom_string)rr\rrrratomrVrVrW _get_atom s$      zDataCol._get_atomcCstj||ddS)NrrrrrrrrVrVrWr szDataCol.get_atom_stringz type[Col]rfr\cCsR|dr$|dd}d|d}n"|dr4d}n|}|d}tt|S)z0return the PyTables column class for this columnuintNZUIntrDperiodInt64Col)rt capitalizerlr)rrfZk4Zcol_nameZkcaprVrVrWget_atom_coltype s    zDataCol.get_atom_coltypecCs|j|d|ddS)Nrrrr*rrrfrVrVrWr szDataCol.get_atom_datacCstj|ddSNrr+rr(rrrVrVrWr szDataCol.get_atom_datetime64cCstj|ddSr.r/r0rVrVrWr szDataCol.get_atom_timedelta64cCst|jddS)Nr)rlr{rrVrVrWr sz DataCol.shapecCs|jSrrrrVrVrWr szDataCol.cvaluescCs`|r\t|j|jd}|dur2|t|jkr2tdt|j|jd}|dur\||jkr\tddS)zAvalidate that we have the same order as the existing & same dtypeNz4appended items do not match existing items in table!z@appended items dtype do not match existing items dtype in table!)rlrrrnr\rrr)rrZexisting_fieldsZexisting_dtyperVrVrWr szDataCol.validate_attrr)r\r[rcCst|tjsJt||jjdur.||j}|jdus.cSsg|]}t|qSrVr fromtimestampr9rVrVrWrm" rrF) categoriesr`validater|Orrr[r)#rQrRrrrrrrrrrfrXrrr`rrtrasarrayrrravelr,Zfloat64r3anyastyperdZcumsum_valuesr4Z from_codesr_unconvert_string_arrayr\)rr\rr[r convertedrrfrrr`rrr=rmaskrVrVrWr sf               zDataCol.convertcCsHt|j|j|jt|j|j|j|jdus2Jt|j|j|jdS)zset the data for this columnN)rrrr\rrrrrrVrVrWrK szDataCol.set_attr) NNNNNNNNNNNN)rrrrrrrrrrrrrrr classmethodr!rr*rrrrrrrr __classcell__rVrVr rWr 7 sX          dr c@sZeZdZdZdZddddZeddZed d d d d ZeddZ eddZ dS)DataIndexableColz+represent a data column that can be indexedTrrcCstt|jjstddS)N-cannot have non-object label DataIndexableCol)r#r,r\rrrrVrVrWrX szDataIndexableCol.validate_namescCstj|dS)N)rr"r#rVrVrWr] sz DataIndexableCol.get_atom_stringrZrDr$cCs|j|dS)Nrr,r-rVrVrWra szDataIndexableCol.get_atom_datacCs tSr]r/r0rVrVrWre sz$DataIndexableCol.get_atom_datetime64cCs tSr]r/r0rVrVrWri sz%DataIndexableCol.get_atom_timedelta64N) rrrrrrrJrrrrrVrVrVrWrLS s  rLc@seZdZdZdS)GenericDataIndexableColz(represent a generic pytables data columnN)rrrrrVrVrVrWrNn srNc@seZdZUdZded<dZded<ded<ded <d ed <d Zd ed<dNd dddddddZed dddZ eddddZ eddZ dddd Z ddd!d"Z ddd#d$Zed%d&Zed'd(Zed)d*Zed+d,Zeddd-d.Zed dd/d0Zed1d2Zddd3d4Zddd5d6Zed7d8Zed dd9d:Zed;d<Zd=dd>d?ZdOdddAdBZd ddCdDZdPdEdEdFdGdHZdddIdJZdQdEdEddKdLdMZ d@S)RFixedz represent an object in my store facilitate read/write of various types of objects this is an abstract base class Parameters ---------- parent : HDFStore group : Node The group node where the table resides. rZ pandas_kindrv format_typetype[DataFrame | Series]obj_typerdrUrrFrr5rPr~rFrYr)rrr[rr\cCsZt|tsJt|tdus"Jt|tjs:Jt|||_||_t||_||_ dSr]) rQrrrrFrrr`r[r)rrrr[rrVrVrWr s  zFixed.__init__rcCs*|jddko(|jddko(|jddkS)Nrrg )versionrrVrVrWis_old_version szFixed.is_old_versionztuple[int, int, int]cCs`tt|jjdd}z0tdd|dD}t|dkrB|d}WntyZd}Yn0|S) zcompute and set our versionpandas_versionNcss|]}t|VqdSr]rdr%rVrVrWrM rz Fixed.version...rUr')rrr)rXrlrrmrorrpr)rrVrVrVrWrV s    z Fixed.versioncCstt|jjddS)Nri)rXrlrrmrrVrVrWri szFixed.pandas_typecCs^||j}|durXt|ttfrDddd|D}d|d}|jdd|d S|jS) (return a pretty representation of myselfNrcSsg|] }t|qSrVr?r%rVrVrWrm rz"Fixed.__repr__..[r$12.12z (shape->))rrrQrnrorri)rrUZjshaperVrVrWr s zFixed.__repr__cCst|j|j_tt|j_dS)zset my pandas type & versionN)rZrPrri_versionrXrrVrVrWr szFixed.set_object_infocCst|}|Sr]r?)rZnew_selfrVrVrWr| s z Fixed.copycCs|jSr])rrrVrVrWr sz Fixed.shapecCs|jjSr]rrrrVrVrWr6 szFixed.pathnamecCs|jjSr])rrrrVrVrWr sz Fixed._handlecCs|jjSr])rrrrVrVrWr szFixed._filterscCs|jjSr])rrrrVrVrWr szFixed._complevelcCs|jjSr])rrrrVrVrWr szFixed._fletcher32cCs|jjSr])rrmrrVrVrWr sz Fixed.attrscCsdSzset our object attributesNrVrrVrVrW set_attrs szFixed.set_attrscCsdS)zget our object attributesNrVrrVrVrW get_attrs szFixed.get_attrscCs|jS)zreturn my storablerrrVrVrWstorable szFixed.storablecCsdSrrVrrVrVrWr szFixed.is_existscCst|jddS)Nr)rlrerrVrVrWr sz Fixed.nrowszLiteral[True] | NonecCs|dur dSdS)z%validate against an existing storableNTrVrrVrVrWr> szFixed.validateNcCsdS)+are we trying to operate on an old version?NrV)rrqrVrVrWvalidate_version szFixed.validate_versioncCs|j}|durdS|dS)zr infer the axes of my storer return a boolean indicating if we have a valid storer or not NFT)rerd)rrUrVrVrWr s zFixed.infer_axesrrrcCs tddS)Nz>cannot read on an abstract storer: subclasses should implementrrrqrrrrVrVrWr  sz Fixed.readcKs tddS)Nz?cannot write on an abstract storer: subclasses should implementrirrrrVrVrWr sz Fixed.writerrr\cCs0t|||r$|jj|jdddStddS)zs support fully deleting the node in its entirety (only) - where specification must be None Tr>Nz#cannot delete on an abstract storer)rBrCrrrr)rrqrrrVrVrWrD sz Fixed.delete)rPr~)N)NNNN)NNN)!rrrrrrQr5rrrWrVrirrr|rr6rrrrrrcrdrerrr>rgrr rrDrVrVrVrWrOr sh                rOc@sFeZdZUdZedediZddeDZgZ de d<dd d d Z d d Z ddZ dd ddZedd ddZdd ddZdd ddZdd ddZd:dddddd Zd;dddd!d"d#d$Zdd!dd%d&d'Zdd(dd%d)d*Zddd5d6dd7d8d9ZdS)? GenericFixedza generified fixed versiondatetimer'cCsi|]\}}||qSrVrV)rir#rPrVrVrWrQ" rzGenericFixed.r attributesrZrcCs|j|dS)N)_index_type_mapr)rrrVrVrW_class_to_alias& szGenericFixed._class_to_aliascCst|tr|S|j|tSr])rQr_reverse_index_maprr,)raliasrVrVrW_alias_to_class) s zGenericFixed._alias_to_classcCs|tt|dd}|tkr.d dd}|}n|tkrFd dd}|}n|}i}d|vrn|d|d<|turnt}d|vrt|dtr|d d|d<n |d|d<|tusJ||fS) N index_classrpcSs>tj|j|j|d}tj|dd}|dur:|d|}|S)N)rrraUTC)r5 _simple_newr\rr+ tz_localize tz_convert)r\rrZdtaresultrVrVrWrx8 s z*GenericFixed._get_index_factory..fcSs$t|}tj||d}tj|ddS)Nr6ra)r(r6rxr.)r\rrrZparrrVrVrWrxE srrzutf-8)NN)NN) rurXrlr+r.r,r1rQbytesrT)rrrvrxrrrVrVrW_get_index_factory/ s*     zGenericFixed._get_index_factoryrcCs$|durtd|dur tddS)zE raise if any keywords are passed which are not-None Nzqcannot pass a column specification when reading a Fixed format store. this store must be selected in its entiretyzucannot pass a where specification when reading from a Fixed format store. this store must be selected in its entirety)r)rrrqrVrVrW validate_read` szGenericFixed.validate_readrcCsdS)NTrVrrVrVrWro szGenericFixed.is_existscCs|j|j_|j|j_dSrb)r[rrrrVrVrWrcs s zGenericFixed.set_attrsc CsRtt|jdd|_tt|jdd|_|jD]}t||tt|j|dq.dS)retrieve our attributesr[Nrr~)r`rlrr[rXrror)rrrVrVrWrdx s zGenericFixed.get_attrscKs |dSr])rcrkrVrVrWr szGenericFixed.writeNrrc Csddl}t|j|}|j}t|dd}t||jrD|d||}ntt|dd} t|dd} | durxtj| | d}n |||}| r| drt|d d} t || d d }n| d krtj |d d}|r|j S|SdS)z2read an array for the specified node (off of grouprN transposedF value_typerr6r1rTr2r4r5) rrlrrmrQZVLArrayrXrRrrtrrBT) rrrrrrrrretrrrrVrVrW read_array s&      zGenericFixed.read_arrayr,)rrrr\cCshtt|j|d}|dkr.|j|||dS|dkrVt|j|}|j|||d}|Std|dS)N_varietymultirhregularzunrecognized index variety: )rXrlrread_multi_indexrread_index_noder)rrrrZvarietyrrrVrVrW read_index s zGenericFixed.read_index)rrr\cCst|tr,t|j|dd|||nt|j|ddtd||j|j}|||j t |j |}|j |j _ |j|j _t|ttfr|t||j _t|tttfr|j|j _t|tr|jdurt|j|j _dS)Nrrrr)rQr-rrwrite_multi_index_convert_indexr[r write_arrayr\rlrrfrmrbr+r.rrrrvr1rr_get_tz)rrrrHrrVrVrW write_index s     zGenericFixed.write_indexr-c Cst|j|d|jtt|j|j|jD]\}\}}}t|j t rNt d|d|}t |||j |j}|||jt|j|} |j| j_|| j_t| j|d|||d|} || |q,dS)N_nlevelsz=Saving a MultiIndex with an extension dtype is not supported._level_name_label)rrr enumerater4levelsrnamesrQrr'rrr[rrr\rlrrfrmrb) rrrilev level_codesrb level_keyZ conv_levelr label_keyrVrVrWr s"   zGenericFixed.write_multi_indexcCst|j|d}g}g}g}t|D]l}|d|} t|j| } |j| ||d} || || j|d|} |j| ||d} || q&t|||ddS)NrrrhrT)rrrr-) rlrrTrrrrbrr-)rrrrrrrrrrrrrrrVrVrWr s      zGenericFixed.read_multi_indexrF)rrrr\c Cs|||}d|jvr>t|jjdkr>tj|jj|jjd}t|jj}d}d|jvrlt|jj }t|}|j}| |\}} |dvr|t |||j |j dfdti| } n |t |||j |j dfi| } || _ | S)Nrrr6rb)rrrr)rmrRprodrrrrXrfrcrbr}_unconvert_indexr[rr) rrrrr{rfrbrrrrrVrVrWr s:      zGenericFixed.read_index_noderH)rrr\cCsJtd|j}|j|j||t|j|}t|j|j _ |j |j _ dS)zwrite a 0-len arrayrgN) rRrrUr create_arrayrrlrZrrmrr)rrrZarrrrVrVrWwrite_array_empty s  zGenericFixed.write_array_emptyrGz Index | None)rrrr\c Csht|dd}||jvr&|j|j||jdk}d}t|jtrHtd|s`t |dr`|j }d}d}|j durt t  tj|j}Wdn1s0Y|dur|s|jj|j|||j|j d}||dd<n |||nh|jjtjkrbtj|dd} |rn,| d krn t| ||f} tj| ttd |j|j|t} | |nt!|jd r|j"|j||#d t$|jt%|j|j&_'nt|jt(r|j"|j||j)t%|j|} t*|j+| j&_+d |jj,d| j&_'n`t!|jdr.|j"|j||#d dt%|j|j&_'n&|rB|||n|j"|j|||t%|j|j&_-dS)NT)Z extract_numpyrFz]Cannot store a category dtype in a HDF5 dataset that uses format="fixed". Use format="table".r)rZskipnarrrr datetime64[r$rr4).r9rrrrrQrr%rrrrrrrZAtomZ from_dtypeZ create_carrayrrrrRZobject_r infer_dtyperurrrrZcreate_vlarray ObjectAtomrrrviewrZrlrmrr&asi8rrunitr) rrrrrZ empty_arrayrr ca inferred_typerZvlarrrrVrVrWr( sf       .    zGenericFixed.write_array)NN)NN)NN)NN)N)rrrrr+r.rqrrsrorrrrur}r~rrrcrdrrrrrrrrrrVrVrVrWrm s2   1#& rmcsVeZdZUdZdgZded<eddZddddd d d Zd d fdd Z Z S)rrrbr@c Cs.zt|jjfWSttfy(YdS0dSr])rprr\rrrrVrVrWr szSeriesFixed.shapeNrr0rlcCs^||||jd||d}|jd||d}t|||jdd}trZt|ddrZ|d}|S) Nrrhr\F)rrbr|Trstring[pyarrow_numpy])r~rrr0rbrrrE)rrqrrrrr\r{rVrVrWr  s  zSeriesFixed.readrrc s<tj|fi||d|j|d||j|j_dS)Nrr\)r rrrrrbrrkr rVrWr s zSeriesFixed.write)NNNN) rrrrProrrrr rrKrVrVr rWr s  rcsZeZdZUddgZded<eddddZdd d d d d d Zddfdd ZZ S)BlockManagerFixedrUnblocksrdz Shape | NonercCsz|j}d}t|jD]8}t|jd|d}t|dd}|dur||d7}q|jj}t|dd}|durt|d|d}ng}|||WStyYdS0dS)Nrblock_itemsrrg) rUrTrrlrZ block0_valuesrnrr)rrUrrrrrVrVrWr s"    zBlockManagerFixed.shapeNrr*rlcCsH||||d}g}t|jD]<}||kr<||fnd\}} |jd||| d} || q(|d} g} t|jD]x}|d|d} |jd|d|| d}| | | }t |j ||dd d }t rt |d d r|d }| |q|t| dkr4t| dd d}tr"|}|j| d d}|St |d|ddS)Nr)NNr,rhrrrFrgFrrr|Trr)r,r|)rr|rr)r~rSZ_get_block_manager_axisrTrUrrrrrZr*rrrrErpr2rr|r_)rrqrrrZ select_axisrFrr r axrdfs blk_itemsr\dfoutrVrVrWr  s0    zBlockManagerFixed.readrc stj|fi|t|jtr*|d}|j}|s@|}|j|j _t |j D]0\}}|dkrr|j srt d|d||qTt|j|j _t |jD]D\}}|j|j}|jd|d|j|d|d|d|qdS)Nrrz/Columns index has to be unique for fixed formatr,rF)rr)r rrQ_mgrr; _as_managerZis_consolidatedZ consolidaterUrrrFZ is_uniquerrrpblocksrrr[mgr_locsrr\)rrrr{rrblkrr rVrWr s    zBlockManagerFixed.write)NNNN) rrrrorrrr rrKrVrVr rWr s )rc@seZdZdZeZdS)rrN)rrrrPr*rSrVrVrVrWr src seZdZUdZdZdZded<ded<dZded <d Zd ed <dddddddddddd fdd Z e ddddZ ddddZ dd d!d"Z ddd#d$Ze d%dd&d'Zd(d)d*d+d,Ze d-dd.d/Ze d%dd0d1Ze d2d3Ze d4d5Ze d6d7Ze d8d9Ze d:dd;d<Ze d-dd=d>Ze d%dd?d@Ze dAddBdCZdDddEdFZdGdHZdIddJdKZdddLdMdNZddOddPdQdRZddSdTdUZ dddVdWZ!dddXdYZ"ddddZd[Z#ddd\d]Z$e%d^d_Z&dddd`dadbZ'ddcdcdddedfdgZ(e)d%dhdidjZ*dkdlZ+ddmd%dndodpZ,e-dmd%dqdrdsZ.ddtdmdudvdwZ/dcd%dcdDdxdydzZ0ddcdcd{d|d}Z1dddcdcd~ddZ2Z3S)raa represent a table: facilitate read/write of various types of tables Attrs in Table Node ------------------- These are attributes that are store in the main table node, they are necessary to recreate these tables when read back in. index_axes : a list of tuples of the (original indexing axis and index column) non_index_axes: a list of tuples of the (original index axis and columns on a non-indexing axis) values_axes : a list of the columns which comprise the data of this table data_columns : a list of the columns that we are allowing indexing (these become single columns in values_axes) nan_rep : the string to use for nan representations for string objects levels : the names of levels metadata : the names of the metadata columns Z wide_tablerwrZrQrrgzint | list[Hashable]rTrnrNr~rrFrYzlist[IndexCol] | Nonez list[tuple[AxisInt, Any]] | Nonezlist[DataCol] | Nonez list | Nonez dict | Noner) rrr[r index_axesr( values_axesrrr\c sPtj||||d|pg|_|p$g|_|p.g|_|p8g|_| pBi|_| |_dS)Nr)r rrr(rrrr) rrrr[rrr(rrrrr rVrWr. s      zTable.__init__rcCs|jddS)N_r)rrrrVrVrWtable_type_shortC szTable.table_type_shortc Cs|t|jrd|jnd}d|d}d}|jrZddd|jD}d|d}dd d|jD}|jd |d |jd |j d |j d|d|d S)r[rrpz,dc->[r$rZcSsg|] }t|qSrVrZr%rVrVrWrmO rz"Table.__repr__..r]cSsg|] }|jqSrVrarzrVrVrWrmR rr^z (typ->z,nrows->z,ncols->z ,indexers->[r_) rrprrrWrVrrirrncols)rZjdcraverZjverZ jindex_axesrVrVrWrG s(  zTable.__repr__)rcCs"|jD]}||jkr|SqdS)zreturn the axis for cN)rFrb)rrrrVrVrWrY s   zTable.__getitem__c Cs|dur dS|j|jkr2td|jd|jddD]~}t||d}t||d}||kr6t|D]4\}}||}||krbtd|d|d|dqbtd|d|d|dq6dS) z"validate against an existing tableNz'incompatible table_type with existing [rr$)rr(rzinvalid combination of [z] on appending data [z] vs current table [)rrrlrrrA)rrrsvovrsaxZoaxrVrVrWr>` s:   zTable.validatercCs t|jtS)z@the levels attribute is 1 or a list in the case of a multi-index)rQrrnrrVrVrWis_multi_index szTable.is_multi_indexrz tuple[DataFrame, list[Hashable]])rr\c Cs`t|jj}z |}Wn.tyH}ztd|WYd}~n d}~00t|tsXJ||fS)ze validate that we can store the multi-index; reset and return the new object zBduplicate names/columns in the multi-index when storing as a tableN)rBZfill_missing_namesrrZ reset_indexrrQr*)rrrZ reset_objrErVrVrWvalidate_multiindex s zTable.validate_multiindexrdcCstdd|jDS)z-based on our axes, compute the expected nrowscSsg|]}|jjdqSr')rrrirrVrVrWrm rz(Table.nrows_expected..)rRrrrrVrVrWnrows_expected szTable.nrows_expectedcCs d|jvS)zhas this table been createdrwrrrVrVrWr szTable.is_existscCst|jddSNrwrlrrrVrVrWre szTable.storablecCs|jS)z,return the table group (this is my storable))rerrVrVrWrw sz Table.tablecCs|jjSr])rwrrrVrVrWr sz Table.dtypecCs|jjSr]rrrVrVrWr szTable.descriptionzitertools.chain[IndexCol]cCst|j|jSr])r2r3rrrrVrVrWrF sz Table.axescCstdd|jDS)z.the number of total columns in the values axescss|]}t|jVqdSr])rpr\rzrVrVrWrM rzTable.ncols..)sumrrrVrVrWr sz Table.ncolscCsdSrrVrrVrVrW is_transposed szTable.is_transposedztuple[int, ...]cCs(ttdd|jDdd|jDS)z@return a tuple of my permutated axes, non_indexable at the frontcSsg|]}t|dqSr'rYrzrVrVrWrm rz*Table.data_orientation..cSsg|]}t|jqSrV)rdr,rzrVrVrWrm r)ror2r3r(rrrVrVrWdata_orientation s zTable.data_orientationzdict[str, Any]csRdddddjD}fddjD}fddjD}t|||S)z.csg|]\}}|dfqSr]rV)rir,r\) axis_namesrVrWrm rcs&g|]}|jtjvr|j|fqSrV)rbrSrrr9rrVrWrm s)rr(rrH)rZd1Zd2Zd3rV)rrrW queryables s  zTable.queryablescCsdd|jDS)zreturn a list of my index colscSsg|]}|j|jfqSrV)r,rrrVrVrWrm rz$Table.index_cols..rrrVrVrW index_cols szTable.index_colsrcCsdd|jDS)zreturn a list of my values colscSsg|] }|jqSrVrrrVrVrWrm rz%Table.values_cols..)rrrVrVrW values_cols szTable.values_colsrcCs|jj}|d|dS)z)return the metadata pathname for this keyz/meta/z/metararrVrVrW_get_metadata_path szTable._get_metadata_pathr)rr\r\cCs0|jj||t|ddd|j|j|jddS)z Write out a metadata array to the key as a fixed-format Series. Parameters ---------- key : str values : ndarray Fr?rw)rr[rrN)rrrr0r[rr)rrr\rVrVrWr s  zTable.write_metadatarcCs0tt|jdd|ddur,|j||SdS)z'return the meta data array for this keyrN)rlrrrrrrVrVrWr szTable.read_metadatacCspt|j|j_||j_||j_|j|j_|j|j_|j|j_|j|j_|j |j_ |j |j_ |j |j_ dS)zset our table type & indexablesN) rZrrrrr(rrr[rrrrrVrVrWrc s        zTable.set_attrscCst|jddpg|_t|jddp$g|_t|jddp8i|_t|jdd|_tt|jdd|_tt|jdd|_ t|jd dpg|_ d d |j D|_ d d |j D|_ dS) rr(Nrrrr[rr~rcSsg|]}|jr|qSrVrrzrVrVrWrmrz#Table.get_attrs..cSsg|]}|js|qSrVrrzrVrVrWrmr)rlrr(rrrr`r[rXrr indexablesrrrrVrVrWrd szTable.get_attrscCs>|dur:|jr:tddd|jD}tj|ttddS)rfNrZcSsg|] }t|qSrVrr%rVrVrWrmrz*Table.validate_version..r)rWrsrrVrrrr)rrqrrVrVrWrgszTable.validate_versioncCsR|dur dSt|tsdS|}|D]&}|dkr4q&||vr&td|dq&dS)z validate the min_itemsize doesn't contain items that are not in the axes this needs data_columns to be defined Nr\zmin_itemsize has the key [z%] which is not an axis or data_column)rQrHrr)rrqr#rVrVrWvalidate_min_itemsize!s  zTable.validate_min_itemsizec sg}jjjtjjD]j\}\}}t|}|}|durJdnd}|d}t|d} t|||| |j||d} || qt j t |fdd| fddtjj D|S) z/create/cache the indexables if they don't existNrr)rbr,rrfrrwrrc st|tsJt}|vrt}t|}t|j}t|dd}t|dd}t|}|}t|dd} |||||||j | ||d } | S)Nrr r) rbrr\rfrrrwrrr) rQrZr rLrl_maybe_adjust_namerVrrrw) rrklassr adj_namer\rrfmdrr)base_posradescr table_attrsrVrWrxYs0    zTable.indexables..fcsg|]\}}||qSrVrV)rirr)rxrVrWrm~rz$Table.indexables..)rrwrrrrlrrrrSrrprWr) r _indexablesrr,rbr rrrrf index_colrV)rrarrxrrrWr6s2      % zTable.indexablesr$c CsV|s dS|durdS|dus(|dur8dd|jD}t|ttfsL|g}i}|dur`||d<|durp||d<|j}|D]}t|j|d}|dur"|jr|j }|j } |j } |dur| |kr| n| |d<|dur| |kr| n| |d<|jsP|j drtd |jfi|qz||jd d vrztd |d |d|dqzdS)aZ Create a pytables index on the specified columns. Parameters ---------- columns : None, bool, or listlike[str] Indicate which columns to create an index on. * False : Do not create any indexes. * True : Create indexes on all columns. * None : Create indexes on all columns. * listlike : Create indexes on the given columns. optlevel : int or None, default None Optimization level, if None, pytables defaults to 6. kind : str or None, default None Kind of index, if None, pytables defaults to "medium". Raises ------ TypeError if trying to create an index on a complex-type column. Notes ----- Cannot index Time64Col or ComplexCol. Pytables must be >= 3.0. NFTcSsg|]}|jr|jqSrV)rrrzrVrVrWrmrz&Table.create_index..rerfcomplexzColumns containing complex values can be stored but cannot be indexed when using table format. Either use fixed format, set index=False, or do not include the columns containing complex values to data_columns when initializing the table.rrgzcolumn z/ is not a data_column. In order to read column z: you must reload the dataframe into HDFStore and include z with the data_columns argument.)rrFrQrornrwrlrLryrrerfZ remove_indexrrtrrgr(r) rrrerfkwrwrrPrZ cur_optlevelZcur_kindrVrVrWrgsR   zTable.create_indexrz9list[tuple[np.ndarray, np.ndarray] | tuple[Index, Index]]rlc CsZt||||d}|}g}|jD]2}||j|j||j|j|jd}| |q"|S)a Create the axes sniffed from the table. Parameters ---------- where : ??? start : int or None, default None stop : int or None, default None Returns ------- List[Tuple[index_values, column_values]] rrA) SelectionrrFrrrrr[rr) rrqrr selectionr\rrresrVrVrW _read_axess   zTable._read_axesrcCs|S)zreturn the data for this objrVrrrrVrVrW get_objectszTable.get_objectcst|s gS|d\}|j|i}|ddkrL|rLtd|d||dur^t}n |durjg}t|trt|t|}|fdd | Dfd d |DS) zd take the input data_columns and min_itemize and create a data columns spec rrr-z"cannot use a multi-index on axis [z] with data_columns TNcs g|]}|dkr|vr|qSrrVr")existing_data_columnsrVrWrmsz/Table.validate_data_columns..csg|]}|vr|qSrVrV)rir) axis_labelsrVrWrm#r) rprrrrnrQrHrSrWr)rrrr(r,rrV)rrrWvalidate_data_columnss.     zTable.validate_data_columnsr*)rr>c/stts,|jj}td|dtddur:dgfddD|rzd}d d|jDt|j }|j }nd }|j } |j d ksJt |j d krtd g} |durd}tfdddD} j| } t| } |rHt | }|j|d }tt| t|dddsHttt| tt|dddrH|} | | i}t| j|d<t| j|d<| | | fd} j| } | }t|| |j|j}| |_|d| | |!||g}t |}|d ksJt | d ksJ| D]} t"| d| d q|jd k}|#||| }|$|%}|&||| |j'|\}}g}t(t)||D]\}\}}t*}d}|rt |d kr|d|vrt+}|d}|dust|t,std|r4|r4z|j'|}WnDt-t.fy0} z&td|d|j'd| WYd} ~ n d} ~ 00nd}|pFd|}!t/|!|j0||||j|j|d}"t1|!|j2}#|3|"}$t4|"j5j6}%d}&t7|"dddurt8|"j9}&d}'}(})t|"j5t:r|"j;})d}'t<|"j=>}(t?|"\}*}+||#|!t||$||%|&|)|'|(|+|*d },|, | ||,|d 7}qpdd|D}-t||j@|j|j|j|| ||-| |d }.tA|drv|jB|._B|.C||r|r|.D||.S) a0 Create and return the axes. Parameters ---------- axes: list or None The names or numbers of the axes to create. obj : DataFrame The object to create axes on. validate: bool, default True Whether to validate the obj against an existing object already written. nan_rep : A value to use for string column nan_rep. data_columns : List[str], True, or None, default None Specify the columns that we want to create to allow indexing on. * True : Use all available columns. * None : Use no columns. * List[str] : Use the specified columns. min_itemsize: Dict[str, int] or None, default None The min itemsize for a column in bytes. z/cannot properly create the storer for: [group->rr$Nrcsg|]}|qSrV)_get_axis_numberrz)rrVrWrmQrz&Table._create_axes..TcSsg|] }|jqSrVrNrzrVrVrWrmVrFrUrgz.rrrrrMzIncompatible appended table [z]with existing table [Z values_block_) existing_colrrr[rrrr) rbrr\rrrfrr`rrrr{cSsg|]}|jr|jqSrV)rrb)rirrVrVrWrmr) rrr[rrr(rrrrr)ErQr*rrrrrrrnrrrrUrprrRrFr(r)rRarrayrYrrrrZ_get_axis_namerr[rr,rrr _reindex_axisrrr._get_blocks_and_itemsrrr4r rLrZ IndexErrorr_maybe_convert_for_string_atomr\rrVr!rrrbrlrrr%r`rBr=rCrrrrrr>)/rrFrr>rrrr table_existsZnew_infonew_non_index_axesrrZ append_axisZindexerZ exist_axisr axis_nameZ new_indexZnew_index_axesjrrrrZvaxesrrb_itemsrrbrrEnew_namedata_convertedrrrfrrrr`r{rrZdcsZ new_tablerV)rFrrW _create_axes%s,             "              zTable._create_axes)rrc Cst|jtr|d}dd}|j}tt|}t|j}||}t|r|d\} } t |  t |} |j | | dj}tt|}t|j}||}|D]:} |j | g| dj}tt|}| |j| ||q|rddt ||D} g}g}|D]}t|j}z&| |\}}||||Wqttfy|}z2dd d |D}td |d |WYd}~qd}~00q|}|}||fS) NrcsfddjDS)Ncsg|]}j|jqSrV)rr[r)rirmgrrVrWrm$rzFTable._get_blocks_and_items..get_blk_items..)rrrVrrW get_blk_items#sz2Table._get_blocks_and_items..get_blk_itemsrrNcSs"i|]\}}t|||fqSrV)rotolist)ribrrVrVrWrQBsz/Table._get_blocks_and_items..rcSsg|] }t|qSrVr\)riitemrVrVrWrmOrz/Table._get_blocks_and_items..z+cannot match existing table structure for [z] on appending data)rQrr;rr r<rnrrpr,rXr_rWr4ror\r7rrrrr)rrrrrrrrrr,rZ new_labelsrZby_itemsZ new_blocksZ new_blk_itemsZearrrrEZjitemsrVrVrWrsR            zTable._get_blocks_and_itemsr)rr\c s|durt|}|durNjrNtjts.JjD]}||vr4|d|q4jD]$\}}t|||fdd}qT|jdur|jD]\}} } ||| | qS)zprocess axes filtersNrcsjD]}|}|}|dus*J||krfjrH|tj}|||}j|d|S||vrtt |j }t|}t t rd|}|||}j|d|Sqt d|ddS)NrNrgzcannot find the field [z] for filtering!)Z _AXIS_ORDERSr _get_axisrunionr,rr^r:rlr\rQr*r)fieldfiltoprZ axis_numberZ axis_valuesZtakersr\rrrVrWprocess_filterjs"       z*Table.process_axes..process_filter) rnrrQrinsertr(rfilterr) rrrrrr,labelsr rr rrVr rW process_axesYs  zTable.process_axes)rrrGr\cCs|durt|jd}d|d}dd|jD|d<|rj|durH|jpFd}tj|||pZ|jd }||d <n|jdur~|j|d <|S) z:create the description of the table from the axes & valuesNi'rw)rbrGcSsi|]}|j|jqSrV)rrrzrVrVrWrQrz,Table.create_description..r )rrrr)maxrrFrrrrr)rrrrrGrIrrVrVrWcreate_descriptions       zTable.create_descriptionrhc Cs|||sdSt||||d}|}|jdur|jD]D\}}}|j|||dd} ||| j |||j }qBt |S)zf select coordinates (row numbers) from a table; return the coordinates object FrNrgrh) rgrr select_coordsr rr rrilocr\r,) rrqrrrZcoordsrr rr{rVrVrWrs   zTable.read_coordinatesrcCs||sdS|dur$td|jD]|}||jkr*|jsNtd|dt|jj |}| |j |j ||||j |j|jd}tt|d|j|ddSq*td|d dS) zj return a single column from the table, generally only indexables are interesting FNz4read_column does not currently accept a where clausezcolumn [z=] can not be extracted individually; it is not data indexablerArg)rbr|z] not found in the table)rgrrrFrbrrrlrwrLrrrrr[rr0rrr)rrrqrrrrZ col_valuesrVrVrWr s*       zTable.read_column)Nr~NNNNNN)N)NNN)NN)TNNN)N)NNN)NNN)4rrrrrPrQrrr5rrrrrr>rrrrrerwrrrFrrrrrrrrrrcrdrgrrrrgrrJrrr staticmethodrrrrr rKrVrVr rWr s   &!         LW"*qC7 rc@s4eZdZdZdZd dddddZdd d d ZdS) rz a write-once read-many table: this format DOES NOT ALLOW appending to a table. writing is a one-time operation the data are stored in a format that allows for searching the data on disk rNrrhcCs tddS)z[ read the indices and the indexing array, calculate offset rows and return z!WORMTable needs to implement readNrirjrVrVrWr s zWORMTable.readrrcKs tddS)z write in a format that we can search later on (but cannot append to): write out the indices and the values using _write_array (e.g. a CArray) create an indexing table so that we can search z"WORMTable needs to implement writeNrirkrVrVrWr szWORMTable.write)NNNN)rrrrrr rrVrVrVrWrs rc @sheZdZdZdZddddddd d d Zddddd d dZddddddddZddddddZdS)r(support the new appendable table formatsZ appendableNFTrrr)rrrr:r\cCs|s|jr|j|jd|j||||| | d}|jD] }|q6|js|j|||| d}|| |d<|jj |jfi||j |j _ |jD]}| ||q|j || ddS)Nrw)rFrr>rrr)rrrrGr:)r)rrrrrrFrrrcZ create_tablerrr write_data)rrrFrrrrrrrGrrrr:rwroptionsrVrVrWrs4     zAppendableTable.write)rrr\cs|jj}|j}g}|rT|jD]6}t|jjdd}t|tj r| |j dddqt |r|d}|ddD] }||@}qp| }nd}dd |jD} t | } | dksJ| d d |jD} d d | D} g} t| D]2\} }|f|j|| | j}| ||q|dur d }tjt|||jd }||d}t|D]x} | |t| d||krxq|j|fdd | D|dur|ndfdd | DdqJdS)z` we form the data into a 2-d including indexes,values,mask write chunk-by-chunk rrNu1Fr?rgNcSsg|] }|jqSrV)rrzrVrVrWrmorz.AppendableTable.write_data..cSsg|] }|qSrV)rrzrVrVrWrmurc Ss,g|]$}|tt|j|jdqSr)Z transposerRZrollarangerUr9rVrVrWrmvrrr6csg|]}|qSrVrVrzZend_iZstart_irVrWrmrcsg|]}|qSrVrVr9rrVrWrmr)indexesrIr\)rrrrr3r{rJrQrRrrrErprCrrrreshaperrrTwrite_data_chunk)rrrrrZmasksrrIrrnindexesr\bvaluesrrPZ new_shaperowschunksrVrrWrTsL        zAppendableTable.write_datarzlist[np.ndarray]znpt.NDArray[np.bool_] | None)r!rrIr\r\c Cs|D]}t|jsdSq|djd}|t|krFtj||jd}|jj}t|}t|D]\} } | ||| <q^t|D]\} }|||| |<q||dur|j t dd} | s|| }t|r|j ||j dS)z Parameters ---------- rows : an empty memory space where we are putting the chunk indexes : an array of the indexes mask : an array of the masks values : an array of the values Nrr6Fr?)rRrrrprrrrrCrErrJrwrr) rr!rrIr\rPrrrrrrrVrVrWrs&   z AppendableTable.write_data_chunkrhcCsf|dust|sf|dur:|dur:|j}|jj|jddn(|durH|j}|jj||d}|j|S|srdS|j}t ||||d}| }t |dd }t|} | rb| } t| | dkj} t| sdg} | d| kr| | | ddkr| dd| } t| D]@} |t| | }|j||jd||jddd| } q|j| S) NTr>rhFr?rgrr<)rprrrrrwZ remove_rowsrrrrr0Z sort_valuesdiffrnrrr r7reversedr[rT)rrqrrrrwrr\Z sorted_serieslnr#rZpgrr!rVrVrWrDsD      zAppendableTable.delete) NFNNNNNNFNNT)F)NNN) rrrrrrrrrDrVrVrVrWrs$;;,rc@s`eZdZUdZdZdZdZeZde d<e ddd d Z e dd d d Z ddddddZdS)rrrrrUrRrSrrcCs|jdjdkS)Nrrg)rr,rrVrVrWrsz"AppendableFrameTable.is_transposedrcCs|r |j}|S)zthese are written transposed)rrrVrVrWrszAppendableFrameTable.get_objectNrrhcsh|sdSj|||d}tjrHjjddini}fddtjD}t|dkstJ|d}||d} g} tjD]\} } | j vrq|| \} }|ddkrt | }n t | }|d}|dur|j |d d jr |}|}t | t| d dd }n|j}t | t| d dd }|}|jdkrlt|tjrl|d|jdf}t|tjrt|j||d d}n.t|t rt|||d}ntj|g||d}tr|jjdks|j|jksJ|j|jftrt|d dr|d}| |qt| dkr8| d}n t!| dd}t"|||d}j#|||d}|S)Nrrcs"g|]\}}|jdur|qSr'r)rirrrrVrWrmrz-AppendableFrameTable.read..rgrr-rTZinplacerbraFrrr@rrrN)rr)$rgrrrpr(rrrrFrr,r- from_tuples set_namesrrlrrUrQrRrrrr*Z _from_arraysrrrfZdtypesrJrrErr2rr)rrqrrrr{rZindsindrframesrrZ index_valsrrLrr\Zindex_Zcols_rrrVrrWr  sf        "     zAppendableFrameTable.read)NNNN)rrrrrPrrUr*rSrrrrJrr rVrVrVrWrs  rcsteZdZdZdZdZdZeZe ddddZ e dd d d Z dd dfdd Z dddddfdd ZZS)rrrrrUrrcCsdSrrVrrVrVrWrfsz#AppendableSeriesTable.is_transposedrcCs|Sr]rVrrVrVrWrjsz AppendableSeriesTable.get_objectNrc s@t|ts|jpd}||}tjf||jd|dS)+we are going to write this as a frame tabler\rrN)rQr*rbZto_framer rrr)rrrrrbr rVrWros   zAppendableSeriesTable.writerr0rlcs|j}|durB|rBt|jts"J|jD]}||vr(|d|q(tj||||d}|rj|j|jdd|jdddf}|j dkrd|_ |S)Nrr+Tr&r\) rrQrrnr r r  set_indexrrb)rrqrrrrrrUr rVrWr vs   zAppendableSeriesTable.read)N)NNNN)rrrrrPrrUr0rSrrrJrrr rKrVrVr rWr^s rcs.eZdZdZdZdZddfdd ZZS)rrrrrrc sb|jpd}||\}|_t|jts*Jt|j}||t||_tj fd|i|dS)r+r\rN) rbrrrQrnrr,rr r)rrrrbZnewobjrLr rVrWrs    z AppendableMultiSeriesTable.write)rrrrrPrrrKrVrVr rWrsrc@speZdZUdZdZdZdZeZde d<e ddd d Z e d d Z d dddZ eddZd dddZdS)rz:a table that read/writes the generic pytables table formatrrrUzlist[Hashable]rrZrcCs|jSr])rPrrVrVrWriszGenericTable.pandas_typecCst|jddp|jSrrrrVrVrWreszGenericTable.storablercCsLg|_d|_g|_dd|jD|_dd|jD|_dd|jD|_dS)rNcSsg|]}|jr|qSrVrrzrVrVrWrmrz*GenericTable.get_attrs..cSsg|]}|js|qSrVrrzrVrVrWrmrcSsg|] }|jqSrVrarzrVrVrWrmr)r(rrrrrrrrVrVrWrds zGenericTable.get_attrsc Cs|j}|d}|durdnd}tdd|j||d}|g}t|jD]^\}}t|tsZJt||}||}|durzdnd}t |||g||j||d} | | qD|S)z0create the indexables from the table descriptionrNrr)rbr,rwrr)rbrr\rrwrr) rrrrwrZ_v_namesrQrZrlrNr) rrIrrrrrrr rarVrVrWrs.     zGenericTable.indexablescKs tddS)Nz cannot write on an generic tableri)rrrVrVrWrszGenericTable.writeN)rrrrrPrrUr*rSrrrirerdrrrrVrVrVrWrs    #rcsfeZdZdZdZeZdZe dZ e ddddZ dd dfd d Z dd d dfdd ZZS)rza frame with a multi-indexrrUz ^level_\d+$rZrcCsdS)NZappendable_multirVrrVrVrWrsz*AppendableMultiFrameTable.table_type_shortNrc s||durg}n|dur |j}||\}|_t|jts@J|jD]}||vrF|d|qFtjf||d|dS)NTrr,) rrrrrQrnr r r)rrrrrr rVrWrs  zAppendableMultiFrameTable.writerrhcsDtj||||d}|j}|jfdd|jjD|_|S)Nr+cs g|]}j|rdn|qSr]) _re_levelssearch)rirbrrVrWrm rz2AppendableMultiFrameTable.read..)r r r-rrr(r)rrqrrrrr rrWr s  zAppendableMultiFrameTable.read)N)NNNN)rrrrrr*rSrUrecompiler.rrrr rKrVrVr rWrs rr*rIr,)rr,rr\cCs||}t|}|dur"t|}|dus4||rB||rB|St|}|durlt|j|dd}||stddg|j}|||<|jt|}|S)NF)sort) rr:equalsuniquer]slicerUr^ro)rr,rrrZslicerrVrVrWrs   rrz str | tzinfo)rr\cCst|}|S)z+for a tz-aware type, return an encoded zone)rZ get_timezone)rzonerVrVrWr)s rznp.ndarray | Indexr+)r\rr3r\cCsdSr]rVr\rr3rVrVrWr/srrcCsdSr]rVr7rVrVrWr6szstr | tzinfo | Noneznp.ndarray | DatetimeIndexcCst|tr0|jdus"|j|ks"J|jdur0|S|dur|t|trJ|j}n d}|}t|}t||d}|d|}n|rtj |dd}|S)a coerce the values to a DatetimeIndex if tz is set preserve the input shape if possible Parameters ---------- values : ndarray or Index tz : str or tzinfo coerce : if we do not have a passed timezone, coerce to M8[ns] ndarray NrarwM8[ns]r6) rQr+rrbrCrXryrzrRrB)r\rr3rbrVrVrWr;s    )rbrr[rr\c Cst|tsJ|j}t|\}}t|}t|}t|j dsTt |j sTt |j rzt ||||t |ddt |dd|dSt|trtdtj|dd} t|} | dkrtjd d | Dtjd }t ||dt|d S| d krt| ||}|j j} t ||d t| |d S| dvr2t |||||dSt|tjrL|j tksPJ|dksbJ|t}t |||||d SdS)Niurr)r\rfrrrrzMultiIndex not supported here!FrrcSsg|] }|qSrV) toordinalr9rVrVrWrmrz"_convert_index..r6)rr)integerZfloating)r\rfrrr)rQrZrbrrrLr!rrrr$r rrlr-rrrRrBZint32rZ Time32Col_convert_string_arrayrrrrr) rbrr[rrrHrrfr rr\rrVrVrWrbs^             r)rfr[rr\cCs|dr,|dkrt|}qt||}n|dkr>t|}n|dkrztjdd|Dtd}Wqtytjdd|Dtd}Yq0nT|dvrt|}n@|d vrt|d||d }n&|d krt|d }ntd ||S)Nr1r4rcSsg|]}t|qSrVr7r9rVrVrWrmrz$_unconvert_index..r6cSsg|]}t|qSrVr:r9rVrVrWrmr)r;floatrrrArrzunrecognized index type ) rtr+rr1rRrBrrrG)r{rfr[rrrVrVrWrs*      rrHr)rbr rcCs|jtkr|Sttj|}|jj}tj|dd} | dkr@td| dkrPtd| dksd|dksd|St |} | } || | <tj| dd} | dkrt | j d D]V} | | } tj| dd} | dkrt || kr|| nd | }td |d | d qt| ||| j }|j}t|tr>t||p:|dp:d }t|pHd |}|durz||}|durz||krz|}|jd|dd}|S)NFrrz+[date] is not implemented as a table columnrnz>too many timezones in this block, create separate data columnsrrrzNo.zCannot serialize the column [z2] because its data contents are not [string] but [z] object dtyper\z|Sr?)rrr rRrrbrrrr3r|rTrrpr<rrrQrHrdrrrrE)rbr rrrr[rrrrrIr{rrZerror_column_labelrrZecirVrVrWrsN      r)r{r[rr\cCs`t|r,t|ddj||j|j}t|}t dt |}t j |d|d}|S)a Take a string-like that is object dtype and coerce to a fixed size string type. Parameters ---------- data : np.ndarray[object] encoding : str errors : str Handler for encoding errors. Returns ------- np.ndarray[fixed-length-string] Fr?rgSr6)rpr0rCrZencoderFrrrr libwritersmax_len_string_arrayrRrB)r{r[rensuredrrVrVrWr<s r<cCs|j}tj|td}t|rztt|}d|}t |dt rbt |ddj j ||dj}n|j|ddjtdd}|durd}t||||S) a* Inverse of _convert_string_array. Parameters ---------- data : np.ndarray[fixed-length-string] nan_rep : the storage repr of NaN encoding : str errors : str Handler for encoding errors. Returns ------- np.ndarray[object] Decoded data. r6UrFr?)rNr)rrRrBrCrrpr@rArrQr|r0rZrTrFrEZ!string_array_replace_from_nan_repr)r{rr[rrrrrVrVrWrG's  rG)r\rr[rcCs6t|tsJt|t|r2t|||}||}|Sr])rQrZr _need_convert_get_converter)r\rr[rconvrVrVrWrMs  rrfr[rcsLdkrddSdvr$fddSdkr:fddStddS)Nr1cSstj|ddS)Nr8r6rRrBr&rVrVrWrWrz _get_converter..cstj|dS)Nr6rHrIrrVrWrYrrcst|ddS)NrA)rGrIrrVrWr[sz invalid kind )rrGrV)r[rrfrWrEUs rEr$cCs|dvsd|vrdSdS)N)r1rr1TFrVrrVrVrWrDbsrDz Sequence[int])rbrVr\cCslt|tst|dkrtd|ddkrh|ddkrh|ddkrhtd|}|rh|d}d|}|S) z Prior to 0.10.1, we named values blocks like: values_block_0 an the name values_0, adjust the given name if necessary. Parameters ---------- name : str version : Tuple[int, int, int] Returns ------- str z6Version is incorrect, expected sequence of 3 integers.rrgrTrUzvalues_block_(\d+)Zvalues_)rQrZrprr0r/r)rbrVrgrprVrVrWrhs$   r) dtype_strr\cCst|}|drd}n|dr(d}n|dr8d}n~|drHd}nn|drX|}n^|drhd }nN|d rxd }n>|d rd }n.|d rd}n|d krd }ntd|d|S)zA Find the "kind" string describing the given dtype name. )rr|rr=r)rdr%r;r1 timedeltar4rrr'rzcannot interpret dtype of [r$)rXrtr)rLrfrVrVrWrs.         rrcCsvt|tr|j}t|jtr.d|jjd}n|jj}|jjdvrTt | d}nt|t rd|j }t |}||fS)zJ Convert the passed data into a storable form and a dtype string. rr$ZmMr) rQr4rrr&rrbrfrRrBrr.r)r{rrVrVrWrs     rc@s>eZdZdZddddddddZd d Zd d Zd dZdS)rz Carries out a selection operation on a tables.Table object. Parameters ---------- table : a Table object where : list of Terms (or convertible to) start, stop: indices to start and/or stop selection Nrrr)rwrrr\cCsV||_||_||_||_d|_d|_d|_d|_t|rt t t j |dd}|dvrt |}|jt jkr|j|j}}|durd}|dur|jj}t ||||_nNt|jjt jr|jdur||jks|jdur||jkrt d||_Wdn1s0Y|jdurR|||_|jdurR|j\|_|_dS)NFr)r;booleanrz3where must have index locations >= start and < stop)rwrqrr conditionr Ztermsr1r"rrrrrRrBrZbool_rr issubclassrr;rDgenerateevaluate)rrwrqrrinferredrVrVrWrsD     &   zSelection.__init__c Cs|dur dS|j}zt|||jjdWSty|}z:d|}td|d|d}t||WYd}~n d}~00dS)z'where can be a : dict,list,tuple,stringN)rr[rz- The passed where expression: a* contains an invalid variable reference all of the variable references must be a reference to an axis (e.g. 'index' or 'columns'), or a data_column The currently defined references are: z ) rwrr7r[ NameErrorrrrr)rrqrrEZqkeysrrVrVrWrQs  zSelection.generatecCsX|jdur(|jjj|j|j|jdS|jdurB|jj|jS|jjj|j|jdS)( generate the selection Nrh) rOrwZ read_whererrrr1rr rrVrVrWr s  zSelection.selectcCs|j|j}}|jj}|dur$d}n|dkr4||7}|durB|}n|dkrR||7}|jdurx|jjj|j||ddS|jdur|jSt ||S)rUNrT)rrr2) rrrwrrOZget_where_listrr1rRr)rrrrrVrVrWrs"  zSelection.select_coords)NNN)rrrrrrQrrrVrVrVrWrs- r) rNNFNTNNNNr~rP) Nrr~NNNNFN)N)F)F)F)r __future__r contextlibrr|rnrrr2rr0textwraprtypingrrr r r r r rnumpyrRZpandas._configrrrrZ pandas._libsrrr@Zpandas._libs.librZpandas._libs.tslibsrZpandas.compat._optionalrZpandas.compat.pickle_compatrZ pandas.errorsrrrrrZpandas.util._decoratorsrZpandas.util._exceptionsrZpandas.core.dtypes.commonrr r!r"r#r$Zpandas.core.dtypes.dtypesr%r&r'r(Zpandas.core.dtypes.missingr)rr*r+r,r-r.r/r0r1r2r3Zpandas.core.arraysr4r5r6Zpandas.core.commoncorecommonrBZ pandas.core.computation.pytablesr7r8Zpandas.core.constructionr9Zpandas.core.indexes.apir:Zpandas.core.internalsr;r<Zpandas.io.commonr=Zpandas.io.formats.printingr>r?collections.abcr@rArBtypesrCrrDrErFZpandas._typingrGrHrIrJrKrLrMrNrOr`r^rXr`rcrhrrrsrrtrurrVrzr{Z config_prefixZregister_optionZis_boolZis_one_of_factoryrrrrrrrrrrr rLrNrOrmrrrrrrrrrrrrrrrrrr<rGrrErDrrrrrVrVrVrWs8   $         0     (          &,: (p*-g#cndh1C,'@I& !