a Of%@s>dZddlmZddlZddlmZddlZddlm Z ddl m Z ddl m Z ddlmZmZmZmZmZerdd lmZmZmZdd lmZmZdd lmZmZmZmZd Z d dddddZ!dde dfdddddddddZ"de fddddddd Z#de dfd!ddddd"d#d$Z$de dfd%ddddd"d&d'Z%dS)(z" data hash pandas / numpy objects ) annotationsN) TYPE_CHECKING)hash_object_array) is_list_like)CategoricalDtype) ABCDataFrameABCExtensionArrayABCIndex ABCMultiIndex ABCSeries)HashableIterableIterator) ArrayLikenpt) DataFrameIndex MultiIndexSeriesZ0123456789123456zIterator[np.ndarray]intznpt.NDArray[np.uint64])arrays num_itemsreturnc Csz t|}Wn"ty.tjgtjdYS0t|g|}td}t|td}d}t|D]:\}}||}||N}||9}|td||7}|}qh|d|ksJd|td7}|S) z Parameters ---------- arrays : Iterator[np.ndarray] num_items : int Returns ------- np.ndarray[uint64] Should be the same as CPython's tupleobject.c )dtypeiCBixV4riXBzFed in wrong num_itemsi|) next StopIterationnparrayuint64 itertoolschainZ zeros_like enumerate) rrfirstZmultoutZlast_iiaZ inverse_ir'Q/var/www/ai-form-bot/venv/lib/python3.9/site-packages/pandas/core/util/hashing.pycombine_hash_arrays/s"   r)Tutf8zIndex | DataFrame | Seriesboolstrz str | Noner)objindexencodinghash_key categorizercsddlm}durtttr8|tdddSttrptjj ddd}||ddd}n tt rtjj ddd}|rȇfd d d D}t |g|} t | d }||jddd}nttrjfd d D} tj} |rLfdd d D} | d7} t | | } dd | D} t | | }||jddd}ntdt|S)a> Return a data hash of the Index/Series/DataFrame. Parameters ---------- obj : Index, Series, or DataFrame index : bool, default True Include the index in the hash (if Series/DataFrame). encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- Series of uint64, same length as the object Examples -------- >>> pd.util.hash_pandas_object(pd.Series([1, 2, 3])) 0 14639053686158035780 1 3869563279212530728 2 393322362522515241 dtype: uint64 r)rNrF)rcopyr2)r.rr2c3s$|]}tjddjVqdSF)r.r/r0r1Nhash_pandas_objectr._values.0_r1r/r0r-r'r( sz%hash_pandas_object..Nc3s"|]\}}t|jVqdSr=) hash_arrayr7)r9r:Zseries)r1r/r0r'r(r<sc3s$|]}tjddjVqdSr4r5r8r;r'r(r<srcss|] }|VqdSr=r')r9xr'r'r(r<zUnexpected type for hashing )pandasr_default_hash_key isinstancer hash_tuplesr r?r7astyper r r!r)r.ritemslencolumns TypeErrortype)r-r.r/r0r1rhZserZ index_iterrhashesrZindex_hash_generator_hashesr'r;r(r6SsJ#           r6z+MultiIndex | Iterable[tuple[Hashable, ...]])valsr/r0rcszt|stdddlmm}t|ts6||n|fddtj D}fdd|D}t |t |}|S)a Hash an MultiIndex / listlike-of-tuples efficiently. Parameters ---------- vals : MultiIndex or listlike-of-tuples encoding : str, default 'utf8' hash_key : str, default _default_hash_key Returns ------- ndarray[np.uint64] of hashed values z'must be convertible to a list-of-tuplesr) Categoricalrc s,g|]$}j|tj|ddqS)F categoriesordered) _simple_newcodesrZlevels)r9level)rPmir'r( s zhash_tuples..c3s|]}|jddVqdS)Fr/r0r1N)_hash_pandas_object)r9cat)r/r0r'r(r<szhash_tuples..) rrJrBrPrrDr from_tuplesrangeZnlevelsr)rH)rOr/r0rZcat_valsrMrLr')rPr/r0rWr(rEs    rEr)rOr/r0r1rcCs\t|dstdt|tr,|j|||dSt|tjsNtdt|jdt ||||S)a Given a 1d array, return an array of deterministic integers. Parameters ---------- vals : ndarray or ExtensionArray encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- ndarray[np.uint64, ndim=1] Hashed values, same length as the vals. Examples -------- >>> pd.util.hash_array(np.array([1, 2, 3])) array([ 6238072747940578789, 15839785061582574730, 2185194620014831856], dtype=uint64) rzmust pass a ndarray-likerYz6hash_array requires np.ndarray or ExtensionArray, not z!. Use hash_pandas_object instead.) hasattrrJrDrrZrZndarrayrK__name__ _hash_ndarray)rOr/r0r1r'r'r(r?s   r?z np.ndarrayc Cs|j}t|tjr@t|j|||}t|j|||}|d|S|tkrT|d}nt |j tj tj fr~| djddd}nt |j tjr|jdkr| d|jjd}n|rdd lm}m}m} | |dd \} } t|| dd }|| |} | j||dd Szt|||}Wn,tyDt|tt||}Yn0||d ?N}|td9}||d?N}|td9}||d?N}|S)z! See hash_array.__doc__. u8i8Fr3ur)rPr factorize)sortrQrYle9z lb&&& )rrZ issubdtypeZ complex128r`realimagr+rF issubclassrKZ datetime64Z timedelta64viewnumberitemsizerBrPrrfrrTrZrrJr,objectr) rOr/r0r1rZ hash_realZ hash_imagrPrrfrUrRr[r'r'r(r`s>        r`)&__doc__ __future__rr typingrnumpyrZpandas._libs.hashingrZpandas.core.dtypes.commonrZpandas.core.dtypes.dtypesrZpandas.core.dtypes.genericrrr r r collections.abcr r rZpandas._typingrrrBrrrrrCr)r6rEr?r`r'r'r'r(s<      &f43