U Cv fHg @sddlmZddlmZmZmZmZmZmZddl Z ddl m Z m Z ddlmZddlmZddlmZmZddlmZmZmZmZdd lmZmZdd lmZdd lmZ erdd lm!Z!m"Z"m#Z#m$Z$ed ddde j%ddfddddddddddd ddZed ddde j%ddfddddddddddd d dZd ddde j%ddfdddd!d"dZd#d$d%d&d'Z&d(d)Z'dd*d+d,d-Z(e j%ddfd.d/d0d0ddddddd1 d2d3Z)e j%ddfdd/d0d0ddddddd4 d5d6Z*dS)7) annotations) TYPE_CHECKINGAnyHashableIterablecastoverloadN)dtypesutils)align)lazy_array_equiv)Index PandasIndex) _VALID_COMPATcollect_variables_and_indexes merge_attrsmerge_collected) T_DataArray T_Dataset)Variable)concat)CombineAttrsOptions CompatOptions ConcatOptions JoinOptionsall differentequalsouteroverridezIterable[T_Dataset]z!Hashable | T_DataArray | pd.IndexzConcatOptions | list[Hashable]rzIterable[Iterable[int]] | Noneobjectrrr) objsdim data_varscoordscompat positions fill_valuejoin combine_attrsreturnc CsdSN r!r"r#r$r%r&r'r(r)r,r,6/tmp/pip-unpacked-wheel-h316xyqg/xarray/core/concat.pyrs rzIterable[T_DataArray]rc CsdSr+r,r-r,r,r.r-s )r%r(r)c Csddlm} ddlm} zt|\} }Wntk rFtdYnX|tkr`td|dt | | rt |||||||||d St | | rt |||||||||d St dt | d S) aConcatenate xarray objects along a new or existing dimension. Parameters ---------- objs : sequence of Dataset and DataArray xarray objects to concatenate together. Each object is expected to consist of variables and coordinates with matching shapes except for along the concatenated dimension. dim : Hashable or DataArray or pandas.Index Name of the dimension to concatenate along. This can either be a new dimension name, in which case it is added along axis=0, or an existing dimension name, in which case the location of the dimension is unchanged. If dimension is provided as a DataArray or Index, its name is used as the dimension to concatenate along and the values are added as a coordinate. data_vars : {"minimal", "different", "all"} or list of Hashable, optional These data variables will be concatenated together: * "minimal": Only data variables in which the dimension already appears are included. * "different": Data variables which are not equal (ignoring attributes) across all datasets are also concatenated (as well as all for which dimension already appears). Beware: this option may load the data payload of data variables into memory if they are not already loaded. * "all": All data variables will be concatenated. * list of dims: The listed data variables will be concatenated, in addition to the "minimal" data variables. If objects are DataArrays, data_vars must be "all". coords : {"minimal", "different", "all"} or list of Hashable, optional These coordinate variables will be concatenated together: * "minimal": Only coordinates in which the dimension already appears are included. * "different": Coordinates which are not equal (ignoring attributes) across all datasets are also concatenated (as well as all for which dimension already appears). Beware: this option may load the data payload of coordinate variables into memory if they are not already loaded. * "all": All coordinate variables will be concatenated, except those corresponding to other dimensions. * list of Hashable: The listed coordinate variables will be concatenated, in addition to the "minimal" coordinates. compat : {"identical", "equals", "broadcast_equals", "no_conflicts", "override"}, optional String indicating how to compare non-concatenated variables of the same name for potential conflicts. This is passed down to merge. - "broadcast_equals": all values must be equal when variables are broadcast against each other to ensure common dimensions. - "equals": all values and dimensions must be the same. - "identical": all values, dimensions and attributes must be the same. - "no_conflicts": only values which are not null in both datasets must be equal. The returned dataset then contains the combination of all non-null values. - "override": skip comparing and pick variable from first dataset positions : None or list of integer arrays, optional List of integer arrays which specifies the integer positions to which to assign each dataset along the concatenated dimension. If not supplied, objects are concatenated in the provided order. fill_value : scalar or dict-like, optional Value to use for newly missing values. If a dict-like, maps variable names to fill values. Use a data array's name to refer to its values. join : {"outer", "inner", "left", "right", "exact"}, optional String indicating how to combine differing indexes (excluding dim) in objects - "outer": use the union of object indexes - "inner": use the intersection of object indexes - "left": use indexes from the first object with each dimension - "right": use indexes from the last object with each dimension - "exact": instead of aligning, raise `ValueError` when indexes to be aligned are not equal - "override": if indexes are of same size, rewrite indexes to be those of the first object with that dimension. Indexes for the same dimension must have the same size in all objects. combine_attrs : {"drop", "identical", "no_conflicts", "drop_conflicts", "override"} or callable, default: "override" A callable or a string indicating how to combine attrs of the objects being merged: - "drop": empty attrs on returned Dataset. - "identical": all attrs must be the same on every object. - "no_conflicts": attrs from all objects are combined, any that have the same name must also have the same value. - "drop_conflicts": attrs from all objects are combined, any that have the same name but different values are dropped. - "override": skip comparing and copy attrs from the first dataset to the result. If a callable, it must expect a sequence of ``attrs`` dicts and a context object as its only parameters. Returns ------- concatenated : type of objs See also -------- merge Examples -------- >>> da = xr.DataArray( ... np.arange(6).reshape(2, 3), [("x", ["a", "b"]), ("y", [10, 20, 30])] ... ) >>> da array([[0, 1, 2], [3, 4, 5]]) Coordinates: * x (x) >> xr.concat([da.isel(y=slice(0, 1)), da.isel(y=slice(1, None))], dim="y") array([[0, 1, 2], [3, 4, 5]]) Coordinates: * x (x) >> xr.concat([da.isel(x=0), da.isel(x=1)], "x") array([[0, 1, 2], [3, 4, 5]]) Coordinates: * x (x) >> xr.concat([da.isel(x=0), da.isel(x=1)], "new_dim") array([[0, 1, 2], [3, 4, 5]]) Coordinates: x (new_dim) >> xr.concat([da.isel(x=0), da.isel(x=1)], pd.Index([-90, -100], name="new_dim")) array([[0, 1, 2], [3, 4, 5]]) Coordinates: x (new_dim) 9s z$_calc_concat_over..c sHt|tr|dkrdkr.tddtdD]bkrHs zA_calc_concat_over..process_subset_opt..rMz@ not present in all datasets and coords='different'. Either add z= to datasets where it is missing or specify coords='minimal'.)equivTFrZminimalzunexpected value for z: cs"g|]}|tdkr|qS)r)rBrHrI)datasetssubsetr,r.rRsr$zCsome variables in coords are not coordinates on the first dataset: zIsome variables in data_vars are not data variables on the first dataset: )r7rAr6rBlenr addrNloadZcomputeappendzipdataupdatesetrC) optrVrNvarZv_lhsZcomputedZds_rhsZv_rhsrPrJZ invalid_vars)r% concat_overrUr)rIrVr.process_subset_opt<sr                 z-_calc_concat_over..process_subset_optr#r$) r^rXrC set_coordsr]rNitemsrZget) rUr" dim_namesr#r$r%Zconcat_over_existing_dimconcat_dim_lengthsrPrbr,)r%rarUr"rr._calc_concat_over%s$   V  rhzRtuple[dict[Hashable, Variable], dict[Hashable, int], set[Hashable], set[Hashable]])rUr*cCst}t}t}i}i}|D]d}||j||j||j|jD]&}||krZqL||krL|j|j||<qL|t|jB}q||||fSr+)r^r]rCr$r#variable)rUrCZall_coord_namesr# dim_coords dims_sizesrPr"r,r,r._parse_datasetss     rlzlist[T_Dataset]zstr | T_DataArray | pd.Indexzstr | list[str]) rUr"r#r$r%r&r'r(r)r*c ' s4ddlm} ddlmttfddDs>tdt| rPj} ntt r`} nd} t \} dd Dtt |d g|d t \} } }t | }|| }| |@}|rt|d | dd| ks|kr|krfd d Dt||||\}}| |B||i}i}rfddttD}t|||d\}}|||||| tdd D|}dj}ddD](}|dkrt|j|stdqƇfdd}fdd}djD]t|krt|krtz|fdd D}Wn$tk rptdYnXt|}|rZt|tkrtdt|dtd|d||djkrdj } ndji} |fdd| D!| }!|!D].\}"tfdd D|d |"_|"|<q(nt"|||d }#|#|<n|kr||<qt#d||d!}$| t |$j}%|%rtd"|%d#|$$| }$||$_|$j%|d$d%}$| dk r&| dk r | !| i}&n| !}&|&|$<| |<|$&|}$|$S)&zN Concatenate a sequence of datasets along a new or existing dimension rr/r1c3s|]}t|VqdSr+r7)rHZdatasetr1r,r.rLsz"_dataset_concat..SThe elements in the input list need to be either all 'Dataset's or all 'DataArray'sNcSsg|] }|qSr,)copyrOr,r,r.rRsz#_dataset_concat..F)r(roexcluder'z1 is a coordinate in some datasets but not others.csg|]}tt|qSr,)rrZ expand_dimsrOrKr,r.rRscsi|]\}}|kr||qSr,r,rG)variables_to_merger,r. sz#_dataset_concat..)r%rcSsg|] }|jqSr,attrsrOr,r,r.rRsrM identicalz$Dataset global attributes not equal.c3svttdd|D}|kr*f|}t|D]<\}|j|krjtfdd|D}|||}|Vq4dS)NcSsg|]}|jD]}|qqSr,rF)rHrJdr,r,r.rRsz?_dataset_concat..ensure_common_dims..c3s|]}|VqdSr+)re)rHrv)dim_lenrkr,r.rLsz>_dataset_concat..ensure_common_dims..)tuplepduniquer[rCset_dims)varsZ common_dimsr`Z common_shape)rgr"rk)rwr.ensure_common_dimss   z+_dataset_concat..ensure_common_dimsc3s\D]R}||jkr |j|Vq|kr|j|}|js|j}t||jdVqdS)Nr?)_indexesZ _variablesrCr{valuesrr>)r=rPr`r\)rUr"r,r. get_indexes"s   z$_dataset_concat..get_indexescsg|]}|jqSr,)rirO)r=r,r.rR1sz is not present in all datasets.z> must have either an index or no index in all datasets, found /z datasets with an index.csi|] }|qSr,r,rT) combined_idxr,r.rrDscsg|]}|jjqSr,)rNrtrOrQr,r.rRHs)r)rsz Variables z1 are coordinates in some datasets but not others.ignore)errors)'r3r0r4r2listrr:r7rirrEr rlr^r6poprhrrdrr]rencodingr Z dict_equivrtrNKeyErrorrWrr~ZxindexesZget_all_coordsZcreate_variables concat_varsr;rcZ drop_varsZ_overwrite_indexes)'rUr"r#r$r%r&r'r(r)r0Zdim_varrDrjZ coord_namesZ data_namesrfZunlabeled_dimsZboth_data_and_coordsrarZ result_varsZresult_indexesZgroupedZ merged_varsZmerged_indexesZ result_attrsZresult_encodingrPr}rr|ZindexesZidx_varsZcombined_idx_varsrJZ combined_varresultZabsent_coord_namesZ index_varsr,) r2rrgrUr"rkrIr=rqr.r9s                          r9) arraysr"r#r$r%r&r'r(r)r*c  sddlmt|}tfdd|Ds2td|dkrBtdg} t|D]R\} } | dkrf| j} n,| | jkr|dkrtd ntt | | } | | qNt | ||||||||d } td d |D|}|d| | }||_|S) Nrr/c3s|]}t|VqdSr+rm)rHarrayr/r,r.rLsz$_dataarray_concat..rnrzFdata_vars is not a valid argument when concatenating DataArray objectsruzarray names not identical)r'r(r)cSsg|] }|jqSr,rs)rHdar,r,r.rRsz%_dataarray_concat..)r3r0rrr:r6 enumerater=rrrenamerZZ_to_temp_datasetr9rZ_from_temp_datasetrt)rr"r#r$r%r&r'r(r)rUnZarrr=rPZ merged_attrsrr,r/r.r8qsD    r8)+ __future__rtypingrrrrrrZpandasryZ xarray.corer r Zxarray.core.alignmentr Zxarray.core.duck_array_opsr Zxarray.core.indexesr rZxarray.core.mergerrrrZxarray.core.typesrrZxarray.core.variablerrrrrrrZNArErhrlr9r8r,r,r,r.sf      $$Os""F