Skip to content

Fix result_type handling for dtype specifiers and pandas ExtensionDtype#2

Open
alippai wants to merge 2 commits intomainfrom
codex/fix-categoricaldtype-exceptions-b5dorl
Open

Fix result_type handling for dtype specifiers and pandas ExtensionDtype#2
alippai wants to merge 2 commits intomainfrom
codex/fix-categoricaldtype-exceptions-b5dorl

Conversation

@alippai
Copy link
Copy Markdown
Owner

@alippai alippai commented Feb 7, 2026

Motivation

  • Prevent misclassification of dtype-specifier strings/types as pandas extension dtypes and preserve correct NumPy-style promotion for explicit dtype inputs.
  • Preserve and return pandas ExtensionDtype results (e.g., CategoricalDtype) where appropriate and avoid unwanted promotion to object.
  • Fix downstream typing/usage sites that assume result_type returns np.dtype and ensure string/bytes promotion matches pandas semantics for indexing and alignment.

Description

  • Refined xarray.core.dtypes.result_type to detect extension-dtype inputs only for actual extension types and not for dtype-specifier str/bytes/type objects, and to return ExtensionDtype when appropriate.
  • Added a fast-path that collects explicit dtype-like inputs (np.dtype, type, str/bytes specifiers), runs maybe_promote_to_variable_width on them, and calls np.result_type to preserve fixed/variable-width string semantics.
  • Updated logic for should_return_str_or_bytes so extension-array dtype detection excludes plain type objects.
  • Improved union_unordered_categorical_and_scalar in xarray.core.extension_array to concatenate category Index objects and produce a unique pd.CategoricalDtype (preserving order and uniqueness).
  • Guarded maybe_coerce_to_str in xarray/core/utils.py to only check .kind for actual np.dtype instances.
  • Added a cast(np.dtype, ...) where result_type is used in a sparse conversion path (xarray/namedarray/core.py) to satisfy typing when a concrete np.dtype is required.
  • Minor test updates: tighten typing in xarray/tests/test_dtypes.py with cast, and use pd.NA in categorical-null tests in xarray/tests/test_dataarray.py.

Testing

  • Ran python -m pytest -c /dev/null xarray/tests/test_dtypes.py -k "result_type" and it passed (19 passed, 1 skipped).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant