Skip to content

BUG: fix infinite recursion in np.ma.flatten_structured_array#30855

Merged
seberg merged 2 commits intonumpy:mainfrom
antareepsarkar:core
Feb 25, 2026
Merged

BUG: fix infinite recursion in np.ma.flatten_structured_array#30855
seberg merged 2 commits intonumpy:mainfrom
antareepsarkar:core

Conversation

@antareepsarkar
Copy link
Copy Markdown
Contributor

Towards resolving #29349

Avoids recursion for both strings and objects.

@charris charris added the 09 - Backport-Candidate PRs tagged should be backported label Feb 20, 2026
@seberg
Copy link
Copy Markdown
Member

seberg commented Feb 23, 2026

Thanks! This is always a bit trickier with regressions in masked arrays and not being quite sure how this is used.

I actually, think a slightly more maximal approach may be nice, but with a release note and no backporting.
If this is important to you to backport, I might try to go more minimal and just skip recursion into strings for now.

The more maximal approach would be to do the following (sorry, it is a bit more complicated, in the sense of changing the pattern!):

  1. Rather than flattening each element, flatten the arr.dtype (if it is structured). There is a function that does this in iotools: flatten_dtype.
  2. We can just use dtype = np.result_type() on the flattened dtype.
  3. We have to create the final array, without moving at least a bit to C, I suspect the easiest thing will be to iterate the fields and assign one by one, like:
    • for i, name in enumerate(dtype.names): out[i+something] = input[name]
    • Unfortunately, subarray fields are a bit tricky and need some thought there.

This approach will also change the result dtype, e.g. for a struct array of float32 where the dtype will now still be float32 (not just for dtypes that include objects as you have it).

The numpy.lib.recfunctions probably has some related functionality, although similar to masked arrays, some of the code may not be something we should adopt exactly.

@antareepsarkar
Copy link
Copy Markdown
Contributor Author

antareepsarkar commented Feb 23, 2026

Thanks! This is always a bit trickier with regressions in masked arrays and not being quite sure how this is used.

I actually, think a slightly more maximal approach may be nice, but with a release note and no backporting. If this is important to you to backport, I might try to go more minimal and just skip recursion into strings for now.

The more maximal approach would be to do the following (sorry, it is a bit more complicated, in the sense of changing the pattern!):

  1. Rather than flattening each element, flatten the arr.dtype (if it is structured). There is a function that does this in iotools: flatten_dtype.

  2. We can just use dtype = np.result_type() on the flattened dtype.

  3. We have to create the final array, without moving at least a bit to C, I suspect the easiest thing will be to iterate the fields and assign one by one, like:

    • for i, name in enumerate(dtype.names): out[i+something] = input[name]
    • Unfortunately, subarray fields are a bit tricky and need some thought there.

This approach will also change the result dtype, e.g. for a struct array of float32 where the dtype will now still be float32 (not just for dtypes that include objects as you have it).

The numpy.lib.recfunctions probably has some related functionality, although similar to masked arrays, some of the code may not be something we should adopt exactly.

Thanks for replying.
Should I implement the maximal approach you mentioned?

@antareepsarkar
Copy link
Copy Markdown
Contributor Author

@seberg @charris
The last changes skip recursion for strings. Maybe, it can be just used for backporting now.
Then, another PR with the maximal approach mentioned can be done.

@ngoldbaum
Copy link
Copy Markdown
Member

#30872 fixes the test failures seen here, a rebase should fix it.

numpy/ma/core.py Outdated
"""
for elm in iter(iterable):
if hasattr(elm, '__iter__'):
if not np.ndim(elm) == 0:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do this minimal thing, I am tempted to just add not isinstance(elem, (str, bytes)), TBH.
np.ndim() may be slow and I don't consider this the true fix either way, so I am probably more happy with a practical 98% fix than something that looks like a neat fix but isn't really.

@seberg seberg merged commit 41f3673 into numpy:main Feb 25, 2026
78 checks passed
@seberg
Copy link
Copy Markdown
Member

seberg commented Feb 25, 2026

Thanks, let's get this minimal approach in then! FWIW, I think the real fix is still to revamp the approach to walk the dtype fields.

charris pushed a commit to charris/numpy that referenced this pull request Mar 4, 2026
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Mar 4, 2026
charris added a commit that referenced this pull request Mar 4, 2026
BUG: fix infinite recursion in np.ma.flatten_structured_array (#30855)
sabasiddique1 pushed a commit to sabasiddique1/numpy that referenced this pull request Mar 4, 2026
@antareepsarkar antareepsarkar deleted the core branch March 7, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants