API: boolean ops with float dtype

[jbrockmendel]

# from #60234 OP

ser1 = pd.Series([False, False])
ser2 = pd.Series([0.0, 0.1])

ser1 | ser2  # <- works by casting floats to bool
ser2 | ser1  # <- raises TypeError

# We also have special-casing for NaNs among floats

ser3 = pd.Series([np.nan, 1.0])

ser3 & ser3            # <- raises bc you can't do float & float
ser3[:-1] & ser3[:-1]  # has no non-NaN floats so we special-case

logical ops (&, |, ^) have inconsistent behavior. I think we should simplify this significantly and more closely resemble the numpy behavior.

[jorisvandenbossche]

I think we should simplify this significantly and more closely resemble the numpy behavior.

And the numpy behaviour is to simply raise an error in all those cases?

That was also my first reaction to seeing this issue: personally I don't really see a reason that those ops are allowed to work. If you want to do logical ops on floats, you can always cast the float to bool manually first.

[jbrockmendel]

And the numpy behaviour is to simply raise an error in all those cases?

Yes.

personally I don't really see a reason that those ops are allowed to work

Looking at what tests fail when i add a deprecation warning, i think it is mostly about ops that do silent reindexing which casts int/bool to float/object.

[jbrockmendel]

Looks like a bunch of issues tentatively related to the silent casting/filling we do: #51267, #23191, #40565, #60234, #52597, #41764. (Some of these are for arithmetic ops which go through a different path than logical ops).

If I'm right that the main motivation for this behavior is reindex-based, then two options come to mind:

1. wait for PDEP16 when after which reindexing won't cast ints/bools and then deprecate this behavior entirely (or if numpy dtypes aren't supported at all, we'll never go through these paths and can rip them out)
2. In silent-alignment cases, instead of doing an outer-join and then operating, we could do an inner-join, operate, then reindex to the outer-join index. Then deprecate/change this behavior for all non-alignment cases.

Update: looks like we already do something similar to 2 for some DataFrame ops in _arith_method_with_reindex

[aijams]

I'd be willing to work on this issue if it's decided that something needs to be done.