What is the: ''Cannot mask with non-boolean array containing NA / NaN values'' problem, and how to solve it?

Question

I encountered the following problem in completing my work:

ValueError: cannot index with vector containing NA / NaN values

Below is the code I ran:

DF[DF.col.str.contains("foo")]

DF[DF.col.notnull()][DF.col.dropna().str.contains("foo")]

What's causing it, and how can it be resolved in the “Cannot mask with non-boolean array containing NA / NaN values“ in the python?

lyytutoria · Accepted Answer

The cause:

I have a flag for that:

In [11]: df = pd.DataFrame([["foo1"], ["foo2"], ["bar"], [np.nan]], columns=['a'])

In [12]: df.a.str.contains("foo")
Out[12]:
0 True
1 True
2 False
3 NaN
Name: a, dtype: object

In [13]: df.a.str.contains("foo", na=False)
Out[13]:
0 True
1 True
2 False
3 False
Name: a, dtype: bool

From the above program, you can see that the data in your program are not strings, they are NA or NaN. However str.contains has a requirement that it must be a string, cannot numbers or mix with numbers. This is the reason of your error.

Solution:

To fix this error, let's do as my following suggestion:

In [21]: df.loc[df.a.str.contains("foo", na=False)]
Out[21]:
a
0 foo1
1 foo2

Another way for this error is:

df[df['Product ID'].str.contains("foo") == True]

Or

df[df.col.str.contains("foo").fillna(False)]

If you follow these above suggestions, your error will be solved effectively. Good lucks!

Aaron Cohen · Answer

You can use the following for columns that do not contain a single word:

df[df['Product ID'].str.contains("foo") == True]

We hope you find this helpful.

Sign Up

Sign In

Forgot Password

ITtutoria Latest Questions

What is the: ”Cannot mask with non-boolean array containing NA / NaN values” problem, and how to solve it?