Is it okay to fix the typeerror: cannot use a string pattern on a bytes-like object problem?

Question

Here is the program I run:

import urllib.request
 import re
 
 url = "http://www.google.com"
 regex = r'<title>(,+?)</title>'
 pattern = re.compile(regex)
 
 with urllib.request.urlopen(url) as response:
  html = response.read()
 
 title = re.findall(pattern, html)
 print(title)

After I run, it returns an error:

Traceback (most recent call last):
  File "path	o\file\Crawler.py", line 11, in <module>
  title = re.findall(pattern, html)
  File "C:\Python33\lib
e.py", line 201, in findall
  return _compile(pattern, flags).findall(string)
 TypeError: can't use a string pattern on a bytes-like object

Does anyone have any suggestions for the problem below: typeerror: cannot use a string pattern on a bytes-like object in the python - How to correct it?

dttutoria · Accepted Answer

The cause: The cause of the "typeerror: cannot use a string pattern on a bytes-like object in the python" is simply because your regex is a string, however html is made up of bytes: Solution:

You need to use .decode to convert html (a byte-like object) to a string. For example, this:

html = response. read(). decode('utf-8').

Mila Tournier · Answer

Your regex is a string but html bytes.

>>> type(html)

Because python does not know the encoded bytes, it throws an error when you attempt to use string regex.

You can decode add bytes to a string.

html = html.decode('ISO-8859-1') # encoding may vary!
title = re.findall(pattern, html) # no more error

You can also use a bytes regex

regex = rb'(,+?)'
# ^

This particular context allows you to get the encoding using the response headers.

with urllib.request.urlopen(url) as response:
 encoding = response.info().get_param('charset', 'utf8')
 html = response.read().decode(encoding)

For more information, see the urlopen documentation .

Evan Evans · Answer

Your regex is a string but html bytes.

>>> type(html)

Because python does not know the encoded bytes, it throws an error when you attempt to use string regex.

You can decode add bytes to a string.

html = html.decode('ISO-8859-1') # encoding may vary!
title = re.findall(pattern, html) # no more error

You can also use a bytes regex

regex = rb'(,+?)'
# ^

This particular context allows you to get the encoding using the response headers.

with urllib.request.urlopen(url) as response:
 encoding = response.info().get_param('charset', 'utf8')
 html = response.read().decode(encoding)

For more information, see the urlopen documentation .

Sign Up

Sign In

Forgot Password

ITtutoria Latest Questions

Is it okay to fix the typeerror: cannot use a string pattern on a bytes-like object problem?