BeautifulSoup връща TypeError: обект на „NoneType“ няма len()

Използвам BeautifulSoup, за да изтрия данни и да ми върне списък с всички div в моя списък, но ми дава следната грешка:

Traceback (most recent call last):
  File "C:\Users\intel\Desktop\One page\test.py", line 16, in <module>
    soup = BeautifulSoup(div.html,'html5lib')
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\lib\site-packages\bs4\__init__.py", line 287, in __init__
    elif len(markup) <= 256 and (
TypeError: object of type 'NoneType' has no len()

Ето моят код:

from bs4 import BeautifulSoup
import requests as req

resp = req.get('https://medium.com/@daranept27')

x = resp.text

soup = BeautifulSoup(x, "lxml")
 
divs = soup.find_all("div")
#print(divs)

lst = []

for div in divs:
    soup = BeautifulSoup(div.html,'html5lib')
    div_tag = soup.find()
    try:
        title = div_tag.section.div.h1.a['href']
        if title not in lst: lst.append(title)
    except:
        pass

print("\n".join(lst))

python-3.x python web-scraping python-requests beautifulsoup
person Community    schedule 06.11.2020    source източник

Отговори (1)


arrow_upward
0
arrow_downward

Опитайте да използвате str(div), за да конвертирате div в str. Ето пълния код:

from bs4 import BeautifulSoup
import requests as req

resp = req.get('https://medium.com/@daranept27')

x = resp.text

soup = BeautifulSoup(x, "lxml")

divs = soup.find_all("div")
# print(divs)

lst = []

for div in divs:
    soup = BeautifulSoup(str(div), 'html5lib')
    div_tag = soup.find()
    try:
        title = div_tag.section.div.h1.a['href']
        if title not in lst: lst.append(title)
    except:
        pass

print("\n".join(lst))

Изход:

/read-rosy/if-the-whole-world-is-compelled-to-forget-everything-cde200c0ad98
/wordsmith-library/seven-days-between-life-and-death-dffb639fb245
/an-idea/have-you-ever-encountered-a-fake-friend-if-so-try-these-simple-tips-to-overcome-it-d8473d755ab8
person Sushil    schedule 06.11.2020