lxml报错

• 前言

前言

爬虫小白，因为第三方库lxml焦头烂额了一个下午，开始时pip版本报错，setting中安装失败，后来在terminal中安装成功，把安装包人工转移到site-packages，还是报错``

bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml.

---

解决方法：将BeautifulSoup(html,‘lxml’)改为BeautifulSoup(html,‘lxml’)BeautifulSoup(html,‘html_parser’)

代码：

# 实战：爬取小说

main_url = 'https://www.pilibook.com/book/1922.html'
headers = {
    'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Mobile Safari/537.36'
}
page_text = requests.get(url=main_url, headers=headers).text
fp = open('./《兼职无常后我红了》.txt', 'w', encoding='utf-8')
soup = BeautifulSoup(page_text, 'html.parser')
a_list = soup.select('.list_xm > ul > li > a')

for a in a_list:
    title = a.text
    detail_url = a['href']

    page_text_detail = requests.get(url=detail_url, headers=headers).text
    soup = BeautifulSoup(page_text_detail, 'html.parser')
    div_tag = soup.find('div', class_="content_novel")
    content = div_tag.text
    fp.write(title + ':' + content + '\n')
    print(title, '保存成功！！！')
fp.close()

运行结果：