环境:ubuntu 16.04 , python3.5.1+
安装顺序如下:
sudo apt-get install build-essential
sudo apt-get install python3-dev
sudo apt-get install libxml2-de;
sudo apt-get install libxslt1-dev
sudo apt-get install libssl-dev
apt-get install -y libffi-dev
pip3 install Twisted
pip3 install w3lib
pip3 install lxm
然后安装Scrapy:
scrapy运行必备组件需要(OpenSSL,lxml(lxml一般默认有安装,可以import xml验证是否有安装,因为我这里已经默认安装了xml,所以就不附上xml安装命令了))
sudo python3 -m pip install Scrapy
安装OpenSSL:
sudo python3 -m pip install pyopenssl
之前安装了python-dev和python-setuptools也要重新安装python3版本的这两个软件,即运行:
sudo apt-get install python3-dev
sudo apt-get install python3-setuptools