puppeteer新手遇到的坑

puppeteer安装以及遇到的坑

1. 环境和安装

Puppeteer 至少需要 Node v6.4.0,如要使用 async / await,只有 Node v7.6.0 或更高版本才支持。 node下载地址: https://nodejs.org/zh-cn/

2. 创建项目

2.1 创建test目录,进入目录执行npm init,生成项目package.json文件
2.2 安装 puppeteer
yarn add puppeteer 或者 npm i puppeteer

在安装的过程中遇到如下错误

weifandeMacBook-Pro:example weifan$ npm i puppeteer --save

> puppeteer@1.6. install /Users/weifan/Desktop/example/node_modules/puppeteer
> node install.js ERROR: Failed to download Chromium r571375! Set "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" env variable to skip download.
{ Error: connect ETIMEDOUT 172.217.25.16:
at Object._errnoException (util.js::)
at _exceptionWithHostPort (util.js::)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js::)
errno: 'ETIMEDOUT',
code: 'ETIMEDOUT',
syscall: 'connect',
address: '172.217.25.16',
port: }
npm WARN example@1.0. No description
npm WARN example@1.0. No repository field. npm ERR! code ELIFECYCLE
npm ERR! errno
npm ERR! puppeteer@1.6. install: `node install.js`
npm ERR! Exit status
npm ERR!
npm ERR! Failed at the puppeteer@1.6. install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in:
npm ERR! /Users/weifan/.npm/_logs/--16T09_49_23_441Z-debug.log

报错的原因是:因为在执行安装的过程中需要执行install.js,这里会下载Chromium,我们这里先跳过进行跳过,

看来需要设置PUPPETEER_SKIP_CHROMIUM_DOWNLOAD,这个环境变量了,设置方法有多种,这里如下:

env PUPPETEER_SKIP_CHROMIUM_DOWNLOAD="true" npm i --save puppeteer

你会看到安装成功

2.3 手动下载Chromium

下载地址:https://download-chromium.appspot.com/

把下载刚刚下载的文件解压到项目的chromium文件夹下,在chromium文件夹下你会看到chrome-mac文件,你可以点击爱看下问价内容。

2.4 在项目的根目录的src文件夹下新建index.js(截图功能), 代码如下:
const puppeteer = require('puppeteer');

async function getPic() {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://google.com');
await page.screenshot({path: 'google.png'}); await browser.close();
} getPic();

运行代码:node index.js,出现了如下错误

(node:38213) UnhandledPromiseRejectionWarning: Error: Chromium revision is not downloaded. Run "npm install" or "yarn install"
at assert (/Users/weifan/Desktop/example/node_modules/puppeteer/lib/helper.js:282:11)
at Function.launch (/Users/weifan/Desktop/example/node_modules/puppeteer/lib/Launcher.js:106:7)
at <anonymous>

显示chromium 未下载错误,因为chromium默认的下载路径是在node_modules/puppeteer/.local-chromium/目录,这时候我们的chromium是在项目根目录,所以需要配置指定路径,修改index.js文件:

const puppeteer = require('puppeteer');

async function getPic() {
const browser = await puppeteer.launch({
executablePath: '../chromium/chrome-mac/Chromium.app',
headless: false
});
const page = await browser.newPage();
await page.goto('https://google.com');
await page.screenshot({path: 'google.png'}); await browser.close();
} getPic();

再次运行index.js,又报如下错误:

(node:38246) UnhandledPromiseRejectionWarning: Error: spawn EACCES

在puppeteer的Git issues找到如下解决方法,https://github.com/GoogleChrome/puppeteer/issues/1649,把executablePath改为如下:

executablePath: '../chromium/chrome-mac/Chromium.app/Contents/MacOS/Chromium',

再次node index.js 运行文件,可以跑通了。

参考如下:

1、https://www.jianshu.com/p/a89d8d6c007b

2、https://blog.fundebug.com/2017/11/01/guide-to-automating-scraping-the-web-with-js/

3、https://github.com/GoogleChrome/puppeteer/issues/1649

上一篇:LOJ 164 【清华集训2015】V——线段树维护历史最值


下一篇:实践 HTML5 的 CSS3 Media Queries