我正在GAE上运行每日报告任务,因为最近使用了太多内存来完成.因此,我想将其设置为后端任务.我将后端设置如下:
backends:
- name: reporting
class: B4_1G
options: dynamic
start: reporting.app
在reporting.py中,定义了许多类,它们调用不同的报告.我的cron.yaml目前看起来像这样:
cron:
- description: update report 1
url: /reports/report1
schedule: every day 03:00
- description: update report 2
url: /reports/report2
schedule: every day 03:30
但从逻辑上讲,这只是通过app.yaml调用前端实例上的作业,目前看起来像这样:
application: appname
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /(robots\.txt)
static_files: \1
upload: (robots\.txt)
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /sitemap\.xml
static_files: sitemap.xml
upload: sitemap\.xml
- url: /images
static_dir: images
- url: /js
static_dir: js
- url: /css
static_dir: css
- url: /reports/.*
script: reporting.app
login: admin
我必须每天更改为在后端实例上调用这些作业?
解决方法:
取决于您是否需要persistent or dynamic后端
对于动态的
计划是:
> A cron在特定时间开火.
>添加一个将启动后端的task on a queue
>后端开始
例:
app.yaml中:
- url: /crons/startgooglepluscrawler/
script: crons.startgooglepluscrawler.app
login: admin
backends.yaml:
backends:
- name: google-plus-crawler
class: B2
start: backends.googlepluscrawler.app
options: dynamic, failfast
instances: 1
crons.yaml:
cron:
- description: get daily google plus user followers and followings
url: /crons/startgooglepluscrawler/
schedule: every day 09:00
queue.yaml中:
total_storage_limit: 10M
queue:
- name: google-plus-daily-crawling
rate: 1/s
retry_parameters:
task_retry_limit: 0
task_age_limit: 1s
在startgooglepluscrawler.app上,您需要使用taskqueue启动后端:
class StartGooglePlusCrawlerHandler(webapp2.RequestHandler):
def get(self):
logging.info("Running daily Cron")
taskqueue.add(queue_name = "google-plus-daily-crawling",
url="/_ah/start",
method='GET',
target=(None if self.is_dev_server() else 'google-plus-crawler'),
headers={"X-AppEngine-FailFast":"true"}
)
logging.info("Daily Cron finished")
def is_dev_server(self):
return os.environ['SERVER_SOFTWARE'].startswith('Dev')
app = webapp2.WSGIApplication([
("/crons/startgooglepluscrawler/",StartGooglePlusCrawlerHandler)
],debug=True)
在后端/ googlepluscrawler.py通常就像一个应用程序,以及/ _ah / start的处理程序:
app = webapp2.WSGIApplication(
[('/_ah/start', StartHandler)],
debug=True,
config=config.config)
上面的示例将启动后端实例.