Python的包管理工具
作者:尹正杰
版权声明:原创作品,谢绝转载!否则将追究法律责任。
一.为什么使用包管理
Python的模块或者源文件直接可以复制到目标项目目录中,就可以导入使用了。 但是为了更多项目调用使用,或者共享给别人,就需要打包,或发布到网络,以便供人使用。 目的也是为了复用。
Pypi(Python Package Index),公共的模块存储中心,https://pypi.python.org/pypi
二.包管理的常见工具
1>.distutils
官方库distutils,使用安装脚本setup.py 来构建、安装包。 从1998年就是标准库的一部分,直到2000年停止开发。
2>.setuptools
它是替代distutils的增强版工具集,包含easy_install工具,使用ez_setup.py文件。支持egg格式的构建和安装。
提供查询、下载、安装、构建、发布、管理等包管理功能。
setuptools是包管理的核心模块。
后来,setuptools开发缓慢了,出现基于setuptools的distribute来替代setuptools。2013年,这两个 项目重新合并,distribute被废弃,setuptools依然是Python安装打包的标准方式。
3>.pip
pip目前包管理的事实标准。 构建在setuptools之上,替代easy_install的。同样提供丰富的包管理功能。 Python3.4之前,需要单独安装,从Python3.4开始直接包含在安装文件中。
4>.wheel
wheel格式定义在PEP427中。 wheel是zip打包的扩展名为.whl的文件,文件中不包含.pyc文件。
提供 bdist_wheel 作为 setuptools 的扩展命令,这个命令可以用来生成新打包格式 wheel。
pip 从1.4版本开始 提供了一个 wheel 子命令来安装 wheel 包。当然,需要先安装 wheel 模块。 它可以让Python库以二进制形式安装,而不需要在本地编译。
三.使用setup.py打包
1>.编写setup.py文件内容
setup.py创建一个源代码分发包的例子,参照例子 https://docs.python.org/3.5/distutils/setupscript. html。
可以在帮助文档chm上搜索索引setup,点击最上面的Distributing Python Modules (Legacy version) ,然后选择 Writing the Setup Script。 包结构即代码如下图所示。
#!/usr/bin/env python
#_*_conding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie from distutils.core import setup """ 导入setup函数并传参
name名字
version 版本
packages=[] 打包列表,
packages=['hadoop','hadoop.hdfs','hadoop.mapreduce','hadoop.mapreduce.yarn'] 逐级建立目录进行打包
description 描述信息
author 作者
author_email 作者邮件
url 包的主页,可以不写
"""
setup(name='hadoop',
version='v1.0',
packages=['hadoop','hadoop.hdfs','hadoop.mapreduce','hadoop.mapreduce.yarn'],
description='Python test hadoop',
author='Jason YIN',
author_email='y1053419035@qq.com',
url='https://www.cnblogs.com/yinzhengjie/tag/Python%E5%9F%BA%E7%A1%80%E7%9F%A5%E8%AF%86/',
)
setup.py文件内容
2>.查询命令的帮助信息
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ pwd
/yinzhengjie/python/devops/python基础/08.模块化
(venv) JasonYin最强王者:08.模块化 yinzhengjie$
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py --help-commands
Standard commands:
build build everything needed to install
build_py "build" pure Python modules (copy to build directory)
build_ext build C/C++ extensions (compile/link to build directory)
build_clib build C/C++ libraries used by Python extensions
build_scripts "build" scripts (copy and fixup #! line)
clean clean up temporary files from 'build' command
install install everything from build directory
install_lib install all Python modules (extensions and pure Python)
install_headers install C/C++ header files
install_scripts install scripts (Python or otherwise)
install_data install data files
sdist create a source distribution (tarball, zip file, etc.)
register register the distribution with the Python package index
bdist create a built (binary) distribution
bdist_dumb create a "dumb" built distribution
bdist_rpm create an RPM distribution
bdist_wininst create an executable installer for MS Windows
check perform some checks on the package
upload upload binary package to PyPI usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help (venv) JasonYin最强王者:08.模块化 yinzhengjie$
3>.build命令,编译
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll
total 8
drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__
drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34 hadoop
-rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33 setup.py
(venv) JasonYin最强王者:08.模块化 yinzhengjie$
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py build #创建一个build目录并将打包的内容放在里面。
running build
running build_py
creating build
creating build/lib
creating build/lib/hadoop
copying hadoop/__init__.py -> build/lib/hadoop
creating build/lib/hadoop/hdfs
copying hadoop/hdfs/__init__.py -> build/lib/hadoop/hdfs
copying hadoop/hdfs/datanode.py -> build/lib/hadoop/hdfs
copying hadoop/hdfs/namenode.py -> build/lib/hadoop/hdfs
creating build/lib/hadoop/mapreduce
copying hadoop/mapreduce/__init__.py -> build/lib/hadoop/mapreduce
creating build/lib/hadoop/mapreduce/yarn
copying hadoop/mapreduce/yarn/__init__.py -> build/lib/hadoop/mapreduce/yarn
(venv) JasonYin最强王者:08.模块化 yinzhengjie$
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll
total 8
drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__
drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 build
drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34 hadoop
-rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33 setup.py
(venv) JasonYin最强王者:08.模块化 yinzhengjie$
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll -R build
total 0
drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 lib build/lib:
total 0
drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:45 hadoop build/lib/hadoop:
total 8
-rw-r--r-- 1 yinzhengjie wheel 127 Nov 15 19:29 __init__.py
drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:45 hdfs
drwxr-xr-x 4 yinzhengjie wheel 136 Nov 15 20:45 mapreduce build/lib/hadoop/hdfs:
total 24
-rw-r--r-- 1 yinzhengjie wheel 132 Nov 15 19:29 __init__.py
-rw-r--r-- 1 yinzhengjie wheel 142 Nov 15 19:32 datanode.py
-rw-r--r-- 1 yinzhengjie wheel 238 Nov 15 19:35 namenode.py build/lib/hadoop/mapreduce:
total 8
-rw-r--r-- 1 yinzhengjie wheel 132 Nov 15 19:29 __init__.py
drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 yarn build/lib/hadoop/mapreduce/yarn:
total 8
-rw-r--r-- 1 yinzhengjie wheel 107 Nov 15 19:26 __init__.py
(venv) JasonYin最强王者:08.模块化 yinzhengjie$
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll -R build
4>.install命令,安装 (将安装包安装到python的“site-packages”)
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py install #build后就可以install,直接运行,如果没有build,会先build编译,然后安装。
running install
running build
running build_py
running install_lib
creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop
copying build/lib/hadoop/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop
creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs
copying build/lib/hadoop/hdfs/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs
copying build/lib/hadoop/hdfs/datanode.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs
copying build/lib/hadoop/hdfs/namenode.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs
creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce
copying build/lib/hadoop/mapreduce/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce
creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn
copying build/lib/hadoop/mapreduce/yarn/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn
byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/__init__.py to __init__.cpython-37.pyc
byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/__init__.py to __init__.cpython-37.pyc
byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/datanode.py to datanode.cpython-37.pyc
byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/namenode.py to namenode.cpython-37.pyc
byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/__init__.py to __init__.cpython-37.pyc
byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn/__init__.py to __init__.cpython-37.pyc
running install_egg_info
Writing /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop-v1.0-py3.7.egg-info
(venv) JasonYin最强王者:08.模块化 yinzhengjie$
5>.sdist命令
(venv) JasonYin最强王者:.模块化 yinzhengjie$ ll
total
drwxr-xr-x yinzhengjie wheel Nov : __pycache__
drwxr-xr-x yinzhengjie wheel Nov : build
drwxr-xr-x yinzhengjie wheel Nov : hadoop
-rw-r--r-- yinzhengjie wheel Nov : setup.py
(venv) JasonYin最强王者:.模块化 yinzhengjie$
(venv) JasonYin最强王者:.模块化 yinzhengjie$ python setup.py sdist #创建源代码的分发包,产生一个dist目录,里面生成一个带版本号的压缩包。
running sdist
running check
warning: sdist: manifest template 'MANIFEST.in' does not exist (using default file list) warning: sdist: standard file not found: should have one of README, README.txt, README.rst writing manifest file 'MANIFEST'
creating hadoop-v1.
creating hadoop-v1./hadoop
creating hadoop-v1./hadoop/hdfs
creating hadoop-v1./hadoop/mapreduce
creating hadoop-v1./hadoop/mapreduce/yarn
making hard links in hadoop-v1....
hard linking setup.py -> hadoop-v1.
hard linking hadoop/__init__.py -> hadoop-v1./hadoop
hard linking hadoop/hdfs/__init__.py -> hadoop-v1./hadoop/hdfs
hard linking hadoop/hdfs/datanode.py -> hadoop-v1./hadoop/hdfs
hard linking hadoop/hdfs/namenode.py -> hadoop-v1./hadoop/hdfs
hard linking hadoop/mapreduce/__init__.py -> hadoop-v1./hadoop/mapreduce
hard linking hadoop/mapreduce/yarn/__init__.py -> hadoop-v1./hadoop/mapreduce/yarn
creating dist
Creating tar archive
removing 'hadoop-v1.0' (and everything under it)
(venv) JasonYin最强王者:.模块化 yinzhengjie$
(venv) JasonYin最强王者:.模块化 yinzhengjie$ ll
total
-rw-r--r-- yinzhengjie wheel Nov : MANIFEST
drwxr-xr-x yinzhengjie wheel Nov : __pycache__
drwxr-xr-x yinzhengjie wheel Nov : build
drwxr-xr-x yinzhengjie wheel Nov : dist
drwxr-xr-x yinzhengjie wheel Nov : hadoop
-rw-r--r-- yinzhengjie wheel Nov : setup.py
(venv) JasonYin最强王者:.模块化 yinzhengjie$
(venv) JasonYin最强王者:.模块化 yinzhengjie$ cat MANIFEST
# file GENERATED by distutils, do NOT edit
setup.py
hadoop/__init__.py
hadoop/hdfs/__init__.py
hadoop/hdfs/datanode.py
hadoop/hdfs/namenode.py
hadoop/mapreduce/__init__.py
hadoop/mapreduce/yarn/__init__.py
(venv) JasonYin最强王者:.模块化 yinzhengjie$
(venv) JasonYin最强王者:.模块化 yinzhengjie$ ll dist/ #在其他地方解压缩这个文件,里面有setup.py,就可以使用"python setup.py install"安装了, 也可以"pip hadoop-v1.0.tar.gz"直接使用pip安装这个压缩包。
total
-rw-r--r-- yinzhengjie wheel Nov : hadoop-v1..tar.gz
(venv) JasonYin最强王者:.模块化 yinzhengjie$
(venv) JasonYin最强王者:.模块化 yinzhengjie$
6>.bdist命令
二进制分发包,或称作安装程序。它可以生成目标操作系统的安装程序。 可以把自己写好的模块发布到公共的Pypi上,也可以搭建Pypi私服,供企业内部使用。 注意:
Pypi里面的模块没有太好的审核机制,不保证安全,请慎重使用。 制作windows下的安装包
$ python setup.py bdist_wininst
$ python setup.py bdist_msi
$ python setup.py bdist --format=msi
rpm包
$ python setup.py bdist_rpm
$ python setup.py bdist --format=rpm
压缩文件
$ python setup.py bdist --format=zip
$ python setup.py bdist --format=gztar
四.wheel包
1>.需要安装依赖包
(venv) JasonYin最强王者:.模块化 yinzhengjie$ pip install wheel
Collecting wheel
Downloading https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Installing collected packages: wheel
Successfully installed wheel-0.33.
(venv) JasonYin最强王者:.模块化 yinzhengjie$
2>.相关命令
$ python setup.py bdist_egg
$ python setup.py bdist_wheel