Python字节码与解释器学习

参考:http://blog.jobbole.com/55327/

http://blog.jobbole.com/56300/

http://blog.jobbole.com/56761/

 

1. 在交互式命令行中执行命令的内部过程

当你敲下return键的时候,python完成了以下四步:词法分析、句法分析、编译、解释。词法分析的工作就是将你刚才输入的那行代码分解为一些符号token(译者注:包括标示符,关键字,数字, 操作符等)。句法分析程序再接收这些符号,并用一种结构来展现它们之间的关系(在这种情况下使用的抽象语法树)。然后编译器接收这棵抽象语法树,并将它转化为一个(或多个)代码对象。最后,解释器逐个接收这些代码对象,并执行它们所代表的代码。

 

每一行我们输入的命令,都要经过上面的四个步骤,才能够被执行。

 

2. 函数对象

对象是面向对象理论中的基本元素,在一些动态或者解释性语言中,函数也可以看作是一种对象,比如在JavaScript,以及功能性编程语言Haskell/Ocaml中,函数都是一种特殊的对象。

 

函数是对象,就意味着函数可以像对象一样被执行各种操作,比如分配,释放,复制,赋值......

“函数是最好的对象”说明函数是一种对象。它就如同一个列表或者举个例子来说 :MyObject 就是一个对象。既然 foo 是一个对象,那么我们就能在不调用它的情况下使用它(也就是说,foo 和 foo() 是大相径庭的)。我们能够将 foo 当作一个参数传递给另一个函数或者赋值给一个新函数名( other_function = foo )。有了如此棒的函数,一切皆为可能!

 

另外,函数作为对象出现的时候,就是和函数调用有区别的,函数调用是一个动态的过程;而函数作为一个对象,是一个静态的实体概念,意思是你可以对这个对象施予一些操作,这与这个对象的类型有关,或者以面向对象的思想来说,你可以执行这个对象提供的各种接口操作(函数)。

 

既然是对象,那么函数对象有哪些成员呢?

>>> dir
<built-in function dir>
>>> dir(dir)
[‘__call__‘, ‘__class__‘, ‘__cmp__‘, ‘__delattr__‘, ‘__doc__‘, ‘__eq__‘, ‘__format__‘, ‘__ge__‘, ‘__getattribute__‘, ‘__gt__‘, ‘__hash__‘, ‘__init__‘, ‘__le__‘, ‘__lt__‘, ‘__module__‘, ‘__name__‘, ‘__ne__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__self__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘]
>>> dir(dir.func_code)

Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    dir(dir.func_code)
AttributeError: ‘builtin_function_or_method‘ object has no attribute ‘func_code‘
>>> def foo(a):
	x = 3
	return x + a

>>> foo
<function foo at 0x0000000002E8F128>
>>> dir(foo)
[‘__call__‘, ‘__class__‘, ‘__closure__‘, ‘__code__‘, ‘__defaults__‘, ‘__delattr__‘, ‘__dict__‘, ‘__doc__‘, ‘__format__‘, ‘__get__‘, ‘__getattribute__‘, ‘__globals__‘, ‘__hash__‘, ‘__init__‘, ‘__module__‘, ‘__name__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘func_closure‘, ‘func_code‘, ‘func_defaults‘, ‘func_dict‘, ‘func_doc‘, ‘func_globals‘, ‘func_name‘]
>>> 

其中,内置函数dir的功能描述如下:

dir([object])

Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object.

If the object has a method named __dir__(), this method will be called and must return the list of attributes. This allows objects that implement a custom __getattr__() or __getattribute__() function to customize the way dir() reports their attributes.

If the object does not provide __dir__(), the function tries its best to gather information from the object’s __dict__ attribute, if defined, and from its type object. The resulting list is not necessarily complete, and may be inaccurate when the object has a custom __getattr__().

The default dir() mechanism behaves differently with different types of objects, as it attempts to produce the most relevant, rather than complete, information:

  • If the object is a module object, the list contains the names of the module’s attributes.
  • If the object is a type or class object, the list contains the names of its attributes, and recursively of the attributes of its bases.
  • Otherwise, the list contains the object’s attributes’ names, the names of its class’s attributes, and recursively of the attributes of its class’s base classes.

The resulting list is sorted alphabetically.

  

除此之外,help内置函数也很重要,可以查看内置函数的帮助内容。

 

首先,查看当前Python程序加载了哪些模块

>>> for i in sys.modules.keys():
...     print "%20s:\t%s\n" % (i, sys.modules[i])
...     print "*"*100

 

            copy_reg:	<module ‘copy_reg‘ from ‘/usr/lib/python2.7/copy_reg.pyc‘>

****************************************************************************************************
         sre_compile:	<module ‘sre_compile‘ from ‘/usr/lib/python2.7/sre_compile.pyc‘>

****************************************************************************************************
                _sre:	<module ‘_sre‘ (built-in)>

****************************************************************************************************
           encodings:	<module ‘encodings‘ from ‘/usr/lib/python2.7/encodings/__init__.pyc‘>

****************************************************************************************************
                site:	<module ‘site‘ from ‘/usr/lib/python2.7/site.pyc‘>

****************************************************************************************************
         __builtin__:	<module ‘__builtin__‘ (built-in)>

****************************************************************************************************
           sysconfig:	<module ‘sysconfig‘ from ‘/usr/lib/python2.7/sysconfig.pyc‘>

****************************************************************************************************
            __main__:	<module ‘__main__‘ (built-in)>

****************************************************************************************************
 encodings.encodings:	None

****************************************************************************************************
                 abc:	<module ‘abc‘ from ‘/usr/lib/python2.7/abc.pyc‘>

****************************************************************************************************
           posixpath:	<module ‘posixpath‘ from ‘/usr/lib/python2.7/posixpath.pyc‘>

****************************************************************************************************
         _weakrefset:	<module ‘_weakrefset‘ from ‘/usr/lib/python2.7/_weakrefset.pyc‘>

****************************************************************************************************
               errno:	<module ‘errno‘ (built-in)>

****************************************************************************************************
    encodings.codecs:	None

****************************************************************************************************
       sre_constants:	<module ‘sre_constants‘ from ‘/usr/lib/python2.7/sre_constants.pyc‘>

****************************************************************************************************
                  re:	<module ‘re‘ from ‘/usr/lib/python2.7/re.pyc‘>

****************************************************************************************************
             _abcoll:	<module ‘_abcoll‘ from ‘/usr/lib/python2.7/_abcoll.pyc‘>

****************************************************************************************************
               types:	<module ‘types‘ from ‘/usr/lib/python2.7/types.pyc‘>

****************************************************************************************************
             _codecs:	<module ‘_codecs‘ (built-in)>

****************************************************************************************************
encodings.__builtin__:	None

****************************************************************************************************
           _warnings:	<module ‘_warnings‘ (built-in)>

****************************************************************************************************
         genericpath:	<module ‘genericpath‘ from ‘/usr/lib/python2.7/genericpath.pyc‘>

****************************************************************************************************
                stat:	<module ‘stat‘ from ‘/usr/lib/python2.7/stat.pyc‘>

****************************************************************************************************
           zipimport:	<module ‘zipimport‘ (built-in)>

****************************************************************************************************
      _sysconfigdata:	<module ‘_sysconfigdata‘ from ‘/usr/lib/python2.7/_sysconfigdata.pyc‘>

****************************************************************************************************
            warnings:	<module ‘warnings‘ from ‘/usr/lib/python2.7/warnings.pyc‘>

****************************************************************************************************
            UserDict:	<module ‘UserDict‘ from ‘/usr/lib/python2.7/UserDict.pyc‘>

****************************************************************************************************
     encodings.utf_8:	<module ‘encodings.utf_8‘ from ‘/usr/lib/python2.7/encodings/utf_8.pyc‘>

****************************************************************************************************
                 sys:	<module ‘sys‘ (built-in)>

****************************************************************************************************
              codecs:	<module ‘codecs‘ from ‘/usr/lib/python2.7/codecs.pyc‘>

****************************************************************************************************
            readline:	<module ‘readline‘ from ‘/usr/lib/python2.7/lib-dynload/readline.i386-linux-gnu.so‘>

****************************************************************************************************
   _sysconfigdata_nd:	<module ‘_sysconfigdata_nd‘ from ‘/usr/lib/python2.7/plat-i386-linux-gnu/_sysconfigdata_nd.pyc‘>

****************************************************************************************************
             os.path:	<module ‘posixpath‘ from ‘/usr/lib/python2.7/posixpath.pyc‘>

****************************************************************************************************
       sitecustomize:	<module ‘sitecustomize‘ from ‘/usr/lib/python2.7/sitecustomize.pyc‘>

****************************************************************************************************
              signal:	<module ‘signal‘ (built-in)>

****************************************************************************************************
           traceback:	<module ‘traceback‘ from ‘/usr/lib/python2.7/traceback.pyc‘>

****************************************************************************************************
           linecache:	<module ‘linecache‘ from ‘/usr/lib/python2.7/linecache.pyc‘>

****************************************************************************************************
               posix:	<module ‘posix‘ (built-in)>

****************************************************************************************************
   encodings.aliases:	<module ‘encodings.aliases‘ from ‘/usr/lib/python2.7/encodings/aliases.pyc‘>

****************************************************************************************************
          exceptions:	<module ‘exceptions‘ (built-in)>

****************************************************************************************************
           sre_parse:	<module ‘sre_parse‘ from ‘/usr/lib/python2.7/sre_parse.pyc‘>

****************************************************************************************************
                  os:	<module ‘os‘ from ‘/usr/lib/python2.7/os.pyc‘>

****************************************************************************************************
            _weakref:	<module ‘_weakref‘ (built-in)>

****************************************************************************************************

  

可以通过下面代码查看__builtin__模块中的成员

>>> num = 0
>>> for i in dir(sys.modules["__builtin__"]):
...     print "%20s\t" % i,
...     num += 1
...     if num == 5:
...             print ""
...             num = 0
... 
     ArithmeticError	      AssertionError	      AttributeError	       BaseException	         BufferError	
        BytesWarning	  DeprecationWarning	            EOFError	            Ellipsis	    EnvironmentError	
           Exception	               False	  FloatingPointError	       FutureWarning	       GeneratorExit	
             IOError	         ImportError	       ImportWarning	    IndentationError	          IndexError	
            KeyError	   KeyboardInterrupt	         LookupError	         MemoryError	           NameError	
                None	      NotImplemented	 NotImplementedError	             OSError	       OverflowError	
PendingDeprecationWarning	      ReferenceError	        RuntimeError	      RuntimeWarning	       StandardError	
       StopIteration	         SyntaxError	       SyntaxWarning	         SystemError	          SystemExit	
            TabError	                True	           TypeError	   UnboundLocalError	  UnicodeDecodeError	
  UnicodeEncodeError	        UnicodeError	UnicodeTranslateError	      UnicodeWarning	         UserWarning	
          ValueError	             Warning	   ZeroDivisionError	                   _	           __debug__	
             __doc__	          __import__	            __name__	         __package__	                 abs	
                 all	                 any	               apply	          basestring	                 bin	
                bool	              buffer	           bytearray	               bytes	            callable	
                 chr	         classmethod	                 cmp	              coerce	             compile	
             complex	           copyright	             credits	             delattr	                dict	
                 dir	              divmod	           enumerate	                eval	            execfile	
                exit	                file	              filter	               float	              format	
           frozenset	             getattr	             globals	             hasattr	                hash	
                help	                 hex	                  id	               input	                 int	
              intern	          isinstance	          issubclass	                iter	                 len	
             license	                list	              locals	                long	                 map	
                 max	          memoryview	                 min	                next	              object	
                 oct	                open	                 ord	                 pow	               print	
            property	                quit	               range	           raw_input	              reduce	
              reload	                repr	            reversed	               round	                 set	
             setattr	               slice	              sorted	        staticmethod	                 str	
                 sum	               super	               tuple	                type	              unichr	
             unicode	                vars	              xrange	                 zip	>>> 

  

  

 3. dir内置命令是怎么实现的

在/Python-2.7.8/Objects/object.c中

1963 /* Implementation of dir() -- if obj is NULL, returns the names in the current
1964    (local) scope.  Otherwise, performs introspection of the object: returns a
1965    sorted list of attribute names (supposedly) accessible from the object
1966 */
1967 PyObject *
1968 PyObject_Dir(PyObject *obj)
1969 {
1970     PyObject * result;
1971 
1972     if (obj == NULL)
1973         /* no object -- introspect the locals */
1974         result = _dir_locals();
1975     else
1976         /* object -- introspect the object */
1977         result = _dir_object(obj);
1978 
1979     assert(result == NULL || PyList_Check(result));
1980 
1981     if (result != NULL && PyList_Sort(result) != 0) {
1982         /* sorting the list failed */
1983         Py_DECREF(result);
1984         result = NULL;
1985     }
1986 
1987     return result;
1988 }

  

可见,与help(dir)描述的基本一致。

 

>>> def foo(a):
...     if a > x:
...             return a/1024
...     else:
...             return a
... 
>>> type(foo)
<type ‘function‘>
>>> dir(foo)
[‘__call__‘, ‘__class__‘, ‘__closure__‘, ‘__code__‘, ‘__defaults__‘, ‘__delattr__‘, ‘__dict__‘, ‘__doc__‘, ‘__format__‘, ‘__get__‘, ‘__getattribute__‘, ‘__globals__‘, ‘__hash__‘, ‘__init__‘, ‘__module__‘, ‘__name__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘func_closure‘, ‘func_code‘, ‘func_defaults‘, ‘func_dict‘, ‘func_doc‘, ‘func_globals‘, ‘func_name‘]
>>> foo.__call__
<method-wrapper ‘__call__‘ of function object at 0xb7420df4>
>>> foo.__str__
<method-wrapper ‘__str__‘ of function object at 0xb7420df4>
>>> foo
<function foo at 0xb7420df4>
>>> foo.func_closure
>>> type(foo.func_closure)
<type ‘NoneType‘>
>>> type(foo.func_code)
<type ‘code‘>
>>> foo.func_code
<code object foo at 0xb7409d10, file "<stdin>", line 1>
>>> dir(foo.func_code)
[‘__class__‘, ‘__cmp__‘, ‘__delattr__‘, ‘__doc__‘, ‘__eq__‘, ‘__format__‘, ‘__ge__‘, ‘__getattribute__‘, ‘__gt__‘, ‘__hash__‘, ‘__init__‘, ‘__le__‘, ‘__lt__‘, ‘__ne__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘co_argcount‘, ‘co_cellvars‘, ‘co_code‘, ‘co_consts‘, ‘co_filename‘, ‘co_firstlineno‘, ‘co_flags‘, ‘co_freevars‘, ‘co_lnotab‘, ‘co_name‘, ‘co_names‘, ‘co_nlocals‘, ‘co_stacksize‘, ‘co_varnames‘]
>>> foo.func_code.co_argcount
1
>>> foo.func_code.co_cellvars
()
>>> foo.func_code.co_code
‘|\x00\x00t\x00\x00k\x04\x00r\x14\x00|\x00\x00d\x01\x00\x15S|\x00\x00Sd\x00\x00S‘
>>> foo.func_code.co_consts
(None, 1024)
>>> foo.func_code.co_filename
‘<stdin>‘
>>> foo.func_code.co_firstlineno
1
>>> foo.func_code.co_flags
67
>>> foo.func_code.co_freevars
()
>>> foo.func_code.co_lnotab
‘\x00\x01\x0c\x01\x08\x02‘
>>> foo.func_code.co_name
‘foo‘
>>> foo.func_code.co_names
(‘x‘,)
>>> foo.func_code.co_nlocals
1
>>> foo.func_code.co_stacksize
2
>>> foo.func_code.co_varnames
(‘a‘,)
>>> 

  

其中,foo.func_code.co_code打印出来的就是Python的字节码。

 

Help on built-in function ord in module __builtin__:

ord(...)
    ord(c) -> integer
    
    Return the integer ordinal of a one-character string.

  

>>> [ord(i) for i in foo.func_code.co_code]
[124, 0, 0, 116, 0, 0, 107, 4, 0, 114, 20, 0, 124, 0, 0, 100, 1, 0, 21, 83, 124, 0, 0, 83, 100, 0, 0, 83]

 

这就是那些组成python字节码的字节。解释器会循环接收各个字节,查找每个字节的指令然后执行这个指令。需要注意的是,字节码本身并不包括任何python对象,或引用任何对象。

如果你想知道python字节码的意思,可以去找到CPython解释器文件(ceval.c),然后查阅100的意思、1的意思、0的意思,等等。  

 

>>> import dis
>>> dir(dis)
[‘EXTENDED_ARG‘, ‘HAVE_ARGUMENT‘, ‘__all__‘, ‘__builtins__‘, ‘__doc__‘, ‘__file__‘, ‘__name__‘, ‘__package__‘, ‘_have_code‘, ‘_test‘, ‘cmp_op‘, ‘dis‘, ‘disassemble‘, ‘disassemble_string‘, ‘disco‘, ‘distb‘, ‘findlabels‘, ‘findlinestarts‘, ‘hascompare‘, ‘hasconst‘, ‘hasfree‘, ‘hasjabs‘, ‘hasjrel‘, ‘haslocal‘, ‘hasname‘, ‘opmap‘, ‘opname‘, ‘sys‘, ‘types‘]
>>> type(dis.dis)
<type ‘function‘>
>>> dir(dis.dis)
[‘__call__‘, ‘__class__‘, ‘__closure__‘, ‘__code__‘, ‘__defaults__‘, ‘__delattr__‘, ‘__dict__‘, ‘__doc__‘, ‘__format__‘, ‘__get__‘, ‘__getattribute__‘, ‘__globals__‘, ‘__hash__‘, ‘__init__‘, ‘__module__‘, ‘__name__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘func_closure‘, ‘func_code‘, ‘func_defaults‘, ‘func_dict‘, ‘func_doc‘, ‘func_globals‘, ‘func_name‘]
>>> [ord(i) for i in dis.dis.func_code.co_code]
[124, 0, 0, 100, 1, 0, 107, 8, 0, 114, 23, 0, 116, 1, 0, 131, 0, 0, 1, 100, 1, 0, 83, 116, 2, 0, 124, 0, 0, 116, 3, 0, 106, 4, 0, 131, 2, 0, 114, 53, 0, 124, 0, 0, 106, 5, 0, 125, 0, 0, 110, 0, 0, 116, 6, 0, 124, 0, 0, 100, 2, 0, 131, 2, 0, 114, 80, 0, 124, 0, 0, 106, 7, 0, 125, 0, 0, 110, 0, 0, 116, 6, 0, 124, 0, 0, 100, 3, 0, 131, 2, 0, 114, 107, 0, 124, 0, 0, 106, 8, 0, 125, 0, 0, 110, 0, 0, 116, 6, 0, 124, 0, 0, 100, 4, 0, 131, 2, 0, 114, 246, 0, 124, 0, 0, 106, 9, 0, 106, 10, 0, 131, 0, 0, 125, 1, 0, 124, 1, 0, 106, 11, 0, 131, 0, 0, 1, 120, 174, 0, 124, 1, 0, 68, 93, 85, 0, 92, 2, 0, 125, 2, 0, 125, 3, 0, 116, 2, 0, 124, 3, 0, 116, 12, 0, 131, 2, 0, 114, 154, 0, 100, 5, 0, 124, 2, 0, 22, 71, 72, 121, 14, 0, 116, 13, 0, 124, 3, 0, 131, 1, 0, 1, 87, 110, 28, 0, 4, 116, 14, 0, 107, 10, 0, 114, 234, 0, 1, 125, 4, 0, 1, 100, 6, 0, 71, 124, 4, 0, 71, 72, 110, 1, 0, 88, 72, 113, 154, 0, 113, 154, 0, 87, 110, 78, 0, 116, 6, 0, 124, 0, 0, 100, 7, 0, 131, 2, 0, 114, 18, 1, 116, 15, 0, 124, 0, 0, 131, 1, 0, 1, 110, 50, 0, 116, 2, 0, 124, 0, 0, 116, 16, 0, 131, 2, 0, 114, 46, 1, 116, 17, 0, 124, 0, 0, 131, 1, 0, 1, 110, 22, 0, 116, 14, 0, 100, 8, 0, 116, 18, 0, 124, 0, 0, 131, 1, 0, 106, 19, 0, 22, 130, 2, 0, 100, 1, 0, 83]

  

>>> dir(dis.dis.func_code)
[‘__class__‘, ‘__cmp__‘, ‘__delattr__‘, ‘__doc__‘, ‘__eq__‘, ‘__format__‘, ‘__ge__‘, ‘__getattribute__‘, ‘__gt__‘, ‘__hash__‘, ‘__init__‘, ‘__le__‘, ‘__lt__‘, ‘__ne__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘co_argcount‘, ‘co_cellvars‘, ‘co_code‘, ‘co_consts‘, ‘co_filename‘, ‘co_firstlineno‘, ‘co_flags‘, ‘co_freevars‘, ‘co_lnotab‘, ‘co_name‘, ‘co_names‘, ‘co_nlocals‘, ‘co_stacksize‘, ‘co_varnames‘]
>>> dis.dis.func_code.co_filename
‘/usr/lib/python2.7/dis.py‘
>>> dis.dis.func_code.co_consts
(‘Disassemble classes, methods, functions, or code.\n\n    With no argument, disassemble the last traceback.\n\n    ‘, None, ‘im_func‘, ‘func_code‘, ‘__dict__‘, ‘Disassembly of %s:‘, ‘Sorry:‘, ‘co_code‘, "don‘t know how to disassemble %s objects")
>>> dis.dis.func_code.co_names
(‘None‘, ‘distb‘, ‘isinstance‘, ‘types‘, ‘InstanceType‘, ‘__class__‘, ‘hasattr‘, ‘im_func‘, ‘func_code‘, ‘__dict__‘, ‘items‘, ‘sort‘, ‘_have_code‘, ‘dis‘, ‘TypeError‘, ‘disassemble‘, ‘str‘, ‘disassemble_string‘, ‘type‘, ‘__name__‘)
>>> dis.dis.func_code.co_varnames
(‘x‘, ‘items‘, ‘name‘, ‘x1‘, ‘msg‘)
>>> dis.dis.func_code.co_stacksize
6
>>> dis.dis.func_code.co_nlocals
5

  

其实dis.dis也不过就是是一连串的字节码而已,它被Python解释器执行,从而完成指定的功能。

下面我们就使用dis.dis来反汇编一下字节码

>>> dis.dis(foo.func_code.co_code)
          0 LOAD_FAST           0 (0)
          3 LOAD_GLOBAL         0 (0)
          6 COMPARE_OP          4 (>)
          9 POP_JUMP_IF_FALSE    20
         12 LOAD_FAST           0 (0)
         15 LOAD_CONST          1 (1)
         18 BINARY_DIVIDE  
         19 RETURN_VALUE   
    >>   20 LOAD_FAST           0 (0)
         23 RETURN_VALUE   
         24 LOAD_CONST          0 (0)
         27 RETURN_VALUE  

  

 

 

 

 

 

 

  

 

 

Python字节码与解释器学习,布布扣,bubuko.com

Python字节码与解释器学习

上一篇:超慢速移动动画使用CSS3实现流畅效果


下一篇:Spring 常用工具类