python之GIL官方文档 global interpreter lock 全局解释器锁

2023-03-04 07:20:28

0.目录

2. 术语 global interpreter lock 全局解释器锁
3. C-API 还有更多没有仔细看
4. 定期切换线程
5. wiki.python
6. python.doc FAQ

1.参考

2. 术语 global interpreter lock 全局解释器锁

https://docs.python.org/2/glossary.html#term-global-interpreter-lock

https://docs.python.org/3/glossary.html#term-global-interpreter-lock

http://python.usyiyi.cn/documents/python_278/glossary.html#term-global-interpreter-lock

http://python.usyiyi.cn/documents/python_352/glossary.html#term-global-interpreter-lock

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time.
Cpython解释器所使用的机制：确保每一时刻只有一个线程可以执行python字节码

This simplifies the CPython implementation by making the object model (including critical built-in types such as dict)
implicitly safe against concurrent access.
这个机制隐式地解决了对象模型（包括诸如dict的关键内建类型）在并发访问中的安全问题，进而简化了CPython的实现，

Locking the entire interpreter makes it easier for the interpreter to be multi-threaded,
at the expense of much of the parallelism afforded by multi-processor machines.
锁定整个解释器使得解释器更容易实现多线程，代价是牺牲了多处理器机器的并行能力。

However, some extension modules, either standard or third-party, are designed so as to release the GIL
when doing computationally-intensive tasks such as compression or hashing.
Also, the GIL is always released when doing I/O.
然而，一些扩展模块，无论是标准还是第三方，被设计为在进行计算密集型任务（例如压缩或散列）时释放GIL。
此外，当执行I/O操作时，GIL总是被释放。

Past efforts to create a “free-threaded” interpreter (one which locks shared data at a much finer granularity)
have not been successful because performance suffered in the common single-processor case.
It is believed that overcoming this performance issue would make the implementation much more complicated
and therefore costlier to maintain.
过去创建“*线程”解释器（以更精细的粒度锁定共享数据）的努力没有成功，因为在常见的单处理器情况下性能会受限。
相信克服这种性能问题将使得实现更复杂，因此维护成本更高。

3. C-API 还有更多没有仔细看

https://docs.python.org/2/c-api/init.html#thread-state-and-the-global-interpreter-lock
https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock
http://python.usyiyi.cn/documents/python_278/c-api/init.html#thread-state-and-the-global-interpreter-lock
http://python.usyiyi.cn/documents/python_352/c-api/init.html#thread-state-and-the-global-interpreter-lock

The Python interpreter is not fully thread-safe.
CPython解释不是完全线程安全的。
In order to support multi-threaded Python programs, there’s a global lock, called the global interpreter lock or GIL,
that must be held by the current thread before it can safely access Python objects.
为了支持多线程Python程序，设计了一个全局锁GIL，当前线程必须持有GIL才能安全访问Python对象。

Without the lock, even the simplest operations could cause problems in a multi-threaded program:
for example, when two threads simultaneously increment the reference count of the same object,
the reference count could end up being incremented only once instead of twice.
如果没有GIL，在多线程程序中，即使是最简单的操作也可能引发问题：
例如，当两个线程同时增加同一对象的引用计数时，引用计数可能最终只增加一次而不是两次。

Therefore, the rule exists that only the thread that has acquired the GIL may operate on Python objects or call Python/C API functions.
In order to emulate concurrency of execution, the interpreter regularly tries to switch threads (see sys.setswitchinterval()).
The lock is also released around potentially blocking I/O operations like reading or writing a file,
so that other Python threads can run in the meantime.
因此，设定了这样一个规则，只有获取GIL的线程可以对Python对象操作或调用Python/C API函数。
为了模拟执行的并发性，解释器定期尝试切换线程（参见后续章节的sys.setswitchinterval() Python 2.x为sys.setcheckinterval()）。
潜在的阻塞I/O操作（例如读取或写入文件）也会释放GIL，以便其他Python线程可以同时运行。

Note Calling system I/O functions is the most common use case for releasing the GIL,
but it can also be useful before calling long-running computations which don’t need access to Python objects,
such as compression or cryptographic functions operating over memory buffers.
For example, the standard zlib and hashlib modules release the GIL when compressing or hashing data.
注意调用系统I/O函数是释放GIL的最常见的用例，
但在调用不需要访问Python对象的长时间运行的计算（例如，通过内存缓冲区操作的压缩或加密函数）之前，它也很有用。
例如，标准的zlib和hashlib模块在压缩或散列数据时释放GIL。

Note When only the main thread exists, no GIL operations are needed.
This is a common situation (most Python programs do not use threads), and the lock operations slow the interpreter down a bit.
Therefore, the lock is not created initially. This situation is equivalent to having acquired the lock:
when there is only a single thread, all object accesses are safe.
注意当只有主线程存在时，不需要GIL操作。
这是一个常见的情况（大多数Python程序不使用线程），锁操作会减慢解释器。
因此，最初不会创建锁定。这种情况相当于获得了锁：
当只有一个线程时，所有对象访问都是安全的。

4. 定期切换线程

(1)Python 2.x为sys.setcheckinterval()

https://docs.python.org/2/library/sys.html#sys.setcheckinterval

http://python.usyiyi.cn/documents/python_278/library/sys.html#sys.setcheckinterval

sys.setcheckinterval(interval)
Set the interpreter’s “check interval”.
This integer value determines how often the interpreter checks for periodic things such as thread switches and signal handlers.
The default is 100, meaning the check is performed every 100 Python virtual instructions.
Setting it to a larger value may increase performance for programs using threads.
Setting it to a value <= 0 checks every virtual instruction, maximizing responsiveness as well as overhead.
设置解释器的"检查间隔"。此整数值决定解释器检查周期性事情的频率，比如线程切换和信号处理程序。
默认值为100，意味着每100个Python虚拟指令执行一次检查。
将其设置为一个较大的值可能会增加使用线程的应用程序的性能。
将其设置为<= 0的一个值将检查每个虚拟机指令，这会最大限度地提高响应能力，同时会增加开销。
In [86]: sys.getcheckinterval()
Out[86]: 100

(2)Python 3.2开始为sys.setswitchinterval(interval)

https://docs.python.org/3/library/sys.html#sys.setswitchinterval

http://python.usyiyi.cn/documents/python_352/library/sys.html#sys.setswitchinterval

sys.setswitchinterval(interval)
Set the interpreter’s thread switch interval (in seconds).
This floating-point value determines the ideal duration of the “timeslices” allocated to concurrently running Python threads.
Please note that the actual value can be higher, especially if long-running internal functions or methods are used.
Also, which thread becomes scheduled at the end of the interval is the operating system’s decision.
The interpreter doesn’t have its own scheduler.
New in version 3.2.
sys.setswitchinterval(interval)
设置解释器的线程切换间隔（以秒为单位）。
此浮点值确定分配给并发运行的Python线程的“时间片”的理想持续时间。
请注意，实际值可以更高，特别是如果使用长期运行的内部函数或方法。
此外，在间隔结束时哪个线程被调度是操作系统的决定。
解释器没有自己的调度器。
>>> sys.getswitchinterval()
0.005

5. wiki.python

https://wiki.python.org/moin/GlobalInterpreterLock

In CPython, the global interpreter lock, or GIL, is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once. This lock is necessary mainly because CPython's memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.)

CPython extensions must be GIL-aware in order to avoid defeating threads. For an explanation, see Global interpreter lock.

The GIL is controversial because it prevents multithreaded CPython programs from taking full advantage of multiprocessor systems in certain situations. Note that potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.

However the GIL can degrade performance even when it is not a bottleneck. Summarizing those slides: The system call overhead is significant, especially on multicore hardware. Two threads calling a function may take twice as much time as a single thread calling the function twice. The GIL can cause I/O-bound threads to be scheduled ahead of CPU-bound threads. And it prevents signals from being delivered.

Non-CPython implementations

Jython and IronPython have no GIL and can fully exploit multiprocessor systems
PyPy currently has a GIL like CPython
in Cython the GIL exists, but can be released temporarily using a "with" statement

Understanding the Python GIL: David Beazley at PyCon 2010

里面的PDF值得一看