CPython

See also Compile CPython on Windows.

Developer mode

https://mail.python.org/pipermail/python-ideas/2016-March/039314.html

Enable all runtime debug checks in strict mode:

PYTHONMALLOC=debug python3.6 -Wd -bb -X faulthandler script.py

CPython infra

Services used by unit tests

  • pythontest.net services:

  • snakebite.net:

    # Testing connect timeout is tricky: we need to have IP connectivity
    # to a host that silently drops our packets.  We can't simulate this
    # from Python because it's a function of the underlying TCP/IP stack.
    # So, the following Snakebite host has been defined:
    blackhole = resolve_address('blackhole.snakebite.net', 56666)
    
    # Blackhole has been configured to silently drop any incoming packets.
    # No RSTs (for TCP) or ICMP UNREACH (for UDP/ICMP) will be sent back
    # to hosts that attempt to connect to this address: which is exactly
    # what we need to confidently test connect timeout.
    
    # However, we want to prevent false positives.  It's not unreasonable
    # to expect certain hosts may not be able to reach the blackhole, due
    # to firewalling or general network configuration.  In order to improve
    # our confidence in testing the blackhole, a corresponding 'whitehole'
    # has also been set up using one port higher:
    whitehole = resolve_address('whitehole.snakebite.net', 56667)
    
  • news.trigofacile.com:

  • ipv6.google.com:

    • test_ssl uses it to test IPv6: HTTP (tcp/80) and HTTPS (tcp/443)
  • sha256.tbs-internet.com:

    • test_ssl uses it to test x509 certificate signed by SHA256: HTTPS (tcp/443)

Package Index (PyPI)

cpython GitHub project

Misc

Documentation

Python developer mode

https://mail.python.org/pipermail/python-ideas/2016-March/039314.html

Strict developer mode:

PYTHONMALLOC=debug python3.6 -Werror -bb -X faulthandler script.py

Developer mode:

PYTHONMALLOC=debug python3.6 -Wd -b -X faulthandler script.py
  • Show DeprecationWarning and ResourceWarning warnings: python -Wd
  • Show BytesWarning warning: python -b
  • Enable faulthandler to get a Python traceback on segfault and fatal errors: python -X faulthandler
  • Debug hooks on Python memory allocators: PYTHONMALLOC=debug
  • Enable Python assertions (assert) and set __debug__ to True: remove (or just ignore) -O or -OO command line arguments

See also PYTHONASYNCIODEBUG=1 for asyncio.

Embedded libraries

See my external_versions.py script: external version of embedded libraries from CPython source code (locally).

On security branches, some dependencies are outdated because no more macOS nor Windows installer is built. It was decided to not upgrade outdated zlib 1.2.5 in Python 3.3.7, since it’s specific to Windows, and no Windows user is expected to build his/her own Python 3.3 anymore.

See also cpython-bin-deps and cpython-source-deps.

Contribute to CPython: where should you start?

Documentation:

How can you start? Where? That’s an hard question. CPython is old and widely used: any change must be carefully discussed to remain Python homogenous. For bug fixes, the most complex part is the backward compatibility.

It’s very hard to find “easy issue”. First, just watch the current activity to get some ideas.

Contribute to CPython? What is CPython? CPython is made of many parts:

  • CPython source code: basically made of 50% Python and 50% C code
  • Build system: complex tools to build Python on all platforms, Visual Studio project for Windows
  • Windows and macOS installer
  • 233,000 lines of documentation written with Sphinx
  • Documentation translated to multiple languages: french, japanese, and other languages
  • Bug tracker: bugs.python.org which has its own “meta” bug tracker for bugs in the bug tracker :-)
  • GitHub project: pull requests, host the Git repository
  • Travis CI to run the test suite on Linux and check the documentation
  • AppVeyor CI to run the test suite on Windows
  • Buildbot master and many workers to run the test suite as post-commit on many variables architectures and operating systems

There are also many things around Python:

  • Devguide: documentation of the CPython development workflow
  • python-dev and python-committers mailing lists
  • #python-dev IRC channel
  • PyPI

Contributing to CPython is hard and it can take up to 2 years until your work is released. Are you sure that CPython itself is the best project for you? Depending on your interests and skills, you may enjoy better to contribute to another popular project like Django or requests.

Contributing to CPython is harder because CPython developers require a very high quality for contributions: every change must be carefully documented, tested and implemented. The implementation can take several rounds until it reachs the expected quality and respect a long list of requirements.

Ok, I understand and I am well aware of all these things. But I want to contribute, how should I start?

  • Bug triage: complete bug report to adjust the title, complete the description, identify impacted Python version and impacted platforms, help to reproduce the bug, or even help to analyze and find the root bug.
  • Review pull requests: https://github.com/python/cpython/pulls/
    • Make sure that a change is carefully documented. For example, behaviour changes and enhancements must have the ”.. versionchanged:: x.y” markup in the documentation of the module. New features must be have the ”.. versionadded:: x.y” tag and maybe also documented in “What’s New in Python X.Y?” documentation.
    • The NEWS entry and commit message must first explain the solved problem, then maybe explain the change itself. While the commit message can be technical and very detailed, the NEWS entry should be short and written for end-users who don’t care of technical details.
    • Every change must be tested: exceptions are rare and should usually be justified. Example of exception which doesn’t have to be justified: changes only impacting the documentation, changes fixing a typo in a comment.
    • The backward compatibility is very important. A change must be carefully reviewed to make sure that it doesn’t break the backward compatibility. If it does break it, the change must be discussed with enough people to make sure that there is a smooth transition plan.
    • Feature removals require a DeprecationWarning in Python version N to remove it in version N+1. It’s common that a feature is kept longer to avoid breaking the world just being a developer wants the perfection. Python developers must be pragmatic: balance between perfection and usability, very hard choices should be made and usually it takes a a long to time to discuss them :-)
    • While Python 2 end of line is near (2020), Python 3 changes which make writing code working on Python 2 and Python 3 harder are usually rejected.
  • Propose to write a bugfix change (a pull request) for an existing bug report. This task is very hard since usually if a bug report doesn’t have a patch, it’s because there is a disagrement on the bug itself, or because the bugfix is very hard to implement.

Supported platforms

PEP 11 lists removed platforms:

  • MS-DOS
  • Python 3.4: VMS, OS/2, Windows 2000
  • Python 3.7: IRIX

Windows:

  • Windows XP supported in Python 2.7, not supported in Python 3.6
  • Windows 2000 support dropped in Python 3.4

Well supported platforms on Python 3.6 and 2.7:

  • Linux
  • Windows: XXX min version?
  • macOS: XXX min version?
  • FreeBSD

Tested by Travis CI and buildbots.

Supported platform with best effort support:

  • Android API 24
  • OpenBSD
  • NetBSD
  • Solaris, OpenIndiana
  • AIX

Platforms not supported officially:

  • Cygwin
  • MinGW
  • HP-UX