From bd05dc17dd0e82b41a37763104c28d173f902e36 Mon Sep 17 00:00:00 2001 From: "Ernest W. Durbin III" Date: Wed, 18 Sep 2019 17:25:57 -0400 Subject: [PATCH 001/871] Add @pablogsal to CPython Repository Administrators (#536) As requested by Pablo on infrastructure-staff mailing list, for integrations with the buildbot infrastructure. If I could get @brettcannon to confirm, we'll merge this and add them to the appropriate team. --- devcycle.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/devcycle.rst b/devcycle.rst index 608e6ddeb..add010f6c 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -322,6 +322,8 @@ Current Administrators +-------------------+----------------------------------------------------------+-----------------+ | Mariatta Wijaya | Maintainer of blurb_it and miss-islington | Mariatta | +-------------------+----------------------------------------------------------+-----------------+ +| Pablo Galindo | Maintainer of buildbot.python.org | pablogsal | ++-------------------+----------------------------------------------------------+-----------------+ Repository Release Manager Role Policy -------------------------------------- From 4cad57d555eac01d33a558121ac7816329f37480 Mon Sep 17 00:00:00 2001 From: Brett Cannon <54418+brettcannon@users.noreply.github.com> Date: Wed, 25 Sep 2019 11:04:11 -0700 Subject: [PATCH 002/871] Update developer log (#538) --- developers.csv | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/developers.csv b/developers.csv index d5b89945e..5c4ea8216 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Joannah Nanjekye,2019-09-23 Abhilash Raj,2019-08-06 Paul Ganssle,2019-06-15 Stéphane Wirtel,2019-04-08 @@ -8,15 +9,15 @@ Emily Morehouse,2018-09-14 Pablo Galindo,2018-06-06 Mark Shannon,2018-05-15 Petr Viktorin,2018-04-16 -Julien Palard,2017-12-08 Nathaniel J. Smith,2018-01-25 +Julien Palard,2017-12-08 Ivan Levkivskyi,2017-12-06 Carol Willing,2017-05-24 Mariatta,2017-01-27 Xiang Zhang,2016-11-21 Inada Naoki,2016-09-26 -Davin Potts,2016-03-06 Xavier de Gaye,2016-06-03 +Davin Potts,2016-03-06 Martin Panter,2015-08-10 Paul Moore,2015-03-15 Robert Collins,2014-10-16 @@ -122,11 +123,12 @@ Tony Lownds,2002-09-22 Steve Holden,2002-06-14 Christian Tismer,2002-05-17 Jason Tishler,2002-05-15 -Raymond Hettinger,2001-12-10 Walter Dörwald,2002-03-21 Andrew MacIntyre,2002-02-17 +Gregory P. Smith,2002-01-08 Anthony Baxter,2001-12-21 Neal Norwitz,2001-12-19 +Raymond Hettinger,2001-12-10 Chui Tey,2001-10-31 Michael W. Hudson,2001-08-27 Finn Bock,2001-08-23 @@ -151,7 +153,6 @@ Mark Hammond,2000-06-09 Marc-André Lemburg,2000-06-07 Trent Mick,2000-06-06 Eric S. Raymond,2000-06-02 -Gregory P. Smith,2002-01-08 Greg Stein,1999-11-07 Just van Rossum,1999-01-22 Greg Ward,1998-12-18 From f68d63a217e83086f49d6d9648df19bebb705390 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jes=C3=BAs=20Cea?= Date: Sat, 28 Sep 2019 02:00:12 +0200 Subject: [PATCH 003/871] Update autoconf version (#539) --- setup.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.rst b/setup.rst index 16cf82d30..a12822155 100644 --- a/setup.rst +++ b/setup.rst @@ -450,7 +450,7 @@ example, ``autoconf`` by itself will not regenerate ``pyconfig.h.in``. appropriate. Python's ``configure.ac`` script typically requires a specific version of -Autoconf. At the moment, this reads: ``AC_PREREQ(2.65)``. +Autoconf. At the moment, this reads: ``AC_PREREQ(2.69)``. If the system copy of Autoconf does not match this version, you will need to install your own copy of Autoconf. From 26176c8fd5264432fa2883f478ed35ac2bda18a4 Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Tue, 15 Oct 2019 15:26:27 +0200 Subject: [PATCH 004/871] 3.8 is now stable. (#544) --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 4fb1baadf..2bac654a5 100644 --- a/index.rst +++ b/index.rst @@ -95,7 +95,7 @@ Status of Python branches +==================+==============+=============+================+================+===================+ | master | :pep:`596` | features | *TBD* | *TBD* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-------------------+ -| 3.8 | :pep:`569` | prerelease | *2019-10-21* | *2024-10* | Łukasz Langa | +| 3.8 | :pep:`569` | stable | 2019-10-14 | *2024-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-------------------+ | 3.7 | :pep:`537` | bugfix | 2018-06-27 | *2023-06-27* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-------------------+ From c09d54616b93b858cb0e046be2bd0f2c6854dc1b Mon Sep 17 00:00:00 2001 From: Ned Deily Date: Tue, 15 Oct 2019 15:33:28 -0400 Subject: [PATCH 005/871] use bugfix instead of stable --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 2bac654a5..4fe736900 100644 --- a/index.rst +++ b/index.rst @@ -95,7 +95,7 @@ Status of Python branches +==================+==============+=============+================+================+===================+ | master | :pep:`596` | features | *TBD* | *TBD* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-------------------+ -| 3.8 | :pep:`569` | stable | 2019-10-14 | *2024-10* | Łukasz Langa | +| 3.8 | :pep:`569` | bugfix | 2019-10-14 | *2024-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-------------------+ | 3.7 | :pep:`537` | bugfix | 2018-06-27 | *2023-06-27* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-------------------+ From 6457c8a8a6a3d6a675a2d6b31fd435a1d478849f Mon Sep 17 00:00:00 2001 From: Ned Deily Date: Thu, 17 Oct 2019 23:45:36 -0400 Subject: [PATCH 006/871] add *atable* as synonym since it is used in docset sidebar --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 4fe736900..f5feda008 100644 --- a/index.rst +++ b/index.rst @@ -118,7 +118,7 @@ Status: :prerelease: feature fixes, bugfixes, and security fixes are accepted for the upcoming feature release. :bugfix: bugfixes and security fixes are accepted, new binaries are still - released. (Also called **maintenance** mode) + released. (Also called **maintenance** mode or **stable** release) :security: only security fixes are accepted and no more binaries are released, but new source-only versions can be released :end-of-life: release cycle is frozen; no further changes can be pushed to it. From e1e04d8e0d33d1d928ec9c47d55896123f4addff Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Sun, 20 Oct 2019 15:42:48 +0200 Subject: [PATCH 007/871] Python docs translations: add Indonesian (#546) --- documenting.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/documenting.rst b/documenting.rst index e9bb8149e..6e5c0089b 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1619,6 +1619,10 @@ in production, other are work in progress: | | | python-hu/>`__ | | | | | +--------------+--------------------+------------------------------------------+ +| Indonesian | `Oon Arfiandwi | `github `__ | +| | /oonid>`__ | | ++--------------+--------------------+------------------------------------------+ | Italian (it) | | `mail `__ | | | | | From e4eeb574dbab0ea350955c862163f2151c245fd3 Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Sun, 20 Oct 2019 22:34:07 +0200 Subject: [PATCH 008/871] Doc translations: FIX: Indonesian. (#547) --- documenting.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/documenting.rst b/documenting.rst index 6e5c0089b..b859598b3 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1620,8 +1620,9 @@ in production, other are work in progress: | | | | +--------------+--------------------+------------------------------------------+ | Indonesian | `Oon Arfiandwi | `github `__ | -| | /oonid>`__ | | +| (id) | `__ | +| | on.org/user32660 | | +| | />`__ | | +--------------+--------------------+------------------------------------------+ | Italian (it) | | `mail `__ | From 524ce4ad5da79a8192dca2195577a9b9363ea8d7 Mon Sep 17 00:00:00 2001 From: Subhrajyoti Sen Date: Tue, 22 Oct 2019 22:48:16 +0530 Subject: [PATCH 009/871] triaging: update dead or redirected links (#541) --- triaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/triaging.rst b/triaging.rst index fe40fe720..2c104ab3f 100644 --- a/triaging.rst +++ b/triaging.rst @@ -598,9 +598,9 @@ Checklist for Triaging .. _Tools: https://github.com/python/cpython/tree/master/Tools/ .. _Tools/demo: https://github.com/python/cpython/tree/master/Tools/demo/ .. _Developer's guide: https://github.com/python/devguide/ -.. _GSoC: https://developers.google.com/open-source/gsoc/ +.. _GSoC: https://summerofcode.withgoogle.com/ .. _issue tracker: https://bugs.python.org -.. _GitHub pull requests: https://github.com/python/cpython/pulls> +.. _GitHub pull requests: https://github.com/python/cpython/pulls .. _Python source code repositories: https://github.com/python/cpython/ .. _Reporting security issues in Python: https://www.python.org/news/security/ .. _python-ideas: https://mail.python.org/mailman/listinfo/python-ideas From f847394515dbf161851b501fc22f6508e0117d7b Mon Sep 17 00:00:00 2001 From: Brett Cannon <54418+brettcannon@users.noreply.github.com> Date: Fri, 15 Nov 2019 16:27:49 -0800 Subject: [PATCH 010/871] Create a TODO list for onboarding (#551) --- coredev.rst | 53 ++++++++++++++++++----------------------------------- 1 file changed, 18 insertions(+), 35 deletions(-) diff --git a/coredev.rst b/coredev.rst index a6810f662..2ab2a680c 100644 --- a/coredev.rst +++ b/coredev.rst @@ -16,7 +16,8 @@ Typically a core developer will offer you the chance to gain commit privilege. The person making the offer will become your mentor and watch your commits for a while to make sure you understand the development process. If other core developers agree that you should gain commit privileges you are then extended -an official offer. +an official offer. How core developers come to that agreement are outlined in +:pep:`13`. What it Means ------------- @@ -61,7 +62,22 @@ Gaining Commit Privileges ------------------------- When you have been extended an official offer to become a Python core -developer, there are several things you must do. +developer, there are several things you and the person handling your onboarding +must do. + +1. Find out who is handling your onboarding (your mentor should know who this + is; at worst ask the steering council) +2. Email the person handling your onboarding +3. The person onboarding you will ask you for various account details to record + them at https://github.com/python/voters/ +4. They will ask what email address you would like to subscribe to + python-committers with +5. They will turn on various permissions based on the information you provided + in the previous steps +6. They will update the devguide to publicly list your team membership at + :ref:`developers` +7. They will announce your membership to python-committers + Mailing Lists ''''''''''''' @@ -71,39 +87,6 @@ python-checkins, and one of new-bugs-announce or python-bugs-list. See :ref:`communication` for links to these mailing lists. -Issue Tracker -''''''''''''' - -If you did not gain the Developer role in the `issue tracker`_ before gaining -commit privileges, please say so. The role allows issues to be assigned to you, -and it allows you to triage (e.g. close or reassign) issues. - -A tracker admin should also flip your "is committer" bit in the tracker's -account screen. This will add the Python logo next to your name in discussions -on the tracker. Note that this is separate from the Developer role. - -It is expected that on the issue tracker you have a username in the form of -"first_name.last_name". If your initial issue tracker username is not of this -form, please change it. This is so that it is easier to assign issues to the -right person. - - -GitHub -'''''' - -You will be added to the ``Python core team`` on GitHub. This will give you -rights to commit to various repositories under the `Python organization`_ -on GitHub. When you are initially added you will be emailed by GitHub with -an invitation to join the team. Please accept the invite in the email or -go to https://github.com/python and accept the invite there. - -An entry in the :ref:`developers` should also be entered for you. -Typically the person who sponsored your application to become a core developer -makes sure an entry is created for you. - -.. _Python organization: https://github.com/python - - .. _contributor_agreement: Sign a Contributor Agreement From 3871715e03c7adaf2720eafb4c622677fb0d3c36 Mon Sep 17 00:00:00 2001 From: Brett Cannon <54418+brettcannon@users.noreply.github.com> Date: Fri, 15 Nov 2019 17:02:47 -0800 Subject: [PATCH 011/871] Add GitHub sernames for core dev team list (#552) --- developers.csv | 334 ++++++++++++++++++++++++------------------------- developers.rst | 2 +- 2 files changed, 168 insertions(+), 168 deletions(-) diff --git a/developers.csv b/developers.csv index 5c4ea8216..2bb4ccd33 100644 --- a/developers.csv +++ b/developers.csv @@ -1,167 +1,167 @@ -Joannah Nanjekye,2019-09-23 -Abhilash Raj,2019-08-06 -Paul Ganssle,2019-06-15 -Stéphane Wirtel,2019-04-08 -Stefan Behnel,2019-04-08 -Cheryl Sabella,2019-02-19 -Lisa Roach,2018-09-14 -Emily Morehouse,2018-09-14 -Pablo Galindo,2018-06-06 -Mark Shannon,2018-05-15 -Petr Viktorin,2018-04-16 -Nathaniel J. Smith,2018-01-25 -Julien Palard,2017-12-08 -Ivan Levkivskyi,2017-12-06 -Carol Willing,2017-05-24 -Mariatta,2017-01-27 -Xiang Zhang,2016-11-21 -Inada Naoki,2016-09-26 -Xavier de Gaye,2016-06-03 -Davin Potts,2016-03-06 -Martin Panter,2015-08-10 -Paul Moore,2015-03-15 -Robert Collins,2014-10-16 -Berker Peksağ,2014-06-26 -Steve Dower,2014-05-10 -Kushal Das,2014-04-14 -Steven D'Aprano,2014-02-08 -Yury Selivanov,2014-01-23 -Zachary Ware,2013-11-02 -Donald Stufft,2013-08-14 -Ethan Furman,2013-05-11 -Serhiy Storchaka,2012-12-26 -Chris Jerdonek,2012-09-24 -Eric Snow,2012-09-05 -Peter Moody,2012-05-20 -Hynek Schlawack,2012-05-14 -Richard Oudkerk,2012-04-29 -Andrew Svetlov,2012-03-13 -Petri Lehtinen,2011-10-22 -Meador Inge,2011-09-19 -Jeremy Kloth,2011-09-12 -Sandro Tosi,2011-08-01 -Alex Gaynor,2011-07-18 -Charles-François Natali,2011-05-19 -Nadeem Vawda,2011-04-10 -Jason R. Coombs,2011-03-14 -Ross Lagerwall,2011-03-13 -Eli Bendersky,2011-01-11 -Ned Deily,2011-01-09 -David Malcolm,2010-10-27 -Tal Einat,2010-10-04 -Łukasz Langa,2010-09-08 -Daniel Stutzbach,2010-08-22 -Éric Araujo,2010-08-10 -Brian Quinlan,2010-07-26 -Alexander Belopolsky,2010-05-25 -Tim Golden,2010-04-21 -Giampaolo Rodolà,2010-04-17 -Jean-Paul Calderone,2010-04-06 -Brian Curtin,2010-03-24 -Florent Xicluna,2010-02-25 -Dino Viehland,2010-02-23 -Larry Hastings,2010-02-22 -Victor Stinner,2010-01-30 -Stefan Krah,2010-01-05 -Doug Hellmann,2009-09-20 -Frank Wierzbicki,2009-08-02 -Ezio Melotti,2009-06-07 -Philip Jenvey,2009-05-07 -Michael Foord,2009-04-01 -R. David Murray,2009-03-30 -Chris Withers,2009-03-08 -Tarek Ziadé,2008-12-21 -Hirokazu Yamamoto,2008-08-12 -Armin Ronacher,2008-07-23 -Antoine Pitrou,2008-07-16 -Senthil Kumaran,2008-06-16 -Jesse Noller,2008-06-16 -Jesús Cea,2008-05-13 -Guilherme Polo,2008-04-24 -Jeroen Ruigrok van der Werven,2008-04-12 -Benjamin Peterson,2008-03-25 -David Wolever,2008-03-17 -Trent Nelson,2008-03-17 -Mark Dickinson,2008-01-06 -Amaury Forgeot d'Arc,2007-11-09 -Christian Heimes,2007-10-31 -Bill Janssen,2007-08-28 -Jeffrey Yasskin,2007-08-09 -Mark Summerfield,2007-08-01 -Alexandre Vassalotti,2007-05-21 -Travis E. Oliphant,2007-04-17 -Eric V. Smith,2007-02-28 -Josiah Carlson,2007-01-06 -Collin Winter,2007-01-05 -Richard Jones,2006-05-23 -Kristján Valur Jónsson,2006-05-17 -Jack Diederich,2006-05-17 -Steven Bethard,2006-04-27 -Gerhard Häring,2006-04-23 -George Yoshida,2006-04-17 -Ronald Oussoren,2006-03-03 -Nick Coghlan,2005-10-16 -Georg Brandl,2005-05-28 -Terry Jan Reedy,2005-04-07 -Bob Ippolito,2005-03-02 -Peter Astrand,2004-10-21 -Facundo Batista,2004-10-16 -Sean Reifschneider,2004-09-17 -Johannes Gijsbers,2004-08-14 -Matthias Klose,2004-08-04 -PJ Eby,2004-03-24 -Vinay Sajip,2004-02-20 -Hye-Shik Chang,2003-12-10 -Armin Rigo,2003-10-24 -Andrew McNamara,2003-06-09 -Samuele Pedroni,2003-05-16 -Alex Martelli,2003-04-22 -Brett Cannon,2003-04-18 -David Goodger,2003-01-02 -Gustavo Niemeyer,2002-11-05 -Tony Lownds,2002-09-22 -Steve Holden,2002-06-14 -Christian Tismer,2002-05-17 -Jason Tishler,2002-05-15 -Walter Dörwald,2002-03-21 -Andrew MacIntyre,2002-02-17 -Gregory P. Smith,2002-01-08 -Anthony Baxter,2001-12-21 -Neal Norwitz,2001-12-19 -Raymond Hettinger,2001-12-10 -Chui Tey,2001-10-31 -Michael W. Hudson,2001-08-27 -Finn Bock,2001-08-23 -Piers Lauder,2001-07-20 -Kurt B. Kaiser,2001-07-03 -Steven M. Gava,2001-06-25 -Steve Purcell,2001-03-22 -Jim Fulton,2000-10-06 -Ka-Ping Yee,2000-10-03 -Lars Gustäbel,2000-09-21 -Neil Schemenauer,2000-09-15 -Martin v. Löwis,2000-09-08 -Thomas Heller,2000-09-07 -Moshe Zadka,2000-07-29 -Thomas Wouters,2000-07-14 -Peter Schneider-Kamp,2000-07-10 -Paul Prescod,2000-07-01 -Tim Peters,2000-06-30 -Skip Montanaro,2000-06-30 -Fredrik Lundh,2000-06-29 -Mark Hammond,2000-06-09 -Marc-André Lemburg,2000-06-07 -Trent Mick,2000-06-06 -Eric S. Raymond,2000-06-02 -Greg Stein,1999-11-07 -Just van Rossum,1999-01-22 -Greg Ward,1998-12-18 -Andrew Kuchling,1998-04-09 -Ken Manheimer,1998-03-03 -Jeremy Hylton,1997-08-13 -Roger E. Masse,1996-12-09 -Fred Drake,1996-07-23 -Barry Warsaw,1994-07-25 -Jack Jansen,1992-08-13 -Sjoerd Mullender,1992-08-04 -Guido van Rossum,1989-12-25 +Joannah Nanjekye,nanjekyejoannah,2019-09-23 +Abhilash Raj,maxking,2019-08-06 +Paul Ganssle,pganssle,2019-06-15 +Stéphane Wirtel,matrixise,2019-04-08 +Stefan Behnel,scoder,2019-04-08 +Cheryl Sabella,csabella,2019-02-19 +Lisa Roach,lisroach,2018-09-14 +Emily Morehouse,emilyemorehouse,2018-09-14 +Pablo Galindo,pablogsal,2018-06-06 +Mark Shannon,markshannon,2018-05-15 +Petr Viktorin,encukou,2018-04-16 +Nathaniel J. Smith,njsmith,2018-01-25 +Julien Palard,JulienPalard,2017-12-08 +Ivan Levkivskyi,ilevkivskyi,2017-12-06 +Carol Willing,willingc,2017-05-24 +Mariatta,Mariatta,2017-01-27 +Xiang Zhang,zhangyangyu,2016-11-21 +Inada Naoki,methane,2016-09-26 +Xavier de Gaye,xdegaye,2016-06-03 +Davin Potts,applio,2016-03-06 +Martin Panter,vadmium,2015-08-10 +Paul Moore,pfmoore,2015-03-15 +Robert Collins,rbtcollins,2014-10-16 +Berker Peksağ,berkerpeksag,2014-06-26 +Steve Dower,zooba,2014-05-10 +Kushal Das,kushaldas,2014-04-14 +Steven D'Aprano,stevendaprano,2014-02-08 +Yury Selivanov,1st1,2014-01-23 +Zachary Ware,zware,2013-11-02 +Donald Stufft,dstufft,2013-08-14 +Ethan Furman,ethanfurman,2013-05-11 +Serhiy Storchaka,serhiy-storchaka,2012-12-26 +Chris Jerdonek,cjerdonek,2012-09-24 +Eric Snow,ericsnowcurrently,2012-09-05 +Peter Moody,,2012-05-20 +Hynek Schlawack,hynek,2012-05-14 +Richard Oudkerk,,2012-04-29 +Andrew Svetlov,asvetlov,2012-03-13 +Petri Lehtinen,akheron,2011-10-22 +Meador Inge,meadori,2011-09-19 +Jeremy Kloth,jkloth,2011-09-12 +Sandro Tosi,sandrotosi,2011-08-01 +Alex Gaynor,alex,2011-07-18 +Charles-François Natali,,2011-05-19 +Nadeem Vawda,,2011-04-10 +Jason R. Coombs,jaraco,2011-03-14 +Ross Lagerwall,,2011-03-13 +Eli Bendersky,eliben,2011-01-11 +Ned Deily,ned-deily,2011-01-09 +David Malcolm,davidmalcolm,2010-10-27 +Tal Einat,taleinat,2010-10-04 +Łukasz Langa,ambv,2010-09-08 +Daniel Stutzbach,,2010-08-22 +Éric Araujo,merwok,2010-08-10 +Brian Quinlan,brianquinlan,2010-07-26 +Alexander Belopolsky,abalkin,2010-05-25 +Tim Golden,tjguk,2010-04-21 +Giampaolo Rodolà,giampaolo,2010-04-17 +Jean-Paul Calderone,,2010-04-06 +Brian Curtin,briancurtin,2010-03-24 +Florent Xicluna,,2010-02-25 +Dino Viehland,DinoV,2010-02-23 +Larry Hastings,larryhastings,2010-02-22 +Victor Stinner,vstinner,2010-01-30 +Stefan Krah,skrah,2010-01-05 +Doug Hellmann,dhellmann,2009-09-20 +Frank Wierzbicki,,2009-08-02 +Ezio Melotti,ezio-melotti,2009-06-07 +Philip Jenvey,pjenvey,2009-05-07 +Michael Foord,voidspace,2009-04-01 +R. David Murray,bitdancer,2009-03-30 +Chris Withers,cjw296,2009-03-08 +Tarek Ziadé,,2008-12-21 +Hirokazu Yamamoto,,2008-08-12 +Armin Ronacher,mitsuhiko,2008-07-23 +Antoine Pitrou,pitrou,2008-07-16 +Senthil Kumaran,orsenthil,2008-06-16 +Jesse Noller,,2008-06-16 +Jesús Cea,jcea,2008-05-13 +Guilherme Polo,,2008-04-24 +Jeroen Ruigrok van der Werven,,2008-04-12 +Benjamin Peterson,benjaminp,2008-03-25 +David Wolever,wolever,2008-03-17 +Trent Nelson,tpn,2008-03-17 +Mark Dickinson,mdickinson,2008-01-06 +Amaury Forgeot d'Arc,amauryfa,2007-11-09 +Christian Heimes,tiran,2007-10-31 +Bill Janssen,,2007-08-28 +Jeffrey Yasskin,,2007-08-09 +Mark Summerfield,,2007-08-01 +Alexandre Vassalotti,avassalotti,2007-05-21 +Travis E. Oliphant,,2007-04-17 +Eric V. Smith,ericvsmith,2007-02-28 +Josiah Carlson,,2007-01-06 +Collin Winter,,2007-01-05 +Richard Jones,,2006-05-23 +Kristján Valur Jónsson,,2006-05-17 +Jack Diederich,jackdied,2006-05-17 +Steven Bethard,,2006-04-27 +Gerhard Häring,,2006-04-23 +George Yoshida,,2006-04-17 +Ronald Oussoren,ronaldoussoren,2006-03-03 +Nick Coghlan,ncoghlan,2005-10-16 +Georg Brandl,birkenfeld,2005-05-28 +Terry Jan Reedy,terryjreedy,2005-04-07 +Bob Ippolito,,2005-03-02 +Peter Astrand,,2004-10-21 +Facundo Batista,facundobatista,2004-10-16 +Sean Reifschneider,,2004-09-17 +Johannes Gijsbers,,2004-08-14 +Matthias Klose,doko42,2004-08-04 +PJ Eby,pjeby,2004-03-24 +Vinay Sajip,vsajip,2004-02-20 +Hye-Shik Chang,,2003-12-10 +Armin Rigo,,2003-10-24 +Andrew McNamara,,2003-06-09 +Samuele Pedroni,,2003-05-16 +Alex Martelli,aleaxit,2003-04-22 +Brett Cannon,brettcannon,2003-04-18 +David Goodger,,2003-01-02 +Gustavo Niemeyer,,2002-11-05 +Tony Lownds,,2002-09-22 +Steve Holden,,2002-06-14 +Christian Tismer,ctismer,2002-05-17 +Jason Tishler,,2002-05-15 +Walter Dörwald,doerwalter,2002-03-21 +Andrew MacIntyre,,2002-02-17 +Gregory P. Smith,gpshead,2002-01-08 +Anthony Baxter,,2001-12-21 +Neal Norwitz,,2001-12-19 +Raymond Hettinger,rhettinger,2001-12-10 +Chui Tey,,2001-10-31 +Michael W. Hudson,,2001-08-27 +Finn Bock,,2001-08-23 +Piers Lauder,,2001-07-20 +Kurt B. Kaiser,kbkaiser,2001-07-03 +Steven M. Gava,,2001-06-25 +Steve Purcell,,2001-03-22 +Jim Fulton,,2000-10-06 +Ka-Ping Yee,,2000-10-03 +Lars Gustäbel,gustaebel,2000-09-21 +Neil Schemenauer,nascheme,2000-09-15 +Martin v. Löwis,,2000-09-08 +Thomas Heller,theller,2000-09-07 +Moshe Zadka,,2000-07-29 +Thomas Wouters,Yhg1s,2000-07-14 +Peter Schneider-Kamp,,2000-07-10 +Paul Prescod,,2000-07-01 +Tim Peters,tim-one,2000-06-30 +Skip Montanaro,smontanaro,2000-06-30 +Fredrik Lundh,,2000-06-29 +Mark Hammond,mhammond,2000-06-09 +Marc-André Lemburg,malemburg,2000-06-07 +Trent Mick,,2000-06-06 +Eric S. Raymond,,2000-06-02 +Greg Stein,,1999-11-07 +Just van Rossum,,1999-01-22 +Greg Ward,,1998-12-18 +Andrew Kuchling,akuchling,1998-04-09 +Ken Manheimer,,1998-03-03 +Jeremy Hylton,jeremyhylton,1997-08-13 +Roger E. Masse,,1996-12-09 +Fred Drake,freddrake,1996-07-23 +Barry Warsaw,warsaw,1994-07-25 +Jack Jansen,jackjansen,1992-08-13 +Sjoerd Mullender,sjoerdmullender,1992-08-04 +Guido van Rossum,gvanrossum,1989-12-25 diff --git a/developers.rst b/developers.rst index 5b61c706c..7c922ece2 100644 --- a/developers.rst +++ b/developers.rst @@ -8,7 +8,7 @@ master list is kept in a private repository due to containing sensitive contact information.) .. csv-table:: - :header: "Name", "Joined" + :header: "Name", "GitHub username", "Joined" :file: developers.csv :encoding: "utf-8" From 765b0aba74a8c9c7d4629c4859b1118383342be0 Mon Sep 17 00:00:00 2001 From: Tal Einat Date: Sun, 17 Nov 2019 19:52:49 +0200 Subject: [PATCH 012/871] add taleinat as an expert for the idlelib and statistics modules --- experts.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/experts.rst b/experts.rst index fd783d917..6aadf1e12 100644 --- a/experts.rst +++ b/experts.rst @@ -130,7 +130,8 @@ heapq rhettinger*, stutzbach hmac christian.heimes, gregory.p.smith html ezio.melotti http -idlelib kbk (inactive), terry.reedy*, roger.serwy (inactive) +idlelib kbk (inactive), terry.reedy*, roger.serwy (inactive), + taleinat imaplib imghdr imp @@ -213,7 +214,7 @@ spwd sqlite3 ghaering ssl janssen, christian.heimes, dstufft, alex stat christian.heimes -statistics steven.daprano, rhettinger +statistics steven.daprano, rhettinger, taleinat string stringprep struct mark.dickinson, meador.inge From c9eb59a78597e5fbbe50d916391f714168e0a83d Mon Sep 17 00:00:00 2001 From: Inada Naoki Date: Mon, 18 Nov 2019 02:55:23 +0900 Subject: [PATCH 013/871] Prefer merge over rebase --- pullrequest.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pullrequest.rst b/pullrequest.rst index cdeab54b3..0cd00253a 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -97,8 +97,8 @@ You should have already :ref:`set up your system `, * If someone else added new changesets and you get an error:: git fetch upstream - git rebase upstream/master - git push --force-with-lease origin + git merge upstream/master + git push origin * Finally go on :samp:`https://github.com/{}/cpython`: you will see a box with the branch you just pushed and a green button that allows From 907ebaf5066fdc7bac36a67c4df100d51e5ebc26 Mon Sep 17 00:00:00 2001 From: Tal Einat Date: Wed, 20 Nov 2019 10:32:12 +0200 Subject: [PATCH 014/871] improve instructions about merging git branches in the step-by-step guide --- pullrequest.rst | 53 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 47 insertions(+), 6 deletions(-) diff --git a/pullrequest.rst b/pullrequest.rst index 0cd00253a..859261948 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -94,12 +94,6 @@ You should have already :ref:`set up your system `, git push origin -* If someone else added new changesets and you get an error:: - - git fetch upstream - git merge upstream/master - git push origin - * Finally go on :samp:`https://github.com/{}/cpython`: you will see a box with the branch you just pushed and a green button that allows you to create a pull request against the official CPython repository. @@ -114,6 +108,24 @@ You should have already :ref:`set up your system `, git commit -m '' git push origin + * If a core developer reviewing your PR pushed one or more commits to your + PR branch, then after checking out your branch and before editing, run:: + + git pull origin # pull = fetch + merge + + If you have made local changes that have not been pushed to your fork and + there are merge conflicts, git will warn you about this and enter conflict + resolution mode. See :ref:`resolving-merge-conflicts` below. + +* If time passes and there are merge conflicts with the master branch, GitHub + will show a warning to this end and you may be asked to address this. Merge + the changes from the master branch while resolving the conflicts locally:: + + git checkout + git pull upstream master # pull = fetch + merge + # resolve conflicts: see "Resolving Merge Conflicts" below + git push origin + * After your PR has been accepted and merged, you can :ref:`delete the branch `:: @@ -125,6 +137,35 @@ You should have already :ref:`set up your system `, workflow is **strongly** preferred. +.. _resolving-merge-conflicts: + +Resolving Merge Conflicts +''''''''''''''''''''''''' + +When merging changes from different branches (or variants of a branch on +different repos), the two branches may contain incompatible changes to one +or more files. These are called "merge conflicts" and need to be manually +resolved as follows: + +#. Check which files have merge conflicts:: + + git status + +#. Edit the affected files and bring them to their intended final state. + Make sure to remove the special "conflict markers" inserted by git. + +#. Commit the affected files:: + + git add + git merge --continue + +When running the final command, git may open an editor for writing a commit +message. It is usually okay to leave that as-is and close the editor. + +See `the merge command's documentation `_ +for a detailed technical explanation. + + .. _good-prs: Making Good PRs From bf7c924a1a6dbfe20853727999694ea073c6acde Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Sun, 5 Jan 2020 10:17:08 +0100 Subject: [PATCH 015/871] Translating: FIX inconsistency with PEP 545. (#545) --- documenting.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documenting.rst b/documenting.rst index b859598b3..6207c8d1f 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1715,7 +1715,7 @@ The important steps looks like this: way, it's up to you. - Ensure we updated this page to reflect your work and progress, either via a PR, or by asking on the doc-sig mailing list. -- When ``tutorial/``, ``library/stdtypes`` and ``library/functions`` +- When ``tutorial/``, ``bugs.py`` and ``library/functions`` are complete, ask on doc-sig for your language to be added in the language picker on docs.python.org. From 11137d4f4471f6fc1236e48ebc7fe28ff41c9e32 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 7 Jan 2020 22:23:38 +0100 Subject: [PATCH 016/871] Remove myself from Unicode experts I'm getting too many notifications. I prefer to poll for Unicode issues, rather than being notified of all Unicode issues. --- experts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/experts.rst b/experts.rst index 6aadf1e12..688eb1429 100644 --- a/experts.rst +++ b/experts.rst @@ -353,7 +353,7 @@ testing michael.foord, ezio.melotti test coverage threads time and dates lemburg, belopolsky, p-ganssle -unicode lemburg, ezio.melotti, vstinner, benjamin.peterson, +unicode lemburg, ezio.melotti, benjamin.peterson, version control eric.araujo, ezio.melotti ================== ========================================================== From fb3df5a3379bb565a1b62eb8fe188b3a8a619cd3 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 13 Jan 2020 12:14:50 +0000 Subject: [PATCH 017/871] Add myself to the relevant areas in experts.rst --- experts.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/experts.rst b/experts.rst index 688eb1429..7c54c3919 100644 --- a/experts.rst +++ b/experts.rst @@ -55,7 +55,7 @@ abc aifc r.david.murray argparse rhettinger* array -ast benjamin.peterson +ast benjamin.peterson, pablogsal asynchat josiahcarlson, giampaolo.rodola*, stutzbach asyncio yselivanov, asvetlov asyncore josiahcarlson, giampaolo.rodola*, stutzbach @@ -118,7 +118,7 @@ fpectl twouters fractions mark.dickinson, rhettinger ftplib giampaolo.rodola* functools rhettinger* -gc pitrou +gc pitrou, pablogsal getopt getpass gettext @@ -167,7 +167,7 @@ optparse aronacher os os.path serhiy.storchaka ossaudiodev -parser benjamin.peterson +parser benjamin.peterson, pablogsal pathlib pdb pickle alexandre.vassalotti @@ -313,11 +313,11 @@ Interest Area Maintainers ================== ========================================================== algorithms rhettinger* argument clinic larry -ast/compiler benjamin.peterson, brett.cannon, yselivanov +ast/compiler benjamin.peterson, brett.cannon, yselivanov, pablogsal autoconf/makefiles twouters* bsd bug tracker ezio.melotti -buildbots zach.ware +buildbots zach.ware, pablogsal bytecode benjamin.peterson, yselivanov context managers ncoghlan core workflow mariatta From a46588fc149f000e14b5066a8c9b81149477b8e5 Mon Sep 17 00:00:00 2001 From: ngie-eign <1574099+ngie-eign@users.noreply.github.com> Date: Tue, 14 Jan 2020 21:37:45 -0800 Subject: [PATCH 018/871] Fix a typo in the Quick Links section (#561) The phrase "Get Setup" should be spelled like "Git Setup". The former doesn't make any semantic sense, whereas the latter does. Signed-off-by: Enji Cooper --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index f5feda008..3ff5fca6c 100644 --- a/index.rst +++ b/index.rst @@ -19,7 +19,7 @@ patch. This is meant as a checklist, once you know the basics. For complete instructions please see the :ref:`setup guide `. 1. Install and set up :ref:`Git ` and other dependencies - (see the :ref:`Get Setup ` page for detailed information). + (see the :ref:`Git Setup ` page for detailed information). 2. Fork `the CPython repository `_ to your GitHub account and :ref:`get the source code ` using:: From d43d6e9cbf72199a7d55825bb4298aa207a71a5a Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 21 Jan 2020 14:03:57 +0000 Subject: [PATCH 019/871] Add section about the design of CPython's garbage collector (#562) Co-Authored-By: Tim Peters Co-Authored-By: Terry Jan Reedy Co-authored-by: Brett Cannon --- appendix.rst | 3 +- garbage_collector.rst | 487 +++++++++++++++++++++++++ images/python-cyclic-gc-1-new-page.png | Bin 0 -> 4415 bytes images/python-cyclic-gc-2-new-page.png | Bin 0 -> 4337 bytes images/python-cyclic-gc-3-new-page.png | Bin 0 -> 4876 bytes images/python-cyclic-gc-4-new-page.png | Bin 0 -> 4863 bytes images/python-cyclic-gc-5-new-page.png | Bin 0 -> 5712 bytes index.rst | 2 + 8 files changed, 491 insertions(+), 1 deletion(-) create mode 100644 garbage_collector.rst create mode 100644 images/python-cyclic-gc-1-new-page.png create mode 100644 images/python-cyclic-gc-2-new-page.png create mode 100644 images/python-cyclic-gc-3-new-page.png create mode 100644 images/python-cyclic-gc-4-new-page.png create mode 100644 images/python-cyclic-gc-5-new-page.png diff --git a/appendix.rst b/appendix.rst index f2db2ac2e..793f615c8 100644 --- a/appendix.rst +++ b/appendix.rst @@ -51,6 +51,7 @@ Language development in depth * :doc:`exploring` * :doc:`grammar` * :doc:`compiler` +* :doc:`garbage_collector` * :doc:`stdlibchanges` * :doc:`langchanges` * :doc:`porting` @@ -64,4 +65,4 @@ Testing and continuous integration * :doc:`buildbots` * :doc:`buildworker` * :doc:`coverity` - \ No newline at end of file + diff --git a/garbage_collector.rst b/garbage_collector.rst new file mode 100644 index 000000000..98ceff178 --- /dev/null +++ b/garbage_collector.rst @@ -0,0 +1,487 @@ +.. _gc: + +Design of CPython's Garbage Collector +===================================== + +:Author: Pablo Galindo Salgado + +.. highlight:: none + +Abstract +-------- + +The main garbage collector system of CPython is reference count. The basic idea is +that CPython counts how many different places there are that have a reference to an +object. Such a place could be another object, or a global (or static) C variable, or +a local variable in some C function. When an object’s reference count becomes zero, +the object is deallocated. If it contains references to other objects, their +reference count is decremented. Those other objects may be deallocated in turn, if +this decrement makes their reference count become zero, and so on. The reference +count field can be examined using the ``sys.getrefcount`` function (notice that the +value returned by this function is always 1 more as the function also has a reference +to the object when called): + +.. code-block:: python + + >>> x = object() + >>> sys.getrefcount(x) + 2 + >>> y = x + >>> sys.getrefcount(x) + 3 + del y + >>> sys.getrefcount(x) + 2 + +The main problem with the reference count schema is that reference counting +does not handle reference cycles. For instance, consider this code: + +.. code-block:: python + + >>> container = [] + >>> container.append(container) + >>> sys.getrefcount(container) + 3 + >>> del container + +In this example, ``container`` holds a reference to itself, so even when we remove +our reference to it (the variable "container") the reference count never falls to 0 +because it still has its own internal reference and therefore it will never be +cleaned just by simple reference counting. For this reason some additional machinery +is needed to clean these reference cycles between objects once they become +unreachable. This is the cyclic garbage collector, usually called just Garbage +Collector (GC), even though reference counting is also a form of garbage collection. + +Memory layout and object structure +---------------------------------- + +Normally the C structure supporting a regular Python object looks as follows: + +.. code-block:: none + + object -----> +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ \ + | ob_refcnt | | + +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | PyObject_HEAD + | *ob_type | | + +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ / + | ... | + + +In order to support the garbage collector, the memory layout of objects is altered +to accommodate extra information **before** the normal layout: + +.. code-block:: none + + +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ \ + | *_gc_next | | + +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | PyGC_Head + | *_gc_prev | | + object -----> +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ / + | ob_refcnt | \ + +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | PyObject_HEAD + | *ob_type | | + +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ / + | ... | + + +In this way the object can be treated as a normal python object and when the extra +information associated to the GC is needed the previous fields can be accessed by a +simple type cast from the original object: :code:`((PyGC_Head *)(the_object)-1)`. + +As is explained later in the `Optimization: reusing fields to save memory`_ section, +these two extra fields are normally used to keep doubly linked lists of all the +objects tracked by the garbage collector (these lists are the GC generations, more on +that in the `Optimization: generations`_ section), but they are also +reused to fullfill other pourposes when the full doubly linked list structure is not +needed as a memory optimization. + +Doubly linked lists are used because they efficiently support most frequently required operations. In +general, the collection of all objects tracked by GC are partitioned into disjoint sets, each in its own +doubly linked list. Between collections, objects are partitioned into "generations", reflecting how +often they've survived collection attempts. During collections, the generations(s) being collected +are further partitioned into, e.g., sets of reachable and unreachable objects. Doubly linked lists +support moving an object from one partition to another, adding a new object, removing an object +entirely (objects tracked by GC are most often reclaimed by the refcounting system when GC +isn't running at all!), and merging partitions, all with a small constant number of pointer updates. +With care, they also support iterating over a partition while objects are being added to - and +removed from - it, which is frequently required while GC is running. + +Specific APIs are offered to allocate, deallocate, initialize, track, and untrack +objects with GC support. These APIs can be found in the `Garbage Collector C API +documentation `_. + +Apart from this object structure, the type object for objects supporting garbage +collection must include the ``Py_TPFLAGS_HAVE_GC`` in its ``tp_flags`` slot and +provide an implementation of the ``tp_traverse`` handler. Unless it can be proven +that the objects cannot form reference cycles with only objects of its type or unless +the type is immutable, a ``tp_clear`` implementation must also be provided. + + +Identifiying reference cycles +---------------------------------------------- + +The algorithm that CPython uses to detect those reference cycles is +implemented in the ``gc`` module. The garbage collector **only focuses** +on cleaning container objects (i.e. objects that can contain a reference +to one or more objects). These can be arrays, dictionaries, lists, custom +class instances, classes in extension modules, etc. One could think that +cycles are uncommon but the truth is that many internal references needed by +the interpreter create cycles everywhere. Some notable examples: + + * Exceptions contain traceback objects that contain a list of frames that + contain the exception itself. + * Module-level functions reference the module's dict (which is needed to resolve globals), + which in turn contains entries for the module-level functions. + * Instances have references to their class which itself references its module, and the module + contains references to everything that is inside (and maybe other modules) + and this can lead back to the original instance. + * When representing data structures like graphs, it is very typical for them to + have internal links to themselves. + +To correctly dispose of these objects once they become unreachable, they need to be +identified first. Inside the function that identifies cycles, two double-linked +lists are maintained: one list contains all objects to be scanned, and the other will +contain all objects "tentatively" unreachable. + +To understand how the algorithm works, Let’s take the case of a circular linked list +which has one link referenced by a variable A, and one self-referencing object which +is completely unreachable + +.. code-block:: python + + >>> import gc + + >>> class Link: + ... def __init__(self, next_link=None): + ... self.next_link = next_link + + >>> link_3 = Link() + >>> link_2 = Link(link3) + >>> link_1 = Link(link2) + >>> link_3.next_link = link_1 + + >>> link_4 = Link() + >>> link_4.next_link = link_4 + + >>> del link_4 + >>> gc.collect() + 2 + +When the GC starts, it has all the container objects it wants to scan +on the first linked list. The objective is to move all the unreachable +objects. Since most objects turn out to be reachable, it is much more +efficient to move the unreachable as this involves fewer pointer updates. + +Every object that supports garbage collection will have an extra reference +count field initialized to the reference count (``gc_ref`` in the figures) +of that object when the algorithm starts. This is because the algorithm needs +to modify the reference count to do the computations and in this way the +interpreter will not modify the real reference count field. + +.. figure:: images/python-cyclic-gc-1-new-page.png + +The GC then iterates over all containers in the first list and decrements by one the +``gc_ref`` field of any other object that container is referencing. Doing +this makes use of the ``tp_traverse`` slot in the container class (implemented +using the C API or inherited by a superclass) to know what objects are referenced by +each container. After all the objects have been scanned, only the objects that have +references from outside the “objects to scan” list will have ``gc_refs > 0``. + +.. figure:: images/python-cyclic-gc-2-new-page.png + +Notice that having ``gc_refs == 0`` does not imply that the object is unreachable. +This is because another object that is reachable from the outside (``gc_refs > 0``) +can still have references to it. For instance, the ``link_2`` object in our example +ended having ``gc_refs == 0`` but is referenced still by the ``link_1`` object that +is reachable from the outside. To obtain the set of objects that are really +unreachable, the garbage collector scans again the container objects using the +``tp_traverse`` slot with a different traverse function that marks objects with +``gc_refs == 0`` as "tentatively unreachable" and then moves them to the +tentatively unreachable list. The following image depicts the state of the lists in a +moment when the GC processed the ``link 3`` and ``link 4`` objects but has not +processed ``link 1`` and ``link 2`` yet. + +.. figure:: images/python-cyclic-gc-3-new-page.png + +Then the GC scans the next ``link 1`` object. Because its has ``gc_refs == 1`` +the gc does not do anything special because it knows it has to be reachable (and is +already in what will become the reachable list): + +.. figure:: images/python-cyclic-gc-4-new-page.png + +When the GC encounters an object which is reachable (``gc_refs > 0``), it traverses +its references using the ``tp_traverse`` slot to find all the objects that are +reachable from it, moving them to the end of the list of reachable objects (where +they started originally) and setting its ``gc_refs`` field to 1. This is what happens +to ``link 2`` and ``link 3`` below as they are reachable from ``link 1``. From the +state in the previous image and after examining the objects referred to by ``link1`` +the GC knows that ``link 3`` is reachable after all, so it is moved back to the +original list and its ``gc_refs`` field is set to one so if the GC visits it again, it +does know that is reachable. To avoid visiting a object twice, the GC marks all +objects that are already visited once (by unsetting the ``PREV_MASK_COLLECTING`` flag) +so if an object that has already been processed is referred by some other object, the +GC does not process it twice. + +.. figure:: images/python-cyclic-gc-5-new-page.png + +Notice that once a object that was marked as "tentatively unreachable" and later is +moved back to the reachable list, it will be visited again by the garbage collector +as now all the references that that objects has need to be processed as well. This +process is really a breadth first search over the object graph. Once all the objects +are scanned, the GC knows that all container objects in the tentatively unreachable +list are really unreachable and can thus be garbage collected. + +Pragmatically, it's important to note that no recursion is required by any of this, +and neither does it in any other way require additional memory proportional to the +number of objects, number of pointers, or the lengths of pointer chains. Apart from +``O(1)`` storage for internal C needs, the objects themselves contain all the storage +the GC algorithms require. + +Why moving unreachable objects is better +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It sounds logical to move the unreachable objects under the premise that most objects +are usually reachable, until you think about it: the reason it pays isn't actually +obvious. + +Suppose we create objects A, B, C in that order. They appear in the young generation +in the same order. If B points to A, and C to B, and C is reachable from outside, +then the adjusted refcounts after the first step of the algorithm runs will be 0, 0, +and 1 respectively because the only reachable object from the outside is C. + +When the next step of the algorithm finds A, A is moved to the unreachable list. The +same for B when it's first encountered. Then C is traversed, B is moved *back* to +the reachable list. B is eventually traversed, and then A is moved back to the reachable +list. + +So instead of not moving at all, the reachable objects B and A are each moved twice. +Why is this a win? A straightforward algorithm to move the reachable objects instead +would move A, B, and C once each. The key is that this dance leaves the objects in +order C, B, A - it's reversed from the original order. On all *subsequent* scans, +none of them will move. Since most objects aren't in cycles, this can save an +unbounded number of moves across an unbounded number of later collections. The only +time the cost can be higher is the first time the chain is scanned. + +Destroying unreachable objects +------------------------------ + +Once the GC knows the list of unreachable objects, a very delicate process starts +with the objective of completely destroying these objects. Roughly, the process +follows these steps in order: + +1. Handle and clean weak references (if any). If an object that is in the unreachable + set is going to be destroyed and has weak references with callbacks, these + callbacks need to be honored. This process is **very** delicate as any error can + cause objects that will be in an inconsistent state to be resurrected or reached + by some python functions invoked from the callbacks. To avoid these weak references + that also are part of the unreachable set (the object and its weak reference + are in a cycles that are unreachable) then the weak reference needs to be cleaned + immediately and the callback must not be executed so it does not trigger later + when the ``tp_clear`` slot is called, causing havoc. This is fine because both + the object and the weakref are going away, so it's legitimate to pretend the + weak reference is going away first so the callback is never executed. + +2. If an object has legacy finalizers (``tp_del`` slot) move them to the + ``gc.garbage`` list. +3. Call the finalizers (``tp_finalize`` slot) and mark the objects as already + finalized to avoid calling them twice if they resurrect of if other finalizers + have removed the object first. +4. Deal with resurrected objects. If some objects have been resurrected the GC + finds the new subset of objects that are still unreachable by running the cycle + detection algorithm again and continues with them. +5. Call the ``tp_clear`` slot of every object so all internal links are broken and + the reference counts fall to 0, triggering the destruction of all unreachable + objects. + +Optimization: generations +------------------------- + +In order to limit the time each garbage collection takes, the GC is uses a popular +optimization: generations. The main idea behind this concept is the assumption that +most objects have a very short lifespan and can thus be collected shortly after their +creation. This has proven to be very close to the reality of many Python programs as +many temporarily objects are created and destroyed very fast. The older an object is +the less likely it is to become unreachable. + +To take advantage of this fact, all container objects are segregated across +three spaces/generations. Every new +object starts in the first generation (generation 0). The previous algorithm is +executed only over the objects of a particular generation and if an object +survives a collection of its generation it will be moved to the next one +(generation 1), where it will be surveyed for collection less often. If +the same object survives another GC round in this new generation (generation 1) +it will be moved to the last generation (generation 2) where it will be +surveyed the least often. + +Generations are collected when the number of objects that they contain reach some +predefined threshold which is unique for each generation and is lower than the older +generations are. These thresholds can be examined using the ``gc.get_threshold`` +function: + +.. code-block:: python + + >>> import gc + >>> gc.get_threshold() + (700, 10, 10) + + +The content of these generations can be examined using the +``gc.get_objects(generation=NUM)`` function and collections can be triggered +specifically in a generation by calling ``gc.collect(generation=NUM)``. + +.. code-block:: python + + >>> import gc + >>> class MyObj: + ... pass + ... + + # Move everything to the last generation so its easier to inspect + # the younger generations. + + >>> gc.collect() + 0 + + # Create a reference cycle + + >>> x = MyObj() + >>> x.self = x + + # Initially the object is in the younguest generation. + + >>> gc.get_objects(generation=0) + [..., <__main__.MyObj object at 0x7fbcc12a3400>, ...] + + # After a collection of the younguest generation the object + # moves to the next generation. + + >>> gc.collect(generation=0) + 0 + >>> gc.get_objects(generation=0) + [] + >>> gc.get_objects(generation=1) + [..., <__main__.MyObj object at 0x7fbcc12a3400>, ...] + + + +Collecting the oldest generation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In addition to the various configurable thresholds, the GC only triggers a full +collection of the oldest generation if the ratio ``long_lived_pending / long_lived_total`` +is above a given value (hardwired to 25%). The reason is that, while "non-full" +collections (i.e., collections of the young and middle generations) will always +examine roughly the same number of objects (determined by the aforementioned +thresholds) the cost of a full collection is proportional to the total +number of long-lived objects, which is virtually unbounded. Indeed, it has +been remarked that doing a full collection every of object +creations entails a dramatic performance degradation in workloads which consist +of creating and storing lots of long-lived objects (e.g. building a large list +of GC-tracked objects would show quadratic performance, instead of linear as +expected). Using the above ratio, instead, yields amortized linear performance +in the total number of objects (the effect of which can be summarized thusly: +"each full garbage collection is more and more costly as the number of objects +grows, but we do fewer and fewer of them"). + +Optimization: reusing fields to save memory +------------------------------------------- + +In order to save memory, the two linked list pointers in every object with GC +support are reused for several purposes. This is a common optimization known +as "fat pointers" or "tagged pointers": pointers that carry additional data, +"folded" into the pointer, meaning stored inline in the data representing the +address, taking advantage of certain properties of memory addressing. This is +possible as most architectures align certain types of data +to the size of the data, often a word or multiple thereof. This discrepancy +leaves a few of the least significant bits of the pointer unused, which can be +used for tags or to keep other information – most often as a bit field (each +bit a separate tag) – as long as code that uses the pointer masks out these +bits before accessing memory. E.g., on a 32-bit architecture (for both +addresses and word size), a word is 32 bits = 4 bytes, so word-aligned +addresses are always a multiple of 4, hence end in ``00``, leaving the last 2 bits +available; while on a 64-bit architecture, a word is 64 bits word = 8 bytes, so +word-aligned addresses end in ``000``, leaving the last 3 bits available. + +The CPython GC makes use of two fat pointers that corresponds to the extra fields +of ``PyGC_Head`` discussed in the `Memory layout and object structure`_ section: + + .. warning:: + + Because the presence of extra information, "tagged" or "fat" pointers cannot be + dereferenced directly and the extra information must be stripped off before + obtaining the real memory address. Special care needs to be taken with + functions that directly manipulate the linked lists, as these functions + normally asume the pointers inside the lists are in a consistent state. + + +* The ``_gc_prev``` field is normally used as the "previous" pointer to maintain the + doubly linked list but its lowest two bits are used to keep the flags + ``PREV_MASK_COLLECTING`` and ``_PyGC_PREV_MASK_FINALIZED``. Between collections, + the only flag that can be present is ``_PyGC_PREV_MASK_FINALIZED`` that indicates + if an object has been already finalized. During collections ``_gc_prev`` is + temporarily used for storing a copy of the reference count (``gc_refs``), in + addition to two flags, and the GC linked list becomes a singly linked list until + ``_gc_prev`` is restored. + +* The ``_gc_next`` field is used as the "next" pointer to maintain the doubly linked + list but during collection its lowest bit is used to keep the + ``NEXT_MASK_UNREACHABLE`` flag that indicates if an object is tentatively + unreachable during the cycle detection algorithm. This is a drawback to using only + doubly linked lists to implement partitions: while most needed operations are + constant-time, there is no efficient way to determine which partition an object is + currently in. Instead, when that's needed, ad hoc tricks (like the + ``NEXT_MASK_UNREACHABLE`` flag) are employed. + +Optimization: delay tracking containers +--------------------------------------- + +Certain types of containers cannot participate in a reference cycle, and so do +not need to be tracked by the garbage collector. Untracking these objects +reduces the cost of garbage collections. However, determining which objects may +be untracked is not free, and the costs must be weighed against the benefits +for garbage collection. There are two possible strategies for when to untrack +a container: + +1. When the container is created. +2. When the container is examined by the garbage collector. + +As a general rule, instances of atomic types aren't tracked and instances of +non-atomic types (containers, user-defined objects...) are. However, some +type-specific optimizations can be present in order to suppress the garbage +collector footprint of simple instances. Some examples of native types that +benefit from delayed tracking: + +* Tuples containing only immutable objects (integers, strings etc, + and recursively, tuples of immutable objects) do not need to be tracked. The + interpreter creates a large number of tuples, many of which will not survive + until garbage collection. It is therefore not worthwhile to untrack eligible + tuples at creation time. Instead, all tuples except the empty tuple are tracked + when created. During garbage collection it is determined whether any surviving + tuples can be untracked. A tuple can be untracked if all of its contents are + already not tracked. Tuples are examined for untracking in all garbage collection + cycles. It may take more than one cycle to untrack a tuple. + +* Dictionaries containing only immutable objects also do not need to be tracked. + Dictionaries are untracked when created. If a tracked item is inserted into a + dictionary (either as a key or value), the dictionary becomes tracked. During a + full garbage collection (all generations), the collector will untrack any dictionaries + whose contents are not tracked. + +The garbage collector module provides the python function is_tracked(obj), which returns +the current tracking status of the object. Subsequent garbage collections may change the +tracking status of the object. + +.. code-block:: python + + >>> gc.is_tracked(0) + False + >>> gc.is_tracked("a") + False + >>> gc.is_tracked([]) + True + >>> gc.is_tracked({}) + False + >>> gc.is_tracked({"a": 1}) + False + >>> gc.is_tracked({"a": []}) + True diff --git a/images/python-cyclic-gc-1-new-page.png b/images/python-cyclic-gc-1-new-page.png new file mode 100644 index 0000000000000000000000000000000000000000..2ddac50f4b5575888d8d19cd9b6f2e3863132776 GIT binary patch literal 4415 zcmZ`-cUV)~vfn9?Pz*(Sks?SF4DHY{v_R-ZsUjdq5iyj2R83TRQ>ubMBya=-k*@S2 z3Mdj0L~1}mA_z$D@Zve|-FwgdzW2whS$p=LHM3{dH*0Xq9R2J5_h(<=`5s@q#(=Bn-Jf z-S2tM!qOTyM=GQ_F{MtvGeD=0kzPv=j*Bz!5MIOG|JTF)DzhYW(esjJ z8f05Ovk=8t?*$Q51LL3qw;pW~)XD##-`3zt|49Z~^5l;x(40MO)Z%!tAv7mko!A0n zmG#6YS#c$AIH392w1k_$~Q!)rSP0RDWQ!sH=4h`Nb>*26BS5k7PBYnJF~ zbT4h4YeC+M?a%Iny|4wBj~)hRJUqkjTZ&4xuGN!2BhK(*%O4{t+91coptKo@;$BTd z8tby(2ExM#R#22;W_ASN3no4_?B;Et>+u|1uR`eH#BUM!{+2a9O!2SDGaL z+x{Le<&h{En)7esAe?j22{GF+P z8UCG4tAF_PcUmjyag17G>?K%Ygo#Mdu;kiXRH~!owO1KUka-VjL$^T~iu-|^u@FXV z*t)x+@Ra1!*_1hMv}rjUt#m~jP)}G?nl@feRtmFl^L;= z$w$i9gtOE2uM7y16@5beh=!|xal#WO^1>CK3c3wRQPDQmnCt>Q7sNH$b#l;) zb)+(&o){N*nz=H6uIKq_vQ?(XgJ1~pri#cb=ka%tsO0yaD6c9ES|SKSB*m4xoHb>m z$_9BqP2~1b(B(d}1jpSZTH$@p%HrR&7YP!UPvAn@1#CgiS#>w|7F~|cuC?9XU99#x z$Ghr;dli0#a>^~k9QBm<6${xGj~lH05#tcZbhGmK?PS*qcRgf-H#pL+UL?|%V(#{G z2t*ZTug-}8@;so^DG_%kkv4?~h09NmYa(`53@k#7>4#RGi|03&*kkMp#~^v#%ugi8 zVN|f|WyD79+FkpDyJhG7UTz1r=5@BeD6gcRA-165an3iQxUae=c-m{u@q88<9TY1d zVULl6)!4o>M8jSY;fHcTREPMUfU3z8isR+49X@S`b$x;2*nBH&sw~D3yFabBtx!&T z+E|`(IYttvd@BxUOacm%qfsZRSw2l^cFZ~a`1)#@SN?VQEc|w^)J*8LlYu5Lv^VlP zD=zu+Q#4av36X_j0`^}w(C@vtH|w!vEkl%u@A=ABGz3vElJN@3$n|>t$FF`S#v{ia z(KvSA2ym1#Gm*+KapVLNWJVh6T%OAQw)M!2x74+-Nzuug{(c4)v7ziD;r_=Ur9o2p zZ93zl?Pq4T;F-FG znp{`A*bDVz|?V`}Lei!y3+2TSH}ph&!A zqox-2Aik@nulVg@6stEGr=~R$?}brw=3a8WNqbPFO#abgyQu-3IQMDXl5LMrx2?>w z)?K8J3+ExnBJ%-ipr2LO2uOYI@cit($04`Xa2K(v*0-i}Pt7t3V{@BE7$I|S@&<2& zd3&{qlKEddrbr;%za?J$9saHoq;8Wr-4UulLT9nhTbY z9SN)Jx0p)gW9q~zR1dP;uG321FcX!EgNsl#2wYVGTmi;w*G;8S@_jPOULQ%ByN90T zt9Rem1pENF0O;*+?-P1v^bW21RVGh8NssFpsG8X7E>Aj{EEzI%ZF}l-r}W7r-L!yw z%9fH7%f{C(EZ-YX;q zyHp$UHq+e_sIbgZM)L_y^y8AW%r(cWs&mP= zVDI9!p>wm5%dMV>s5WdZ`n@cLWuBbf>QJh4_KdtaFUl7?q=9jqn8{tN$VwNCSn8Ln zltHvz0N@b_Ua7XLt zzuuy7V$k}qZN^HneP)UTI&xm_ht zoVnB^x#18saQk(B+R-HN+#ek^h&34t(t)<>88N$x}nsZE#MGvQXeL9`VO$Vv~|fgxZUxG(+O(b+i^Jc6irFph_OD& zd5tV7qf*%xa&Q@f(S5)cdK_qf5sA_H*fm-&^kEp}TZ~mE+U?bY8x46b;`4`*QnFEw zgo?@Lqp?+|#0Ke!Pi@Ot(`BCypCHD?uhaYHVYUA6^WbU$H4*;p$GIae8b=*K&ArG* zzuLN{vNw|*EfNDmmePc}z6MdUOq3I7@^q51Pg9KnWuP|fuI`fmx^2fv>X|_XHTKLlue~_{KBZ$4PXp$#_#s+x*UPa zwMM%4Ar|Wf!v}}16|3E($eVS;^&qlVXGwajZC^~edaw{D+i3G!azPsw71~7cTNguU-QSiIGhDAhI#41P8wN=Z;8Jr4N3QG@%5e zw&Ao_myO|hIm%$EP}6Zdnq#vbKKmlFw;r|_9Vx^~+XpqF^JuLQ&{(2<$>%zO6NMT{P!G%_z@~aqz z3x2?YHLXaGd`21UD+LSO~EC;)9T#smRQaXX?+8EXbl_n_`chHawo* zh`N#(Wmj6bLNPy8zn9OV?$`Otm}3mGgnKGwXcTtr>t8!<{yDh&!1A~;V(01A0b6ZS z%8L5%dPPL@JPl8E6>p&odBe@s0;PdMm+Q7y>9pQ#l!xNmCyxQ$zj_0}aj;22_vQ!)t#z_Gs7bqTm{2b&`T*6Y zNj@A55xJ+s;$|r~0#kRv9SD;v;Tw^yLI!$7$|sdlZE_QTej6*58y4Ocn~ZSoGh93r zxI~>_MtOq!+0=_w1qQU8o=a`$(Hi2Z%9yCq*MfwjS@FiVqN7E$2FY>(vb2pl79&H$ zRp^$04WfI8rg1q-ZEN*_5aE^^G0%quH=3@ggc+|Z|0+&)VWzSdcZrU^T+!6V+<@Y4 ziRaf)y;OshyGpX>?w`s_cZ6uQ+R7Tf9P4S>FMY;~rXy}EUzBnDu7vDU$vR2C;%eK literal 0 HcmV?d00001 diff --git a/images/python-cyclic-gc-2-new-page.png b/images/python-cyclic-gc-2-new-page.png new file mode 100644 index 0000000000000000000000000000000000000000..159aeeb05024a3247381e1c3aec2b986035b5e44 GIT binary patch literal 4337 zcmai2c|26z|GzV1UlU^N*_SX`LzplkQPwOGiO9~_w~?}rl$3;wY$aRPjEpP|m2?rJ zmR-!@|}8q%k%v4d;R`+pL1U4-uLI6d*1h+^LgL&^H!%h*@V~t0N^w?GqC{x zC`fM)v(O8`)bGhCG@A?yWf?=9IGv3e#VmmljFh{G#RiPKn)u5jej&5;;o z8)=E~t&T2Aeq=+&9f}n5;8Aw~cqzZYcC67;rB*^zaJuDiizk4>< zkypd{HI5{l{iu!+E4ww&`yu@rH~L6v^a0C;#QkF4Q8Ax;&#ka-P(%4D9neZqraj+Q zW0Mc($GG5_SwfG4WBmty56JNFpLpyhNPcn$X?uy;M@}@U!ax)}3Z!MI`Sv4OtRaBY zOKGTOycVtl^4GCKM=8d^Gzh@KG{)<**a#`QVs!s_I|OBVLg&h$_bYccI@^USK;QI@ zd{wgimy)Z7W=4co$(u;qI_4%JwEkb%D*3{KgJxcywn!H<+|zNG?mKzNDcLe!&^$%3 zbh=Gh8vM8f$r==tNaG%SfCeTlAm_1i;K!>l-8AMisw(HkL@@@pmpSR<2!nOo1qx70 z+&{qM^LrWpi3}vK&;R20O0#ItLBWci>v~K$1fHzCy<>yikak+MfMlIAIU;)cV0?G3 zGt+ILk?>Rwl2sis&_T?*$^N%$e`5WP_upD2$%1@;@LN!8 zhKD=e$kgd(u-nvjv3b;Ra-v<;yJVMx>~rI>Lw_%G@U(d2njA8A!S?RSB!#}b^UJRd zf3|QVVi~X-D9gN{&PI`YNZWBW@4cPOY(y*vc0X#E$`9_Wh1x+aUHzim88=Qp&}Sl2 zsHhN5G=1r+%8V%x+}yN#n$&;ILl*qE)_-*8CC@|kpH^L?Gx7j@@#hD*bnzkNP^t5D zrKjPOR`iJ*Ay4PAXxQ_Ut*4u9$6?M@-@vJmd`QUlH1gbR(e^NlHgt+iK;Qd}fq z9}=u|!6ri)!vvINzVwvCZEWfy7Gt4Pk&OLTA3l^+Go!@{!%P(iz`jB$p43%N^n>FA z0p31Wb)V-AtuRsw!*(S&Ng`x_Ql6gTSc&IhlM67i&OF(?C$cV7;%f>r`!>`?>f3`b zE-`KkT+nSNPeVD1J{?tNo-}`|-Vmup+Y2PrbjxRUe|lgn>{p|& zdgvus6?CDdb7E-AtLGC7dU%jaxk}`<9G_<3Bq#X+4=JZXq#$YYTihij><63F6`3xJ z$Tq`YvN%)-K0-hC2ieB5KHLA+_WO=OL*#H1yA+gUvcE|XQvEl7bHk2Wz#AGf$lz~X(ebpht!FA-G%PP zXCf`aP1-Z;K-S5tVb&c{S?^!z?r^DH3@9BTY72*W&Of>rD$?Vqb~mnE7Y)PlTw%bi zYKc7K80#80cq7o(N}R8R#B*s{t`>R>~q{WFM91BD}6(_0CSe9 zQY!U@qJi_r==;?s#;T%t05$IUI=AQ6aY}5Nh}UrV=LFXuK^yXp+}rmnx81a-S7GP^ zgqDL&sL4v)-3HgiExZE9?8+_wlW>7ZSBVJAxI4AK6qI({mfPW2I{^UCn0ytq6HI#AuHG>9V2f&L`;r@dQn?sE zpXpv^A)kEDU{Gp7q)<4^IxS7F0U_gRF`V(P4o>vO$t`@Yc03h4TiEXWv7X6ih_9Tz zNcx2_Z*qZde$IB;;dAb7=3durk>_Yxo#eXRr95HM+4cDuvH3cO}C9oW9>YXC# zlHO`24BX0H2FbT_+41q4NYjWFUs~9}tGafYN`A@wh(-!}Tp$?|dMwaQZp}0&KVGF; z3=%W*?E9dEd9opOC^T^YgH!u}UOn+yO>zQTQ`)Z1ewa4t zH8Qh5D5h^~+n5hSUT9A+#N}N~7)8{##WyVpm5XqHNg}`|#ShO!oArG8xT1_1{Gj@R z^S<`TX3)N)9&H+kNnTw`tYOZ$Avry@O!p@?O64_ie#YxpdJV)(Thh8M9!N{R zV_33YhC8ZART#JVcFW?#DN`Jja#h*%^1CNLVI^cNkU8o(DWTW)oln~w2zkV{G)zU{ zGI^CMUdn`Be*O5wLN7nZuu~+|;@oJ;zH8+=vqE3z9;L1*ExpI9}&va*iwfp#1wAGRCS@cn6sruI%E+egyACipOqeFPO&jPVT)Wk{Our zU-~AT_fmgT~-chP>VgCCMMsy~E3Yp9#qXJ&cnyM4) z{yYkENC_`0h-jr^dW6WBX!boNI_7+{L%2Y_1I<|{(OQpmAjne|-gqqm^6U zN8qP3HeHJOOy?Wr-}?Lr*5R4&a=e5Kb*4si>|Zvf`DdwKUr>Mamy2DX!O(n_59y}a1%DtW@5GEg!`_iv3d{*&|RO_WoiAM3A(Rjn~R#&T3 zed;2t2z3-A%Bqo6%hdERe9PCxn>>9);S?MF+!)1J5zifewCZ~?M9-Cn2TnhJ&>e96 z!l3!x6s#t4UKdu-JG=$MDc!td$mLlqqB~f{g+6n_v8dR7`}xH7t8(Q+1!#OC8@d@` zN8GW$1sfhsidbpt@@fUE)3({}qCz9RGgiOHGNQ{74FU?INh&SYZ$q?>Tvu|<;(?4EBJIyTKq;GWM4jJ}#1Lo-yg*&Xha^Jsk} zfsRiqQ4;lF4=ibuB)zj+%?0wke>0yp1?z@-=j{YB|EVMiN3Y*Bg^;*{#~M zkQBDPRjUD;q{3O09OCPSf&|_LvQQ!!V;Y8a=AAXYx&-IKfH|-mX}qZyd0^_T7)r7z8UaBAyYKJ-@ss0Wt`kSe~+jlY17oB%%{>_Vx!- z2O~jt-yO#?+W6k?vUw~6=#1mU6U`?)Wki`RiF-+pI=u+?qk&FJ-kT+#)vlvO?V^e}-zAd-rILnC+_S|gFq8e+tJ&n|HZU>X)^NSG6c*b}kTvC~@ zxOxV%6AHHH(_PfR%<8}W?B?wXx`o+gAnO&SGd#rK@4Wc2$xPgXD(E>AAhq$V=vbBl z!$W6vO@sO?e@P99c~-}|P=Yt9r)(WWo1%pV$T7c literal 0 HcmV?d00001 diff --git a/images/python-cyclic-gc-3-new-page.png b/images/python-cyclic-gc-3-new-page.png new file mode 100644 index 0000000000000000000000000000000000000000..29fab0498e5b106f813e08aa2799602269cd74d1 GIT binary patch literal 4876 zcma)AcT`i$w?1hgARUw{9f{J5KxmO50qIqw$(14qh(JsbDTxXJq)U?;L_n&b6afQr z0YPa&q)JgiiPEkRq)6li@9)05)_d!%_5L{Lo3-cLduH~WS!ebaZ*7TYXBA=v0D#@p z#K;x^z!XON?GeTcF!q0ZnE@=I1^lB3#Xj=55SyDZAlKN{sm-} zJ!`}HQU+LYVz}2x);DtOQ-JVC`l|Pn?_Z53u?fOgU7U=cA}lNEgk!9c(^U93WCfNv zx5BQ@1(V>Y2c^0vdAZ%QVT=KDCOE<=jBo@;lEdFo`Ft5eAy5C)cWR6mQXmbdR6c>I z-?t+no<$G6!9ftS7>WN=Nm21(6Aj*m6v zG2hDuqamMe-%5tDQD8b8dg`&H6ie9asgw+x+5!_BlEW}_gTQl;hm;}s%kGopIO?=# zF@vj5VuK~bsY{3vhSbY47T^5G_%RVAx!d<@5$0>fkJNDgJ$jn1HS;Ici|%-LlOM*v zEyFvn|JUOFULzy9Bz|8;a*bH>u~!>IB84v;#f9V zNQ%3rHTl<*-On9WbJe)WsuNK4gh_|ykptC^{lKxjz{d#lN+Lk+sa14+%!tlN>;K5^ z;nw{-kAJpx{wVd-3!z6;5{PvYSHck}5l`TR43srigzkDc_A5_*TL^<{I$?Oy`Zcr&|kzYj#A( zNYIw8U}&K&?%gcy@XecV)vS0>1ox_js6DRptTcsb*YtCYN%1pw{{I8*!c-w9pD#8H zZU>Z-R?SFdv?z&q7R3SEV*}~cgJ(h1l|Fvh9te3{#}ZyTGq=8WUF4$}kqR zh=Ll1VI$~A(QqJyoiUcrzmk6tg{p#$F+T=e;)eff_YqI;$IA7vaTK85e-adGoA);q^dHQ|Y~0^jk8QZk|`$x&Eo zg;gSU-CnNQsgP?f5(*4XyO^(+IBw&Xcy7A_MJ7DIdKyk?0|t-Q%|5NSb#zpu(|q_f?BK_{umQ}r1r-xiD$by9ao*7+s?1p9L5{XI8wJk zOP(zk%r;AGf==55*XTzT$mwePpJkTOQ{)@5P5DQ5y|{`->u+f+*v5}!q3h&TMwvR) z#`HxMv#+ExIQ+g6XmjBv53%vig52PWZr^_5pYpnQpZ2V$+m6n zVCrIA-#H9c9(XxjfJ(N?m5|j3Q5;kKz};tlv_Y8IiUW3%?&;xw`Qm+gzfr9qDh>CaRVzuCj?JGXjg)+ zQE?gCedj;DHk)Rr-kD7^PHS7ypFq%v9LG>1A_k`xwbVTaGqyDF#*gqQg6tG0wWpT+ zel1cbrUHwVf8&v)#0lA>Br37dME9IqL>|Ls{4`P9%O7~vQG}={QV$pR)I;PtJBA~j zpZW!vC-MAxE3_854r-8*yxQFQ2&ps*&GfRa29wn&ycWo(X9?GIQKP{5&oGgnJ@^y4 zfKe4vlPQ-aqLp6E@S^4Sd=w6doG@+5q*qq3P&6lNZdHdEAzfcWL~a#^%`;I82YoME zX3xmYT)qM4xwLv6F}vjzGAK86+o;QH(wGcze#ZR*&VYTW=WT*Wm z*`k=Y-13EYW$S>D{f(Xbmts4jb{EmPzg#2eKRaBe^qBW}+bptH$M1^7Q;9|67%twV z5o>ni4J10fubgm7uK300>Hwy2a?wxq5<$w>RFv8r0tG37sK%6b@p`|r?*^0?xT}w* z;;kl4%3QyW%{(_ifuaI^a7BXQai3Hu7&+#ha}&*Jomd6uAW0LzX0b@ag32gf+Aam`T{eZkZZ zK_R7gh{t&V>E8K0Eb__s&?27(*UwH{{b4M=9ne2RtrqhA*A#%ICCvzqEX~o~lb0N& z+KK?Wbn9eiS9A^YsU9PixDNma6xU-ec^`^z&VVz1^x1-{li%RKU4eP8-H z?~rbODeZoA-Qx3U<%mA}cCL{9eyf83uM6Xofn2BymRjV*ueN>PyL$aatIe}?pK4?o zzf4=e*-43IAK!@}r#^AioBgn)I2a~GT4J7-w1a)(;hs<)jFtydQ5LtQH}m|@6USm( zILul(Xelj*Gf$FM$~3Spx_q|AD)tuudikP;sL_K_$x+ueY(gBj%8;8PTrBJOr<*at z?ao`6dqua@OZ@6zZCoq^gth$5}!nij!AJ@`(Hq0S#r z$Ic#l9XD9Vm}D7hpWdi_ZEAr}*5XqOgZH*agrrwoPWOhjs!-+81RK(eso{s4_GS;#ngx?sTjvB1d}_M zpnJf?>gBqgi&55BzMTCQg&ba9X?C;}LVGB(_DAhT`}myjEa>M`**rqpuob*2G}YT` zFRr7I62VF7?@j6x$vQy$_&fj-LvgZY(m2M*JG@{9mVjMx4PznqWshGgr&~jnDNv6``*I|Iul@n~?MxN9aV^=i(&%0_wtzmuYk@qT`;WTj;JF}j&zcd? zy{d`QNi~<9VW)Y;N>SqY=0F90Je5FRPopTok}nzAC8qWW1kJ5FrqtXsP8_5l&wFd@ zg)&nN`3ca)w3PTjbHNxxi_n_w&5^4B?O2=PPO9Q4udl?Bz&2nblw?PO4x8}VvKoEn z3h{dB!p9mU4ifJl2tWWyo%T)y>+`SgN3v-HPAMjWU)>6HdI$Ca(o+BOZaeW$0>Lod zikCVGA{ceIS|Wq$PctDWIeq!WlbUN$9n3-jRez=TF7kYlb9_X~cOa9FJM=`Q#+KZ3J$p-~2#B_2$*`y`1PX&zkmIi}m3f*%3A`Fr@6Ot?Hy51)D z#)zl_X72l|$P}g=6A$pW?|mQgZF2~Thm*~N7tH$29jAK5WvS|GYMrs})ctM?z8fnO zYuRs;e{sAtBmH|Qy$X>Y#7^@7c_o-Y@iq5RD_O}4PJQJWlIIf3fSBy=gI<8VwXruG zfc``^QO@~5g~|&gN5y+DH9EvOoqh|YT3{8((RK$nd>69SlA5)e(gl00a|yLs9Go$Q z8y}a@S+4q4{8_GbUU^FIExhiWw7Wu5BkS=wyKqXNGj>fo5niY}0zxOWP#64j*k@Z% zd6UO^-g&HsA-J!AmoLG z>sl(Gvt8hF!0Y?MW}B&y`<92S&obG&NeIW8G(aJXOjezuXW`(E(Olo_b(d zF$;pZjt-W#q&!IADK;bCeHX~!nMWD64y0`uVwCzK=P|OmuvqxgiwfkE2sshViRpSJxM!2BN33vB1Ghrxmfy7kFsq{_ZVu*~ z*}I?DtL6eOtPrak+Ds=_=W_c0Xc~vltxR?_e*J*X`4MEi?8;$^RIH-6O9VWOyD(8B zK@4DSoq99iE;d#vXn|v8e>TPSEpQ~r=vNz9FZJS(DV%Z(yc*bZ@lB*!#}&!F?`{P5 z%V=2m6_R(m&`s*c?X)<#XgiXDFbvjeKpC?vm30ZQ_h~QbUubCoLb^=80+QJ#Mt5DJsz~qyZn7m+C#ZWkHH&RY`Hg2ikKXapFy2~KYn6zhDBSgSmsi22X& zG1HeqSiPTROyGM{Y`T2Z6N^Ke1@``O^)hY5y;)ZDd{-x!bU|BW%lY7dHT;chvQpWX zWy(2vEi#&gG!j&9&8R)azzutD|JJKxd548NL%2AEs0;sAvkv~Gg1yMbPsd0lh#$wM zbO>MWjj6s{zcO^*e?E7seEpSJ=SO|gLJy~PN5=$4@U30Ty}znFqaPB=e&S;#)XIwD z=^lbK1_hOXduZ1m6yPup(-*PFjO9{;S{?(mC5IWarm}j=>y)|Xr zBRmxB_gC4T?QcQhw_5X$ymF`tRQxRt90EgZAV2>l*7sDPT7U=so}n|zV9q&Vy#By= zBRoUA833p%tEej|YbvNJ+p8)gG?Wo4XHF|CBb1ei4-yVltA8B=gS`BFZvNj6Ke|;L Q7!H8xIZLBjLyz141yIB0(EtDd literal 0 HcmV?d00001 diff --git a/images/python-cyclic-gc-4-new-page.png b/images/python-cyclic-gc-4-new-page.png new file mode 100644 index 0000000000000000000000000000000000000000..51a2b1065ea64eaf009a96ce6fecd3132424ea0f GIT binary patch literal 4863 zcma)AXIN8Nw?0WE)W8S|NY{Xl^r8ZRK$LC)DN+R-YA{j_C}1ENg@{5B5k^!X6k+HJ z0Y+jdA{`}y6oCY000}}2NNACA<9Fw4KkoC~ALqRL+2^di)_T|8d+oJTF526`MC3#O z0D#%qS~>v$n9gqx3h`fnHRg9eJ_twHdR*tD?LQvt9*h~k0rop*V+rgL)B9ThK=QPm z82#6q+FnvrGKR}ilyOU0y zY+V+9bjatnv44;l36k}t$h^E*-KqRb^RdToT1q``4)%zjQR5@b*weiPHH>@Y2F`{^8;(ZxEGL1ZeRPC@)!%C-*AK#zFskKZ|H-ou3>c?44}EH&+LWMIAxb5(i}$3dChM z?u`r63*AJGT^zoUyxq_U-b}OlaGjeKUYaILw>LQa;PyuE-~eVFB5Cj|pcSV_hC9{R zrZX|;%nd_<=}yW;yfjjo{*e~b&$5dKBMQ>>c#A`4_{#5~Hbhbm1z|;gfsPkE(Cp=_ zk%1sAGMBPQpRPc8s9I+*h1G#i=+wBs56x%t-%kFgum5!NKYc~<1%B}QzjWli$aH>T z=_`abF1^Toj?c~vu<{jUk%78dBc8c~a^hud4DDfEv(XjQ_CW@946;K;FMHEb+L=q}txm9*+-at9*R-lvs|Z zDHpH4%Qk@THrX_p3Q|K`Wiyuc^EP;cUcKBv zYl`&7?`qmp-_L}Sch=%nQNFutI?RL5YVxx*P>D)qbAkjCm>VU(6tzwRaOtku0sHy! z{xKK+KY#%gOV>#JS+b*s*S`1zqw(V9w%vzOi3R&TMd>iSwxY`c@&?e^fknEfqf{;LTXW)tP| zTBuxr1;d+}9+pp~U+>quWkWXE_t46k#U`xF0WJ`-gq{tLgi05AI5#H1N_{_WKPOGv z695vnvpvMKRZtu-_M(EO#nhV>Bwi=Nv~sq(#DvU|YoY^v+Kr~?mI{`z%PvF3x} zV^$V9tY3NK>V`e}l7l6Js(>bHv;C%}6PkVeDC+c#;?esytg)++)_>&HGAkJIgqU&_ zr=$+l{K@F}3b{Yh@@ukWs}5P1$X2{COfY{af$)mG3r?l;$jSzYV zn2x(ZHa|)CcJTiwVQj{-e!D4#k{>kEABM^$L89KGNhuW<@VjL1!Bi7~ou%BB&>>nu zg;88#@?CTe)e+lu3_!GbrQTY-SPf9r;C^@7j)C0ZWOH<*d!0?TMf}61%YkHbUApHV zRfL4mG@svsfhUU7BJtcbyjgr;-En$pOy;RT2{H|Crzj7?t!Q8SV^_cUF#WorB!Ux( zeE0H#5+eBFF9C4!fKt;_Fz%XR#n(2ko8j4z+>OpzYm_pK(xB@*y>+_ioihqSeETWDiZgv{QiSDCMe+=L>a_6u_$7ps2}+1dF<>5_VTb)u6hXuVi{iu^ z@$wKH+Rc3YtADtNa~PD{);cJU&7gvE?bBOzgt-u&AI4bLUtuwg4ueg$$b-475Tw1p zEo)F5hJ1aw2=G>9y-yYLrpai4xo60wB3ORcA@sq!mbn6~P=a=EaLIcidKRIf$=$cf z9?z6!k?r)|yq>9|@~gk-_$CU`y@1>YTkFMHCInK#bB(i=V6GxN8&<2Jcl`ve48UR_ z8yxF*vg7D^lQ|^`WBq@yo(k#WQXX`KYokM;nWqpQzol1LxDrL5Ae*1}@ zqY5$OEo>=k_I%yWnM}GLr?Nj{QfE%{vcp|0~DGD^%qpU zm(TGTya+dzeXL7Eek0p_WT8R*_&WrJchMD-A>YI5V}@iKf$`&w0bTcu!}7=Z3u5PT zoiPH-kU#zA2=Bn%n}>t2 zb{fGm9K1NuZ$a>!#2)rZapaw2j4r6j$>vwzqKB_;(2DXc*iLaBx9{v16g6KPz~i+8CD(>WRG04|)W1HoEBY9_yaJ=C<~f~q4z|GK6d zk48(2LcGS>(ee6Y`9W=riL73?GbCl|a(?~zn&@~{1~CS*0G3HsV9*C!2f}Y8%k{b@ zBqWe6RC@I9obQ}zW49Y2d-d%~3M|cc9F#r$o&~pYL|)#4QWPbp&bumw7;2*%JBxpU zgy?Dmy!nJ{*RH&dG9j|JUklUKYQ_GvCs%x_*<1eFyFSI+3U1k5-Mcm#CP1!I^Y~ch zJ;nNL$*tMjY_x{@Py?j{HO#~JQ5sI*abR-G*i~xC=tdpK{pHQ%Fc;)VXsRr4ZvKD> zy$>%Q!Sp%+it{2D6<+EeRzD_8uZx3W`wZ=c=+Sul7ZFR2`&hlG%p`?EfEFo+RL{K` zTb&GeLs&~R&S5?72+AOHoAZYx8dDW*rhNAESQ)gT(OESQIswZO%T~?2pgO}9FKZn6 zo)KUks!ZEllSXX>Kd*6~?+ZdXloox$9;0U_{+b*78_s;ITq4O-L3J!sb6r%!QuVq1KM1U=m5FS>Wb9}noiWkq~c-baijNv(*p@K z7=5{Gq1vJpJE__C;o$5qvFof0hD$5WS9%Y4SsQU+2Noh#k1H^jjDKgo=|3o?V}t`( zX-$q2Xw0Mfk@8_w{6i39YwsH&3d}+uA7mO~0vNT&v6Qh^Ky8fizv(71LMD{S@OO@jO$IB3FA`v@1IO znTTKbi8B?~sOY9Hg3_fMM^gTzoX81y!*aB(_#A>Z_%ZYsQdSmmR2d(mM-pjQFt`&^ zQ2DaBeW7B+6Z4ZDTlRr%MdhXJa0^m|GJQW>O0(*CumHpq>O{Rf+_<8JWB*JO9}bzOTiQ}Y7t=6%Ev4DBm(@i&h!DtSmhr3Owd;4 z%_iao)5Ov`wK$3&g&Qw|QTeCTs|4c!yU;RTU$}HhJ0|V^UG#2RXsyC?-HjXb1p{`4 z!yR7M06wM89~b1fGrKvY5#!FGjl%F@V`)#|6#W2K%#F&{Oyry9>n%PpM+6zKD(%L|6(d26|LCG|-`)5adrs-6C+dNE&?gAwXDFrvLfw!J?xy+E7}mlV~c zk`%TDKFfCBiga>3^Gy8FCn&3sVDvcN7bnsE>~e~29F{nJQmNstLW$PVPT8!PyBV5D ze(p5<(wRpb&A%XGvl;s{*W<>}960{>W=N+LJ;77G5LUI`ET~^)mP#i2wurLC6f0{L z=+q%&xRR;~a??UgQ5~qm7#cQr0{_V{hbH(7{_#~qYmlI2RdpVzjY3Z59;z@Er zbb9?zjC^#_`k!mz_K&?pPMFNzbeG$m03Bpxe8R9h>okcC#Q0~LS1Qk~4(`Yf#Mn({ z%yqv%>fduEt4{{?+vy5zy}YXP!56)mOC9&tEoXP?*-kZC^07vdI!bm=z7-H@_0DY- zcL=HCB?I*qB>u{XMp8J}d^xFEf)pjzU+ojQ4Z{<(7I*mmvV2+tL*~}ng=WUF?+A>s z0yo6WB;R^l0VQZ;1n;vk6pW+1II(h)@Oj!l&Yrvgd!(t%P(nleLq6UIGn#*#_i_T? zWjf%iv*v7>uBNYj@OgF~uEuuwYBBEJ4-*!rJn_{+6gOO|UJxj5=7EKLFcdDjU6D@D zl3*Sn%YYS60GY0U;`o@?iEmO3@jCS5o%L|;i`xWcfYCkhdN8digsRYY&s|5Wr3JHx zSImFGAc`|DL$IA|5YmL~rZ_7ofy5e9Kr634Taxip*QFOxlxd59fNbzuO z+f#zHRHdas%p)1B2%{38=M)X||NouRD08~fs7FgmDk5z2UoYP`HR`qa3;?wz!G=EmDUuNZx%pi$CZIprNv1WZM zCJrK@$M^$aB^X5e!=r+6k7v5MIKuk}6h;GTad*3@eCGDXuFmH0_iY#aFe4>U%`A{$0@ko)B3wL z`e#ayX0qka>ACrcM#OG}z3vxcpx@0mym2_!vc=gPt&llWuU50>WOKW@(d!`95t-~- zyC8I+rUK`*P2O#)ju=CxP=0qxM|x%Z$lPOR5FjL6g*{N)SNp0ANMvs zUV9RccxLc;?>f@1$~0rl&0%X|UA&dT1h`plJI{|Ce<0u%rM literal 0 HcmV?d00001 diff --git a/images/python-cyclic-gc-5-new-page.png b/images/python-cyclic-gc-5-new-page.png new file mode 100644 index 0000000000000000000000000000000000000000..fe67a6896fe4b07c03291ca8b21781fde8785757 GIT binary patch literal 5712 zcmZ{oc|25W{Ksd(U>H&MeTi&kD-6RRYgw`{4T`ZO6oaB+CgB#@ibBH?ne3G1rjSZx zJ@y(GNf>)HBujpC@9*CG+g`sv&iOp&`99z0oaghL=e(ZtzHMh?&dn*p34uVkEiF#l zLm)7aSsr9(o*-sHPcJfsUr3MiX71YkS z46$;&&OfOq;KkV&VAeD?n6jEu#b}*Z-deqM^y;-f+eey7Npi>dh~_sRV&$;Or(ty# ze+x-XsESiz@uTdi-t6t>$w z?tqJflHqqz=fsXi>wp3X)|aM<0RvPBaoE)TL;G*cej%#ZWR?J^yBrn?$}oiwFoo4I zy9ZeRiDl4llWn9qz{hKd0HiE7*%L(=Sj$wrCK|Z07epI#mLCA%?Z=F3hX(xC6KHE1K*+6MMs3EA3G_fVf7p{ zopk@Yk-5&Z$QnEt$r3I1N%5~5!x`=WJ}!~~(=Vv|pTcyOB=FBD|Ic8VvHxrm^6`VB z>*F={09~pb0|!-A_;9C18CZZ;pi_TQx8`}uCsq83$p#z&*?isjtt#SQ%+;dOS2DFwE<5o-+B=AxsXVWaCp= z>uK#VnD~V+CSO=Mbe)xQgOOnL!)eyumioLPa)XO4*Ac+bNI0^4{AAaeT%wP*3ye}- zu%LvEOXTVjINl_9aZWghyCAVQ9Ky+Tg86L7f8;NFnA85JOaBU; z3P&d$GPYng-N!*D8fBhJF*BxE!KQM)Fm`Z?2Rv@YrHYL+$Bs2d)WaA^C{EQ;0PqJ^ zE~qmCol;&$OM%Xa&AH|HYAynoIbQdDM=wRGiIpCOaaFHzy;dTF?JvXqF(rm)X%ciz zgEvW~FY~8$IW&86O?U)tO3w-Tq^i_N2!#o8$t9#;n*$E5wE5IP{+%ZV(AxBoaIc&mHr&Em0-=x)oWFQeB*m-=|q-OvgM*1<0eb6?4 zo|Rro&R?#a!o&L4s&*?Zc0X^bD&|M-^Ohj}i)YTpY4%7q!E^6MNCKLquTHwJ z}eWu5d}US28D?~qLlHJ)ZF6RE|R!%`wp#f%FnG3piiS#QU56_MC1|Hs1uSEH`7QG~=s;SL4?8@2ZHAE9Y(MErPitJDJ_t(^F24pmdi^G*yy>7k1HP;Lg|#G zVmRiSK8T4nqks2UP3j@n9P8yy{;_pR5?NPY7rFXnZ}Mw!r#Y08E)d)+_Ye=^UyG;8;o%muPF(SRyRID8m(z8 z$%=%a6D$|Jc`{WN-=Ab=TF6BYj*Iv5@|h?aoC^P%ZYh6Q5<*BxDNStAkgiQ6%jsn; z43n`B?5w;uN}-rrR$(N6#|amW_a{5G)2(_Kz?|{Ot-H~veS;#3@d$0tIAkJ{~2Vz3@gi9(ZmU^{1ER9l+K3i-@yi%yla|zcZc~=i&FwoAB+n4 zw=6qr%{$K})dX_{S;s&rZ_Jc&!v|9c!fg-B)~5$kLG4x-Lbs0D(3ab}cMjH;3f10B zYMJN8IIQ3!w?hDA&&o(*a!JmKMX5D7t@LE~mf*qaiLPe_!R=ilJQ~L*`S|lu^z%92 z&4)G;UL7-{c%J|D>%;7EDio*hWlQ7%pXlDq%k8U?WNAtl+{DTS5yvlio_D8i@?z(0 zh&M{U;dnf0wV7WpioWyAnTONlGo2;^QLcDl?qzIpgZ$B;r{v}FR|kP_6axu$5P9wQ z`C>iR;W`CV^$90M&WaN%E`-g~O1`Iuz%Y$3>zs{~?BWx<^(jeT$yFM9PtFIP(Lcn# zR+z*hb#`jfC58ob-sM+z?s=86m5mVQKkK(Cg8QP+K~v^ue{ECAqpL3p4k$fex407? zF7=h=F!#pqdryz)32_M@M-=BM>9KItk-5T&$$H)SzZlP>^4{{udc^H@9C);>vKb?i zV%-($3Kx*T!Cv+Zf2PZ%kUv))BI>LEGtjJQ&Ut4(V@72b4u#h871(rK!uSlidkFxLm` zXMOZoW>>!>o^Sd_oMg%K+>Rf;1F4kVHvbL7KC=KPUEtYbh{jT4$y5p$cfA!Ss(cW3 z$`~6CcR;emUYMGkr@mp6K8NE(||!$-fpFm?WiRI|*PvIber#KT}&&dSB8 zNAulb_tW(4rOsBZnT4m|>KgWa&dqEjmi@X|UaX9TZ?_#R34@Vk*X$kgL~}inTBOx} ze=PO(-B?)DzNLk)F&Zi^0V8m>DL2Y|)p45L|9G(2grFL-%kgGMxKu+J9r+o&7XI0j zcY-}6sFosQliv)SRyST*#Dr_UMgkoT^3SV6cT_=_2q@$JVG*y;1GhsTphp2p{gxEq zlAV_QUBYwoGoyf(!CxEkQyD6L?qQQwZjlTqjj~V?dMaM-X@~E2MDkC)i_Ie+|4jw= zsHdrEfd{s77Qfb|m^gY=*?Pzc{r0yvu^a?n-J{Jb3Qty!H;Ohd8lfykfNMCRb4VFY z0qm~3?YP*D2Ur&*x05Zekd{_?0=bf*M(cA-&>aDrguolooRGx)7u3e(HxR60B!Zb6$wJIiCd_@55hyfgfL zdQ4m2RaHyuxwA;VmRh8`@wjgHg{c&ceh7i6iEo6`1zu#SIclVys@w-XRpfGZb72kv zp;e03rHPH>MSH-#X%VxIB`W4i>#Nw9)AqQ$C2qPdiZelwCt>Y+`jeG+8MLaGCcY^twGpkw`gmPh zEp7G%$wmp!+B_be#6zcgy&C%?E%w=K$%BPB)TN*O2kDaydcoBjW331M@wpRINIq1#+HfGkO z<_?VTl5I%ZMRm651X8Ox5j{m`D8nh-J?%%5Ap7pvdkM?;AQ(S38QtZ^=@*dwD?m8= z^kUl$FHXAi_VWm+ZNsSh)L27dYW?UvU+qzZLMX1H_=lXZs^wOslx5J%LmHOjVl{i) zXrQuwG(p2NUR>di4&%u`#;Z;^(ss2+C{*#^skD##>VjSof4q8~;(PpGfzqD0YB`F=+QB)vGaF0-SA9(v2m zD7)v%!zd0)X~7LYu?QzickxFz^{mHK4O-X&%1yp;=HD{l4fM9qP!_7@rE*x@id4Pr zi~S}No$u&z;|BF)LW@*PTqxH>%LjLPsqjbfoYKb##ZC%a7=~koIYM zmop%$v>&!H6j>ts>Fro#j5h2piQNFh+hXu41F>}BO1;FV0y1*BV+8w|&LD6jir zy*q@`a7%17vnRQ1U@I=&Qi)>W?v#xcrA~ZoyOiM%yzP@lu-zQ~IfD(-gHFUg!K=F% z=G4B$4$2xTfDWBTlW@W#^wjFh-*^+2-S65>^U?bpXLYksU(;ZeEF51c&PZ-IGU?cP z*hJAvNswxe(}6Tl-j-=WEfvohUnv}&Wv8gK4D$vIt?K}H%dTk4JC8ysU_W-Kmw)h4 zuTDO{+s_JYsKKjPmL}U$#KsX#-$7+(mM~BM;Sg7m%9PI1^+dhVqsL8BUqJ&N3<997 zy2<4P@O>ugy9e`?kjkLdd!;*#Ds^|K%T}?{itlqZDf)f$HMOl)nkTMwteuvk_!&)A zoi2|S4%^-*tvcO-L<$vtDzy?9O1BRKsp;_wnkDT9QI*yGY=kLPwPXW~8zNuZ?je0Y z5|S}gY@jM$kG^$CrIXZ^@=?Y+z6KIqg>dE2f;`oEzb-BJY^d_$<;ms-Vx`H&H{kK! z#Y5Rv%1V)4VfVY1hfwNT z<%(24Jfird_~p8}*k?aLcISR( zfzGM#J5P2=RWy~~xPcaud|-@NBe2KYrVfhtTwezL=fu_epP{7nBmfqLFxC36^5HbD9{^+0UkD5UyMIH=sPI4l(<6B?wq-}UIHOP zz%^XE3oF_^2M<+yRIVcz-16yE!62_qYSrLwN zA{pp^6;dgKOq`t_{xbXrhFmoz0-hro7$hD~PcqIdzPZ?A3cNfF&X|dNDPR2&$diC( z2jvi4a^)@v2u3{hhc%N8(CaRU#;*y}*Dq&1D|{Wd|0L&FU4NowN}Xq4nq06 zh0RUVXHjD^m8I-Nd4Ss49k~m)mT)cIE`IidI$@YSt2QR^A@(&gNXfW>gTCd41;HmdEn~#_NihACS>I&lmmz4Qi;myqczM4#*1lG3RG~EIHVNI_Y z_;0ig=AO_+DVApk+=!PYw~QeXDwlQ8!DFJ3uGtwTOM`>epL+F3@@EWRrRp*P)Q$jn z0+0Ha_56+h{u6%vONICVFL=NStYO~X7=$JXW!#ln`Zt>VTmFWl{Y3|ef%~pM z3N_a7Ojdi8VCqWny+FXDeZufefoQ0yYpJSft7Ulni@p(`R@Nn d2oCiP^t=B5CtNi{nllq1mZmnRtBk#G{s-d0T~Yu5 literal 0 HcmV?d00001 diff --git a/index.rst b/index.rst index 3ff5fca6c..72a81eb3f 100644 --- a/index.rst +++ b/index.rst @@ -260,6 +260,7 @@ Additional Resources * :doc:`exploring` * :doc:`grammar` * :doc:`compiler` + * :doc:`garbage_collector` * Tool support * :doc:`gdb` * :doc:`clang` @@ -317,6 +318,7 @@ Full Table of Contents exploring grammar compiler + garbage_collector extensions coverity clang From fe06a99a3d2c1625425aac6b35f3d832ee5bdffd Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 21 Jan 2020 14:59:12 +0000 Subject: [PATCH 020/871] Fix minor typo in garbage_collector.rst --- garbage_collector.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index 98ceff178..d03c09d91 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -29,7 +29,7 @@ to the object when called): >>> y = x >>> sys.getrefcount(x) 3 - del y + >>> del y >>> sys.getrefcount(x) 2 From 407141c06d100522d9a11a819f378f3d3023df00 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 21 Jan 2020 15:12:18 +0000 Subject: [PATCH 021/871] Add CODEOWNERS file --- .github/CODEOWNERS | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 000000000..668bb5ace --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,8 @@ +# See https://help.github.com/articles/about-codeowners/ +# for more info about CODEOWNERS file + +# It uses the same pattern rule for gitignore file +# https://git-scm.com/docs/gitignore#_pattern_format + + +garbage_collector.rst @pablogsal From 373762d7b02d3a4f1c1f379f827966d1197e4943 Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Tue, 21 Jan 2020 19:33:08 +0100 Subject: [PATCH 022/871] Minor editorial pass over garbage_collector.rst (#563) --- garbage_collector.rst | 51 ++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index d03c09d91..6e6952926 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -15,7 +15,7 @@ that CPython counts how many different places there are that have a reference to object. Such a place could be another object, or a global (or static) C variable, or a local variable in some C function. When an object’s reference count becomes zero, the object is deallocated. If it contains references to other objects, their -reference count is decremented. Those other objects may be deallocated in turn, if +reference counts are decremented. Those other objects may be deallocated in turn, if this decrement makes their reference count become zero, and so on. The reference count field can be examined using the ``sys.getrefcount`` function (notice that the value returned by this function is always 1 more as the function also has a reference @@ -46,7 +46,7 @@ does not handle reference cycles. For instance, consider this code: In this example, ``container`` holds a reference to itself, so even when we remove our reference to it (the variable "container") the reference count never falls to 0 -because it still has its own internal reference and therefore it will never be +because it still has its own internal reference. Therefore it would never be cleaned just by simple reference counting. For this reason some additional machinery is needed to clean these reference cycles between objects once they become unreachable. This is the cyclic garbage collector, usually called just Garbage @@ -92,7 +92,7 @@ As is explained later in the `Optimization: reusing fields to save memory`_ sect these two extra fields are normally used to keep doubly linked lists of all the objects tracked by the garbage collector (these lists are the GC generations, more on that in the `Optimization: generations`_ section), but they are also -reused to fullfill other pourposes when the full doubly linked list structure is not +reused to fullfill other purposes when the full doubly linked list structure is not needed as a memory optimization. Doubly linked lists are used because they efficiently support most frequently required operations. In @@ -144,8 +144,8 @@ lists are maintained: one list contains all objects to be scanned, and the other contain all objects "tentatively" unreachable. To understand how the algorithm works, Let’s take the case of a circular linked list -which has one link referenced by a variable A, and one self-referencing object which -is completely unreachable +which has one link referenced by a variable ``A``, and one self-referencing object which +is completely unreachable: .. code-block:: python @@ -156,14 +156,15 @@ is completely unreachable ... self.next_link = next_link >>> link_3 = Link() - >>> link_2 = Link(link3) - >>> link_1 = Link(link2) + >>> link_2 = Link(link_3) + >>> link_1 = Link(link_2) >>> link_3.next_link = link_1 + >>> A = link_1 + >>> del link_1, link_2, link_3 >>> link_4 = Link() >>> link_4.next_link = link_4 - >>> del link_4 >>> gc.collect() 2 @@ -194,16 +195,16 @@ This is because another object that is reachable from the outside (``gc_refs > 0 can still have references to it. For instance, the ``link_2`` object in our example ended having ``gc_refs == 0`` but is referenced still by the ``link_1`` object that is reachable from the outside. To obtain the set of objects that are really -unreachable, the garbage collector scans again the container objects using the -``tp_traverse`` slot with a different traverse function that marks objects with +unreachable, the garbage collector re-scans the container objects using the +``tp_traverse`` slot; this time with a different traverse function that marks objects with ``gc_refs == 0`` as "tentatively unreachable" and then moves them to the tentatively unreachable list. The following image depicts the state of the lists in a -moment when the GC processed the ``link 3`` and ``link 4`` objects but has not -processed ``link 1`` and ``link 2`` yet. +moment when the GC processed the ``link_3`` and ``link_4`` objects but has not +processed ``link_1`` and ``link_2`` yet. .. figure:: images/python-cyclic-gc-3-new-page.png -Then the GC scans the next ``link 1`` object. Because its has ``gc_refs == 1`` +Then the GC scans the next ``link_1`` object. Because its has ``gc_refs == 1`` the gc does not do anything special because it knows it has to be reachable (and is already in what will become the reachable list): @@ -213,9 +214,9 @@ When the GC encounters an object which is reachable (``gc_refs > 0``), it traver its references using the ``tp_traverse`` slot to find all the objects that are reachable from it, moving them to the end of the list of reachable objects (where they started originally) and setting its ``gc_refs`` field to 1. This is what happens -to ``link 2`` and ``link 3`` below as they are reachable from ``link 1``. From the -state in the previous image and after examining the objects referred to by ``link1`` -the GC knows that ``link 3`` is reachable after all, so it is moved back to the +to ``link_2`` and ``link_3`` below as they are reachable from ``link_1``. From the +state in the previous image and after examining the objects referred to by ``link_1`` +the GC knows that ``link_3`` is reachable after all, so it is moved back to the original list and its ``gc_refs`` field is set to one so if the GC visits it again, it does know that is reachable. To avoid visiting a object twice, the GC marks all objects that are already visited once (by unsetting the ``PREV_MASK_COLLECTING`` flag) @@ -273,13 +274,13 @@ follows these steps in order: set is going to be destroyed and has weak references with callbacks, these callbacks need to be honored. This process is **very** delicate as any error can cause objects that will be in an inconsistent state to be resurrected or reached - by some python functions invoked from the callbacks. To avoid these weak references + by some python functions invoked from the callbacks. In addition, weak references that also are part of the unreachable set (the object and its weak reference - are in a cycles that are unreachable) then the weak reference needs to be cleaned - immediately and the callback must not be executed so it does not trigger later - when the ``tp_clear`` slot is called, causing havoc. This is fine because both - the object and the weakref are going away, so it's legitimate to pretend the - weak reference is going away first so the callback is never executed. + are in a cycles that are unreachable) need to be cleaned + immediately, without executing the callback. Otherwise it will be triggered later, + when the ``tp_clear`` slot is called, causing havoc. Ignoring the weak reference's + callback is fine because both the object and the weakref are going away, so it's + legitimate to say the weak reference is going away first. 2. If an object has legacy finalizers (``tp_del`` slot) move them to the ``gc.garbage`` list. @@ -296,7 +297,7 @@ follows these steps in order: Optimization: generations ------------------------- -In order to limit the time each garbage collection takes, the GC is uses a popular +In order to limit the time each garbage collection takes, the GC uses a popular optimization: generations. The main idea behind this concept is the assumption that most objects have a very short lifespan and can thus be collected shortly after their creation. This has proven to be very close to the reality of many Python programs as @@ -314,7 +315,7 @@ it will be moved to the last generation (generation 2) where it will be surveyed the least often. Generations are collected when the number of objects that they contain reach some -predefined threshold which is unique for each generation and is lower than the older +predefined threshold, which is unique for each generation and is lower the older generations are. These thresholds can be examined using the ``gc.get_threshold`` function: @@ -411,7 +412,7 @@ of ``PyGC_Head`` discussed in the `Memory layout and object structure`_ section: dereferenced directly and the extra information must be stripped off before obtaining the real memory address. Special care needs to be taken with functions that directly manipulate the linked lists, as these functions - normally asume the pointers inside the lists are in a consistent state. + normally assume the pointers inside the lists are in a consistent state. * The ``_gc_prev``` field is normally used as the "previous" pointer to maintain the From fc6b1d88fcbc233816052d38b0234d0c796f2e77 Mon Sep 17 00:00:00 2001 From: Joannah Nanjekye <33177550+nanjekyejoannah@users.noreply.github.com> Date: Wed, 22 Jan 2020 07:23:22 -0400 Subject: [PATCH 023/871] Editorial fixes to garbage_collector.rst (#564) --- garbage_collector.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index 6e6952926..444064919 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -10,7 +10,7 @@ Design of CPython's Garbage Collector Abstract -------- -The main garbage collector system of CPython is reference count. The basic idea is +The main garbage collection algorithm used by CPython is reference counting. The basic idea is that CPython counts how many different places there are that have a reference to an object. Such a place could be another object, or a global (or static) C variable, or a local variable in some C function. When an object’s reference count becomes zero, @@ -33,8 +33,8 @@ to the object when called): >>> sys.getrefcount(x) 2 -The main problem with the reference count schema is that reference counting -does not handle reference cycles. For instance, consider this code: +The main problem with the reference counting scheme is that it does not handle reference +cycles. For instance, consider this code: .. code-block:: python @@ -98,7 +98,7 @@ needed as a memory optimization. Doubly linked lists are used because they efficiently support most frequently required operations. In general, the collection of all objects tracked by GC are partitioned into disjoint sets, each in its own doubly linked list. Between collections, objects are partitioned into "generations", reflecting how -often they've survived collection attempts. During collections, the generations(s) being collected +often they've survived collection attempts. During collections, the generation(s) being collected are further partitioned into, e.g., sets of reachable and unreachable objects. Doubly linked lists support moving an object from one partition to another, adding a new object, removing an object entirely (objects tracked by GC are most often reclaimed by the refcounting system when GC @@ -204,7 +204,7 @@ processed ``link_1`` and ``link_2`` yet. .. figure:: images/python-cyclic-gc-3-new-page.png -Then the GC scans the next ``link_1`` object. Because its has ``gc_refs == 1`` +Then the GC scans the next ``link_1`` object. Because it has ``gc_refs == 1``, the gc does not do anything special because it knows it has to be reachable (and is already in what will become the reachable list): @@ -225,7 +225,7 @@ GC does not process it twice. .. figure:: images/python-cyclic-gc-5-new-page.png -Notice that once a object that was marked as "tentatively unreachable" and later is +Notice that once an object that was marked as "tentatively unreachable" and later is moved back to the reachable list, it will be visited again by the garbage collector as now all the references that that objects has need to be processed as well. This process is really a breadth first search over the object graph. Once all the objects @@ -276,7 +276,7 @@ follows these steps in order: cause objects that will be in an inconsistent state to be resurrected or reached by some python functions invoked from the callbacks. In addition, weak references that also are part of the unreachable set (the object and its weak reference - are in a cycles that are unreachable) need to be cleaned + are in cycles that are unreachable) need to be cleaned immediately, without executing the callback. Otherwise it will be triggered later, when the ``tp_clear`` slot is called, causing havoc. Ignoring the weak reference's callback is fine because both the object and the weakref are going away, so it's @@ -287,7 +287,7 @@ follows these steps in order: 3. Call the finalizers (``tp_finalize`` slot) and mark the objects as already finalized to avoid calling them twice if they resurrect of if other finalizers have removed the object first. -4. Deal with resurrected objects. If some objects have been resurrected the GC +4. Deal with resurrected objects. If some objects have been resurrected, the GC finds the new subset of objects that are still unreachable by running the cycle detection algorithm again and continues with them. 5. Call the ``tp_clear`` slot of every object so all internal links are broken and @@ -316,7 +316,7 @@ surveyed the least often. Generations are collected when the number of objects that they contain reach some predefined threshold, which is unique for each generation and is lower the older -generations are. These thresholds can be examined using the ``gc.get_threshold`` +the generations are. These thresholds can be examined using the ``gc.get_threshold`` function: .. code-block:: python From bfebb53ee9b50e804a76bd32c2a8d8327d09d1f2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Batuhan=20Ta=C5=9Fkaya?= <47358913+isidentical@users.noreply.github.com> Date: Tue, 28 Jan 2020 02:30:09 +0300 Subject: [PATCH 024/871] Adapt release cycle timelines for PEP 602 (#565) --- devcycle.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index add010f6c..f7a902cbb 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -14,13 +14,12 @@ version* of 3, a *minor version* of 1, and a *micro version* of 2. incompatible changes are deemed necessary, and are planned very long in advance; -* new *minor versions* are feature releases; they get released roughly - every 18 months, from the current :ref:`in-development ` - branch; +* new *minor versions* are feature releases; they get released annually, + from the current :ref:`in-development ` branch; * new *micro versions* are bugfix releases; they get released roughly - every 6 months, although they can come more often if necessary; they are - prepared in :ref:`maintenance ` branches. + every 2 months; they are prepared in :ref:`maintenance ` + branches. We also publish non-final versions which get an additional qualifier: :ref:`alpha`, :ref:`beta`, :ref:`release candidate `. These versions From 9de87eda00f0d6533a9dedcfe3ddf3b982a0232d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Batuhan=20Ta=C5=9Fkaya?= <47358913+isidentical@users.noreply.github.com> Date: Tue, 28 Jan 2020 14:04:03 +0300 Subject: [PATCH 025/871] Experts index cleanup for removed modules (#566) --- experts.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/experts.rst b/experts.rst index 7c54c3919..f5839f371 100644 --- a/experts.rst +++ b/experts.rst @@ -107,14 +107,12 @@ encodings lemburg ensurepip ncoghlan, dstufft, pradyunsg enum eli.bendersky*, barry, ethan.furman* errno twouters -exceptions faulthandler vstinner fcntl twouters filecmp fileinput fnmatch formatter -fpectl twouters fractions mark.dickinson, rhettinger ftplib giampaolo.rodola* functools rhettinger* @@ -184,7 +182,6 @@ pstats pty twouters* pwd py_compile -pybench lemburg pyclbr pydoc queue rhettinger* @@ -283,7 +280,6 @@ Tools Tool Maintainers ================== =========== Argument Clinic larry -pybench lemburg ================== =========== From c8c0f2b082c92958457a78504f2fcbb838927b7c Mon Sep 17 00:00:00 2001 From: Reece Dunham Date: Tue, 28 Jan 2020 21:38:09 -0500 Subject: [PATCH 026/871] Mark python 2.7 as dead. (#560) --- devcycle.rst | 16 +++++++--------- index.rst | 7 ++----- 2 files changed, 9 insertions(+), 14 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index f7a902cbb..40b14aae5 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -36,8 +36,7 @@ Branches '''''''' There is a branch for each *feature version*, whether released or not (e.g. -2.7, 3.7, 3.8). Development is handled separately for Python 2 and Python 3: -no merging happens between 2.x and 3.x branches. +3.7, 3.8). .. _indevbranch: @@ -68,12 +67,11 @@ Maintenance branches -------------------- A branch for a previous feature release, currently being maintained for bug -fixes. There are usually two maintenance branches at any given time: one for -Python 3.x and one for Python 2.x. Only during the beta/rc phase of a new +fixes. There are usually two maintenance branches at any given time for +Python 3.x. Only during the beta/rc phase of a new minor/feature release will there be three active maintenance branches, e.g. -during the beta phase for Python 3.8 there are master, 3.8, 3.7, and 2.7 -branches open. At some point in the future, Python 2.x will be closed for bug -fixes and there will be only one maintenance branch left. Releases +during the beta phase for Python 3.8 there are master, 3.8, 3.7, and 3.6 +branches open. Releases produced from a maintenance branch are called **maintenance** or **bugfix** releases; the terms are used interchangeably. These releases have a **micro version** number greater than zero. @@ -140,6 +138,8 @@ For reference, here are the Python versions that most recently reached their end +------------------+--------------+----------------+----------------+----------------------------------+ | 2.6 | :pep:`361` | 2008-10-01 | 2013-10-29 | Barry Warsaw | +------------------+--------------+----------------+----------------+----------------------------------+ +| 2.7 | :pep:`373` | 2010-07-03 | 2020-01-01 | Benjamin Peterson | ++------------------+--------------+----------------+----------------+----------------------------------+ The latest release for each Python version can be found on the `download page `_. @@ -307,8 +307,6 @@ Current Administrators +===================+==========================================================+=================+ | Łukasz Langa | Python 3.8 and 3.9 Release Manager | ambv | +-------------------+----------------------------------------------------------+-----------------+ -| Benjamin Peterson | Python 2.7 Release Manager | benjaminp | -+-------------------+----------------------------------------------------------+-----------------+ | Ned Deily | Python 3.7 Release Manager | ned-deily | +-------------------+----------------------------------------------------------+-----------------+ | Lary Hastings | Python 3.5 Release Manager | larryhastings | diff --git a/index.rst b/index.rst index 72a81eb3f..c62b3754b 100644 --- a/index.rst +++ b/index.rst @@ -99,8 +99,6 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-------------------+ | 3.7 | :pep:`537` | bugfix | 2018-06-27 | *2023-06-27* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-------------------+ -| 2.7 | :pep:`373` | bugfix | 2010-07-03 | *2020-01-01* | Benjamin Peterson | -+------------------+--------------+-------------+----------------+----------------+-------------------+ | 3.6 | :pep:`494` | security | 2016-12-23 | *2021-12-23* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-------------------+ | 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-13* | Larry Hastings | @@ -126,9 +124,8 @@ Status: Dates in *italic* are scheduled and can be adjusted. By default, the end-of-life is scheduled 5 years after the first release, -but can be adjusted by the release manager of each branch. The support for -Python 2.7 has currently been extended to 2020-01-01. Versions older than -2.7 have reached end-of-life. +but can be adjusted by the release manager of each branch. All Python 2 +versions have reached end-of-life. See also the :ref:`devcycle` page for more information about branches. From 35b9c5927623deb0063cb470ecc8a7bc780aca33 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Wed, 29 Jan 2020 07:18:59 +0200 Subject: [PATCH 027/871] Update Python 3.0's EOL date (#543) > Python 3.0 is end-of-lifed with the release of Python 3.1. https://www.python.org/download/releases/3.0/ > Python 3.1 final was released on June 27th, 2009. https://www.python.org/download/releases/3.1/ --- devcycle.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/devcycle.rst b/devcycle.rst index 40b14aae5..181d00315 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -134,7 +134,7 @@ For reference, here are the Python versions that most recently reached their end +------------------+--------------+----------------+----------------+----------------------------------+ | 3.1 | :pep:`375` | 2009-06-27 | 2012-04-09 | Benjamin Peterson | +------------------+--------------+----------------+----------------+----------------------------------+ -| 3.0 | :pep:`361` | 2008-12-03 | 2009-01-13 | Barry Warsaw | +| 3.0 | :pep:`361` | 2008-12-03 | 2009-06-27 | Barry Warsaw | +------------------+--------------+----------------+----------------+----------------------------------+ | 2.6 | :pep:`361` | 2008-10-01 | 2013-10-29 | Barry Warsaw | +------------------+--------------+----------------+----------------+----------------------------------+ From e1d2dac399bb65007b4b99432e08cdc8e3ffa324 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Batuhan=20Ta=C5=9Fkaya?= <47358913+isidentical@users.noreply.github.com> Date: Wed, 29 Jan 2020 18:55:17 +0300 Subject: [PATCH 028/871] Add ast.unparse to grammar change checklist (#567) * Add ast.unparse to grammar change checklist * Apply suggestions from code review Co-Authored-By: Pablo Galindo * fix whitespace Co-authored-by: Pablo Galindo --- grammar.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/grammar.rst b/grammar.rst index 912dbaef5..fb17ab9b3 100644 --- a/grammar.rst +++ b/grammar.rst @@ -54,6 +54,9 @@ Note: sometimes things mysteriously don't work. Before giving up, try ``make cl * The :mod:`parser` module. Add some of your new syntax to ``test_parser``, bang on :file:`Modules/parsermodule.c` until it passes. +* ``_Unparser`` in the :file:`Lib/ast.py` file may need changes to accommodate + any modifications in the AST nodes. + * Add some usage of your new syntax to ``test_grammar.py``. * Certain changes may require tweaks to the library module :mod:`pyclbr`. From 5bec32326868f99f2c4b6f77a43c7cc122af0a88 Mon Sep 17 00:00:00 2001 From: Florian Dahlitz Date: Fri, 31 Jan 2020 20:12:06 +0100 Subject: [PATCH 029/871] add a note about running make clean before or after re-running configure (#556) --- setup.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/setup.rst b/setup.rst index a12822155..f13272320 100644 --- a/setup.rst +++ b/setup.rst @@ -156,6 +156,10 @@ Configuration is typically: More flags are available to ``configure``, but this is the minimum you should do to get a pydebug build of CPython. +.. note:: + You might need to run ``make clean`` before or after re-running ``configure`` + in a particular build directory. + Once ``configure`` is done, you can then compile CPython with: .. code-block:: bash From e5d726bedbf8568d276cca70a3e1b629e36b2b02 Mon Sep 17 00:00:00 2001 From: Gijs Date: Thu, 13 Feb 2020 18:55:06 +0000 Subject: [PATCH 030/871] Fix security reporting link target --- triaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/triaging.rst b/triaging.rst index 2c104ab3f..7a062a330 100644 --- a/triaging.rst +++ b/triaging.rst @@ -602,7 +602,7 @@ Checklist for Triaging .. _issue tracker: https://bugs.python.org .. _GitHub pull requests: https://github.com/python/cpython/pulls .. _Python source code repositories: https://github.com/python/cpython/ -.. _Reporting security issues in Python: https://www.python.org/news/security/ +.. _Reporting security issues in Python: https://www.python.org/dev/security/ .. _python-ideas: https://mail.python.org/mailman/listinfo/python-ideas .. _release schedule: https://devguide.python.org/#status-of-python-branches .. _PSF Code of Conduct: https://www.python.org/psf/codeofconduct/ From d9cb45c649e97fef3d4e36a1350ce83ea3763b05 Mon Sep 17 00:00:00 2001 From: idomic Date: Sun, 23 Feb 2020 03:10:30 -0500 Subject: [PATCH 031/871] bpo-38101: Update devguide triaging keywords (#570) --- triaging.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/triaging.rst b/triaging.rst index 7a062a330..027a7a01a 100644 --- a/triaging.rst +++ b/triaging.rst @@ -375,6 +375,16 @@ Various informational flags about the issue. Multiple values are possible. | easy | Fixing the issue should not take longer than a day for | | | someone new to contributing to Python to solve. | +---------------+------------------------------------------------------------+ +| easy (C) | Fixing the issue should not take longer than a day for | +| | someone new contributing to Python, focused on C. | ++---------------+------------------------------------------------------------+ +| security_issue| This is a security issue or is related to one. The main | +| | difference from the "security" issue type is that this is | +| | a definite security problem that has to be dealt with. | ++---------------+------------------------------------------------------------+ +| PEP 3121 | The issue is related to PEP `PEP 3121`_. | +| | Extension Module Initialization and Finalization. | ++---------------+------------------------------------------------------------+ | newcomer | Issue suitable for newcomer/first time contributors. | | friendly | Not suitable for experienced contributors. Typically it is | | | straightforward, well-defined, low-risk, and optionally | @@ -606,3 +616,4 @@ Checklist for Triaging .. _python-ideas: https://mail.python.org/mailman/listinfo/python-ideas .. _release schedule: https://devguide.python.org/#status-of-python-branches .. _PSF Code of Conduct: https://www.python.org/psf/codeofconduct/ +.. _PEP 3121: https://www.python.org/dev/peps/pep-3121/ From 74214c90033d26f366f65ee62f8931d3e2022b52 Mon Sep 17 00:00:00 2001 From: Brett Cannon <54418+brettcannon@users.noreply.github.com> Date: Fri, 28 Feb 2020 15:35:56 -0800 Subject: [PATCH 032/871] Update the checklist for onboarding a new core dev (#574) --- coredev.rst | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/coredev.rst b/coredev.rst index 2ab2a680c..af8b8579b 100644 --- a/coredev.rst +++ b/coredev.rst @@ -61,22 +61,36 @@ in the :ref:`experts` and :ref:`developers`). Gaining Commit Privileges ------------------------- -When you have been extended an official offer to become a Python core -developer, there are several things you and the person handling your onboarding -must do. - -1. Find out who is handling your onboarding (your mentor should know who this - is; at worst ask the steering council) -2. Email the person handling your onboarding -3. The person onboarding you will ask you for various account details to record - them at https://github.com/python/voters/ -4. They will ask what email address you would like to subscribe to - python-committers with -5. They will turn on various permissions based on the information you provided - in the previous steps -6. They will update the devguide to publicly list your team membership at +The steps to gaining commit privileges are: + +1. A core developer starts a poll at https://discuss.python.org/c/committers/ + + - Open for 7 days + - Results shown upon close + +2. The poll is announced on python-committers +3. Wait for the poll to close and see if the results confirm your membership + as per the voting results requied by PEP 13 +4. The person who nominated you emails the steering council with your email + address and a request that the council either accept or reject the proposed + membership +5. Assuming the steering council does not object, a member of the council will + email you asking for: + + - Account details as required by + https://github.com/python/voters/ + - Your preferred email address to + subscribe to python-committers with + - A reminder about the Code of Conduct and to report issues to the PSF + Conduct WG + +6. Once you have provided the pertinent details, your various new privileges + will be turned on +7. Your details will be added to https://github.com/python/voters/ +8. They will update the devguide to publicly list your team membership at :ref:`developers` -7. They will announce your membership to python-committers +9. An announcement email by the steering council member handling your new + membership will be sent to python-committers Mailing Lists From 1c65b17ac7ab677bced841c2500b4604a87e068f Mon Sep 17 00:00:00 2001 From: Brandt Bucher Date: Fri, 6 Mar 2020 10:54:55 -0800 Subject: [PATCH 033/871] Add language reference to Grammar checklist. (#575) --- grammar.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/grammar.rst b/grammar.rst index fb17ab9b3..4bdf3c54e 100644 --- a/grammar.rst +++ b/grammar.rst @@ -65,4 +65,5 @@ Note: sometimes things mysteriously don't work. Before giving up, try ``make cl * :file:`Lib/lib2to3/Grammar.txt` may need changes to match the Grammar. -* Documentation must be written! +* Documentation must be written! Specifically, one or more of the pages in + :file:`Doc/reference/` will need to be updated. From 931f8724cc996089122b3209f0729f9a1ade921f Mon Sep 17 00:00:00 2001 From: Brett Cannon <54418+brettcannon@users.noreply.github.com> Date: Fri, 6 Mar 2020 16:05:59 -0800 Subject: [PATCH 034/871] Add Karthikeyan to the developer log (#576) --- developers.csv | 335 +++++++++++++++++++++++++------------------------ 1 file changed, 168 insertions(+), 167 deletions(-) diff --git a/developers.csv b/developers.csv index 2bb4ccd33..6021e4587 100644 --- a/developers.csv +++ b/developers.csv @@ -1,167 +1,168 @@ -Joannah Nanjekye,nanjekyejoannah,2019-09-23 -Abhilash Raj,maxking,2019-08-06 -Paul Ganssle,pganssle,2019-06-15 -Stéphane Wirtel,matrixise,2019-04-08 -Stefan Behnel,scoder,2019-04-08 -Cheryl Sabella,csabella,2019-02-19 -Lisa Roach,lisroach,2018-09-14 -Emily Morehouse,emilyemorehouse,2018-09-14 -Pablo Galindo,pablogsal,2018-06-06 -Mark Shannon,markshannon,2018-05-15 -Petr Viktorin,encukou,2018-04-16 -Nathaniel J. Smith,njsmith,2018-01-25 -Julien Palard,JulienPalard,2017-12-08 -Ivan Levkivskyi,ilevkivskyi,2017-12-06 -Carol Willing,willingc,2017-05-24 -Mariatta,Mariatta,2017-01-27 -Xiang Zhang,zhangyangyu,2016-11-21 -Inada Naoki,methane,2016-09-26 -Xavier de Gaye,xdegaye,2016-06-03 -Davin Potts,applio,2016-03-06 -Martin Panter,vadmium,2015-08-10 -Paul Moore,pfmoore,2015-03-15 -Robert Collins,rbtcollins,2014-10-16 -Berker Peksağ,berkerpeksag,2014-06-26 -Steve Dower,zooba,2014-05-10 -Kushal Das,kushaldas,2014-04-14 -Steven D'Aprano,stevendaprano,2014-02-08 -Yury Selivanov,1st1,2014-01-23 -Zachary Ware,zware,2013-11-02 -Donald Stufft,dstufft,2013-08-14 -Ethan Furman,ethanfurman,2013-05-11 -Serhiy Storchaka,serhiy-storchaka,2012-12-26 -Chris Jerdonek,cjerdonek,2012-09-24 -Eric Snow,ericsnowcurrently,2012-09-05 -Peter Moody,,2012-05-20 -Hynek Schlawack,hynek,2012-05-14 -Richard Oudkerk,,2012-04-29 -Andrew Svetlov,asvetlov,2012-03-13 -Petri Lehtinen,akheron,2011-10-22 -Meador Inge,meadori,2011-09-19 -Jeremy Kloth,jkloth,2011-09-12 -Sandro Tosi,sandrotosi,2011-08-01 -Alex Gaynor,alex,2011-07-18 -Charles-François Natali,,2011-05-19 -Nadeem Vawda,,2011-04-10 -Jason R. Coombs,jaraco,2011-03-14 -Ross Lagerwall,,2011-03-13 -Eli Bendersky,eliben,2011-01-11 -Ned Deily,ned-deily,2011-01-09 -David Malcolm,davidmalcolm,2010-10-27 -Tal Einat,taleinat,2010-10-04 -Łukasz Langa,ambv,2010-09-08 -Daniel Stutzbach,,2010-08-22 -Éric Araujo,merwok,2010-08-10 -Brian Quinlan,brianquinlan,2010-07-26 -Alexander Belopolsky,abalkin,2010-05-25 -Tim Golden,tjguk,2010-04-21 -Giampaolo Rodolà,giampaolo,2010-04-17 -Jean-Paul Calderone,,2010-04-06 -Brian Curtin,briancurtin,2010-03-24 -Florent Xicluna,,2010-02-25 -Dino Viehland,DinoV,2010-02-23 -Larry Hastings,larryhastings,2010-02-22 -Victor Stinner,vstinner,2010-01-30 -Stefan Krah,skrah,2010-01-05 -Doug Hellmann,dhellmann,2009-09-20 -Frank Wierzbicki,,2009-08-02 -Ezio Melotti,ezio-melotti,2009-06-07 -Philip Jenvey,pjenvey,2009-05-07 -Michael Foord,voidspace,2009-04-01 -R. David Murray,bitdancer,2009-03-30 -Chris Withers,cjw296,2009-03-08 -Tarek Ziadé,,2008-12-21 -Hirokazu Yamamoto,,2008-08-12 -Armin Ronacher,mitsuhiko,2008-07-23 -Antoine Pitrou,pitrou,2008-07-16 -Senthil Kumaran,orsenthil,2008-06-16 -Jesse Noller,,2008-06-16 -Jesús Cea,jcea,2008-05-13 -Guilherme Polo,,2008-04-24 -Jeroen Ruigrok van der Werven,,2008-04-12 -Benjamin Peterson,benjaminp,2008-03-25 -David Wolever,wolever,2008-03-17 -Trent Nelson,tpn,2008-03-17 -Mark Dickinson,mdickinson,2008-01-06 -Amaury Forgeot d'Arc,amauryfa,2007-11-09 -Christian Heimes,tiran,2007-10-31 -Bill Janssen,,2007-08-28 -Jeffrey Yasskin,,2007-08-09 -Mark Summerfield,,2007-08-01 -Alexandre Vassalotti,avassalotti,2007-05-21 -Travis E. Oliphant,,2007-04-17 -Eric V. Smith,ericvsmith,2007-02-28 -Josiah Carlson,,2007-01-06 -Collin Winter,,2007-01-05 -Richard Jones,,2006-05-23 -Kristján Valur Jónsson,,2006-05-17 -Jack Diederich,jackdied,2006-05-17 -Steven Bethard,,2006-04-27 -Gerhard Häring,,2006-04-23 -George Yoshida,,2006-04-17 -Ronald Oussoren,ronaldoussoren,2006-03-03 -Nick Coghlan,ncoghlan,2005-10-16 -Georg Brandl,birkenfeld,2005-05-28 -Terry Jan Reedy,terryjreedy,2005-04-07 -Bob Ippolito,,2005-03-02 -Peter Astrand,,2004-10-21 -Facundo Batista,facundobatista,2004-10-16 -Sean Reifschneider,,2004-09-17 -Johannes Gijsbers,,2004-08-14 -Matthias Klose,doko42,2004-08-04 -PJ Eby,pjeby,2004-03-24 -Vinay Sajip,vsajip,2004-02-20 -Hye-Shik Chang,,2003-12-10 -Armin Rigo,,2003-10-24 -Andrew McNamara,,2003-06-09 -Samuele Pedroni,,2003-05-16 -Alex Martelli,aleaxit,2003-04-22 -Brett Cannon,brettcannon,2003-04-18 -David Goodger,,2003-01-02 -Gustavo Niemeyer,,2002-11-05 -Tony Lownds,,2002-09-22 -Steve Holden,,2002-06-14 -Christian Tismer,ctismer,2002-05-17 -Jason Tishler,,2002-05-15 -Walter Dörwald,doerwalter,2002-03-21 -Andrew MacIntyre,,2002-02-17 -Gregory P. Smith,gpshead,2002-01-08 -Anthony Baxter,,2001-12-21 -Neal Norwitz,,2001-12-19 -Raymond Hettinger,rhettinger,2001-12-10 -Chui Tey,,2001-10-31 -Michael W. Hudson,,2001-08-27 -Finn Bock,,2001-08-23 -Piers Lauder,,2001-07-20 -Kurt B. Kaiser,kbkaiser,2001-07-03 -Steven M. Gava,,2001-06-25 -Steve Purcell,,2001-03-22 -Jim Fulton,,2000-10-06 -Ka-Ping Yee,,2000-10-03 -Lars Gustäbel,gustaebel,2000-09-21 -Neil Schemenauer,nascheme,2000-09-15 -Martin v. Löwis,,2000-09-08 -Thomas Heller,theller,2000-09-07 -Moshe Zadka,,2000-07-29 -Thomas Wouters,Yhg1s,2000-07-14 -Peter Schneider-Kamp,,2000-07-10 -Paul Prescod,,2000-07-01 -Tim Peters,tim-one,2000-06-30 -Skip Montanaro,smontanaro,2000-06-30 -Fredrik Lundh,,2000-06-29 -Mark Hammond,mhammond,2000-06-09 -Marc-André Lemburg,malemburg,2000-06-07 -Trent Mick,,2000-06-06 -Eric S. Raymond,,2000-06-02 -Greg Stein,,1999-11-07 -Just van Rossum,,1999-01-22 -Greg Ward,,1998-12-18 -Andrew Kuchling,akuchling,1998-04-09 -Ken Manheimer,,1998-03-03 -Jeremy Hylton,jeremyhylton,1997-08-13 -Roger E. Masse,,1996-12-09 -Fred Drake,freddrake,1996-07-23 -Barry Warsaw,warsaw,1994-07-25 -Jack Jansen,jackjansen,1992-08-13 -Sjoerd Mullender,sjoerdmullender,1992-08-04 -Guido van Rossum,gvanrossum,1989-12-25 +Karthikeyan Singaravelan,tirkarthi,2020-12-31 +Joannah Nanjekye,nanjekyejoannah,2019-09-23 +Abhilash Raj,maxking,2019-08-06 +Paul Ganssle,pganssle,2019-06-15 +Stéphane Wirtel,matrixise,2019-04-08 +Stefan Behnel,scoder,2019-04-08 +Cheryl Sabella,csabella,2019-02-19 +Lisa Roach,lisroach,2018-09-14 +Emily Morehouse,emilyemorehouse,2018-09-14 +Pablo Galindo,pablogsal,2018-06-06 +Mark Shannon,markshannon,2018-05-15 +Petr Viktorin,encukou,2018-04-16 +Nathaniel J. Smith,njsmith,2018-01-25 +Julien Palard,JulienPalard,2017-12-08 +Ivan Levkivskyi,ilevkivskyi,2017-12-06 +Carol Willing,willingc,2017-05-24 +Mariatta,Mariatta,2017-01-27 +Xiang Zhang,zhangyangyu,2016-11-21 +Inada Naoki,methane,2016-09-26 +Xavier de Gaye,xdegaye,2016-06-03 +Davin Potts,applio,2016-03-06 +Martin Panter,vadmium,2015-08-10 +Paul Moore,pfmoore,2015-03-15 +Robert Collins,rbtcollins,2014-10-16 +Berker Peksağ,berkerpeksag,2014-06-26 +Steve Dower,zooba,2014-05-10 +Kushal Das,kushaldas,2014-04-14 +Steven D'Aprano,stevendaprano,2014-02-08 +Yury Selivanov,1st1,2014-01-23 +Zachary Ware,zware,2013-11-02 +Donald Stufft,dstufft,2013-08-14 +Ethan Furman,ethanfurman,2013-05-11 +Serhiy Storchaka,serhiy-storchaka,2012-12-26 +Chris Jerdonek,cjerdonek,2012-09-24 +Eric Snow,ericsnowcurrently,2012-09-05 +Peter Moody,,2012-05-20 +Hynek Schlawack,hynek,2012-05-14 +Richard Oudkerk,,2012-04-29 +Andrew Svetlov,asvetlov,2012-03-13 +Petri Lehtinen,akheron,2011-10-22 +Meador Inge,meadori,2011-09-19 +Jeremy Kloth,jkloth,2011-09-12 +Sandro Tosi,sandrotosi,2011-08-01 +Alex Gaynor,alex,2011-07-18 +Charles-François Natali,,2011-05-19 +Nadeem Vawda,,2011-04-10 +Jason R. Coombs,jaraco,2011-03-14 +Ross Lagerwall,,2011-03-13 +Eli Bendersky,eliben,2011-01-11 +Ned Deily,ned-deily,2011-01-09 +David Malcolm,davidmalcolm,2010-10-27 +Tal Einat,taleinat,2010-10-04 +Łukasz Langa,ambv,2010-09-08 +Daniel Stutzbach,,2010-08-22 +Éric Araujo,merwok,2010-08-10 +Brian Quinlan,brianquinlan,2010-07-26 +Alexander Belopolsky,abalkin,2010-05-25 +Tim Golden,tjguk,2010-04-21 +Giampaolo Rodolà,giampaolo,2010-04-17 +Jean-Paul Calderone,,2010-04-06 +Brian Curtin,briancurtin,2010-03-24 +Florent Xicluna,,2010-02-25 +Dino Viehland,DinoV,2010-02-23 +Larry Hastings,larryhastings,2010-02-22 +Victor Stinner,vstinner,2010-01-30 +Stefan Krah,skrah,2010-01-05 +Doug Hellmann,dhellmann,2009-09-20 +Frank Wierzbicki,,2009-08-02 +Ezio Melotti,ezio-melotti,2009-06-07 +Philip Jenvey,pjenvey,2009-05-07 +Michael Foord,voidspace,2009-04-01 +R. David Murray,bitdancer,2009-03-30 +Chris Withers,cjw296,2009-03-08 +Tarek Ziadé,,2008-12-21 +Hirokazu Yamamoto,,2008-08-12 +Armin Ronacher,mitsuhiko,2008-07-23 +Antoine Pitrou,pitrou,2008-07-16 +Senthil Kumaran,orsenthil,2008-06-16 +Jesse Noller,,2008-06-16 +Jesús Cea,jcea,2008-05-13 +Guilherme Polo,,2008-04-24 +Jeroen Ruigrok van der Werven,,2008-04-12 +Benjamin Peterson,benjaminp,2008-03-25 +David Wolever,wolever,2008-03-17 +Trent Nelson,tpn,2008-03-17 +Mark Dickinson,mdickinson,2008-01-06 +Amaury Forgeot d'Arc,amauryfa,2007-11-09 +Christian Heimes,tiran,2007-10-31 +Bill Janssen,,2007-08-28 +Jeffrey Yasskin,,2007-08-09 +Mark Summerfield,,2007-08-01 +Alexandre Vassalotti,avassalotti,2007-05-21 +Travis E. Oliphant,,2007-04-17 +Eric V. Smith,ericvsmith,2007-02-28 +Josiah Carlson,,2007-01-06 +Collin Winter,,2007-01-05 +Richard Jones,,2006-05-23 +Kristján Valur Jónsson,,2006-05-17 +Jack Diederich,jackdied,2006-05-17 +Steven Bethard,,2006-04-27 +Gerhard Häring,,2006-04-23 +George Yoshida,,2006-04-17 +Ronald Oussoren,ronaldoussoren,2006-03-03 +Nick Coghlan,ncoghlan,2005-10-16 +Georg Brandl,birkenfeld,2005-05-28 +Terry Jan Reedy,terryjreedy,2005-04-07 +Bob Ippolito,,2005-03-02 +Peter Astrand,,2004-10-21 +Facundo Batista,facundobatista,2004-10-16 +Sean Reifschneider,,2004-09-17 +Johannes Gijsbers,,2004-08-14 +Matthias Klose,doko42,2004-08-04 +PJ Eby,pjeby,2004-03-24 +Vinay Sajip,vsajip,2004-02-20 +Hye-Shik Chang,,2003-12-10 +Armin Rigo,,2003-10-24 +Andrew McNamara,,2003-06-09 +Samuele Pedroni,,2003-05-16 +Alex Martelli,aleaxit,2003-04-22 +Brett Cannon,brettcannon,2003-04-18 +David Goodger,,2003-01-02 +Gustavo Niemeyer,,2002-11-05 +Tony Lownds,,2002-09-22 +Steve Holden,,2002-06-14 +Christian Tismer,ctismer,2002-05-17 +Jason Tishler,,2002-05-15 +Walter Dörwald,doerwalter,2002-03-21 +Andrew MacIntyre,,2002-02-17 +Gregory P. Smith,gpshead,2002-01-08 +Anthony Baxter,,2001-12-21 +Neal Norwitz,,2001-12-19 +Raymond Hettinger,rhettinger,2001-12-10 +Chui Tey,,2001-10-31 +Michael W. Hudson,,2001-08-27 +Finn Bock,,2001-08-23 +Piers Lauder,,2001-07-20 +Kurt B. Kaiser,kbkaiser,2001-07-03 +Steven M. Gava,,2001-06-25 +Steve Purcell,,2001-03-22 +Jim Fulton,,2000-10-06 +Ka-Ping Yee,,2000-10-03 +Lars Gustäbel,gustaebel,2000-09-21 +Neil Schemenauer,nascheme,2000-09-15 +Martin v. Löwis,,2000-09-08 +Thomas Heller,theller,2000-09-07 +Moshe Zadka,,2000-07-29 +Thomas Wouters,Yhg1s,2000-07-14 +Peter Schneider-Kamp,,2000-07-10 +Paul Prescod,,2000-07-01 +Tim Peters,tim-one,2000-06-30 +Skip Montanaro,smontanaro,2000-06-30 +Fredrik Lundh,,2000-06-29 +Mark Hammond,mhammond,2000-06-09 +Marc-André Lemburg,malemburg,2000-06-07 +Trent Mick,,2000-06-06 +Eric S. Raymond,,2000-06-02 +Greg Stein,,1999-11-07 +Just van Rossum,,1999-01-22 +Greg Ward,,1998-12-18 +Andrew Kuchling,akuchling,1998-04-09 +Ken Manheimer,,1998-03-03 +Jeremy Hylton,jeremyhylton,1997-08-13 +Roger E. Masse,,1996-12-09 +Fred Drake,freddrake,1996-07-23 +Barry Warsaw,warsaw,1994-07-25 +Jack Jansen,jackjansen,1992-08-13 +Sjoerd Mullender,sjoerdmullender,1992-08-04 +Guido van Rossum,gvanrossum,1989-12-25 From 7a7dc9286855c3f9e429c0309263937cc58237a1 Mon Sep 17 00:00:00 2001 From: Brett Cannon <54418+brettcannon@users.noreply.github.com> Date: Mon, 9 Mar 2020 16:18:11 -0700 Subject: [PATCH 035/871] Fix the year Karthikeyan became a core dev --- developers.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developers.csv b/developers.csv index 6021e4587..68121c96e 100644 --- a/developers.csv +++ b/developers.csv @@ -1,4 +1,4 @@ -Karthikeyan Singaravelan,tirkarthi,2020-12-31 +Karthikeyan Singaravelan,tirkarthi,2019-12-31 Joannah Nanjekye,nanjekyejoannah,2019-09-23 Abhilash Raj,maxking,2019-08-06 Paul Ganssle,pganssle,2019-06-15 From 2efd6326870cf26a8d0c9cfa678313803aa64144 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 8 Apr 2020 18:22:18 -0700 Subject: [PATCH 036/871] Add Dong-hee Na to the developer log --- developers.csv | 1 + 1 file changed, 1 insertion(+) diff --git a/developers.csv b/developers.csv index 68121c96e..2c5061fe7 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Dong-hee Na,corona10,2020-04-08 Karthikeyan Singaravelan,tirkarthi,2019-12-31 Joannah Nanjekye,nanjekyejoannah,2019-09-23 Abhilash Raj,maxking,2019-08-06 From 8b7e79218839f187788e7908d60020c59f56853c Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 14 Apr 2020 17:45:11 -0700 Subject: [PATCH 037/871] Add Kyle Stanley to the developer log --- developers.csv | 1 + 1 file changed, 1 insertion(+) diff --git a/developers.csv b/developers.csv index 2c5061fe7..acb284cbd 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Kyle Stanley,aeros,2020-04-14 Dong-hee Na,corona10,2020-04-08 Karthikeyan Singaravelan,tirkarthi,2019-12-31 Joannah Nanjekye,nanjekyejoannah,2019-09-23 From 89b12a52369c7ce0b90e70f17a99c9e4bfd701e6 Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Wed, 20 May 2020 18:53:04 +0200 Subject: [PATCH 038/871] Rewrite translations table and fix link (#585) --- documenting.rst | 189 ++++++++++++++++++++++-------------------------- 1 file changed, 87 insertions(+), 102 deletions(-) diff --git a/documenting.rst b/documenting.rst index 6207c8d1f..abb0ab23e 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1592,108 +1592,93 @@ they are built by `docsbuild-scripts docs.python.org. There are several documentation translations already in production, other are work in progress: -+--------------+--------------------+------------------------------------------+ -| Language | Contact | Links | -+==============+====================+==========================================+ -| Arabic (ar) | Abdur-Rahmaan | `github `__ | -+--------------+--------------------+------------------------------------------+ -| Bengali as | `Kushal Das | `github `__ | -| India (bn_IN)| .org/user16382>`__ | | -+--------------+--------------------+------------------------------------------+ -| French (fr) | `Julien Palard | `github `__ | -| | s.python.org/user2 | `doc `__ | -| | 3063>`__ | | -+--------------+--------------------+------------------------------------------+ -| Hindi as | | `github `__ | -| India | | | -| (hi_IN) | | | -+--------------+--------------------+------------------------------------------+ -| Hungarian | `Tamás Bajusz | `github `__ | -| | ugs.python.org/use | | -| | r25857>`__ | `list `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Indonesian | `Oon Arfiandwi | `github `__ | -| | on.org/user32660 | | -| | />`__ | | -+--------------+--------------------+------------------------------------------+ -| Italian (it) | | `mail `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Japanese | `Kinebuchi Tomohiko| `github `__ | -| | `__ | -| | user19001>`__ | | -+--------------+--------------------+------------------------------------------+ -| Korean (ko) | | `github `__ | -| | | | -| | | `doc `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Lithuanian | | `mail `__ | -+--------------+--------------------+------------------------------------------+ -| Polish (pl) | | `mail `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Portuguese | Gustavo Toffo | | -| (pt) | | | -+--------------+--------------------+------------------------------------------+ -| Portuguese | Marco Rougeth | `github `__ | -| in Brasil | | | -| (pt-br) | | `wiki `__ | -| | | | -| | | `telgram `__ | -| | | | -| | | `article `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Russian (ru) | | `mail `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Simplified | `Shengjing | `transifex `__ | -| | 11>`__ | | -| | | `github `__ | -| | | | -| | | `doc `__ | -| | | | -+--------------+--------------------+------------------------------------------+ -| Spanish | Raul Cumplido | `github `__ | -| | | | -| | | `old repo `__ | -+--------------+--------------------+------------------------------------------+ -| Traditional | 廖偉涵 Adrian Liaw | `github `__ | -| (zh-tw) | | | -| | | `transifex `__ | -| | | | -| | | `doc `__ | -+--------------+--------------------+------------------------------------------+ -| Turkish (tr) | | `github `__ | -| | | | -+--------------+--------------------+------------------------------------------+ - ++-----------------+-------------------------------+----------------------------+ +| Language | Contact | Links | ++=================+===============================+============================+ +| Arabic (ar) | Abdur-Rahmaan Janhangeer | `GitHub `_ | ++-----------------+-------------------------------+----------------------------+ +| Bengali as | `Kushal Das `_ | `GitHub `_ | +| spoken in | | | +| India (bn_IN) | | | ++-----------------+-------------------------------+----------------------------+ +| French (fr) | `Julien Palard (mdk) | `GitHub `_ | +| | `_ | | ++-----------------+-------------------------------+----------------------------+ +| Hindi as spoken | | `GitHub `_ | +| in india (hi_IN)| | | ++-----------------+-------------------------------+----------------------------+ +| Hungarian (hu) | `Tamás Bajusz (gbtami) | `GitHub `_ | +| | `_ | `Mailing List `_ | ++-----------------+-------------------------------+----------------------------+ +| Indonesian (id) | `Oon Arfiandwi `_ | `GitHub `_ | ++-----------------+-------------------------------+----------------------------+ +| Italian (it) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Japanese (ja) | `Kinebuchi Tomohiko | `GitHub `_ | +| | (cocoatomo) `_| `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Korean (ko) | | `GitHub `_ | +| | | `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Lithuanian (lt) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Polish (pl) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Portuguese (pt) | Gustavo Toffo | | ++-----------------+-------------------------------+----------------------------+ +| Portuguese | Marco Rougeth | `GitHub `_ | +| as spoken | | `Wiki `_ | +| in Brasil | | `Telegram `_ | +| (pt-br) | | `article `_| ++-----------------+-------------------------------+----------------------------+ +| Russian (ru) | | `mail `_ | ++-----------------+-------------------------------+----------------------------+ +| Simplified | `Shengjing Zhu `_ | `Transifex `_ | +| Chinese | | `GitHub `_ | +| (zh-cn) | | `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Spanish (es) | Raúl Cumplido | `GitHub `_ | ++-----------------+-------------------------------+----------------------------+ +| Traditional | 廖偉涵 Adrian Liaw | `GitHub `_ | +| Chinese | | `Transifex `_ | +| (zh-tw) | | `Doc `_ | ++-----------------+-------------------------------+----------------------------+ +| Turkish (tr) | | `GitHub `_ | ++-----------------+-------------------------------+----------------------------+ + +.. _article_pt_br: http://rgth.co/blog/python-ptbr-cenario-atual +.. _bpo_cocoatomo: https://bugs.python.org/user19001 +.. _bpo_gbtami: https://bugs.python.org/user25857 +.. _bpo_kushal: https://bugs.python.org/user16382 +.. _bpo_mdk: https://bugs.python.org/user23063 +.. _bpo_oonid: https://bugs.python.org/user32660 +.. _bpo_zhsj: https://bugs.python.org/user24811 +.. _chat_pt_br: https://t.me/pybr_i18n +.. _doc_ja: https://docs.python.org/ja/ +.. _doc_ko: https://docs.python.org/ko/ +.. _doc_zh_cn: https://docs.python.org/zh-cn/ +.. _doc_zh_tw: https://docs.python.org/zh-tw/ +.. _github_ar: https://github.com/Abdur-rahmaanJ/py-docs-ar +.. _github_bn_in: https://github.com/python/python-docs-bn-in +.. _github_es: https://github.com/PyCampES/python-docs-es +.. _github_fr: https://github.com/python/python-docs-fr +.. _github_hi_in: https://github.com/CuriousLearner/python-docs-hi-in +.. _github_hu: https://github.com/python/python-docs-hu +.. _github_id: https://github.com/python/python-docs-id +.. _github_ja: https://github.com/python/python-docs-ja +.. _github_ko: https://github.com/python/python-docs-ko +.. _github_pt_br: https://github.com/python/python-docs-pt-br +.. _github_tr: https://github.com/alaeddingurel/python-docs-tr +.. _github_zh_cn: https://github.com/python/python-docs-zh-cn +.. _github_zh_tw: https://github.com/python/python-docs-zh-tw +.. _list_hu: https://mail.python.org/pipermail/python-hu +.. _mail_it: https://mail.python.org/pipermail/doc-sig/2019-April/004114.html +.. _mail_lt: https://mail.python.org/pipermail/doc-sig/2019-July/004138.html +.. _mail_pl: https://mail.python.org/pipermail/doc-sig/2019-April/004106.html +.. _mail_ru: https://mail.python.org/pipermail/doc-sig/2019-May/004131.html +.. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/language/ +.. _tx_zh_tw: https://www.transifex.com/python-tw-doc/python-36-tw +.. _wiki_pt_br: http://python.org.br/traducao Starting a new translation -------------------------- From ec91dc66e354fc83d26b74a5f6d569cc5cea39b9 Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Wed, 20 May 2020 23:59:46 +0200 Subject: [PATCH 039/871] Doc translations: update Polish (#586) --- documenting.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/documenting.rst b/documenting.rst index abb0ab23e..8145ecd7e 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1623,6 +1623,7 @@ in production, other are work in progress: | Lithuanian (lt) | | `mail `_ | +-----------------+-------------------------------+----------------------------+ | Polish (pl) | | `mail `_ | +| | | `Translations `_ | +-----------------+-------------------------------+----------------------------+ | Portuguese (pt) | Gustavo Toffo | | +-----------------+-------------------------------+----------------------------+ @@ -1676,6 +1677,7 @@ in production, other are work in progress: .. _mail_lt: https://mail.python.org/pipermail/doc-sig/2019-July/004138.html .. _mail_pl: https://mail.python.org/pipermail/doc-sig/2019-April/004106.html .. _mail_ru: https://mail.python.org/pipermail/doc-sig/2019-May/004131.html +.. _tx_pl: https://www.transifex.com/python-doc/python-newest/language/pl/ .. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/language/ .. _tx_zh_tw: https://www.transifex.com/python-tw-doc/python-36-tw .. _wiki_pt_br: http://python.org.br/traducao From e0460a9cda187c095dd208c876b0f1036028f6ac Mon Sep 17 00:00:00 2001 From: Florian Dahlitz Date: Sun, 24 May 2020 04:42:15 +0200 Subject: [PATCH 040/871] Add a note about not modifying the git history to the quick guide (GH-588) Co-authored-by: Kyle Stanley Co-authored-by: Terry Jan Reedy Co-authored-by: Carol Willing --- pullrequest.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/pullrequest.rst b/pullrequest.rst index 859261948..bd94cbe12 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -47,6 +47,11 @@ Here is a quick overview of how you can contribute to CPython: .. [*] If an issue is trivial (e.g. typo fixes), or if an issue already exists, you can skip this step. +.. note:: + In order to keep the commit history intact, please avoid squashing or amending + history and then force-pushing to the PR. Reviewers often want to look at + individual commits. + .. _Clear communication: https://opensource.guide/how-to-contribute/#how-to-submit-a-contribution .. _Open Source: https://opensource.guide/ .. _create an issue: https://bugs.python.org/ From 74d5f3ad0c3492b5b2df9949e6ec109b6bb7febc Mon Sep 17 00:00:00 2001 From: Florian Dahlitz Date: Mon, 25 May 2020 12:54:52 +0200 Subject: [PATCH 041/871] Fix good commit example to use imperative title (GH-587) * Explicitly mention the preference for imperative over descriptive titles * Link to the article from Chris Beams --- pullrequest.rst | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/pullrequest.rst b/pullrequest.rst index bd94cbe12..4548e2ccf 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -263,21 +263,30 @@ and for each pull request there may be several commits. In particular: Commit messages should follow the following structure:: - bpo-42: the spam module is now more spammy (GH-NNNN) + bpo-42: Make the spam module more spammy (GH-NNNN) The spam module sporadically came up short on spam. This change raises the amount of spam in the module by making it more spammy. The first line or sentence is meant to be a dense, to-the-point explanation -of what the purpose of the commit is. If this is not enough detail for a -commit, a new paragraph(s) can be added to explain in proper depth what has -happened (detail should be good enough that a core developer reading the -commit message understands the justification for the change). +of what the purpose of the commit is. The imperative form (used in the example +above) is strongly preferred to a descriptive form such as 'the spam module is +now more spammy'. Use ``git log --oneline`` to see existing title lines. +Furthermore, the first line should not end in a period. + +If this is not enough detail for a commit, a new paragraph(s) can be added +to explain in proper depth what has happened (detail should be good enough +that a core developer reading the commit message understands the +justification for the change). Check :ref:`the git bootcamp ` for further instructions on how the commit message should look like when merging a pull request. +.. note:: + `How to Write a Git Commit Message `_ + is a nice article that describes how to write a good commit message. + .. _cla: From b353f815c76697e10390d1a1723de582fa4f5060 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9mi=20Lapeyre?= Date: Wed, 27 May 2020 22:27:16 +0200 Subject: [PATCH 042/871] Update table "Status of Python branches" (GH-590) --- index.rst | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/index.rst b/index.rst index c62b3754b..0eab4ede1 100644 --- a/index.rst +++ b/index.rst @@ -90,19 +90,21 @@ contributing to Python: Status of Python branches ------------------------- -+------------------+--------------+-------------+----------------+----------------+-------------------+ -| Branch | Schedule | Status | First release | End-of-life | Release manager | -+==================+==============+=============+================+================+===================+ -| master | :pep:`596` | features | *TBD* | *TBD* | Łukasz Langa | -+------------------+--------------+-------------+----------------+----------------+-------------------+ -| 3.8 | :pep:`569` | bugfix | 2019-10-14 | *2024-10* | Łukasz Langa | -+------------------+--------------+-------------+----------------+----------------+-------------------+ -| 3.7 | :pep:`537` | bugfix | 2018-06-27 | *2023-06-27* | Ned Deily | -+------------------+--------------+-------------+----------------+----------------+-------------------+ -| 3.6 | :pep:`494` | security | 2016-12-23 | *2021-12-23* | Ned Deily | -+------------------+--------------+-------------+----------------+----------------+-------------------+ -| 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-13* | Larry Hastings | -+------------------+--------------+-------------+----------------+----------------+-------------------+ ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| Branch | Schedule | Status | First release | End-of-life | Release manager | ++==================+==============+=============+================+================+=======================+ +| master | :pep:`619` | features | *2021-10-04* | *TBD* | Pablo Galindo Salgado | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| 3.9 | :pep:`596` | bugfix | *2020-10-05* | *TBD* | Łukasz Langa | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| 3.8 | :pep:`569` | bugfix | 2019-10-14 | *2024-10* | Łukasz Langa | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| 3.7 | :pep:`537` | bugfix | 2018-06-27 | *2023-06-27* | Ned Deily | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| 3.6 | :pep:`494` | security | 2016-12-23 | *2021-12-23* | Ned Deily | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-13* | Larry Hastings | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ .. Remember to update the end-of-life table in devcycle.rst. From 6356ed39466f92850d25cf029cb9dfbb898c3208 Mon Sep 17 00:00:00 2001 From: Manuel Jacob Date: Sat, 20 Jun 2020 06:42:24 +0200 Subject: [PATCH 043/871] Update future Python version number (#595) Co-authored-by: Manuel Jacob --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 0eab4ede1..313e82d24 100644 --- a/index.rst +++ b/index.rst @@ -108,7 +108,7 @@ Status of Python branches .. Remember to update the end-of-life table in devcycle.rst. -The master branch is currently the future Python 3.9, and is the only +The master branch is currently the future Python 3.10, and is the only branch that accepts new features. The latest release for each Python version can be found on the `download page `_. From 85b3ff25c76f76c7d5c7387e251ad2b18d939812 Mon Sep 17 00:00:00 2001 From: Ned Deily Date: Sat, 27 Jun 2020 05:42:38 -0400 Subject: [PATCH 044/871] Move 3.7 to security-fix phase. --- committing.rst | 4 ++-- index.rst | 2 +- setup.rst | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/committing.rst b/committing.rst index 19191b03b..59757f2df 100644 --- a/committing.rst +++ b/committing.rst @@ -232,7 +232,7 @@ repositories means you have to be more careful with your workflow: main repository. * You should not commit directly into the ``master`` branch, or any of the - maintenance branches (currently ``3.7``, ``3.6``, and ``2.7``). + maintenance branches (currently ``3.9`` and ``3.8``). You should commit against your own feature branch, and create a pull request. * For a small change, you can make a quick edit through the GitHub web UI. @@ -288,7 +288,7 @@ message:: Prefix the backport pull request with the branch, and reference the pull request number from ``master``, for example:: - [3.7] bpo-12345: Fix the Spam Module (GH-NNNN) + [3.9] bpo-12345: Fix the Spam Module (GH-NNNN) Note that cherry_picker.py_ adds the branch prefix automatically. diff --git a/index.rst b/index.rst index 313e82d24..0da88ad01 100644 --- a/index.rst +++ b/index.rst @@ -99,7 +99,7 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.8 | :pep:`569` | bugfix | 2019-10-14 | *2024-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.7 | :pep:`537` | bugfix | 2018-06-27 | *2023-06-27* | Ned Deily | +| 3.7 | :pep:`537` | security | 2018-06-27 | *2023-06-27* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.6 | :pep:`494` | security | 2016-12-23 | *2021-12-23* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-----------------------+ diff --git a/setup.rst b/setup.rst index f13272320..0b5e86441 100644 --- a/setup.rst +++ b/setup.rst @@ -96,8 +96,8 @@ in the ``cpython`` directory and two remotes that refer to your own GitHub fork If you want a working copy of an already-released version of Python, i.e., a version in :ref:`maintenance mode `, you can checkout -a release branch. For instance, to checkout a working copy of Python 3.7, -do ``git checkout 3.7``. +a release branch. For instance, to checkout a working copy of Python 3.8, +do ``git checkout 3.8``. You will need to re-compile CPython when you do such an update. From d9f942d4a6871b69329014f93a19cac955964598 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 30 Jun 2020 12:30:25 -0700 Subject: [PATCH 045/871] Add Lysandros Nikolaou to the developer log --- developers.csv | 1 + 1 file changed, 1 insertion(+) diff --git a/developers.csv b/developers.csv index acb284cbd..0a639f983 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Lysandros Nikolaou,lysnikolaou,2020-06-29 Kyle Stanley,aeros,2020-04-14 Dong-hee Na,corona10,2020-04-08 Karthikeyan Singaravelan,tirkarthi,2019-12-31 From 8736d210acc856bc697cd8e8041eb87dd1cd7329 Mon Sep 17 00:00:00 2001 From: Ethan Furman Date: Wed, 22 Jul 2020 11:12:24 -0700 Subject: [PATCH 046/871] Remove Rhodri per his request --- experts.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/experts.rst b/experts.rst index f5839f371..ac67bf3cc 100644 --- a/experts.rst +++ b/experts.rst @@ -69,8 +69,8 @@ bisect rhettinger* builtins bz2 calendar rhettinger* -cgi ethan.furman*, Rhodri James -cgitb ethan.furman*, Rhodri James +cgi ethan.furman* +cgitb ethan.furman* chunk cmath mark.dickinson cmd From da685fb606ba28bb3c2eb4bcd126584c1545a464 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Mon, 27 Jul 2020 15:37:25 -0700 Subject: [PATCH 047/871] Update grammar.rst and compiler.rst to describe the PEG parser (#601) --- compiler.rst | 59 +++++++++++----------------------------------------- grammar.rst | 44 ++++++++++++++++----------------------- 2 files changed, 30 insertions(+), 73 deletions(-) diff --git a/compiler.rst b/compiler.rst index 3338cef4e..40384cb07 100644 --- a/compiler.rst +++ b/compiler.rst @@ -10,8 +10,8 @@ Abstract In CPython, the compilation from source code to bytecode involves several steps: -1. Parse source code into a parse tree (:file:`Parser/pgen.c`) -2. Transform parse tree into an Abstract Syntax Tree (:file:`Python/ast.c`) +1. Tokenize the source code (:file:`Parser/tokenizer.c`) +2. Parse the stream of tokens into an Abstract Syntax Tree (:file:`Parser/parser.c`) 3. Transform AST into a Control Flow Graph (:file:`Python/compile.c`) 4. Emit bytecode based on the Control Flow Graph (:file:`Python/compile.c`) @@ -23,49 +23,18 @@ in terms of the how the entire system works. You will most likely need to read some source to have an exact understanding of all details. -Parse Trees ------------ +Parsing +------- -Python's parser is an LL(1) parser mostly based off of the -implementation laid out in the Dragon Book [Aho86]_. +As of Python 3.9, Python's parser is a PEG parser of a somewhat +unusual design (since its input is a stream of tokens rather than a +stream of characters as is more common with PEG parsers). -The grammar file for Python can be found in :file:`Grammar/Grammar` with the -numeric value of grammar rules stored in :file:`Include/graminit.h`. The -list of types of tokens (literal tokens, such as ``:``, numbers, etc.) can -be found in :file:`Grammar/Tokens` with the numeric value stored in -:file:`Include/token.h`. The parse tree is made up -of ``node *`` structs (as defined in :file:`Include/node.h`). - -Querying data from the node structs can be done with the following -macros (which are all defined in :file:`Include/node.h`): - -``CHILD(node *, int)`` - Returns the nth child of the node using zero-offset indexing -``RCHILD(node *, int)`` - Returns the nth child of the node from the right side; use - negative numbers! -``NCH(node *)`` - Number of children the node has -``STR(node *)`` - String representation of the node; e.g., will return ``:`` for a - ``COLON`` token -``TYPE(node *)`` - The type of node as specified in :file:`Include/graminit.h` -``REQ(node *, TYPE)`` - Assert that the node is the type that is expected -``LINENO(node *)`` - Retrieve the line number of the source code that led to the - creation of the parse rule; defined in :file:`Python/ast.c` - -For example, consider the rule for 'while': - -.. productionlist:: - while_stmt: "while" `expression` ":" `suite` : ["else" ":" `suite`] - -The node representing this will have ``TYPE(node) == while_stmt`` and -the number of children can be 4 or 7 depending on whether there is an -'else' statement. ``REQ(CHILD(node, 2), COLON)`` can be used to access -what should be the first ``:`` and require it be an actual ``:`` token. +The grammar file for Python can be found in +:file:`Grammar/python.gram`. The definitions for literal tokens +(such as ``:``, numbers, etc.) can be found in :file:`Grammar/Tokens`. +Various C files, including :file:`Parser/parser.c` are generated from +these (see :doc:`grammar`). Abstract Syntax Trees (AST) @@ -569,10 +538,6 @@ thanks to having to support both classic and new-style classes. References ---------- -.. [Aho86] Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman. - `Compilers: Principles, Techniques, and Tools`, - https://www.amazon.com/exec/obidos/tg/detail/-/0201100886/104-0162389-6419108 - .. [Wang97] Daniel C. Wang, Andrew W. Appel, Jeff L. Korn, and Chris S. Serra. `The Zephyr Abstract Syntax Description Language.`_ In Proceedings of the Conference on Domain-Specific Languages, pp. diff --git a/grammar.rst b/grammar.rst index 4bdf3c54e..5f0f83d2e 100644 --- a/grammar.rst +++ b/grammar.rst @@ -7,22 +7,14 @@ Abstract -------- There's more to changing Python's grammar than editing -:file:`Grammar/Grammar`. This document aims to be a -checklist of places that must also be fixed. +:file:`Grammar/python.gram`. Here's a checklist. -It is probably incomplete. If you see omissions, submit a bug or patch. - -This document is not intended to be an instruction manual on Python -grammar hacking, for several reasons. - - -Rationale ---------- - -People are getting this wrong all the time; it took well over a -year before someone `noticed `_ -that adding the floor division -operator (``//``) broke the :mod:`parser` module. +NOTE: These instructions are for Python 3.9 and beyond. Earlier +versions use a different parser technology. You probably shouldn't +try to change the grammar of earlier Python versions, but if you +really want to, use GitHub to track down the earlier version of this +file in the devguide. (Python 3.9 itself actually supports both +parsers; the old parser can be invoked by passing ``-X oldparser``.) Checklist @@ -30,29 +22,29 @@ Checklist Note: sometimes things mysteriously don't work. Before giving up, try ``make clean``. -* :file:`Grammar/Grammar`: OK, you'd probably worked this one out. :-) After changing - it, run ``make regen-grammar``, to regenerate :file:`Include/graminit.h` and - :file:`Python/graminit.c`. (This runs Python's parser generator, ``Python/pgen``). +* :file:`Grammar/python.gram`: The grammar, with actions that build AST nodes. After changing + it, run ``make regen-pegen``, to regenerate :file:`Parser/parser.c`. + (This runs Python's parser generator, ``Tools/peg_generator``). * :file:`Grammar/Tokens` is a place for adding new token types. After changing it, run ``make regen-token`` to regenerate :file:`Include/token.h`, :file:`Parser/token.c`, :file:`Lib/token.py` and - :file:`Doc/library/token-list.inc`. If you change both ``Grammar`` and ``Tokens``, - run ``make regen-tokens`` before ``make regen-grammar``. + :file:`Doc/library/token-list.inc`. If you change both ``python.gram`` and ``Tokens``, + run ``make regen-token`` before ``make regen-pegen``. -* :file:`Parser/Python.asdl` may need changes to match the Grammar. Then run ``make +* :file:`Parser/Python.asdl` may need changes to match the grammar. Then run ``make regen-ast`` to regenerate :file:`Include/Python-ast.h` and :file:`Python/Python-ast.c`. * :file:`Parser/tokenizer.c` contains the tokenization code. This is where you would add a new type of comment or string literal, for example. -* :file:`Python/ast.c` will need changes to create the AST objects involved with the - Grammar change. +* :file:`Python/ast.c` will need changes to validate AST objects involved with the + grammar change. -* The :doc:`compiler` has its own page. +* :file:`Python/ast_unparse.c` will need changes to unparse AST objects involved with the + grammar change ("unparsing" is used to turn annotations into strings per :pep:`563`). -* The :mod:`parser` module. Add some of your new syntax to ``test_parser``, - bang on :file:`Modules/parsermodule.c` until it passes. +* The :doc:`compiler` has its own page. * ``_Unparser`` in the :file:`Lib/ast.py` file may need changes to accommodate any modifications in the AST nodes. From 1829b28f03c0ebdf8148df2fad4a8874dc721bc7 Mon Sep 17 00:00:00 2001 From: Lysandros Nikolaou Date: Tue, 28 Jul 2020 19:06:36 +0200 Subject: [PATCH 048/871] Triagers can also apply labels to Github Pull Requests (#602) --- committing.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/committing.rst b/committing.rst index 59757f2df..773dd66cc 100644 --- a/committing.rst +++ b/committing.rst @@ -294,9 +294,11 @@ Note that cherry_picker.py_ adds the branch prefix automatically. Once the backport pull request has been created, remove the ``needs backport to X.Y`` label from the original pull request. (Only Core -Developers can apply labels to GitHub pull requests). +Developers and members of the `Python Triage Team`_ can apply labels to GitHub +pull requests). .. _cherry_picker.py: https://github.com/python/cherry-picker +.. _`Python Triage Team`: https://devguide.python.org/triaging/#python-triage-team Reverting a Merged Pull Request From c67a0776448ad61ce8c3793a67252bba57e83a3d Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Thu, 13 Aug 2020 20:27:44 +0300 Subject: [PATCH 049/871] Escape dot to prevent
    (#604) --- developers.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developers.csv b/developers.csv index 0a639f983..75f08f85f 100644 --- a/developers.csv +++ b/developers.csv @@ -72,7 +72,7 @@ Frank Wierzbicki,,2009-08-02 Ezio Melotti,ezio-melotti,2009-06-07 Philip Jenvey,pjenvey,2009-05-07 Michael Foord,voidspace,2009-04-01 -R. David Murray,bitdancer,2009-03-30 +R\. David Murray,bitdancer,2009-03-30 Chris Withers,cjw296,2009-03-08 Tarek Ziadé,,2008-12-21 Hirokazu Yamamoto,,2008-08-12 From f7f816e0af7ab4b5ad12a713f210ea42f0f8f25e Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Thu, 13 Aug 2020 20:29:27 +0300 Subject: [PATCH 050/871] Sort table by Python version (#600) --- devcycle.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index 181d00315..d4bf5023f 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -136,10 +136,10 @@ For reference, here are the Python versions that most recently reached their end +------------------+--------------+----------------+----------------+----------------------------------+ | 3.0 | :pep:`361` | 2008-12-03 | 2009-06-27 | Barry Warsaw | +------------------+--------------+----------------+----------------+----------------------------------+ -| 2.6 | :pep:`361` | 2008-10-01 | 2013-10-29 | Barry Warsaw | -+------------------+--------------+----------------+----------------+----------------------------------+ | 2.7 | :pep:`373` | 2010-07-03 | 2020-01-01 | Benjamin Peterson | +------------------+--------------+----------------+----------------+----------------------------------+ +| 2.6 | :pep:`361` | 2008-10-01 | 2013-10-29 | Barry Warsaw | ++------------------+--------------+----------------+----------------+----------------------------------+ The latest release for each Python version can be found on the `download page `_. From 17077dc00a58eb8d39d16e7d9d2e5e7875cbc649 Mon Sep 17 00:00:00 2001 From: Stefan Pochmann <609905+pochmann@users.noreply.github.com> Date: Mon, 7 Sep 2020 19:03:05 +0200 Subject: [PATCH 051/871] Make minor editorial corrections in the garbage collector section (#608) --- garbage_collector.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index 444064919..0f93be4af 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -92,7 +92,7 @@ As is explained later in the `Optimization: reusing fields to save memory`_ sect these two extra fields are normally used to keep doubly linked lists of all the objects tracked by the garbage collector (these lists are the GC generations, more on that in the `Optimization: generations`_ section), but they are also -reused to fullfill other purposes when the full doubly linked list structure is not +reused to fulfill other purposes when the full doubly linked list structure is not needed as a memory optimization. Doubly linked lists are used because they efficiently support most frequently required operations. In @@ -117,7 +117,7 @@ that the objects cannot form reference cycles with only objects of its type or u the type is immutable, a ``tp_clear`` implementation must also be provided. -Identifiying reference cycles +Identifying reference cycles ---------------------------------------------- The algorithm that CPython uses to detect those reference cycles is @@ -164,7 +164,9 @@ is completely unreachable: >>> link_4 = Link() >>> link_4.next_link = link_4 + >>> del link_4 + # Collect the unreachable Link object (and its .__dict__ dict). >>> gc.collect() 2 @@ -285,7 +287,7 @@ follows these steps in order: 2. If an object has legacy finalizers (``tp_del`` slot) move them to the ``gc.garbage`` list. 3. Call the finalizers (``tp_finalize`` slot) and mark the objects as already - finalized to avoid calling them twice if they resurrect of if other finalizers + finalized to avoid calling them twice if they resurrect or if other finalizers have removed the object first. 4. Deal with resurrected objects. If some objects have been resurrected, the GC finds the new subset of objects that are still unreachable by running the cycle @@ -337,23 +339,23 @@ specifically in a generation by calling ``gc.collect(generation=NUM)``. ... pass ... - # Move everything to the last generation so its easier to inspect + # Move everything to the last generation so it's easier to inspect # the younger generations. >>> gc.collect() 0 - # Create a reference cycle + # Create a reference cycle. >>> x = MyObj() >>> x.self = x - # Initially the object is in the younguest generation. + # Initially the object is in the youngest generation. >>> gc.get_objects(generation=0) [..., <__main__.MyObj object at 0x7fbcc12a3400>, ...] - # After a collection of the younguest generation the object + # After a collection of the youngest generation the object # moves to the next generation. >>> gc.collect(generation=0) From 085d45d260c6ea9546c8fdc8565d3acfb5ec00eb Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 16 Sep 2020 11:29:14 -0700 Subject: [PATCH 052/871] Add Brandt Bucher as a core developer --- developers.csv | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/developers.csv b/developers.csv index 75f08f85f..5ffa76f86 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Brandt Bucher,brandtbucher,2020-09-14 Lysandros Nikolaou,lysnikolaou,2020-06-29 Kyle Stanley,aeros,2020-04-14 Dong-hee Na,corona10,2020-04-08 @@ -72,7 +73,7 @@ Frank Wierzbicki,,2009-08-02 Ezio Melotti,ezio-melotti,2009-06-07 Philip Jenvey,pjenvey,2009-05-07 Michael Foord,voidspace,2009-04-01 -R\. David Murray,bitdancer,2009-03-30 +R. David Murray,bitdancer,2009-03-30 Chris Withers,cjw296,2009-03-08 Tarek Ziadé,,2008-12-21 Hirokazu Yamamoto,,2008-08-12 From 87bc36f8feb84866db0cd49b7c1831344672eea5 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 30 Sep 2020 11:48:58 +0200 Subject: [PATCH 053/871] developers.csv: Escape dot to prevent
      (GH-614) Re-apply commit 0427168c1ae4f69aff299fa215dd2ae7aaa0ff15. This time the issue was fixed in the script that generates the data, so it shouldn't reappear. Co-authored-by: Hugo van Kemenade --- developers.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developers.csv b/developers.csv index 5ffa76f86..1040bb9a7 100644 --- a/developers.csv +++ b/developers.csv @@ -73,7 +73,7 @@ Frank Wierzbicki,,2009-08-02 Ezio Melotti,ezio-melotti,2009-06-07 Philip Jenvey,pjenvey,2009-05-07 Michael Foord,voidspace,2009-04-01 -R. David Murray,bitdancer,2009-03-30 +R\. David Murray,bitdancer,2009-03-30 Chris Withers,cjw296,2009-03-08 Tarek Ziadé,,2008-12-21 Hirokazu Yamamoto,,2008-08-12 From 334b448782eaef7b85784ade008b2ef59c273e37 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=81lvaro=20Mond=C3=A9jar?= Date: Thu, 1 Oct 2020 15:09:05 +0200 Subject: [PATCH 054/871] Update Spanish translation repository link (GH-610) Spanish translation of Python documentation is located at new repository under `python` organization. --- documenting.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documenting.rst b/documenting.rst index 8145ecd7e..4b1b76bce 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1661,7 +1661,7 @@ in production, other are work in progress: .. _doc_zh_tw: https://docs.python.org/zh-tw/ .. _github_ar: https://github.com/Abdur-rahmaanJ/py-docs-ar .. _github_bn_in: https://github.com/python/python-docs-bn-in -.. _github_es: https://github.com/PyCampES/python-docs-es +.. _github_es: https://github.com/python/python-docs-es .. _github_fr: https://github.com/python/python-docs-fr .. _github_hi_in: https://github.com/CuriousLearner/python-docs-hi-in .. _github_hu: https://github.com/python/python-docs-hu From e94f51612947447c73cbfb82193dc03fb200ec7e Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Thu, 1 Oct 2020 23:27:56 +0300 Subject: [PATCH 055/871] Python 3.5 EOL is end of September 2020 (#609) --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 0da88ad01..3a945498a 100644 --- a/index.rst +++ b/index.rst @@ -103,7 +103,7 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.6 | :pep:`494` | security | 2016-12-23 | *2021-12-23* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-13* | Larry Hastings | +| 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-30* | Larry Hastings | +------------------+--------------+-------------+----------------+----------------+-----------------------+ .. Remember to update the end-of-life table in devcycle.rst. From b0be57c453fe0b50e4966a8ddcf8d21c940f4591 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Sun, 4 Oct 2020 22:18:07 +0300 Subject: [PATCH 056/871] Python 3.5 is now EOL (#616) --- devcycle.rst | 2 ++ index.rst | 2 -- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index d4bf5023f..aa510d145 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -126,6 +126,8 @@ For reference, here are the Python versions that most recently reached their end +------------------+--------------+----------------+----------------+----------------------------------+ | Branch | Schedule | First release | End-of-life | Release manager | +==================+==============+================+================+==================================+ +| 3.5 | :pep:`478` | 2015-09-13 | 2020-09-30 | Larry Hastings | ++------------------+--------------+----------------+----------------+----------------------------------+ | 3.4 | :pep:`429` | 2014-03-16 | 2019-03-18 | Larry Hastings | +------------------+--------------+----------------+----------------+----------------------------------+ | 3.3 | :pep:`398` | 2012-09-29 | 2017-09-29 | Georg Brandl, Ned Deily (3.3.7+) | diff --git a/index.rst b/index.rst index 3a945498a..8248919fd 100644 --- a/index.rst +++ b/index.rst @@ -103,8 +103,6 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.6 | :pep:`494` | security | 2016-12-23 | *2021-12-23* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.5 | :pep:`478` | security | 2015-09-13 | *2020-09-30* | Larry Hastings | -+------------------+--------------+-------------+----------------+----------------+-----------------------+ .. Remember to update the end-of-life table in devcycle.rst. From 9203dc084f764f8be55a2fe477d06961d8a80629 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Tue, 20 Oct 2020 00:24:19 +0300 Subject: [PATCH 057/871] Python 3.9.0 was released on schedule (#621) https://www.python.org/downloads/release/python-390/ --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 8248919fd..ad22ed8ac 100644 --- a/index.rst +++ b/index.rst @@ -95,7 +95,7 @@ Status of Python branches +==================+==============+=============+================+================+=======================+ | master | :pep:`619` | features | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.9 | :pep:`596` | bugfix | *2020-10-05* | *TBD* | Łukasz Langa | +| 3.9 | :pep:`596` | bugfix | 2020-10-05 | *TBD* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.8 | :pep:`569` | bugfix | 2019-10-14 | *2024-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ From 1b481c8f25a79bfc28e5b2155687c655c37f5f0c Mon Sep 17 00:00:00 2001 From: Julien Palard Date: Mon, 19 Oct 2020 23:26:25 +0200 Subject: [PATCH 058/871] pl translation migrated to python org. (#606) --- documenting.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documenting.rst b/documenting.rst index 4b1b76bce..e2bcc12bd 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1622,7 +1622,7 @@ in production, other are work in progress: +-----------------+-------------------------------+----------------------------+ | Lithuanian (lt) | | `mail `_ | +-----------------+-------------------------------+----------------------------+ -| Polish (pl) | | `mail `_ | +| Polish (pl) | | `GitHub `_ | | | | `Translations `_ | +-----------------+-------------------------------+----------------------------+ | Portuguese (pt) | Gustavo Toffo | | @@ -1668,6 +1668,7 @@ in production, other are work in progress: .. _github_id: https://github.com/python/python-docs-id .. _github_ja: https://github.com/python/python-docs-ja .. _github_ko: https://github.com/python/python-docs-ko +.. _github_pl: https://github.com/python/python-docs-pl .. _github_pt_br: https://github.com/python/python-docs-pt-br .. _github_tr: https://github.com/alaeddingurel/python-docs-tr .. _github_zh_cn: https://github.com/python/python-docs-zh-cn @@ -1675,7 +1676,6 @@ in production, other are work in progress: .. _list_hu: https://mail.python.org/pipermail/python-hu .. _mail_it: https://mail.python.org/pipermail/doc-sig/2019-April/004114.html .. _mail_lt: https://mail.python.org/pipermail/doc-sig/2019-July/004138.html -.. _mail_pl: https://mail.python.org/pipermail/doc-sig/2019-April/004106.html .. _mail_ru: https://mail.python.org/pipermail/doc-sig/2019-May/004131.html .. _tx_pl: https://www.transifex.com/python-doc/python-newest/language/pl/ .. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/language/ From b662dafe6e8b156a9885cad7dbc04a8846ef80d4 Mon Sep 17 00:00:00 2001 From: m-aciek Date: Mon, 19 Oct 2020 23:36:14 +0200 Subject: [PATCH 059/871] Add links to GitHub repository and builds on docs.python.org for Polish docs translations (#607) Remove e-mail link Co-authored-by: Maciej Olko Co-authored-by: Carol Willing --- documenting.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/documenting.rst b/documenting.rst index e2bcc12bd..ab96e7f51 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1624,6 +1624,7 @@ in production, other are work in progress: +-----------------+-------------------------------+----------------------------+ | Polish (pl) | | `GitHub `_ | | | | `Translations `_ | +| | | `Doc `_ | +-----------------+-------------------------------+----------------------------+ | Portuguese (pt) | Gustavo Toffo | | +-----------------+-------------------------------+----------------------------+ @@ -1657,6 +1658,7 @@ in production, other are work in progress: .. _chat_pt_br: https://t.me/pybr_i18n .. _doc_ja: https://docs.python.org/ja/ .. _doc_ko: https://docs.python.org/ko/ +.. _doc_pl: https://docs.python.org/pl/ .. _doc_zh_cn: https://docs.python.org/zh-cn/ .. _doc_zh_tw: https://docs.python.org/zh-tw/ .. _github_ar: https://github.com/Abdur-rahmaanJ/py-docs-ar From 82fefa221c654c88c9d2ed0069a1ccc5a3a442bc Mon Sep 17 00:00:00 2001 From: Ned Deily Date: Mon, 19 Oct 2020 17:37:35 -0400 Subject: [PATCH 060/871] Clarify branch/release terminology in Development Cycle page. (#598) Try to be more consistent about use of major.minor.micro version numbers and of feature, bugfix, and security releases. Reflect current policy and practice for branch transitions as of 3.9. Other minor cleanups. --- devcycle.rst | 40 +++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index aa510d145..858fa5514 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -67,35 +67,39 @@ Maintenance branches -------------------- A branch for a previous feature release, currently being maintained for bug -fixes. There are usually two maintenance branches at any given time for -Python 3.x. Only during the beta/rc phase of a new -minor/feature release will there be three active maintenance branches, e.g. -during the beta phase for Python 3.8 there are master, 3.8, 3.7, and 3.6 -branches open. Releases -produced from a maintenance branch are called **maintenance** or **bugfix** +fixes, or for the next feature release in its +:ref:`beta ` or :ref:`release candidate ` stages. +There is usually either one or two maintenance branches at any given time for +Python 3.x. After the final release of a new minor version (3.x.0), releases +produced from a maintenance branch are called **bugfix** or **maintenance** releases; the terms are used interchangeably. These releases have a **micro version** number greater than zero. The only changes allowed to occur in a maintenance branch without debate are bug fixes. Also, a general rule for maintenance branches is that compatibility -must not be broken at any point between sibling minor releases (3.5.1, 3.5.2, +must not be broken at any point between sibling micro releases (3.5.1, 3.5.2, etc.). For both rules, only rare exceptions are accepted and **must** be discussed first. -Sometime after a new maintenance branch is created (after a new *minor version* -is released), the old maintenance branch on that major version will go into -:ref:`security mode `, -usually after one last maintenance release at the discretion of the +A new maintenance branch is normally created when the next feature release +cycle reaches feature freeze, i.e. at its first beta pre-release. +From that point on, changes intended for remaining pre-releases, the final +release (3.x.0), and subsequent bugfix releases are merged to +that maintenance branch. + +Sometime following the final release (3.x.0), the maintenance branch for +the previous minor version will go into :ref:`security mode `, +usually after at least one more bugfix release at the discretion of the release manager. For example, the 3.4 maintenance branch was put into -:ref:`security mode ` after the 3.4.4 final maintenance release -following the release of 3.5.1. +:ref:`security mode ` after the 3.4.4 bugfix release +which followed the release of 3.5.1. .. _secbranch: Security branches ----------------- -A branch less than 5 years old but no longer in maintenance mode is a security +A branch less than 5 years old but no longer in bugfix mode is a security branch. The only changes made to a security branch are those fixing issues exploitable @@ -107,6 +111,7 @@ since it is important to be able to run the tests successfully before releasing. Commits to security branches are to be coordinated with the release manager for the corresponding feature version, as listed in the :ref:`branchstatus`. +Merging of pull requests to security branches is restricted to release managers. Any release made from a security branch is source-only and done only when actual security patches have been applied to the branch. These releases have a **micro version** number greater than the last **bugfix** release. @@ -307,9 +312,12 @@ Current Administrators +-------------------+----------------------------------------------------------+-----------------+ | Name | Role | GitHub Username | +===================+==========================================================+=================+ +| Pablo Galindo | Python 3.10 and 3.11 Release Manager, | pablogsal | +| | Maintainer of buildbot.python.org | | ++-------------------+----------------------------------------------------------+-----------------+ | Łukasz Langa | Python 3.8 and 3.9 Release Manager | ambv | +-------------------+----------------------------------------------------------+-----------------+ -| Ned Deily | Python 3.7 Release Manager | ned-deily | +| Ned Deily | Python 3.6 and 3.7 Release Manager | ned-deily | +-------------------+----------------------------------------------------------+-----------------+ | Lary Hastings | Python 3.5 Release Manager | larryhastings | +-------------------+----------------------------------------------------------+-----------------+ @@ -321,8 +329,6 @@ Current Administrators +-------------------+----------------------------------------------------------+-----------------+ | Mariatta Wijaya | Maintainer of blurb_it and miss-islington | Mariatta | +-------------------+----------------------------------------------------------+-----------------+ -| Pablo Galindo | Maintainer of buildbot.python.org | pablogsal | -+-------------------+----------------------------------------------------------+-----------------+ Repository Release Manager Role Policy -------------------------------------- From 98fb4e2aad4d911d0d5f4641989bfc7fa13096fd Mon Sep 17 00:00:00 2001 From: Eric Cousineau Date: Mon, 19 Oct 2020 17:40:01 -0400 Subject: [PATCH 061/871] help: Add link to Discourse instance (#596) * help: Add link to Discourse instance Add note that IRC channel may not have history, and thus may not be easily searchable for newcomers. (I could not easily find a discussion archive) * Address Mariatta's comments --- help.rst | 58 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 37 insertions(+), 21 deletions(-) diff --git a/help.rst b/help.rst index 2df8022dc..50e3b982a 100644 --- a/help.rst +++ b/help.rst @@ -12,6 +12,30 @@ Should you require help, there are a :ref:`variety of options available usage then please check the rest of this guide first as it should answer your question. +Discourse +--------- + +Python has a hosted `Discourse`_ instance. Be sure to visit the related Core +categories, such as +`Core Development `_ and +`Core Workflow `_. + +.. _Discourse: https://discuss.python.org/ + +Mailing Lists +------------- + +Further options for seeking assistance include the `python-ideas`_ and +`python-dev`_ mailing lists. Python-ideas contains discussion of speculative +Python language ideas for possible inclusion into the language. If an idea +gains traction it can then be discussed and honed to the point of becoming a +solid proposal and presented on python-dev. Python-dev contains discussion +of current Python design issues, release mechanics, and maintenance of +existing releases. These mailing lists are for questions involving the +development *of* Python, **not** for development *with* Python. + +.. _python-ideas: https://mail.python.org/mailman/listinfo/python-ideas +.. _python-dev: https://mail.python.org/mailman/listinfo/python-dev Ask #python-dev --------------- @@ -19,12 +43,16 @@ Ask #python-dev If you are comfortable with IRC you can try asking on ``#python-dev`` (on the `freenode`_ network). Typically there are a number of experienced developers, ranging from triagers to core developers, who can answer -questions about developing for Python. Just remember that ``#python-dev`` -is for questions involving the development *of* Python whereas ``#python`` -is for questions concerning development *with* Python. +questions about developing for Python. As with the mailing lists, +``#python-dev`` is for questions involving the development *of* Python +whereas ``#python`` is for questions concerning development *with* Python. -.. _freenode: https://freenode.net/ +.. note:: + + You may not be able to access the history of this channel, so it cannot + be used as a "knowledge base" of sorts. +.. _freenode: https://freenode.net/ Zulip ----- @@ -34,6 +62,11 @@ for asking help with core development, as well as core developers' office hour stream. It is preferred that you ask questions here first or schedule an office hour, before posting to python-dev mailing list or filing bugs. +.. warning:: + + This is no longer actively monitored by core devs. Consider asking your questions + on Discourse or on the `python-dev`_ mailing list. + .. _Zulip: https://python.zulipchat.com @@ -78,23 +111,6 @@ during office hours. .. _Mariatta's twitter: https://twitter.com/mariatta -Mailing Lists -------------- - -Further options for seeking assistance include the `python-ideas`_ and -`python-dev`_ mailing lists. Python-ideas contains discussion of speculative -Python language ideas for possible inclusion into the language. If an idea -gains traction it can then be discussed and honed to the point of becoming a -solid proposal and presented on python-dev. Python-dev contains discussion -of current Python design issues, release mechanics, and maintenance of -existing releases. As with ``#python-dev``, these mailing lists are for -questions involving the development *of* Python, **not** for development -*with* Python. - -.. _python-ideas: https://mail.python.org/mailman/listinfo/python-ideas -.. _python-dev: https://mail.python.org/mailman/listinfo/python-dev - - File a Bug ---------- From d5419d6a3da54d568789b1901a5e122f9a4e57e0 Mon Sep 17 00:00:00 2001 From: Tal Einat Date: Tue, 20 Oct 2020 00:40:59 +0300 Subject: [PATCH 062/871] Guido is no longer the BDFL :( (#550) --- langchanges.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/langchanges.rst b/langchanges.rst index 6a7d1c304..f7984d608 100644 --- a/langchanges.rst +++ b/langchanges.rst @@ -27,8 +27,8 @@ go through Python's stdlib and find examples of code which would benefit from your proposed change (which helps communicate the usefulness of your change to others). For further guidance, see :ref:`suggesting-changes`. -Your proposed change also needs to be *Pythonic*. While Guido is the only -person who can truly classify something as Pythonic, you can read the `Zen of +Your proposed change also needs to be *Pythonic*. While only the Steering +Council can truly classify something as Pythonic, you can read the `Zen of Python`_ for guidance. .. _Zen of Python: https://www.python.org/dev/peps/pep-0020/ From 234c3e3b9ead01a20a05302b8d9d91a3e228eb92 Mon Sep 17 00:00:00 2001 From: Krishna Date: Tue, 20 Oct 2020 03:13:49 +0530 Subject: [PATCH 063/871] Fix issue467- Use the imperative form for the bullet list (#542) * Fix issue467- Use the imperative form for the bullet list * fix nouns- cleanup, checkout --- buildbots.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/buildbots.rst b/buildbots.rst index c37b1355b..ce1f16845 100644 --- a/buildbots.rst +++ b/buildbots.rst @@ -15,10 +15,10 @@ will schedule a new build to be run as soon as possible. The build steps run by the buildbots are the following: -* Checkout of the source tree for the changeset which triggered the build -* Compiling Python -* Running the test suite using :ref:`strenuous settings ` -* Cleaning up the build tree +* Check out the source tree for the changeset which triggered the build +* Compile Python +* Run the test suite using :ref:`strenuous settings ` +* Clean up the build tree It is your responsibility, as a core developer, to check the automatic build results after you push a change to the repository. It is therefore From d69596dfa44c49624ef6f9b79ff0a8f7dac98a16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pavol=20Babin=C4=8D=C3=A1k?= Date: Mon, 19 Oct 2020 23:53:50 +0200 Subject: [PATCH 064/871] Improve steps to install coverage in Virtualenv on Windows (#336) (#584) --- coverage.rst | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/coverage.rst b/coverage.rst index b0d5e9c3c..a2069c675 100644 --- a/coverage.rst +++ b/coverage.rst @@ -84,14 +84,25 @@ Install Coverage By default, pip will not install into the in-development version of Python you just built, and this built version of Python will not see packages installed into your default version of Python. One option is to use a virtual environment -to install coverage:: +to install coverage. + +On Unix run:: ./python -m venv ../cpython-venv source ../cpython-venv/bin/activate pip install coverage -On :ref:`most ` Mac OS X systems, replace :file:`./python` -with :file:`./python.exe`. On Windows, use :file:`python.bat`. +On :ref:`most ` Mac OS X systems run:: + + ./python.exe -m venv ../cpython-venv + source ../cpython-venv/bin/activate + pip install coverage + +On Windows run:: + + python.bat -m venv ..\\cpython-venv + ..\\cpython-venv\\Scripts\\activate.bat + pip install coverage You can now use python without the ./ for the rest of these instructions, as long as your venv is activated. For more info on venv see `Virtual Environment From 8fb82a40bc92bf5af72ca7b6d88599a5335b0db6 Mon Sep 17 00:00:00 2001 From: lrjball <50599110+lrjball@users.noreply.github.com> Date: Mon, 19 Oct 2020 22:55:29 +0100 Subject: [PATCH 065/871] Updated docs for coverage c extension (#594) --- coverage.rst | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/coverage.rst b/coverage.rst index a2069c675..bc6c73bec 100644 --- a/coverage.rst +++ b/coverage.rst @@ -201,10 +201,15 @@ the encodings module). Do not worry if you can't get this to work or it doesn't make any sense; it's entirely optional and only important for a small number of modules. -If you still choose to try this, the first step is to build coverage.py's C -extension code. Assuming that coverage.py's clone is at ``COVERAGEDIR`` and -your clone of CPython is at ``CPYTHONDIR``, you execute the following in your -coverage.py clone:: +If you still choose to try this, the first step is to make sure coverage.py's +C extension is installed. You can check this with:: + + ./python COVERAGEDIR --version + +If it says 'without C extension', then you will need to build the C extension. +Assuming that coverage.py's clone is at ``COVERAGEDIR`` and your clone of CPython +is at ``CPYTHONDIR``, you can do this by executing the following in your coverage.py +clone:: CPPFLAGS="-I CPYTHONDIR -I CPYTHONDIR/Include" CPYTHONDIR/python setup.py build_ext --inplace From ac8a267f683e56b80303438e3a2f4f2225aebd73 Mon Sep 17 00:00:00 2001 From: Abhijeet Kasurde Date: Tue, 20 Oct 2020 03:34:02 +0530 Subject: [PATCH 066/871] Add information about blurb tool (#599) * Add information about blurb tool Add blurb tool process in quick reference guide. Signed-off-by: Abhijeet Kasurde * Update index.rst Co-authored-by: Carol Willing --- index.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/index.rst b/index.rst index ad22ed8ac..f62a7eef8 100644 --- a/index.rst +++ b/index.rst @@ -67,6 +67,11 @@ instructions please see the :ref:`setup guide `. bpo-12345: Fix some bug in spam module +8. Add a News entry into the ``Misc/NEWS.d`` directory as individual file. The + news entry can be created by using `blurb-it `_, + or the `blurb `_ tool and its ``blurb add`` + command. Please read more about ``blurb`` in :ref:`documentation `. + .. note:: First time contributors will need to sign the Contributor Licensing From 4d695e32d61b601f6b5185cb7244bcc2d33127f2 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 19 Oct 2020 15:17:38 -0700 Subject: [PATCH 067/871] Update the format of the developer log (#626) It now includes when someone left the team and any notes about their membership. --- developers.csv | 345 +++++++++++++++++++++++++------------------------ developers.rst | 2 +- 2 files changed, 174 insertions(+), 173 deletions(-) diff --git a/developers.csv b/developers.csv index 1040bb9a7..561775364 100644 --- a/developers.csv +++ b/developers.csv @@ -1,172 +1,173 @@ -Brandt Bucher,brandtbucher,2020-09-14 -Lysandros Nikolaou,lysnikolaou,2020-06-29 -Kyle Stanley,aeros,2020-04-14 -Dong-hee Na,corona10,2020-04-08 -Karthikeyan Singaravelan,tirkarthi,2019-12-31 -Joannah Nanjekye,nanjekyejoannah,2019-09-23 -Abhilash Raj,maxking,2019-08-06 -Paul Ganssle,pganssle,2019-06-15 -Stéphane Wirtel,matrixise,2019-04-08 -Stefan Behnel,scoder,2019-04-08 -Cheryl Sabella,csabella,2019-02-19 -Lisa Roach,lisroach,2018-09-14 -Emily Morehouse,emilyemorehouse,2018-09-14 -Pablo Galindo,pablogsal,2018-06-06 -Mark Shannon,markshannon,2018-05-15 -Petr Viktorin,encukou,2018-04-16 -Nathaniel J. Smith,njsmith,2018-01-25 -Julien Palard,JulienPalard,2017-12-08 -Ivan Levkivskyi,ilevkivskyi,2017-12-06 -Carol Willing,willingc,2017-05-24 -Mariatta,Mariatta,2017-01-27 -Xiang Zhang,zhangyangyu,2016-11-21 -Inada Naoki,methane,2016-09-26 -Xavier de Gaye,xdegaye,2016-06-03 -Davin Potts,applio,2016-03-06 -Martin Panter,vadmium,2015-08-10 -Paul Moore,pfmoore,2015-03-15 -Robert Collins,rbtcollins,2014-10-16 -Berker Peksağ,berkerpeksag,2014-06-26 -Steve Dower,zooba,2014-05-10 -Kushal Das,kushaldas,2014-04-14 -Steven D'Aprano,stevendaprano,2014-02-08 -Yury Selivanov,1st1,2014-01-23 -Zachary Ware,zware,2013-11-02 -Donald Stufft,dstufft,2013-08-14 -Ethan Furman,ethanfurman,2013-05-11 -Serhiy Storchaka,serhiy-storchaka,2012-12-26 -Chris Jerdonek,cjerdonek,2012-09-24 -Eric Snow,ericsnowcurrently,2012-09-05 -Peter Moody,,2012-05-20 -Hynek Schlawack,hynek,2012-05-14 -Richard Oudkerk,,2012-04-29 -Andrew Svetlov,asvetlov,2012-03-13 -Petri Lehtinen,akheron,2011-10-22 -Meador Inge,meadori,2011-09-19 -Jeremy Kloth,jkloth,2011-09-12 -Sandro Tosi,sandrotosi,2011-08-01 -Alex Gaynor,alex,2011-07-18 -Charles-François Natali,,2011-05-19 -Nadeem Vawda,,2011-04-10 -Jason R. Coombs,jaraco,2011-03-14 -Ross Lagerwall,,2011-03-13 -Eli Bendersky,eliben,2011-01-11 -Ned Deily,ned-deily,2011-01-09 -David Malcolm,davidmalcolm,2010-10-27 -Tal Einat,taleinat,2010-10-04 -Łukasz Langa,ambv,2010-09-08 -Daniel Stutzbach,,2010-08-22 -Éric Araujo,merwok,2010-08-10 -Brian Quinlan,brianquinlan,2010-07-26 -Alexander Belopolsky,abalkin,2010-05-25 -Tim Golden,tjguk,2010-04-21 -Giampaolo Rodolà,giampaolo,2010-04-17 -Jean-Paul Calderone,,2010-04-06 -Brian Curtin,briancurtin,2010-03-24 -Florent Xicluna,,2010-02-25 -Dino Viehland,DinoV,2010-02-23 -Larry Hastings,larryhastings,2010-02-22 -Victor Stinner,vstinner,2010-01-30 -Stefan Krah,skrah,2010-01-05 -Doug Hellmann,dhellmann,2009-09-20 -Frank Wierzbicki,,2009-08-02 -Ezio Melotti,ezio-melotti,2009-06-07 -Philip Jenvey,pjenvey,2009-05-07 -Michael Foord,voidspace,2009-04-01 -R\. David Murray,bitdancer,2009-03-30 -Chris Withers,cjw296,2009-03-08 -Tarek Ziadé,,2008-12-21 -Hirokazu Yamamoto,,2008-08-12 -Armin Ronacher,mitsuhiko,2008-07-23 -Antoine Pitrou,pitrou,2008-07-16 -Senthil Kumaran,orsenthil,2008-06-16 -Jesse Noller,,2008-06-16 -Jesús Cea,jcea,2008-05-13 -Guilherme Polo,,2008-04-24 -Jeroen Ruigrok van der Werven,,2008-04-12 -Benjamin Peterson,benjaminp,2008-03-25 -David Wolever,wolever,2008-03-17 -Trent Nelson,tpn,2008-03-17 -Mark Dickinson,mdickinson,2008-01-06 -Amaury Forgeot d'Arc,amauryfa,2007-11-09 -Christian Heimes,tiran,2007-10-31 -Bill Janssen,,2007-08-28 -Jeffrey Yasskin,,2007-08-09 -Mark Summerfield,,2007-08-01 -Alexandre Vassalotti,avassalotti,2007-05-21 -Travis E. Oliphant,,2007-04-17 -Eric V. Smith,ericvsmith,2007-02-28 -Josiah Carlson,,2007-01-06 -Collin Winter,,2007-01-05 -Richard Jones,,2006-05-23 -Kristján Valur Jónsson,,2006-05-17 -Jack Diederich,jackdied,2006-05-17 -Steven Bethard,,2006-04-27 -Gerhard Häring,,2006-04-23 -George Yoshida,,2006-04-17 -Ronald Oussoren,ronaldoussoren,2006-03-03 -Nick Coghlan,ncoghlan,2005-10-16 -Georg Brandl,birkenfeld,2005-05-28 -Terry Jan Reedy,terryjreedy,2005-04-07 -Bob Ippolito,,2005-03-02 -Peter Astrand,,2004-10-21 -Facundo Batista,facundobatista,2004-10-16 -Sean Reifschneider,,2004-09-17 -Johannes Gijsbers,,2004-08-14 -Matthias Klose,doko42,2004-08-04 -PJ Eby,pjeby,2004-03-24 -Vinay Sajip,vsajip,2004-02-20 -Hye-Shik Chang,,2003-12-10 -Armin Rigo,,2003-10-24 -Andrew McNamara,,2003-06-09 -Samuele Pedroni,,2003-05-16 -Alex Martelli,aleaxit,2003-04-22 -Brett Cannon,brettcannon,2003-04-18 -David Goodger,,2003-01-02 -Gustavo Niemeyer,,2002-11-05 -Tony Lownds,,2002-09-22 -Steve Holden,,2002-06-14 -Christian Tismer,ctismer,2002-05-17 -Jason Tishler,,2002-05-15 -Walter Dörwald,doerwalter,2002-03-21 -Andrew MacIntyre,,2002-02-17 -Gregory P. Smith,gpshead,2002-01-08 -Anthony Baxter,,2001-12-21 -Neal Norwitz,,2001-12-19 -Raymond Hettinger,rhettinger,2001-12-10 -Chui Tey,,2001-10-31 -Michael W. Hudson,,2001-08-27 -Finn Bock,,2001-08-23 -Piers Lauder,,2001-07-20 -Kurt B. Kaiser,kbkaiser,2001-07-03 -Steven M. Gava,,2001-06-25 -Steve Purcell,,2001-03-22 -Jim Fulton,,2000-10-06 -Ka-Ping Yee,,2000-10-03 -Lars Gustäbel,gustaebel,2000-09-21 -Neil Schemenauer,nascheme,2000-09-15 -Martin v. Löwis,,2000-09-08 -Thomas Heller,theller,2000-09-07 -Moshe Zadka,,2000-07-29 -Thomas Wouters,Yhg1s,2000-07-14 -Peter Schneider-Kamp,,2000-07-10 -Paul Prescod,,2000-07-01 -Tim Peters,tim-one,2000-06-30 -Skip Montanaro,smontanaro,2000-06-30 -Fredrik Lundh,,2000-06-29 -Mark Hammond,mhammond,2000-06-09 -Marc-André Lemburg,malemburg,2000-06-07 -Trent Mick,,2000-06-06 -Eric S. Raymond,,2000-06-02 -Greg Stein,,1999-11-07 -Just van Rossum,,1999-01-22 -Greg Ward,,1998-12-18 -Andrew Kuchling,akuchling,1998-04-09 -Ken Manheimer,,1998-03-03 -Jeremy Hylton,jeremyhylton,1997-08-13 -Roger E. Masse,,1996-12-09 -Fred Drake,freddrake,1996-07-23 -Barry Warsaw,warsaw,1994-07-25 -Jack Jansen,jackjansen,1992-08-13 -Sjoerd Mullender,sjoerdmullender,1992-08-04 -Guido van Rossum,gvanrossum,1989-12-25 +Brandt Bucher,brandtbucher,2020-09-14,, +Lysandros Nikolaou,lysnikolaou,2020-06-29,, +Kyle Stanley,aeros,2020-04-14,, +Dong-hee Na,corona10,2020-04-08,, +Karthikeyan Singaravelan,tirkarthi,2019-12-31,, +Joannah Nanjekye,nanjekyejoannah,2019-09-23,, +Abhilash Raj,maxking,2019-08-06,, +Paul Ganssle,pganssle,2019-06-15,, +Stéphane Wirtel,matrixise,2019-04-08,, +Stefan Behnel,scoder,2019-04-08,, +Cheryl Sabella,csabella,2019-02-19,, +Lisa Roach,lisroach,2018-09-14,, +Emily Morehouse,emilyemorehouse,2018-09-14,, +Pablo Galindo,pablogsal,2018-06-06,, +Mark Shannon,markshannon,2018-05-15,, +Petr Viktorin,encukou,2018-04-16,, +Nathaniel J. Smith,njsmith,2018-01-25,, +Julien Palard,JulienPalard,2017-12-08,, +Ivan Levkivskyi,ilevkivskyi,2017-12-06,, +Carol Willing,willingc,2017-05-24,, +Mariatta,Mariatta,2017-01-27,, +Xiang Zhang,zhangyangyu,2016-11-21,, +Inada Naoki,methane,2016-09-26,, +Xavier de Gaye,xdegaye,2016-06-03,2018-01-25,Privileges relinquished on 2018-01-25 +Davin Potts,applio,2016-03-06,, +Martin Panter,vadmium,2015-08-10,, +Paul Moore,pfmoore,2015-03-15,, +Robert Collins,rbtcollins,2014-10-16,,To work on unittest +Berker Peksağ,berkerpeksag,2014-06-26,, +Steve Dower,zooba,2014-05-10,, +Kushal Das,kushaldas,2014-04-14,, +Steven D'Aprano,stevendaprano,2014-02-08,,For statistics module +Yury Selivanov,1st1,2014-01-23,, +Zachary Ware,zware,2013-11-02,, +Donald Stufft,dstufft,2013-08-14,, +Ethan Furman,ethanfurman,2013-05-11,, +Serhiy Storchaka,serhiy-storchaka,2012-12-26,, +Chris Jerdonek,cjerdonek,2012-09-24,, +Eric Snow,ericsnowcurrently,2012-09-05,, +Peter Moody,,2012-05-20,2017-02-10,For ipaddress module; did not make GitHub transition +Hynek Schlawack,hynek,2012-05-14,, +Richard Oudkerk,,2012-04-29,2017-02-10,For multiprocessing module; did not make GitHub transition +Andrew Svetlov,asvetlov,2012-03-13,,At PyCon sprint +Petri Lehtinen,akheron,2011-10-22,, +Meador Inge,meadori,2011-09-19,, +Jeremy Kloth,jkloth,2011-09-12,, +Sandro Tosi,sandrotosi,2011-08-01,, +Alex Gaynor,alex,2011-07-18,,For PyPy compatibility (since expanded scope) +Charles-François Natali,,2011-05-19,2017-02-10,Did not make GitHub transition +Nadeem Vawda,,2011-04-10,2017-02-10,Did not make GitHub transition +Jason R. Coombs,jaraco,2011-03-14,,For sprinting on distutils2 +Ross Lagerwall,,2011-03-13,2017-02-10,Did not make GitHub transition +Eli Bendersky,eliben,2011-01-11,, +Ned Deily,ned-deily,2011-01-09,, +David Malcolm,davidmalcolm,2010-10-27,, +Tal Einat,taleinat,2010-10-04,,For IDLE +Łukasz Langa,ambv,2010-09-08,, +Daniel Stutzbach,,2010-08-22,2017-02-10,Did not make GitHub transition +Éric Araujo,merwok,2010-08-10,, +Brian Quinlan,brianquinlan,2010-07-26,,For work related to PEP 3148 +Alexander Belopolsky,abalkin,2010-05-25,, +Tim Golden,tjguk,2010-04-21,, +Giampaolo Rodolà,giampaolo,2010-04-17,, +Jean-Paul Calderone,,2010-04-06,2017-02-10,Did not make GitHub transition +Brian Curtin,briancurtin,2010-03-24,, +Florent Xicluna,,2010-02-25,2017-02-10,Did not make GitHub transition +Dino Viehland,DinoV,2010-02-23,,For IronPython compatibility +Larry Hastings,larryhastings,2010-02-22,, +Victor Stinner,vstinner,2010-01-30,, +Stefan Krah,skrah,2010-01-05,2020-10-07,For the decimal module +Doug Hellmann,dhellmann,2009-09-20,,For documentation +Frank Wierzbicki,,2009-08-02,2017-02-10,For Jython compatibility; did not make GitHub transition +Ezio Melotti,ezio-melotti,2009-06-07,,For documentation +Philip Jenvey,pjenvey,2009-05-07,,For Jython compatibility +Michael Foord,voidspace,2009-04-01,,For IronPython compatibility +R\. David Murray,bitdancer,2009-03-30,, +Chris Withers,cjw296,2009-03-08,, +Tarek Ziadé,,2008-12-21,2017-02-10,For distutils module; did not make GitHub transition +Hirokazu Yamamoto,,2008-08-12,2017-02-10,For Windows build; did not make GitHub transition +Armin Ronacher,mitsuhiko,2008-07-23,,For documentation toolset and ast module +Antoine Pitrou,pitrou,2008-07-16,, +Senthil Kumaran,orsenthil,2008-06-16,,For GSoC +Jesse Noller,,2008-06-16,2017-02-10,For multiprocessing module; did not make GitHub transition +Jesús Cea,jcea,2008-05-13,,For bsddb module +Guilherme Polo,,2008-04-24,2017-02-10,Did not make GitHub transition +Jeroen Ruigrok van der Werven,,2008-04-12,2017-02-10,For documentation; did not make GitHub transition +Benjamin Peterson,benjaminp,2008-03-25,,For bug triage +David Wolever,wolever,2008-03-17,,For 2to3 module +Trent Nelson,tpn,2008-03-17,, +Mark Dickinson,mdickinson,2008-01-06,,For maths-related work +Amaury Forgeot d'Arc,amauryfa,2007-11-09,, +Christian Heimes,tiran,2007-10-31,, +Bill Janssen,,2007-08-28,2017-02-10,For ssl module; did not make GitHub transition +Jeffrey Yasskin,,2007-08-09,2017-02-10,Did not make GitHub transition +Mark Summerfield,,2007-08-01,2017-02-10,For documentation; did not make GitHub transition +Alexandre Vassalotti,avassalotti,2007-05-21,,For GSoC +Travis E. Oliphant,,2007-04-17,2017-02-10,Did not make GitHub transition +Eric V. Smith,ericvsmith,2007-02-28,,For PEP 3101 in a sandbox +Josiah Carlson,,2007-01-06,2017-02-10,For asyncore and asynchat modules; did not make GitHub transition +Collin Winter,,2007-01-05,2017-02-10,For PEP access; did not make GitHub transition +Richard Jones,,2006-05-23,2017-02-10,For Need for Speed sprint; did not make GitHub transition +Kristján Valur Jónsson,,2006-05-17,2017-02-10,For Need for Speed sprint; did not make GitHub transition +Jack Diederich,jackdied,2006-05-17,,For Need for Speed sprint +Steven Bethard,,2006-04-27,2017-02-10,For PEP access and SourceForge maintenance; did not make GitHub transition +Gerhard Häring,,2006-04-23,2017-02-10,Did not make the GitHub transition +George Yoshida,,2006-04-17,2017-02-10,For tracker administration; did not make GitHub transition +Ronald Oussoren,ronaldoussoren,2006-03-03,,For Mac-related work +Nick Coghlan,ncoghlan,2005-10-16,, +Georg Brandl,birkenfeld,2005-05-28,, +Terry Jan Reedy,terryjreedy,2005-04-07,, +Bob Ippolito,,2005-03-02,2017-02-10,For Mac-related work; did not make GitHub transition +Peter Astrand,,2004-10-21,2017-02-10,Did not make GitHub transition +Facundo Batista,facundobatista,2004-10-16,, +Sean Reifschneider,,2004-09-17,2017-02-10,Did not make GitHub transition +Johannes Gijsbers,,2004-08-14,2005-07-27,Privileges relinquished on 2005-07-27 +Matthias Klose,doko42,2004-08-04,, +PJ Eby,pjeby,2004-03-24,, +Vinay Sajip,vsajip,2004-02-20,, +Hye-Shik Chang,,2003-12-10,2017-02-10,Did not make GitHub transition +Armin Rigo,,2003-10-24,2012,Privileges relinquished in 2012 +Andrew McNamara,,2003-06-09,2017-02-10,Did not make GitHub transition +Samuele Pedroni,,2003-05-16,2017-02-10,Did not make GitHub transition +Alex Martelli,aleaxit,2003-04-22,, +Brett Cannon,brettcannon,2003-04-18,, +David Goodger,,2003-01-02,2017-02-10,Did not make GitHub transition +Gustavo Niemeyer,,2002-11-05,2017-02-10,Did not make GitHub transition +Tony Lownds,,2002-09-22,2017-02-10,Did not make GitHub transition +Steve Holden,,2002-06-14,2017-02-10,"Relinquished privileges on 2005-04-07, + but granted again for Need for Speed sprint; did not make GitHub transition" +Christian Tismer,ctismer,2002-05-17,,For Need for Speed sprint +Jason Tishler,,2002-05-15,2017-02-10,Did not make GitHub transition +Walter Dörwald,doerwalter,2002-03-21,, +Andrew MacIntyre,,2002-02-17,2016-01-02,Privileges relinquished 2016-01-02 +Gregory P. Smith,gpshead,2002-01-08,, +Anthony Baxter,,2001-12-21,2017-02-10,Did not make GitHub transition +Neal Norwitz,,2001-12-19,2017-02-10,Did not make GitHub transition +Raymond Hettinger,rhettinger,2001-12-10,, +Chui Tey,,2001-10-31,2017-02-10,Did not make GitHub transition +Michael W. Hudson,,2001-08-27,2017-02-10,Did not make GitHub transition +Finn Bock,,2001-08-23,2005-04-13,Privileges relinquished on 2005-04-13 +Piers Lauder,,2001-07-20,2017-02-10,Did not make GitHub transition +Kurt B. Kaiser,kbkaiser,2001-07-03,, +Steven M. Gava,,2001-06-25,2017-02-10,Did not make GitHub transition +Steve Purcell,,2001-03-22,2017-02-10,Did not make GitHub transition +Jim Fulton,,2000-10-06,2017-02-10,Did not make GitHub transition +Ka-Ping Yee,,2000-10-03,2017-02-10,Did not make GitHub transition +Lars Gustäbel,gustaebel,2000-09-21,,For tarfile module +Neil Schemenauer,nascheme,2000-09-15,, +Martin v. Löwis,,2000-09-08,2017-02-10,Did not make GitHub transition +Thomas Heller,theller,2000-09-07,, +Moshe Zadka,,2000-07-29,2005-04-08,Privileges relinquished on 2005-04-08 +Thomas Wouters,Yhg1s,2000-07-14,, +Peter Schneider-Kamp,,2000-07-10,2017-02-10,Did not make GitHub transition +Paul Prescod,,2000-07-01,2005-04-30,Privileges relinquished on 2005-04-30 +Tim Peters,tim-one,2000-06-30,, +Skip Montanaro,smontanaro,2000-06-30,2015-04-21,Privileges relinquished 2015-04-21 +Fredrik Lundh,,2000-06-29,2017-02-10,Did not make GitHub transition +Mark Hammond,mhammond,2000-06-09,, +Marc-André Lemburg,malemburg,2000-06-07,, +Trent Mick,,2000-06-06,2017-02-10,Did not make GitHub transition +Eric S. Raymond,,2000-06-02,2017-02-10,Did not make GitHub transition +Greg Stein,,1999-11-07,2017-02-10,Did not make GitHub transition +Just van Rossum,,1999-01-22,2017-02-10,Did not make GitHub transition +Greg Ward,,1998-12-18,2017-02-10,Did not make GitHub transition +Andrew Kuchling,akuchling,1998-04-09,, +Ken Manheimer,,1998-03-03,2005-04-08,Privileges relinquished on 2005-04-08 +Jeremy Hylton,jeremyhylton,1997-08-13,, +Roger E. Masse,,1996-12-09,2017-02-10,Did not make GitHub transition +Fred Drake,freddrake,1996-07-23,, +Barry Warsaw,warsaw,1994-07-25,, +Jack Jansen,jackjansen,1992-08-13,, +Sjoerd Mullender,sjoerdmullender,1992-08-04,, +Guido van Rossum,gvanrossum,1989-12-25,, diff --git a/developers.rst b/developers.rst index 7c922ece2..160b7060e 100644 --- a/developers.rst +++ b/developers.rst @@ -8,7 +8,7 @@ master list is kept in a private repository due to containing sensitive contact information.) .. csv-table:: - :header: "Name", "GitHub username", "Joined" + :header: "Name", "GitHub username", "Joined", "Left", "Notes" :file: developers.csv :encoding: "utf-8" From 1961e2ab7cda0c6ce42b21eb84148c8972bc9543 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Tue, 20 Oct 2020 00:31:12 +0100 Subject: [PATCH 068/871] Fix typo in Triaging documentation (GH-627) --- triaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/triaging.rst b/triaging.rst index 027a7a01a..a7bfec465 100644 --- a/triaging.rst +++ b/triaging.rst @@ -93,7 +93,7 @@ Labels for PRs include: DO-NOT-MERGE Used on PRs to prevent miss-islington from being able - to automatically merge to pull request. This label is appropriate when a PR + to automatically merge the pull request. This label is appropriate when a PR has a non-trivial conflict with the branch it is being merged into. expert-asyncio From dcb8ec24d7f996ea9a9bee2a63f80ffc15f5e203 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Tue, 20 Oct 2020 18:01:26 -0700 Subject: [PATCH 069/871] Update issue templates --- .github/ISSUE_TEMPLATE/bug_report.md | 25 +++++++++++++++++++++++ .github/ISSUE_TEMPLATE/feature_request.md | 22 ++++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/bug_report.md create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 000000000..46c21cbe7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,25 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '' +labels: '' +assignees: '' + +--- + +> Note: This repo is for the Python devguide. If you are requesting an +enhancementfor the Python language or CPython interpreter, +then the CPython issue tracker is better +suited for this report: https://bugs.python.org + +**Describe the bug** +A clear and concise description of what the bug is. + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Additional context** +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 000000000..d9e580720 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,22 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '' +labels: '' +assignees: '' + +--- + +> Note: This repo is for the Python devguide. If you are requesting an +enhancement for the Python language or CPython interpreter, +then the CPython issue tracker is better +suited for this report: https://bugs.python.org + +**Describe the enhancement or feature you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. From 87b65487ae543e8625a53a5bae764d78696becf3 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Tue, 20 Oct 2020 18:35:42 -0700 Subject: [PATCH 070/871] Add contributing, issue/pr templates, and reformat CoC (#482) * Convert CoC to markdown for GH insights * Add contributing file * add PR template * add issue template * add an issue template and format * add .github directory to sphinx exclude * add link * update link and text * update link * remove old style issue template * add ignore .github to conf exclude directory * make contributing markdown to conform to github.amrom.workers.devmunity standard * correct link style * convert to md --- ...CODE_OF_CONDUCT.rst => CODE_OF_CONDUCT.md} | 9 ++-- .github/CONTRIBUTING.md | 45 +++++++++++++++++++ .github/PULL_REQUEST_TEMPLATE.md | 11 +++++ conf.py | 2 +- 4 files changed, 60 insertions(+), 7 deletions(-) rename .github/{CODE_OF_CONDUCT.rst => CODE_OF_CONDUCT.md} (53%) create mode 100644 .github/CONTRIBUTING.md create mode 100644 .github/PULL_REQUEST_TEMPLATE.md diff --git a/.github/CODE_OF_CONDUCT.rst b/.github/CODE_OF_CONDUCT.md similarity index 53% rename from .github/CODE_OF_CONDUCT.rst rename to .github/CODE_OF_CONDUCT.md index 05b033db7..ab0fa8421 100644 --- a/.github/CODE_OF_CONDUCT.rst +++ b/.github/CODE_OF_CONDUCT.md @@ -1,13 +1,10 @@ -:orphan: - Code of Conduct =============== Please note that all interactions on -`Python Software Foundation `__-supported -infrastructure is `covered -`__ -by the `PSF Code of Conduct `__, +[Python Software Foundation](https://www.python.org/psf-landing/)-supported +infrastructure is [covered](https://www.python.org/psf/records/board/minutes/2014-01-06/#management-of-the-psfs-web-properties) +by the [PSF Code of Conduct](https://www.python.org/psf/codeofconduct/), which includes all infrastructure used in the development of Python itself (e.g. mailing lists, issue trackers, GitHub, etc.). diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 000000000..0f8320d35 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,45 @@ +# Contributing to the Devguide + +## Thank You + +First off, thanks for contributing to the devguide of the Python programming +language! Even if your contribution is not ultimately accepted, the fact you +put time and effort into helping out is greatly appreciated. + + +## Contribution Guidelines + +Please read this [devguide](https://devguide.python.org/) for +guidance on how to contribute to this project. The documentation covers +everything from how to build the code to submitting a pull request. There are +also suggestions on how you can most effectively help the project. + +Please be aware that our workflow does deviate slightly from the typical GitHub +project. Details on how to properly submit a pull request are covered in +[Lifecycle of a Pull Request](https://devguide.python.org/pullrequest/). +We utilize various bots and status checks to help with this, so do follow the +comments they leave and their "Details" links, respectively. The key points of +our workflow that are not covered by a bot or status check are: + +- All discussions that are not directly related to the code in the pull request + should happen on bugs.python.org +- Upon your first non-trivial pull request (which includes documentation changes), + feel free to add yourself to [`Misc/ACKS`](https://github.com/python/cpython/blob/master/Misc/ACKS) + + +## Setting Expectations + +Due to the fact that this project is entirely volunteer-run (i.e. no one is paid +to work on Python full-time), we unfortunately can make no guarantees as to if +or when a core developer will get around to reviewing your pull request. +If no core developer has done a review or responded to changes made because of a +"changes requested" review, please feel free to email python-dev to ask if +someone could take a look at your pull request. + + +## Code of Conduct + +All interactions for this project are covered by the +[PSF Code of Conduct](https://www.python.org/psf/codeofconduct/). Everyone is +expected to be open, considerate, and respectful of others no matter their +position within the project. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 000000000..839471ed0 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,11 @@ + \ No newline at end of file diff --git a/conf.py b/conf.py index 6c0cff11d..21b9e020b 100644 --- a/conf.py +++ b/conf.py @@ -67,7 +67,7 @@ # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. -exclude_patterns = ['_build', 'venv*', 'env*', 'README.rst'] +exclude_patterns = ['_build', 'venv*', 'env*', 'README.rst', '.github'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None From 310fa944075e8110f5a4a3b689cf59a2be3b99a6 Mon Sep 17 00:00:00 2001 From: Batuhan Taskaya Date: Wed, 21 Oct 2020 04:55:16 +0300 Subject: [PATCH 071/871] Remove deprecated lib2to3 from the grammar checklist (#620) --- grammar.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/grammar.rst b/grammar.rst index 5f0f83d2e..4a80fc1a6 100644 --- a/grammar.rst +++ b/grammar.rst @@ -55,7 +55,5 @@ Note: sometimes things mysteriously don't work. Before giving up, try ``make cl * :file:`Lib/tokenize.py` needs changes to match changes to the tokenizer. -* :file:`Lib/lib2to3/Grammar.txt` may need changes to match the Grammar. - * Documentation must be written! Specifically, one or more of the pages in :file:`Doc/reference/` will need to be updated. From f83559af26c8c67283319cb43d4cdf7a15d51fbb Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Wed, 21 Oct 2020 02:56:15 +0100 Subject: [PATCH 072/871] Add Mark Shannon to experts list. (#632) --- experts.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/experts.rst b/experts.rst index ac67bf3cc..ce1ed744b 100644 --- a/experts.rst +++ b/experts.rst @@ -309,12 +309,12 @@ Interest Area Maintainers ================== ========================================================== algorithms rhettinger* argument clinic larry -ast/compiler benjamin.peterson, brett.cannon, yselivanov, pablogsal +ast/compiler benjamin.peterson, brett.cannon, yselivanov, pablogsal, Mark.Shannon autoconf/makefiles twouters* bsd bug tracker ezio.melotti buildbots zach.ware, pablogsal -bytecode benjamin.peterson, yselivanov +bytecode benjamin.peterson, yselivanov, Mark.Shannon context managers ncoghlan core workflow mariatta coverity scan christian.heimes, brett.cannon, twouters @@ -338,7 +338,7 @@ memoryview skrah networking giampaolo.rodola, object model benjamin.peterson, twouters packaging tarek, lemburg, alexis, eric.araujo, dstufft, paul.moore -performance brett.cannon, vstinner, serhiy.storchaka, yselivanov, rhettinger +performance brett.cannon, vstinner, serhiy.storchaka, yselivanov, rhettinger, Mark.Shannon pip ncoghlan, dstufft, paul.moore, Marcus.Smith, pradyunsg py3 transition benjamin.peterson release management tarek, lemburg, benjamin.peterson, barry, From e8021d1dffcda2036207eebfa92d00b0602de09b Mon Sep 17 00:00:00 2001 From: Inada Naoki Date: Thu, 22 Oct 2020 07:26:27 +0900 Subject: [PATCH 073/871] Add note about formatting-only PR (#635) * Add note about formatting-only PR * Update pullrequest.rst Co-authored-by: Petr Viktorin * Update pullrequest.rst * Update pullrequest.rst Co-authored-by: Terry Jan Reedy Co-authored-by: Petr Viktorin Co-authored-by: Carol Willing Co-authored-by: Terry Jan Reedy --- pullrequest.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/pullrequest.rst b/pullrequest.rst index 4548e2ccf..ad6d78c1c 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -185,6 +185,11 @@ one or two discrepancies those can be fixed by the core developer who merges your pull request. But if you have systematic deviations from the style guides your pull request will be put on hold until you fix the formatting issues. +.. note:: + Pull requests with only code formatting changes are usually rejected. On the other + hand, fixes for typos and grammar errors in documents and docstrings are + welcome. + Second, be aware of backwards-compatibility considerations. While the core developer who eventually handles your pull request will make the final call on whether something is acceptable, thinking about backwards-compatibility early From d15b128c19384c4dcd25f74486a86f79c800b522 Mon Sep 17 00:00:00 2001 From: kj <28750310+Fidget-Spinner@users.noreply.github.com> Date: Thu, 22 Oct 2020 06:35:12 +0800 Subject: [PATCH 074/871] Update experts index (#634) --- experts.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/experts.rst b/experts.rst index ce1ed744b..2c1ca3bd5 100644 --- a/experts.rst +++ b/experts.rst @@ -50,7 +50,7 @@ __future__ __main__ gvanrossum, ncoghlan _dummy_thread brett.cannon _thread -_testbuffer skrah +_testbuffer abc aifc r.david.murray argparse rhettinger* @@ -96,7 +96,7 @@ curses twouters dataclasses eric.smith datetime belopolsky, p-ganssle dbm -decimal facundobatista, rhettinger, mark.dickinson, skrah +decimal facundobatista, rhettinger, mark.dickinson difflib tim.peters (inactive) dis yselivanov distutils eric.araujo, dstufft @@ -141,7 +141,7 @@ itertools rhettinger* json bob.ippolito (inactive), ezio.melotti, rhettinger keyword lib2to3 benjamin.peterson -libmpdec skrah +libmpdec linecache locale lemburg logging vinay.sajip @@ -334,7 +334,7 @@ io benjamin.peterson, stutzbach locale lemburg mathematics mark.dickinson, lemburg, stutzbach, rhettinger memory management tim.peters, lemburg, twouters -memoryview skrah +memoryview networking giampaolo.rodola, object model benjamin.peterson, twouters packaging tarek, lemburg, alexis, eric.araujo, dstufft, paul.moore From 69d74a6732a478c8a73e6c1c676e79bd9505bec8 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Thu, 22 Oct 2020 01:36:12 +0300 Subject: [PATCH 075/871] Link to bugs.python.org and python-dev (#633) --- .github/CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 0f8320d35..87686d966 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -22,7 +22,7 @@ comments they leave and their "Details" links, respectively. The key points of our workflow that are not covered by a bot or status check are: - All discussions that are not directly related to the code in the pull request - should happen on bugs.python.org + should happen on [bugs.python.org](https://bugs.python.org/) - Upon your first non-trivial pull request (which includes documentation changes), feel free to add yourself to [`Misc/ACKS`](https://github.com/python/cpython/blob/master/Misc/ACKS) @@ -33,7 +33,7 @@ Due to the fact that this project is entirely volunteer-run (i.e. no one is paid to work on Python full-time), we unfortunately can make no guarantees as to if or when a core developer will get around to reviewing your pull request. If no core developer has done a review or responded to changes made because of a -"changes requested" review, please feel free to email python-dev to ask if +"changes requested" review, please feel free to email [python-dev](https://mail.python.org/mailman3/lists/python-dev.python.org/) to ask if someone could take a look at your pull request. From 39e5fbff318bdf58ca41527afe088177997f9bcb Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Thu, 22 Oct 2020 01:37:02 +0300 Subject: [PATCH 076/871] Fix RemovedInSphinx40Warning: The app.add_stylesheet() is deprecated. Please use app.add_css_file() instead (#631) --- conf.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/conf.py b/conf.py index 21b9e020b..407109588 100644 --- a/conf.py +++ b/conf.py @@ -237,7 +237,8 @@ '\/.*', ] + # Use our custom CSS stylesheet to differentiate us from the official python # docs. def setup(app): - app.add_stylesheet('custom.css') + app.add_css_file('custom.css') From 0573e74f69723ae31aceb1d572043a25eaa9df45 Mon Sep 17 00:00:00 2001 From: Aliya Rahmani <65673692+aliya-rahmani@users.noreply.github.com> Date: Thu, 22 Oct 2020 04:42:26 +0530 Subject: [PATCH 077/871] Updated zulip to discourse #625 (#630) * Updated zulip to discourse * Update README.rst * Closes #625 Co-authored-by: Hugo van Kemenade Co-authored-by: Hugo van Kemenade --- README.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/README.rst b/README.rst index 2b4e26dd1..f083d9d0f 100644 --- a/README.rst +++ b/README.rst @@ -1,15 +1,15 @@ The CPython Developer's Guide ============================= -|ReadTheDocs| |Zulip| |Codestyle| +|ReadTheDocs| |Discourse| |Codestyle| .. |ReadTheDocs| image:: https://readthedocs.org/projects/cpython-devguide/badge/ :target: https://devguide.python.org :alt: Documentation Status -.. |Zulip| image:: https://img.shields.io/badge/zulip-join_chat-brightgreen.svg - :alt: Python Zulip chat - :target: https://python.zulipchat.com +.. |Discourse| image:: https://img.shields.io/badge/discourse-join_chat-brightgreen.svg + :alt: Python Discourse chat + :target: https://discuss.python.org/ .. |Codestyle| image:: https://img.shields.io/badge/code%20style-black-000000.svg :target: https://github.com/psf/black @@ -29,4 +29,3 @@ supports the ``venv`` module, because the ``make html`` command will create a virtual environment and will install the ``Sphinx`` package:: make html - From d2b9563a675dd82802ae7c560a9d8e61eeda3ec6 Mon Sep 17 00:00:00 2001 From: Matti Picus Date: Thu, 22 Oct 2020 02:25:25 +0300 Subject: [PATCH 078/871] Add more information about blurb (#629) * Add more information about blurb * Update pullrequest.rst Co-authored-by: Hugo van Kemenade Co-authored-by: Hugo van Kemenade --- pullrequest.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pullrequest.rst b/pullrequest.rst index ad6d78c1c..ef85923c0 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -243,6 +243,8 @@ The automated patch checklist runs through: * Has the documentation been updated? * Has the test suite been updated? * Has an entry under ``Misc/NEWS.d/next`` been added? + (using `blurb-it `_, + or the `blurb `_ tool) * Has ``Misc/ACKS`` been updated? * Has ``configure`` been regenerated, if necessary? * Has ``pyconfig.h.in`` been regenerated, if necessary? From 178606f576033c5566294816904404c712b11a63 Mon Sep 17 00:00:00 2001 From: Lysandros Nikolaou Date: Mon, 26 Oct 2020 18:02:39 +0200 Subject: [PATCH 079/871] Add PEG Parser experts (#637) --- experts.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/experts.rst b/experts.rst index 2c1ca3bd5..e33768417 100644 --- a/experts.rst +++ b/experts.rst @@ -280,6 +280,7 @@ Tools Tool Maintainers ================== =========== Argument Clinic larry +PEG Generator gvanrossum, pablogsal, lysnikolaou ================== =========== @@ -338,6 +339,7 @@ memoryview networking giampaolo.rodola, object model benjamin.peterson, twouters packaging tarek, lemburg, alexis, eric.araujo, dstufft, paul.moore +peg parser gvanrossum, pablogsal, lysnikolaou performance brett.cannon, vstinner, serhiy.storchaka, yselivanov, rhettinger, Mark.Shannon pip ncoghlan, dstufft, paul.moore, Marcus.Smith, pradyunsg py3 transition benjamin.peterson From 99b271cb18be3d1bb051255c4d9f05209d5032b4 Mon Sep 17 00:00:00 2001 From: Lysandros Nikolaou Date: Thu, 29 Oct 2020 00:14:19 +0200 Subject: [PATCH 080/871] Fix typo in my username in the experts file (#640) --- experts.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/experts.rst b/experts.rst index e33768417..52da0c987 100644 --- a/experts.rst +++ b/experts.rst @@ -280,7 +280,7 @@ Tools Tool Maintainers ================== =========== Argument Clinic larry -PEG Generator gvanrossum, pablogsal, lysnikolaou +PEG Generator gvanrossum, pablogsal, lys.nikolaou ================== =========== @@ -339,7 +339,7 @@ memoryview networking giampaolo.rodola, object model benjamin.peterson, twouters packaging tarek, lemburg, alexis, eric.araujo, dstufft, paul.moore -peg parser gvanrossum, pablogsal, lysnikolaou +peg parser gvanrossum, pablogsal, lys.nikolaou performance brett.cannon, vstinner, serhiy.storchaka, yselivanov, rhettinger, Mark.Shannon pip ncoghlan, dstufft, paul.moore, Marcus.Smith, pradyunsg py3 transition benjamin.peterson From 540b48ec540361d34a1306b330cb98824e39f10a Mon Sep 17 00:00:00 2001 From: Batuhan Taskaya Date: Thu, 29 Oct 2020 22:15:49 +0300 Subject: [PATCH 081/871] Add an entry for Doc/library/ast.rst in the grammar update file (#639) Co-authored-by: Petr Viktorin --- grammar.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/grammar.rst b/grammar.rst index 4a80fc1a6..73226e115 100644 --- a/grammar.rst +++ b/grammar.rst @@ -49,6 +49,8 @@ Note: sometimes things mysteriously don't work. Before giving up, try ``make cl * ``_Unparser`` in the :file:`Lib/ast.py` file may need changes to accommodate any modifications in the AST nodes. +* :file:`Doc/library/ast.rst` may need to be updated to reflect changes to AST nodes. + * Add some usage of your new syntax to ``test_grammar.py``. * Certain changes may require tweaks to the library module :mod:`pyclbr`. From 2a274d826c7e031e3d05659a0888c395994dadd9 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 9 Nov 2020 14:01:21 -0800 Subject: [PATCH 082/871] Add Batuhan Taskaya to the developer log --- developers.csv | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/developers.csv b/developers.csv index 561775364..48cc31309 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Batuhan Taskaya,isidentical,2020-11-08,, Brandt Bucher,brandtbucher,2020-09-14,, Lysandros Nikolaou,lysnikolaou,2020-06-29,, Kyle Stanley,aeros,2020-04-14,, @@ -125,7 +126,7 @@ Brett Cannon,brettcannon,2003-04-18,, David Goodger,,2003-01-02,2017-02-10,Did not make GitHub transition Gustavo Niemeyer,,2002-11-05,2017-02-10,Did not make GitHub transition Tony Lownds,,2002-09-22,2017-02-10,Did not make GitHub transition -Steve Holden,,2002-06-14,2017-02-10,"Relinquished privileges on 2005-04-07, +Steve Holden,,2002-06-14,2017-02-10,"Relinquished privileges on 2005-04-07, but granted again for Need for Speed sprint; did not make GitHub transition" Christian Tismer,ctismer,2002-05-17,,For Need for Speed sprint Jason Tishler,,2002-05-15,2017-02-10,Did not make GitHub transition From eb2359d3870b8e05c1c17924dc2c0b440fa1a7e1 Mon Sep 17 00:00:00 2001 From: Batuhan Taskaya Date: Tue, 10 Nov 2020 02:10:45 +0300 Subject: [PATCH 083/871] Add myself to the experts index (#642) --- experts.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/experts.rst b/experts.rst index 52da0c987..473e9f997 100644 --- a/experts.rst +++ b/experts.rst @@ -55,7 +55,7 @@ abc aifc r.david.murray argparse rhettinger* array -ast benjamin.peterson, pablogsal +ast benjamin.peterson, pablogsal, BTaskaya asynchat josiahcarlson, giampaolo.rodola*, stutzbach asyncio yselivanov, asvetlov asyncore josiahcarlson, giampaolo.rodola*, stutzbach @@ -182,7 +182,7 @@ pstats pty twouters* pwd py_compile -pyclbr +pyclbr BTaskaya pydoc queue rhettinger* quopri @@ -310,7 +310,7 @@ Interest Area Maintainers ================== ========================================================== algorithms rhettinger* argument clinic larry -ast/compiler benjamin.peterson, brett.cannon, yselivanov, pablogsal, Mark.Shannon +ast/compiler benjamin.peterson, brett.cannon, yselivanov, pablogsal, Mark.Shannon, BTaskaya autoconf/makefiles twouters* bsd bug tracker ezio.melotti From b6d8676d911bfb915f43003ce957cad142dba476 Mon Sep 17 00:00:00 2001 From: Abdur-Rahmaan Janhangeer Date: Wed, 11 Nov 2020 03:55:28 +0400 Subject: [PATCH 084/871] Changing Arabic coordinator (#641) --- documenting.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documenting.rst b/documenting.rst index ab96e7f51..b1a3151d8 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1595,7 +1595,7 @@ in production, other are work in progress: +-----------------+-------------------------------+----------------------------+ | Language | Contact | Links | +=================+===============================+============================+ -| Arabic (ar) | Abdur-Rahmaan Janhangeer | `GitHub `_ | +| Arabic (ar) | Ibrahim Elbouhissi | `GitHub `_ | +-----------------+-------------------------------+----------------------------+ | Bengali as | `Kushal Das `_ | `GitHub `_ | | spoken in | | | From bc37079135d8c3b30f7ddf3d42bcf3beea82c3cc Mon Sep 17 00:00:00 2001 From: E-Paine <63801254+E-Paine@users.noreply.github.com> Date: Tue, 1 Dec 2020 19:33:13 +0000 Subject: [PATCH 085/871] Improve instruction for updating with upstream (#646) Co-authored-by: Mariatta --- gitbootcamp.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/gitbootcamp.rst b/gitbootcamp.rst index 7a8405a22..bd5f8e441 100644 --- a/gitbootcamp.rst +++ b/gitbootcamp.rst @@ -237,8 +237,8 @@ Creating a Pull Request 6. Press the ``Create pull request`` button. -Syncing with Upstream ---------------------- +Updating your CPython Fork +-------------------------- Scenario: @@ -249,12 +249,19 @@ Scenario: - You now want to update your forked CPython repository to be the same as the upstream CPython repository. +Please do not try to solve this by creating a pull request from +``python:master`` to ``:master`` as the authors of the patches will +get notified unnecessarily. + Solution:: git checkout master git pull upstream master git push origin master +.. note:: For the above commands to work, please follow the instructions found + in the :ref:`checkout` section + Another scenario: - You created ``some-branch`` some time ago. From e5918c0a0558d1c530a49c916fb83de211faca2e Mon Sep 17 00:00:00 2001 From: larryhastings Date: Thu, 17 Dec 2020 09:29:22 -0800 Subject: [PATCH 086/871] Update Larry's status in the Administrators list (#636) --- devcycle.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/devcycle.rst b/devcycle.rst index 858fa5514..06c7eaddd 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -319,7 +319,7 @@ Current Administrators +-------------------+----------------------------------------------------------+-----------------+ | Ned Deily | Python 3.6 and 3.7 Release Manager | ned-deily | +-------------------+----------------------------------------------------------+-----------------+ -| Lary Hastings | Python 3.5 Release Manager | larryhastings | +| Lary Hastings | Retired Release Manager (for Python 3.4 and 3.5) | larryhastings | +-------------------+----------------------------------------------------------+-----------------+ | Berker Peksag | Maintainer of bpo-linkify and cpython-emailer-webhook | berkerpeksag | +-------------------+----------------------------------------------------------+-----------------+ From 5fea5796c1926bda7f728b4099ccb9630cbf5913 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Sat, 19 Dec 2020 10:41:51 -0800 Subject: [PATCH 087/871] Update my motivations entry --- motivations.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/motivations.rst b/motivations.rst index e5ed47930..50085000c 100644 --- a/motivations.rst +++ b/motivations.rst @@ -257,9 +257,10 @@ participating in the CPython core development process: .. topic:: Carol Willing (United States) + * Noteable: ``__ (Technical Evangelist) * Personal site: `Willing Consulting `_ * `Extended bio `__ - * Project Jupyter, Cal Poly SLO (Research Software Engineer) + * Project Jupyter (Steering Council, Core Team for JupyterHub/Binder) * Python Software Foundation (Fellow) Carol is focused on Python's usage in education and scientific research. From bc8823bbfe82d8e8ef6abb15cd5876530b5679f9 Mon Sep 17 00:00:00 2001 From: jablonskidev <36643503+jablonskidev@users.noreply.github.com> Date: Mon, 21 Dec 2020 16:32:04 -0800 Subject: [PATCH 088/871] Made wording more welcoming (#649) Made wording more in the translation Q&A be more welcoming. Closes https://github.com/python/devguide/issues/522 Co-authored-by: Mariatta --- documenting.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/documenting.rst b/documenting.rst index b1a3151d8..ebf5a58e2 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1792,10 +1792,9 @@ Ask on doc-sig, or better, make a PR on the `devguide I have a translation, but not on git, what should I do? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Just ask for help on the doc-sig mailing list and our python-fu, git-fu -and bash-fu combined will help you create an appropriate repository. If -you use a tool like transifex don’t worry. Keeping them in sync is not -that hard. +You can ask for help on the doc-sig mailing list, and the team will help you +create an appropriate repository. You can still use tools like transifex, +if you like. My git hierarchy does not match yours, can I keep it? From b39db20b309e16050625d035a968c9db7f6ddbaa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=91=D0=BE=D1=80=D0=B8=D1=81=20=D0=92=D0=B5=D1=80=D1=85?= =?UTF-8?q?=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9?= Date: Mon, 25 Jan 2021 10:59:56 -0500 Subject: [PATCH 089/871] Minor corrections in garbage_collector.rst (#656) --- garbage_collector.rst | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index 0f93be4af..68a4df028 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -219,17 +219,17 @@ they started originally) and setting its ``gc_refs`` field to 1. This is what ha to ``link_2`` and ``link_3`` below as they are reachable from ``link_1``. From the state in the previous image and after examining the objects referred to by ``link_1`` the GC knows that ``link_3`` is reachable after all, so it is moved back to the -original list and its ``gc_refs`` field is set to one so if the GC visits it again, it -does know that is reachable. To avoid visiting a object twice, the GC marks all -objects that are already visited once (by unsetting the ``PREV_MASK_COLLECTING`` flag) -so if an object that has already been processed is referred by some other object, the -GC does not process it twice. +original list and its ``gc_refs`` field is set to 1 so that if the GC visits it again, +it will know that it's reachable. To avoid visiting an object twice, the GC marks all +objects that have already been visited once (by unsetting the ``PREV_MASK_COLLECTING`` +flag) so that if an object that has already been processed is referenced by some other +object, the GC does not process it twice. .. figure:: images/python-cyclic-gc-5-new-page.png -Notice that once an object that was marked as "tentatively unreachable" and later is -moved back to the reachable list, it will be visited again by the garbage collector -as now all the references that that objects has need to be processed as well. This +Notice that an object that was marked as "tentatively unreachable" and was later +moved back to the reachable list will be visited again by the garbage collector +as now all the references that that object has need to be processed as well. This process is really a breadth first search over the object graph. Once all the objects are scanned, the GC knows that all container objects in the tentatively unreachable list are really unreachable and can thus be garbage collected. @@ -276,7 +276,7 @@ follows these steps in order: set is going to be destroyed and has weak references with callbacks, these callbacks need to be honored. This process is **very** delicate as any error can cause objects that will be in an inconsistent state to be resurrected or reached - by some python functions invoked from the callbacks. In addition, weak references + by some Python functions invoked from the callbacks. In addition, weak references that also are part of the unreachable set (the object and its weak reference are in cycles that are unreachable) need to be cleaned immediately, without executing the callback. Otherwise it will be triggered later, @@ -304,9 +304,9 @@ optimization: generations. The main idea behind this concept is the assumption t most objects have a very short lifespan and can thus be collected shortly after their creation. This has proven to be very close to the reality of many Python programs as many temporarily objects are created and destroyed very fast. The older an object is -the less likely it is to become unreachable. +the less likely it is that it will become unreachable. -To take advantage of this fact, all container objects are segregated across +To take advantage of this fact, all container objects are segregated into three spaces/generations. Every new object starts in the first generation (generation 0). The previous algorithm is executed only over the objects of a particular generation and if an object @@ -316,7 +316,7 @@ the same object survives another GC round in this new generation (generation 1) it will be moved to the last generation (generation 2) where it will be surveyed the least often. -Generations are collected when the number of objects that they contain reach some +Generations are collected when the number of objects that they contain reaches some predefined threshold, which is unique for each generation and is lower the older the generations are. These thresholds can be examined using the ``gc.get_threshold`` function: @@ -402,10 +402,10 @@ bit a separate tag) – as long as code that uses the pointer masks out these bits before accessing memory. E.g., on a 32-bit architecture (for both addresses and word size), a word is 32 bits = 4 bytes, so word-aligned addresses are always a multiple of 4, hence end in ``00``, leaving the last 2 bits -available; while on a 64-bit architecture, a word is 64 bits word = 8 bytes, so +available; while on a 64-bit architecture, a word is 64 bits = 8 bytes, so word-aligned addresses end in ``000``, leaving the last 3 bits available. -The CPython GC makes use of two fat pointers that corresponds to the extra fields +The CPython GC makes use of two fat pointers that correspond to the extra fields of ``PyGC_Head`` discussed in the `Memory layout and object structure`_ section: .. warning:: @@ -440,7 +440,7 @@ Optimization: delay tracking containers Certain types of containers cannot participate in a reference cycle, and so do not need to be tracked by the garbage collector. Untracking these objects -reduces the cost of garbage collections. However, determining which objects may +reduces the cost of garbage collection. However, determining which objects may be untracked is not free, and the costs must be weighed against the benefits for garbage collection. There are two possible strategies for when to untrack a container: @@ -470,7 +470,7 @@ benefit from delayed tracking: full garbage collection (all generations), the collector will untrack any dictionaries whose contents are not tracked. -The garbage collector module provides the python function is_tracked(obj), which returns +The garbage collector module provides the Python function ``is_tracked(obj)``, which returns the current tracking status of the object. Subsequent garbage collections may change the tracking status of the object. From 793046c8b8b8d3fc0fbcabe43aa1a01061494cf2 Mon Sep 17 00:00:00 2001 From: jablonskidev <36643503+jablonskidev@users.noreply.github.com> Date: Fri, 29 Jan 2021 16:23:35 -0800 Subject: [PATCH 090/871] Make https://devguide.python.org/committing/ simpler (#650) Closes #520 --- committing.rst | 390 ++++++++++++++++++++----------------------------- 1 file changed, 162 insertions(+), 228 deletions(-) diff --git a/committing.rst b/committing.rst index 773dd66cc..a242e8aa9 100644 --- a/committing.rst +++ b/committing.rst @@ -5,212 +5,147 @@ Accepting Pull Requests .. highlight:: none -This page is aimed to core developers, and covers the steps required to -accept, merge, and possibly backport a pull request on the main repository. - -Is the PR ready to be accepted? -------------------------------- - -Before a PR is accepted, you must make sure it is ready to enter the public -source tree. Use the following as a checklist of what to check for before -merging (details of various steps can be found later in this document): - -#. Has the submitter signed the CLA? - (delineated by a label on the pull request) -#. Did the test suite pass? (delineated by a pull request check) -#. Did code coverage increase or stay the same? - (delineated by a comment on the pull request) -#. Are the changes acceptable? -#. Was ``configure`` regenerated (if necessary)? -#. Was ``pyconfig.h.in`` regenerated (if necessary)? -#. Was the submitter added to ``Misc/ACKS`` (as appropriate)? -#. Was an entry added under ``Misc/NEWS.d/next`` (as appropriate)? -#. Was "What's New" updated (as appropriate)? -#. Were appropriate labels added to signify necessary backporting of the - pull request? -#. Was the pull request first made against the ``master`` branch? - -.. note:: - If you want to share your work-in-progress code on a feature or bugfix, - either open a ``WIP``-prefixed PR, publish patches on the - `issue tracker`_ or create a public fork of the repository. - -.. _issue tracker: https://bugs.python.org - - -Does the test suite still pass? -''''''''''''''''''''''''''''''' - -You must :ref:`run the whole test suite ` to ensure that it -passes before merging any code changes. - -.. note:: - You really need to run the **entire** test suite. Running a single test - is not enough as the changes may have unforeseen effects on other tests - or library modules. - - Running the entire test suite doesn't guarantee that the changes - will pass the :ref:`continuous integration ` tests, as those - will exercise more possibilities still (such as different platforms or - build options). But it will at least catch non-build specific, - non-platform specific errors, therefore minimizing the chance for - breakage. - - -Patch checklist -''''''''''''''' - -You should also :ref:`run patchcheck ` to perform a quick -sanity check on the changes. - - -Handling Others' Code ---------------------- - -As a core developer you will occasionally want to commit a patch created by -someone else. When doing so you will want to make sure of some things. - -First, make sure the patch is in a good state. Both :ref:`patch` and -:ref:`helptriage` -explain what is to be expected of a patch. Typically patches that get cleared by -triagers are good to go except maybe lacking ``Misc/ACKS`` and ``Misc/NEWS.d`` -entries (which a core developer should make sure are updated appropriately). - -Second, make sure the patch does not break backwards-compatibility without a -good reason. This means :ref:`running the entire test suite ` to -make sure everything still passes. It also means that if semantics do change -there must be a good reason for the breakage of code the change will cause -(and it **will** break someone's code). If you are unsure if the breakage -is worth it, ask on python-dev. - -Third, ensure the patch is attributed correctly with the contributor's -name in ``Misc/ACKS`` if they aren't already there (and didn't add themselves -in their patch) and by mentioning "Patch by " in the ``Misc/NEWS.d`` entry -and the check-in message. If the patch has been heavily modified then "Initial -patch by " is an appropriate alternate wording. GitHub now supports `multiple -authors `_ -in a commit. Add ``Co-authored-by: name `` at the end of the commit message. - -If you omit correct attribution in the initial check-in, then update ``ACKS`` -and ``NEWS.d`` in a subsequent check-in (don't worry about trying to fix the -original check-in message in that case). - -Finally, make sure that the submitter of the -patch has a CLA in place (indicated by an asterisk following their username -in the `issue tracker`_ or by the "CLA Signed" label on the pull request). -If the submitter lacks a signed CLA and the patch is non-trivial, direct them -to the electronic `Contributor Licensing Agreement`_ -to ensure the PSF has the appropriate authorizations in place to relicense -and redistribute their code. - - -Contributor Licensing Agreements --------------------------------- +This page is a step-by-step guide for core developers who need to assess, +merge, and possibly backport a pull request on the main repository. + +Assessing a pull request +------------------------ + +Before you can accept a pull request, you need to make sure that it is ready +to enter the public source tree. Ask yourself the following questions: + +* **Are there ongoing discussions at** ``bugs.python.org`` **(b.p.o.)?** + Read the linked b.p.o. issue. If there are ongoing discussions, then + we need to have a resolution there before we can merge the pull request. + +* **Was the pull request first made against the appropriate branch?** + The only branch that receives new features is ``master``, the + in-development branch. Pull requests should only target bug-fix branches + if an issue appears in only that version and possibly older versions. + +* **Are there comments on the pull request?** + Look for explanations about whether the code coverage increased or + stayed the same. + +* **Are the changes acceptable?** + If you want to share your work-in-progress code on a feature or bugfix, + then you can open a ``WIP``-prefixed pull request, publish patches on + the `issue tracker `_, or create a public + fork of the repository. + +* **Do the checks on the pull request show that the test suite passes?** + Make sure that all of the status checks are passing. + +* **Is the patch in a good state?** + Check :ref:`patch` and :ref:`helptriage` to review what is expected of + a patch. + +* **Does the patch break backwards-compatibility without a strong reason?** + :ref:`Run the entire test suite ` to make sure that everything + still passes. If there is a change to the semantics, then there needs to + be a strong reason, because it will cause some peoples' code to break. + If you are unsure if the breakage is worth it, then ask on python-dev. + +* **Does documentation need to be updated?** + If the pull request introduces backwards-incompatible changes (e.g. + deprecating or removing a feature), then make sure that those changes + are reflected in the documentation before you merge the pull request. + +* **Were appropriate labels added to signify necessary backporting of the pull request?** + If it is determined that a pull request needs to be + backported into one or more of the maintenance branches, then a core + developer can apply the label ``needs backport to X.Y`` to the pull + request. Once the backport pull request has been created, remove the + ``needs backport to X.Y`` label from the original pull request. (Only + core developers and members of the `Python Triage Team`_ can apply + labels to GitHub pull requests). + +* **Does the pull request have a label indicating that the submitter has signed the CLA?** + Make sure that the contributor has signed a `Contributor + Licensing Agreement `_ + (CLA), unless their change has no possible intellectual property + associated with it (e.g. fixing a spelling mistake in documentation). + To check if a contributor’s CLA has been received, go + to `Check Python CLA `_ and + put in their GitHub username. For further questions about the CLA + process, write to contributors@python.org. + +* **Were** ``What's New in Python`` **and** ``Misc/NEWS.d/next`` **updated?** + If the change is particularly interesting for end users (e.g. new features, + significant improvements, or backwards-incompatible changes), then an + entry in the ``What's New in Python`` document (in ``Doc/whatsnew/``) should + be added as well. Changes that affect only documentation generally do not + require a ``NEWS`` entry. (See the following section for more information.) + + +Updating NEWS and What's New in Python +-------------------------------------- -Always get a `Contributor Licensing Agreement`_ (CLA) signed unless the -change has no possible intellectual property associated with it (e.g. fixing -a spelling mistake in documentation). Otherwise it is simply safer from a -legal standpoint to require the contributor to sign the CLA. - -These days, the CLA can be signed electronically through the form linked -above, and this process is strongly preferred to the old mechanism that -involved sending a scanned copy of the signed paper form. - -As discussed on the PSF Contribution_ page, it is the CLA itself that gives -the PSF the necessary relicensing rights to redistribute contributions under -the Python license stack. This is an additional permission granted above and -beyond the normal permissions provided by the chosen open source license. - -Some developers may object to the relicensing permissions granted to the PSF -by the CLA. They're entirely within their rights to refuse to sign the CLA -on that basis, but that refusal *does* mean we **can't accept their patches** -for inclusion. - - -.. _Contribution: https://www.python.org/psf/contrib/ -.. _Contributor Licensing Agreement: - https://www.python.org/psf/contrib/contrib-form/ - - -Checking if the CLA has been received -------------------------------------- - -To check if a contributor's CLA has been received, go to the following website:: +Almost all changes made to the code base deserve an entry in ``Misc/NEWS.d``. +If the change is particularly interesting for end users (e.g. new features, +significant improvements, or backwards-incompatible changes), then an entry in +the ``What's New in Python`` document (in ``Doc/whatsnew/``) should be added +as well. Changes that affect documentation only generally do not require +a ``NEWS`` entry. - https://check-python-cla.herokuapp.com/ +There are two notable exceptions to this general principle, and they +both relate to changes that: -and put in their GitHub username. +* Already have a ``NEWS`` entry +* Have not yet been included in any formal release (including alpha + and beta releases) -For further questions about the CLA process, write to: contributors@python.org. +These are the two exceptions: +#. **If a change is reverted prior to release**, then the corresponding + entry is simply removed. Otherwise, a new entry must be added noting + that the change has been reverted (e.g. when a feature is released in + an alpha and then cut prior to the first beta). -What's New and News Entries ---------------------------- +#. **If a change is a fix (or other adjustment) to an earlier unreleased + change and the original** ``NEWS`` **entry remains valid**, then no additional + entry is needed. -Almost all changes made to the code base deserve an entry in ``Misc/NEWS.d``. -If the change is particularly interesting for end users (e.g. new features, -significant improvements, or backwards-incompatible changes), an entry in -the ``What's New in Python`` document (in ``Doc/whatsnew/``) should be added -as well. Changes that affect documentation only generally do not require -a news entry. +If a change needs an entry in ``What's New in Python``, then it very +likely not suitable for including in a maintenance release. -There are two notable exceptions to this general principle, and they -both relate to changes that *already* have a news entry, and have not yet -been included in any formal release (including alpha and beta releases). -These exceptions are: - -* If a change is reverted prior to release, then the corresponding entry - is simply removed. Otherwise, a new entry must be added noting that the - change has been reverted (e.g. when a feature is released in an alpha and - then cut prior to the first beta). - -* If a change is a fix (or other adjustment) to an earlier unreleased change - and the original news entry remains valid, then no additional entry is - needed. - -Needing a What's New entry almost always means that a change is *not* -suitable for inclusion in a maintenance release. A small number of -exceptions have been made for Python 2.7 due to the long support period - -when implemented, these changes *must* be noted in the "New Additions in -Python 2.7 Maintenance Releases" section of the Python 2.7 What's New -document. - -News entries go into the ``Misc/NEWS.d`` directory as individual files. The -news entry can be created by using `blurb-it `_, +``NEWS`` entries go into the ``Misc/NEWS.d`` directory as individual files. The +``NEWS`` entry can be created by using `blurb-it `_, or the `blurb `_ tool and its ``blurb add`` command. -If you are unable to use the tool you can create the news entry file manually. -The ``Misc/NEWS.d`` directory contains a sub-directory named ``next`` which -itself contains various sub-directories representing classifications for what -was affected (e.g. ``Misc/NEWS.d/next/Library`` for changes relating to the -standard library). The file name itself should be of the format +If you are unable to use the tool, then you can create the ``NEWS`` entry file +manually. The ``Misc/NEWS.d`` directory contains a sub-directory named +``next``, which contains various sub-directories representing classifications +for what was affected (e.g. ``Misc/NEWS.d/next/Library`` for changes relating +to the standard library). The file name itself should be in the format ``.bpo-..rst``: -* ```` is today's date joined with a ``-`` to the current - time, in ``YYYY-MM-DD-hh-mm-ss`` format, e.g. ``2017-05-27-16-46-23`` -* ```` is the issue number the change is for, e.g. ``12345`` - for ``bpo-12345`` -* ```` is some "unique" string to guarantee the file name is - unique across branches, e.g. ``Yl4gI2`` (typically six characters, but it can - be any length of letters and numbers, and its uniqueness can be satisfied by - typing random characters on your keyboard) +* ```` is today's date joined with a hyphen (``-``) to the current + time, in the ``YYYY-MM-DD-hh-mm-ss`` format (e.g. ``2017-05-27-16-46-23``). +* ```` is the issue number the change is for (e.g. ``12345`` + for ``bpo-12345``). +* ```` is a unique string to guarantee that the file name is + unique across branches (e.g. ``Yl4gI2``). It is typically six characters + long, but it can be any length of letters and numbers. Its uniqueness + can be satisfied by typing random characters on your keyboard. -So a file name may be +As a result, a file name can look something like ``Misc/NEWS.d/next/Library/2017-05-27-16-46-23.bpo-12345.Yl4gI2.rst``. -The contents of a news file should be valid reStructuredText. An 80 character +The contents of a ``NEWS`` file should be valid reStructuredText. An 80 character column width should be used. There is no indentation or leading marker in the file (e.g. ``-``). There is also no need to start the entry with the issue -number as it's part of the file name itself. You can use -:ref:`inline markups ` too. Example news entry:: +number since it is part of the file name. You can use +:ref:`inline markups ` too. Here is an example of a ``NEWS`` +entry:: Fix warning message when :func:`os.chdir` fails inside :func:`test.support.temp_cwd`. Patch by Chris Jerdonek. -The inline Sphinx roles like ``:func:`` can be used to assist readers in finding -more information. You can build html and verify that the link target is -appropriate by using :ref:`make html `. +The inline Sphinx roles like ``:func:`` can be used help readers +find more information. You can build HTML and verify that the +link target is appropriate by using :ref:`make html `. While Sphinx roles can be beneficial to readers, they are not required. Inline ````code blocks```` can be used instead. @@ -222,30 +157,28 @@ Working with Git_ .. seealso:: :ref:`gitbootcamp` -As a core developer, the ability to push changes to the official Python -repositories means you have to be more careful with your workflow: +As a core developer, you have the ability to push changes to the official +Python repositories, so you need to be careful with your workflow: -* You should not push new branches to the main repository. You can still use - them in your fork that you use for development of patches; you can - also push these branches to a **separate** public repository that will be - dedicated to maintenance of the work before the work gets integrated in the - main repository. +* **You should not push new branches to the main repository.** You can + still use them in the fork that you use for the development of patches. + You can also push these branches to a separate public repository + for maintenance work before it is integrated into the main repository. -* You should not commit directly into the ``master`` branch, or any of the - maintenance branches (currently ``3.9`` and ``3.8``). - You should commit against your own feature branch, and create a pull request. +* **You should not commit directly into the** ``master`` **branch, or any of the maintenance branches.** + You should commit against your own feature branch, and then create a + pull request. -* For a small change, you can make a quick edit through the GitHub web UI. +* **For a small change, you can make a quick edit through the GitHub web UI.** If you choose to use the web UI, be aware that GitHub will - create a new branch in the **main** CPython repo (not your fork). Please - delete this newly created branch after it has been merged into the + create a new branch in the main CPython repository rather than in your fork. + Delete this newly created branch after it has been merged into the ``master`` branch or any of the maintenance branches. To keep the CPython - repo tidy, please try to limit the existence of the new branch to, at most, - a few days. + repository tidy, remove the new branch within a few days. -It is recommended to keep a fork of the main repository around, as it allows -simple reversion of all local changes (even "committed" ones) if your local -clone gets into a state you aren't happy with. +Keep a fork of the main repository, since it will allow you to revert all +local changes (even committed ones) if you're not happy with your local +clone. .. _Git: https://git-scm.com/ @@ -253,65 +186,66 @@ clone gets into a state you aren't happy with. .. _committing-active-branches: -Active branches -''''''''''''''' +Seeing active branches +'''''''''''''''''''''' -If you do ``git branch`` you will see a :ref:`list of branches `. -``master`` is the in-development branch, and is the only branch that receives -new features. The other branches only receive bug fixes or security fixes. +If you use ``git branch``, then you will see a :ref:`list of branches +`. The only branch that receives new features is +``master``, the in-development branch. The other branches receive only +bug fixes or security fixes. .. _branch-merge: -Backporting Changes to an Older Version +Backporting changes to an older version ''''''''''''''''''''''''''''''''''''''' -When it is determined that a pull request needs to be backported into one or -more of the maintenance branches, a core developer can apply the labels +If it is determined that a pull request needs to be backported into one or +more of the maintenance branches, then a core developer can apply the label ``needs backport to X.Y`` to the pull request. After the pull request has been merged, miss-islington (bot) will first try to -do the backport automatically. In case that miss-islington is unable to do it, +do the backport automatically. If miss-islington is unable to do it, then the pull request author or the core developer who merged it should look into backporting it themselves, using the backport generated by cherry_picker.py_ as a starting point. -The commit hash can be obtained from the original pull request, or by using -``git log`` on the ``master`` branch. -To display the 10 most recent commit hashes and their first line of the commit -message:: +You can get the commit hash from the original pull request, or you can use +``git log`` on the ``master`` branch. To display the 10 most recent commit +hashes and their first line of the commit, use the following command:: git log -10 --oneline .. _backport-pr-title: -Prefix the backport pull request with the branch, and reference the pull request -number from ``master``, for example:: +You can prefix the backport pull request with the branch, and reference +the pull request number from ``master``. Here is an example:: [3.9] bpo-12345: Fix the Spam Module (GH-NNNN) Note that cherry_picker.py_ adds the branch prefix automatically. Once the backport pull request has been created, remove the -``needs backport to X.Y`` label from the original pull request. (Only Core -Developers and members of the `Python Triage Team`_ can apply labels to GitHub -pull requests). +``needs backport to X.Y`` label from the original pull request. (Only +core developers and members of the `Python Triage Team`_ can apply +labels to GitHub pull requests). .. _cherry_picker.py: https://github.com/python/cherry-picker .. _`Python Triage Team`: https://devguide.python.org/triaging/#python-triage-team -Reverting a Merged Pull Request +Reverting a merged pull request ''''''''''''''''''''''''''''''' -To revert a merged pull request, press the ``Revert`` button at the bottom of -the pull request. It will bring up the page to create a new pull request where -the commit can be reverted. It also creates a new branch on the main CPython -repository. Delete the branch once the pull request has been merged. +To revert a merged pull request, press the ``Revert`` button at the +bottom of the pull request. That will bring up the page to create a +new pull request where the commit can be reverted. It will also create +a new branch on the main CPython repository. Delete the branch once +the pull request has been merged. -Always include the reason for reverting the commit to help others understand -why it was done. The reason should be included as part of the commit message, -for example:: +Always include the reason for reverting the commit to help others +understand why it was done. The reason should be included as part of +the commit message. Here is an example:: Revert bpo-NNNN: Fix Spam Module (GH-111) From 41dde3c86da26be2a15e1387be18c46f3c3bbbf1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jes=C3=BAs=20Cea?= Date: Tue, 2 Feb 2021 19:14:54 +0100 Subject: [PATCH 091/871] Typo (#658) --- garbage_collector.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index 68a4df028..0dbac59fc 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -417,7 +417,7 @@ of ``PyGC_Head`` discussed in the `Memory layout and object structure`_ section: normally assume the pointers inside the lists are in a consistent state. -* The ``_gc_prev``` field is normally used as the "previous" pointer to maintain the +* The ``_gc_prev`` field is normally used as the "previous" pointer to maintain the doubly linked list but its lowest two bits are used to keep the flags ``PREV_MASK_COLLECTING`` and ``_PyGC_PREV_MASK_FINALIZED``. Between collections, the only flag that can be present is ``_PyGC_PREV_MASK_FINALIZED`` that indicates From 6bcef3134d63dabc381f0a5a91e31bfdd5f63748 Mon Sep 17 00:00:00 2001 From: Brandt Bucher Date: Mon, 8 Feb 2021 13:55:21 -0800 Subject: [PATCH 092/871] Add myself as a pattern matching expert (#659) --- experts.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/experts.rst b/experts.rst index 473e9f997..619016d1d 100644 --- a/experts.rst +++ b/experts.rst @@ -339,6 +339,7 @@ memoryview networking giampaolo.rodola, object model benjamin.peterson, twouters packaging tarek, lemburg, alexis, eric.araujo, dstufft, paul.moore +pattern matching brandtbucher* peg parser gvanrossum, pablogsal, lys.nikolaou performance brett.cannon, vstinner, serhiy.storchaka, yselivanov, rhettinger, Mark.Shannon pip ncoghlan, dstufft, paul.moore, Marcus.Smith, pradyunsg From 23fb8b53bfac0cf271439441ab1d2a87f492e58b Mon Sep 17 00:00:00 2001 From: Andrew Kuchling Date: Mon, 22 Feb 2021 10:38:05 -0500 Subject: [PATCH 093/871] Note removal of privileges for akuchling (#660) Co-authored-by: Andrew Kuchling --- developers.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/developers.csv b/developers.csv index 48cc31309..ffee4c300 100644 --- a/developers.csv +++ b/developers.csv @@ -163,7 +163,7 @@ Eric S. Raymond,,2000-06-02,2017-02-10,Did not make GitHub transition Greg Stein,,1999-11-07,2017-02-10,Did not make GitHub transition Just van Rossum,,1999-01-22,2017-02-10,Did not make GitHub transition Greg Ward,,1998-12-18,2017-02-10,Did not make GitHub transition -Andrew Kuchling,akuchling,1998-04-09,, +Andrew Kuchling,akuchling,1998-04-09,2021-02-22,Privileges relinquished on 2021-02-22 Ken Manheimer,,1998-03-03,2005-04-08,Privileges relinquished on 2005-04-08 Jeremy Hylton,jeremyhylton,1997-08-13,, Roger E. Masse,,1996-12-09,2017-02-10,Did not make GitHub transition From 29ae8afe247e9eeebba2cad66e2cfc8f9ec3fae2 Mon Sep 17 00:00:00 2001 From: Mariatta Wijaya Date: Wed, 10 Mar 2021 18:19:22 -0800 Subject: [PATCH 094/871] Updated Devcycle doc with Ee's name. (#663) cc @ewdurbin. --- devcycle.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index 06c7eaddd..366636389 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -255,7 +255,7 @@ This role is paramount to the security of the Python Language, Community, and Infrastructure. The Executive Director of the Python Software Foundation delegates authority on -GitHub Organization Owner Status to Ernest W. Durbin III - Python Software +GitHub Organization Owner Status to Ee W. Durbin III - Python Software Foundation Director of Infrastructure. Common reasons for this role are: Infrastructure Staff Membership, Python Software Foundation General Counsel, and Python Software Foundation Staff as fallback. @@ -280,7 +280,7 @@ Current Owners +----------------------+--------------------------------+-----------------+ | Ewa Jodlowska | PSF Executive Director | ejodlowska | +----------------------+--------------------------------+-----------------+ -| Ernest W. Durbin III | PSF Director of Infrastructure | ewdurbin | +| Ee W. Durbin III | PSF Director of Infrastructure | ewdurbin | +----------------------+--------------------------------+-----------------+ | Van Lindberg | PSF General Counsel | VanL | +----------------------+--------------------------------+-----------------+ From 0f2ea524684c428b81b41517d1d150b0e6959509 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 19 Mar 2021 12:00:10 -0600 Subject: [PATCH 095/871] Fix the pyperformance Repo URL. (#664) The repo changed name, from "performance" to "pyperformance", a while back. The devguide did not get updated at that time. Updating the links here means GitHub will no longer need to rewrite the old URL in requests. --- communication.rst | 2 +- runtests.rst | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/communication.rst b/communication.rst index 11c4c5aeb..c2fe9a7a0 100644 --- a/communication.rst +++ b/communication.rst @@ -130,4 +130,4 @@ source of benchmarks for all Python implementations. .. _Python Core Workflow: https://github.com/python/core-workflow .. _cherry_picker: https://pypi.org/project/cherry-picker/ .. _blurb: https://pypi.org/project/blurb -.. _Performance Benchmark: https://github.com/python/performance +.. _Performance Benchmark: https://github.com/python/pyperformance diff --git a/runtests.rst b/runtests.rst index ef410bdd2..74e38ff8b 100644 --- a/runtests.rst +++ b/runtests.rst @@ -129,7 +129,7 @@ Benchmarks ---------- Benchmarking is useful to test that a change does not degrade performance. -`The Python Benchmark Suite `_ +`The Python Benchmark Suite `_ has a collection of benchmarks for all Python implementations. Documentation about running the benchmarks is in the `README.txt -`_ of the repo. +`_ of the repo. From b9f1ae8c60c62d1e709659f016e73b94e0cd4d41 Mon Sep 17 00:00:00 2001 From: Antoine Pitrou Date: Sun, 21 Mar 2021 18:06:23 +0100 Subject: [PATCH 096/871] Fix link to custom builders (GH-666) --- buildbots.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/buildbots.rst b/buildbots.rst index ce1f16845..92662487c 100644 --- a/buildbots.rst +++ b/buildbots.rst @@ -209,7 +209,7 @@ Custom builders When working on a platform-specific issue, you may want to test your changes on the buildbot fleet rather than just on Travis and AppVeyor. To do so, you can make use of the `custom builders -`_. +`_. These builders track the ``buildbot-custom`` short-lived branch of the ``python/cpython`` repository, which is only accessible to core developers. From 73e53b6ddb7152bf9fd74658b3fb802bbf87c502 Mon Sep 17 00:00:00 2001 From: "Eric V. Smith" Date: Thu, 15 Apr 2021 15:18:41 -0400 Subject: [PATCH 097/871] Allow assigning dataclasses issues to eric.smith. (#675) --- experts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/experts.rst b/experts.rst index 619016d1d..1f0c96192 100644 --- a/experts.rst +++ b/experts.rst @@ -93,7 +93,7 @@ csv skip.montanaro (inactive) ctypes theller (inactive), belopolsky, amaury.forgeotdarc, meador.inge curses twouters -dataclasses eric.smith +dataclasses eric.smith* datetime belopolsky, p-ganssle dbm decimal facundobatista, rhettinger, mark.dickinson From 6127a2b84dea97d9ee046f9d40517d2ba06bc399 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Fri, 23 Apr 2021 20:02:51 +0200 Subject: [PATCH 098/871] devcycle: Clarify doc & test changes in Beta & RC (GH-676) --- devcycle.rst | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index 366636389..aba81d71e 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -185,9 +185,10 @@ Beta ---- After a first beta release is published, no new features are accepted. Only -bug fixes can now be committed. This is when core developers should concentrate -on the task of fixing regressions and other new issues filed by users who have -downloaded the alpha and beta releases. +bug fixes and improvements to documentation and tests can now be committed. +This is when core developers should concentrate on the task of fixing +regressions and other new issues filed by users who have downloaded the alpha +and beta releases. Being in beta can be viewed much like being in RC_ but without the extra overhead of needing commit reviews. @@ -207,6 +208,10 @@ severe enough (e.g. crashes) that they deserve fixing before the final release. All other issues should be deferred to the next development cycle, since stability is the strongest concern at this point. +While the goal is to have no code changes between a RC and a final release, +there may be a need for final documentation or test fixes. Any such proposed +changes should be discussed first with the release manager. + You **cannot** skip the peer review during an RC, no matter how small! Even if it is a simple copy-and-paste change, **everything** requires peer review from a core developer. From 206e583c3defb77ff13b320a84a1bf487def51bc Mon Sep 17 00:00:00 2001 From: Mariatta Wijaya Date: Thu, 29 Apr 2021 12:56:37 -0700 Subject: [PATCH 099/871] Add GitHub Actions to run the CI (#680) set continue-on-error true on the linkcheck step --- .github/workflows/ci.yml | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 .github/workflows/ci.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 000000000..09cd927e7 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,35 @@ +name: Tests + +on: + pull_request: + push: + branches: master + +jobs: + test: + name: test w/ Python ${{ matrix.python-version }} + + runs-on: ubuntu-latest + + strategy: + matrix: + python-version: ["3.9"] + + steps: + - uses: actions/checkout@v2 + - uses: actions/cache@v1 + with: + path: ~/.cache/pip + key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }} + restore-keys: | + ${{ runner.os }}-pip- + - uses: actions/setup-python@v2 + with: + python-version: ${{ matrix.python-version }} + - name: Install Dependencies + run: python3 -m pip install -U pip -r requirements.txt + - name: Build Docs + run: sphinx-build -n -W -q -b html -d _build/doctrees . _build/html + - name: Link Check + run: sphinx-build -b linkcheck -d _build/doctrees . _build/linkcheck + continue-on-error: true From fb051b71da938c95d52880e0f964a84db67eb2e1 Mon Sep 17 00:00:00 2001 From: Ammar Askar Date: Thu, 29 Apr 2021 22:05:01 -0400 Subject: [PATCH 100/871] Remove mention of old coverage comment from checklist (#685) --- committing.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/committing.rst b/committing.rst index a242e8aa9..4cb92e23d 100644 --- a/committing.rst +++ b/committing.rst @@ -23,10 +23,6 @@ to enter the public source tree. Ask yourself the following questions: in-development branch. Pull requests should only target bug-fix branches if an issue appears in only that version and possibly older versions. -* **Are there comments on the pull request?** - Look for explanations about whether the code coverage increased or - stayed the same. - * **Are the changes acceptable?** If you want to share your work-in-progress code on a feature or bugfix, then you can open a ``WIP``-prefixed pull request, publish patches on From 0af1395fbe184ab48b9b154054fbe07765965356 Mon Sep 17 00:00:00 2001 From: "dependabot-preview[bot]" <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Thu, 29 Apr 2021 19:05:29 -0700 Subject: [PATCH 101/871] Upgrade to GitHub-native Dependabot (#684) Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> --- .github/dependabot.yml | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 .github/dependabot.yml diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 000000000..491deae0a --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,7 @@ +version: 2 +updates: +- package-ecosystem: pip + directory: "/" + schedule: + interval: daily + open-pull-requests-limit: 10 From 5048d2e5abbee689c2462d73c93e00fb8ccf6fc0 Mon Sep 17 00:00:00 2001 From: Himanshu Date: Mon, 3 May 2021 03:05:45 +0530 Subject: [PATCH 102/871] Fix broken links (#683) --- buildworker.rst | 2 +- communication.rst | 4 ++-- documenting.rst | 4 ++-- help.rst | 2 +- index.rst | 2 +- motivations.rst | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/buildworker.rst b/buildworker.rst index 3e5deabe1..99f758179 100644 --- a/buildworker.rst +++ b/buildworker.rst @@ -198,7 +198,7 @@ Latent workers ^^^^^^^^^^^^^^ We also support running `latent workers -`_ +`_ on the AWS EC2 service. To set up such a worker: * Start an instance of your chosen base AMI and set it up as a diff --git a/communication.rst b/communication.rst index c2fe9a7a0..90b5a1027 100644 --- a/communication.rst +++ b/communication.rst @@ -56,8 +56,8 @@ on Freenode_. mailing list is the place to discuss and work on improvements to the CPython core development workflow. -A complete list of Python mailing lists can be found at https://mail.python.org. -Most lists are also mirrored at `GMANE `_ and can be read and +A complete list of Python mailing lists can be found at https://mail.python.org/mailman/listinfo. +Most lists are also mirrored at `GMANE `_ and can be read and posted to in various ways, including via web browsers, NNTP newsreaders, and RSS feed readers. diff --git a/documenting.rst b/documenting.rst index ebf5a58e2..5c712a001 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1679,8 +1679,8 @@ in production, other are work in progress: .. _mail_it: https://mail.python.org/pipermail/doc-sig/2019-April/004114.html .. _mail_lt: https://mail.python.org/pipermail/doc-sig/2019-July/004138.html .. _mail_ru: https://mail.python.org/pipermail/doc-sig/2019-May/004131.html -.. _tx_pl: https://www.transifex.com/python-doc/python-newest/language/pl/ -.. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/language/ +.. _tx_pl: https://www.transifex.com/python-doc/python-newest/ +.. _tx_zh_cn: https://www.transifex.com/python-doc/python-newest/ .. _tx_zh_tw: https://www.transifex.com/python-tw-doc/python-36-tw .. _wiki_pt_br: http://python.org.br/traducao diff --git a/help.rst b/help.rst index 50e3b982a..b0c726cdc 100644 --- a/help.rst +++ b/help.rst @@ -107,7 +107,7 @@ during office hours. | | | 24 hours before the start. | +------------------+-------------------------------+------------------------------------------------+ -.. _Python's Zulip Chat: https://python.zulipchat.com/#narrow/stream/116503-core/topic/Office.20Hour +.. _Python's Zulip Chat: https://python.zulipchat.com/login/#narrow/stream/116503-core/topic/Office.20Hour .. _Mariatta's twitter: https://twitter.com/mariatta diff --git a/index.rst b/index.rst index f62a7eef8..90cc5de0c 100644 --- a/index.rst +++ b/index.rst @@ -335,7 +335,7 @@ Full Table of Contents .. _python.org maintenance: https://pythondotorg.readthedocs.io/ .. _Python: https://www.python.org/ .. _Core Python Mentorship: https://www.python.org/dev/core-mentorship/ -.. _PyPy: http://www.pypy.org/ +.. _PyPy: http://www.pypy.org .. _Jython: http://www.jython.org/ .. _IronPython: http://ironpython.net/ .. _Stackless: http://www.stackless.com/ diff --git a/motivations.rst b/motivations.rst index 50085000c..f1a1a41cc 100644 --- a/motivations.rst +++ b/motivations.rst @@ -118,7 +118,7 @@ participating in the CPython core development process: * Microsoft (Software Developer) * Personal site: `stevedower.id.au `_ - * Speaking: `stevedower.id.au/speaking `_ + * Speaking: `stevedower.id.au/speaking `_ * Work blog: `aka.ms/pythonblog `_ * Email address: steve.dower@python.org From acc834ccc95364706c9cdc95553d63f3320cda65 Mon Sep 17 00:00:00 2001 From: Mariatta Wijaya Date: Mon, 3 May 2021 14:39:18 -0700 Subject: [PATCH 103/871] Replace CPython default branch name to main branch. (#662) --- .github/CONTRIBUTING.md | 2 +- buildbots.rst | 2 +- committing.rst | 12 ++++----- coverity.rst | 2 +- devcycle.rst | 4 +-- gitbootcamp.rst | 59 +++++++++++++++++++++++++---------------- index.rst | 10 +++---- pullrequest.rst | 8 +++--- setup.rst | 2 +- triaging.rst | 54 ++++++++++++++++++------------------- 10 files changed, 84 insertions(+), 71 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 87686d966..4b29950db 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -24,7 +24,7 @@ our workflow that are not covered by a bot or status check are: - All discussions that are not directly related to the code in the pull request should happen on [bugs.python.org](https://bugs.python.org/) - Upon your first non-trivial pull request (which includes documentation changes), - feel free to add yourself to [`Misc/ACKS`](https://github.com/python/cpython/blob/master/Misc/ACKS) + feel free to add yourself to [`Misc/ACKS`](https://github.com/python/cpython/blob/main/Misc/ACKS) ## Setting Expectations diff --git a/buildbots.rst b/buildbots.rst index 92662487c..58535ca1e 100644 --- a/buildbots.rst +++ b/buildbots.rst @@ -40,7 +40,7 @@ There are three ways of visualizing recent build results: https://code.google.com/archive/p/bbreport. Installing it is trivial: just add the directory containing ``bbreport.py`` to your system path so that you can run it from any filesystem location. For example, if you want - to display the latest build results on the development ("master") branch, + to display the latest build results on the development ("main") branch, type:: bbreport.py -q 3.x diff --git a/committing.rst b/committing.rst index 4cb92e23d..213a2292e 100644 --- a/committing.rst +++ b/committing.rst @@ -19,7 +19,7 @@ to enter the public source tree. Ask yourself the following questions: we need to have a resolution there before we can merge the pull request. * **Was the pull request first made against the appropriate branch?** - The only branch that receives new features is ``master``, the + The only branch that receives new features is ``main``, the in-development branch. Pull requests should only target bug-fix branches if an issue appears in only that version and possibly older versions. @@ -161,7 +161,7 @@ Python repositories, so you need to be careful with your workflow: You can also push these branches to a separate public repository for maintenance work before it is integrated into the main repository. -* **You should not commit directly into the** ``master`` **branch, or any of the maintenance branches.** +* **You should not commit directly into the** ``main`` **branch, or any of the maintenance branches.** You should commit against your own feature branch, and then create a pull request. @@ -169,7 +169,7 @@ Python repositories, so you need to be careful with your workflow: If you choose to use the web UI, be aware that GitHub will create a new branch in the main CPython repository rather than in your fork. Delete this newly created branch after it has been merged into the - ``master`` branch or any of the maintenance branches. To keep the CPython + ``main`` branch or any of the maintenance branches. To keep the CPython repository tidy, remove the new branch within a few days. Keep a fork of the main repository, since it will allow you to revert all @@ -187,7 +187,7 @@ Seeing active branches If you use ``git branch``, then you will see a :ref:`list of branches `. The only branch that receives new features is -``master``, the in-development branch. The other branches receive only +``main``, the in-development branch. The other branches receive only bug fixes or security fixes. @@ -207,7 +207,7 @@ backporting it themselves, using the backport generated by cherry_picker.py_ as a starting point. You can get the commit hash from the original pull request, or you can use -``git log`` on the ``master`` branch. To display the 10 most recent commit +``git log`` on the ``main`` branch. To display the 10 most recent commit hashes and their first line of the commit, use the following command:: git log -10 --oneline @@ -215,7 +215,7 @@ hashes and their first line of the commit, use the following command:: .. _backport-pr-title: You can prefix the backport pull request with the branch, and reference -the pull request number from ``master``. Here is an example:: +the pull request number from ``main``. Here is an example:: [3.9] bpo-12345: Fix the Spam Module (GH-NNNN) diff --git a/coverity.rst b/coverity.rst index 8de943aef..2a9359715 100644 --- a/coverity.rst +++ b/coverity.rst @@ -150,4 +150,4 @@ Dakshesh Vyas .. _Coverity Connect: https://scan.coverity.com/projects/python -.. _coverity_model.c: https://github.com/python/cpython/blob/master/Misc/coverity_model.c +.. _coverity_model.c: https://github.com/python/cpython/blob/main/Misc/coverity_model.c diff --git a/devcycle.rst b/devcycle.rst index aba81d71e..e8bb60934 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -44,7 +44,7 @@ There is a branch for each *feature version*, whether released or not (e.g. In-development (main) branch ---------------------------- -The ``master`` branch is the branch for the next feature release; it is +The ``main`` branch is the branch for the next feature release; it is under active development for all kinds of changes: new features, semantic changes, performance improvements, bug fixes. @@ -57,7 +57,7 @@ release was cut (for example, 3.4.0 final). Starting with the 3.5 release, we create the release maintenance branch (e.g. 3.5) at the time we enter beta (3.5.0 beta 1). This allows -feature development for the release 3.n+1 to occur within the master +feature development for the release 3.n+1 to occur within the main branch alongside the beta and release candidate stabilization periods for release 3.n. diff --git a/gitbootcamp.rst b/gitbootcamp.rst index bd5f8e441..9cb76ac2f 100644 --- a/gitbootcamp.rst +++ b/gitbootcamp.rst @@ -103,17 +103,17 @@ Creating and Switching Branches ------------------------------- .. important:: - Never commit directly to the ``master`` branch. + Never commit directly to the ``main`` branch. Create a new branch and switch to it:: - # creates a new branch off master and switch to it - git checkout -b master + # creates a new branch off main and switch to it + git checkout -b main This is equivalent to:: - # create a new branch from master, without checking it out - git branch master + # create a new branch from main, without checking it out + git branch main # check out the branch git checkout @@ -144,7 +144,7 @@ Deleting Branches To delete a **local** branch that you no longer need:: - git checkout master + git checkout main git branch -D To delete a **remote** branch:: @@ -153,6 +153,19 @@ To delete a **remote** branch:: You may specify more than one branch for deletion. + +Renaming Branch +--------------- + +The CPython repository's default branch was renamed from ``master`` to ``main`` +after the Python 3.10b1 release. If you had cloned the repository before this +change, you can rename your local branch as follows:: + + git branch -m master main + git fetch upstream + git branch -u upstream/main main + + Staging and Committing Files ---------------------------- @@ -230,7 +243,7 @@ Creating a Pull Request 3. Click the ``compare across forks`` link. -4. Select the base repository: ``python/cpython`` and base branch: ``master``. +4. Select the base repository: ``python/cpython`` and base branch: ``main``. 5. Select the head repository: ``/cpython`` and head branch: the branch containing your changes. @@ -250,14 +263,14 @@ Scenario: the upstream CPython repository. Please do not try to solve this by creating a pull request from -``python:master`` to ``:master`` as the authors of the patches will +``python:main`` to ``:main`` as the authors of the patches will get notified unnecessarily. Solution:: - git checkout master - git pull upstream master - git push origin master + git checkout main + git pull upstream main + git push origin main .. note:: For the above commands to work, please follow the instructions found in the :ref:`checkout` section @@ -275,11 +288,11 @@ Solution:: git checkout some-branch git fetch upstream - git merge upstream/master + git merge upstream/main git push origin some-branch You may see error messages like "CONFLICT" and "Automatic merge failed;" when -you run ``git merge upstream/master``. +you run ``git merge upstream/main``. When it happens, you need to resolve conflict. See these articles about resolving conflicts: @@ -308,7 +321,7 @@ Solution: .. code-block:: bash - git checkout $(git rev-list -n 1 --before="yyyy-mm-dd hh:mm:ss" master) + git checkout $(git rev-list -n 1 --before="yyyy-mm-dd hh:mm:ss" main) git apply /path/to/issueNNNN-git.patch If the patch still won't apply, then a patch tool will not be able to @@ -321,7 +334,7 @@ Solution: 5. If the patch was applied to an old revision, it needs to be updated and merge conflicts need to be resolved:: - git rebase master + git rebase main git mergetool 6. Push the changes and open a pull request. @@ -388,7 +401,7 @@ Pull requests can be accepted and merged by a Python Core Developer. bpo-12345: Improve the spam module (#777) * Improve the spam module - * merge from master + * merge from main * adjust code based on review comment * rebased @@ -402,7 +415,7 @@ Backporting Merged Changes -------------------------- A pull request may need to be backported into one of the maintenance branches -after it has been accepted and merged into ``master``. It is usually indicated +after it has been accepted and merged into ``main``. It is usually indicated by the label ``needs backport to X.Y`` on the pull request itself. Use the utility script @@ -411,10 +424,10 @@ from the `core-workflow `_ repository to backport the commit. The commit hash for backporting is the squashed commit that was merged to -the ``master`` branch. On the merged pull request, scroll to the bottom of the +the ``main`` branch. On the merged pull request, scroll to the bottom of the page. Find the event that says something like:: - merged commit into python:master ago. + merged commit into python:main ago. By following the link to ````, you will get the full commit hash. @@ -459,12 +472,12 @@ items like updating ``Misc/ACKS``. .. _Allow edits from maintainers: https://help.github.com/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork/ -To edit an open pull request that targets ``master``: +To edit an open pull request that targets ``main``: 1. In the pull request page, under the description, there is some information about the contributor's forked CPython repository and branch name that will be useful later:: - wants to merge 1 commit into python:master from : + wants to merge 1 commit into python:main from : 2. Fetch the pull request, using the :ref:`git pr ` alias:: @@ -473,13 +486,13 @@ To edit an open pull request that targets ``master``: This will checkout the contributor's branch at ````. 3. Make and commit your changes on the branch. For example, merge in changes - made to ``master`` since the PR was submitted (any merge commits will be + made to ``main`` since the PR was submitted (any merge commits will be removed by the later ``Squash and Merge`` when accepting the change): .. code-block:: bash git fetch upstream - git merge upstream/master + git merge upstream/main git add git commit -m "" diff --git a/index.rst b/index.rst index 90cc5de0c..7da47b14e 100644 --- a/index.rst +++ b/index.rst @@ -52,7 +52,7 @@ instructions please see the :ref:`setup guide `. 5. Create a new branch where your work for the issue will go, e.g.:: - git checkout -b fix-issue-12345 master + git checkout -b fix-issue-12345 main If an issue does not already exist, please `create it `_. Trivial issues (e.g. typo fixes) do not @@ -98,7 +98,7 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-----------------------+ | Branch | Schedule | Status | First release | End-of-life | Release manager | +==================+==============+=============+================+================+=======================+ -| master | :pep:`619` | features | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +| main | :pep:`619` | features | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.9 | :pep:`596` | bugfix | 2020-10-05 | *TBD* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ @@ -111,7 +111,7 @@ Status of Python branches .. Remember to update the end-of-life table in devcycle.rst. -The master branch is currently the future Python 3.10, and is the only +The main branch is currently the future Python 3.10, and is the only branch that accepts new features. The latest release for each Python version can be found on the `download page `_. @@ -244,7 +244,7 @@ Key Resources * `Buildbot status`_ * Source code * `Browse online `_ - * `Snapshot of the *master* branch `_ + * `Snapshot of the *main* branch `_ * `Daily OS X installer `_ * PEPs_ (Python Enhancement Proposals) * :doc:`help` @@ -330,7 +330,7 @@ Full Table of Contents appendix .. _Buildbot status: https://www.python.org/dev/buildbot/ -.. _Misc directory: https://github.com/python/cpython/tree/master/Misc +.. _Misc directory: https://github.com/python/cpython/tree/main/Misc .. _PEPs: https://www.python.org/dev/peps/ .. _python.org maintenance: https://pythondotorg.readthedocs.io/ .. _Python: https://www.python.org/ diff --git a/pullrequest.rst b/pullrequest.rst index ef85923c0..7cb194490 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -75,7 +75,7 @@ You should have already :ref:`set up your system `, * Create a new branch in your local clone:: - git checkout -b upstream/master + git checkout -b upstream/main * Make changes to the code, and use ``git status`` and ``git diff`` to see them. @@ -122,12 +122,12 @@ You should have already :ref:`set up your system `, there are merge conflicts, git will warn you about this and enter conflict resolution mode. See :ref:`resolving-merge-conflicts` below. -* If time passes and there are merge conflicts with the master branch, GitHub +* If time passes and there are merge conflicts with the main branch, GitHub will show a warning to this end and you may be asked to address this. Merge - the changes from the master branch while resolving the conflicts locally:: + the changes from the main branch while resolving the conflicts locally:: git checkout - git pull upstream master # pull = fetch + merge + git pull upstream main # pull = fetch + merge # resolve conflicts: see "Resolving Merge Conflicts" below git push origin diff --git a/setup.rst b/setup.rst index 0b5e86441..385f142f2 100644 --- a/setup.rst +++ b/setup.rst @@ -280,7 +280,7 @@ to build. .. _this documentation: https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html .. _Visual Studio 2017: https://www.visualstudio.com/ -.. _readme: https://github.com/python/cpython/blob/master/PCbuild/readme.txt +.. _readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt .. _PCbuild directory: https://github.com/python/cpython/tree/2.7/PCbuild/ .. _2.7 readme: https://github.com/python/cpython/blob/2.7/PCbuild/readme.txt .. _PC/VS9.0 directory: https://github.com/python/cpython/tree/2.7/PC/VS9.0/ diff --git a/triaging.rst b/triaging.rst index a7bfec465..60025394b 100644 --- a/triaging.rst +++ b/triaging.rst @@ -103,13 +103,13 @@ expert-asyncio invalid Used manually for PRs that do not meet basic requirements and automatically added by bedevere when PR authors attempt to merge maintenace - branches into the master branch. During events such as the October + branches into the main branch. During events such as the October Hacktoberfest, this label will prevent the PR from counting toward the author's contributions. needs backport to X.Y Used for PRs which are appropriate to backport to - branches prior to master. Generally, backports to the maintenance branches + branches prior to main. Generally, backports to the maintenance branches are primarily bugfixes and documentation clarifications. Backports to the security branches are strictly reserved for PRs involving security fixes, such as crashes, privilege escalation, and DoS. The use of this label will cause @@ -582,31 +582,31 @@ Checklist for Triaging .. _CPython: https://github.com/python/cpython/ -.. _Doc: https://github.com/python/cpython/tree/master/Doc/ -.. _Grammar: https://github.com/python/cpython/tree/master/Grammar/ -.. _Lib: https://github.com/python/cpython/tree/master/Lib/ -.. _Lib/lib2to3: https://github.com/python/cpython/tree/master/Lib/lib2to3/ -.. _Lib/ctypes: https://github.com/python/cpython/tree/master/Lib/ctypes/ -.. _Lib/distutils: https://github.com/python/cpython/tree/master/Lib/distutils/ -.. _Lib/doctest.py: https://github.com/python/cpython/tree/master/Lib/doctest.py -.. _Lib/idlelib: https://github.com/python/cpython/tree/master/Lib/idlelib/ -.. _Lib/io.py: https://github.com/python/cpython/tree/master/Lib/io.py -.. _Lib/re.py: https://github.com/python/cpython/tree/master/Lib/re.py -.. _Lib/test: https://github.com/python/cpython/tree/master/Lib/test/ -.. _Lib/test/regrtest.py: https://github.com/python/cpython/tree/master/Lib/test/regrtest.py -.. _Lib/test/support: https://github.com/python/cpython/tree/master/Lib/test/support/ -.. _Lib/tkinter: https://github.com/python/cpython/tree/master/Lib/tkinter/ -.. _Lib/unittest: https://github.com/python/cpython/tree/master/Lib/unittest/ -.. _Lib/xml: https://github.com/python/cpython/tree/master/Lib/xml/ -.. _Modules: https://github.com/python/cpython/tree/master/Modules/ -.. _Modules/_io: https://github.com/python/cpython/tree/master/Modules/_io/ -.. _Modules/_sre.c: https://github.com/python/cpython/tree/master/Modules/_sre.c -.. _Objects: https://github.com/python/cpython/tree/master/Objects/ -.. _Objects/unicodeobject.c: https://github.com/python/cpython/tree/master/Objects/unicodeobject.c -.. _Parser: https://github.com/python/cpython/tree/master/Parser/ -.. _Python: https://github.com/python/cpython/tree/master/Python/ -.. _Tools: https://github.com/python/cpython/tree/master/Tools/ -.. _Tools/demo: https://github.com/python/cpython/tree/master/Tools/demo/ +.. _Doc: https://github.com/python/cpython/tree/main/Doc/ +.. _Grammar: https://github.com/python/cpython/tree/main/Grammar/ +.. _Lib: https://github.com/python/cpython/tree/main/Lib/ +.. _Lib/lib2to3: https://github.com/python/cpython/tree/main/Lib/lib2to3/ +.. _Lib/ctypes: https://github.com/python/cpython/tree/main/Lib/ctypes/ +.. _Lib/distutils: https://github.com/python/cpython/tree/main/Lib/distutils/ +.. _Lib/doctest.py: https://github.com/python/cpython/tree/main/Lib/doctest.py +.. _Lib/idlelib: https://github.com/python/cpython/tree/main/Lib/idlelib/ +.. _Lib/io.py: https://github.com/python/cpython/tree/main/Lib/io.py +.. _Lib/re.py: https://github.com/python/cpython/tree/main/Lib/re.py +.. _Lib/test: https://github.com/python/cpython/tree/main/Lib/test/ +.. _Lib/test/regrtest.py: https://github.com/python/cpython/tree/main/Lib/test/regrtest.py +.. _Lib/test/support: https://github.com/python/cpython/tree/main/Lib/test/support/ +.. _Lib/tkinter: https://github.com/python/cpython/tree/main/Lib/tkinter/ +.. _Lib/unittest: https://github.com/python/cpython/tree/main/Lib/unittest/ +.. _Lib/xml: https://github.com/python/cpython/tree/main/Lib/xml/ +.. _Modules: https://github.com/python/cpython/tree/main/Modules/ +.. _Modules/_io: https://github.com/python/cpython/tree/main/Modules/_io/ +.. _Modules/_sre.c: https://github.com/python/cpython/tree/main/Modules/_sre.c +.. _Objects: https://github.com/python/cpython/tree/main/Objects/ +.. _Objects/unicodeobject.c: https://github.com/python/cpython/tree/main/Objects/unicodeobject.c +.. _Parser: https://github.com/python/cpython/tree/main/Parser/ +.. _Python: https://github.com/python/cpython/tree/main/Python/ +.. _Tools: https://github.com/python/cpython/tree/main/Tools/ +.. _Tools/demo: https://github.com/python/cpython/tree/main/Tools/demo/ .. _Developer's guide: https://github.com/python/devguide/ .. _GSoC: https://summerofcode.withgoogle.com/ .. _issue tracker: https://bugs.python.org From 534b0ec04ad448cac3384a128ff9f61e4ba0c99e Mon Sep 17 00:00:00 2001 From: Ned Deily Date: Mon, 3 May 2021 19:14:47 -0400 Subject: [PATCH 104/871] update branch status as of 3.10.0b1 (#686) --- index.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/index.rst b/index.rst index 7da47b14e..31aa398d4 100644 --- a/index.rst +++ b/index.rst @@ -98,11 +98,13 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-----------------------+ | Branch | Schedule | Status | First release | End-of-life | Release manager | +==================+==============+=============+================+================+=======================+ -| main | :pep:`619` | features | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +| main | *TBD* | features | *TBD* | *TBD* | Pablo Galindo Salgado | ++------------------+--------------+-------------+----------------+----------------+-----------------------+ +| 3.10 | :pep:`619` | prerelase | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.9 | :pep:`596` | bugfix | 2020-10-05 | *TBD* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.8 | :pep:`569` | bugfix | 2019-10-14 | *2024-10* | Łukasz Langa | +| 3.8 | :pep:`569` | security | 2019-10-14 | *2024-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.7 | :pep:`537` | security | 2018-06-27 | *2023-06-27* | Ned Deily | +------------------+--------------+-------------+----------------+----------------+-----------------------+ @@ -111,7 +113,7 @@ Status of Python branches .. Remember to update the end-of-life table in devcycle.rst. -The main branch is currently the future Python 3.10, and is the only +The main branch is currently the future Python 3.11, and is the only branch that accepts new features. The latest release for each Python version can be found on the `download page `_. From 9b6b939e7170c0c56cc30a9718b7a28295a53cc3 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 4 May 2021 17:20:57 +0200 Subject: [PATCH 105/871] Add a page on changing the C API (PEP-652) (GH-682) This repeats points currently said in `Include/README.rst`. I intend to replace most of that README with a link to this document. The main change from that document are: - The three "rings" are not defined by where things are declared. It would be good to have them in the right place, but it's not yet the case, and for technical reasons we might never reach the ideal. (In `Include/README.rst` it makes more sense to make the locations more prominent.) - The order is switched. For changing the Limited API you generally need to read (and follow guidelines in) the preceding sections as well. On top of that, I addes points from PEP 652 (Maintaining the Stable ABI). Co-Authored-By: Erlend Egeberg Aasland --- c-api.rst | 194 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ index.rst | 1 + 2 files changed, 195 insertions(+) create mode 100644 c-api.rst diff --git a/c-api.rst b/c-api.rst new file mode 100644 index 000000000..9198c80cb --- /dev/null +++ b/c-api.rst @@ -0,0 +1,194 @@ +======================= +Changing Python's C API +======================= + +The C API is divided into three sections: + +1. The internal, private API, available with ``Py_BUILD_CORE`` defined. + Ideally declared in ``Include/internal/``. Any API named with a leading + underscore is also considered private. +2. The public C API, available when ``Python.h`` is included normally. + Ideally declared in ``Include/cpython/``. +3. The Limited API, available with ``Py_LIMITED_API`` defined. + Ideally declared directly under ``Include/``. + +Each section has higher stability & maintenance requirements, and you will +need to think about more issues when you add or change definitions in it. + +The compatibility guarantees for public C API are explained in the +user documentation, ``Doc/c-api/stable.rst`` (:ref:`python:stable`). + + +The internal API +================ + +Internal API is defined in ``Include/internal/`` and is only available +for building CPython itself, as indicated by a macro like ``Py_BUILD_CORE``. + +While internal API can be changed at any time, it's still good to keep it +stable: other API or other CPython developers may depend on it. + +With PyAPI_FUNC or PyAPI_DATA +----------------------------- + +Functions or structures in ``Include/internal/`` defined with +``PyAPI_FUNC`` or ``PyAPI_DATA`` are internal functions which are +exposed only for specific use cases like debuggers and profilers. + + +With the extern keyword +----------------------- + +Functions in ``Include/internal/`` defined with the ``extern`` keyword +*must not and can not* be used outside the CPython code base. Only +built-in stdlib extensions (built with the ``Py_BUILD_CORE_BUILTIN`` +macro defined) can use such functions. + +When in doubt, new internal C functions should be defined in +``Include/internal`` using the ``extern`` keyword. + +Private names +-------------- + +Any API named with a leading underscore is also considered internal. +There are two main use cases for using such names rather than putting the +definition in ``Include/internal/`` (or directly in a ``.c`` file): + +* Internal helpers for other public API; users should not use these directly; +* “Provisional” API, included in a Python release to test real-world usage + of new API. Such names should be renamed when stabilized; preferably with + a macro aliasing the old name to the new one. + See `"Finalizing the API" in PEP 590`_ for an example. + +.. _"Finalizing the API" in PEP 590: https://www.python.org/dev/peps/pep-0590/#finalizing-the-api + + +.. _public-capi: + +Public C API +============ + +CPython's public C API is available when ``Python.h`` is included normally +(that is, without defining macros to select the other variants). + +It should be defined in ``Include/cpython/`` (unless part of the Limited API, +see below). + +Guidelines for expanding/changing the public API: + +- Make sure the new API follows reference counting conventions. + (Following them makes the API easier to reason about, and easier use + in other Python implementations.) + + - Functions *must not* steal references + - Functions *must not* return borrowed references + - Functions returning references *must* return a strong reference + +- Make sure the ownership rules and lifetimes of all applicable struct + fields, arguments and return values are well defined. + + +Limited API +=========== + +The Limited API is a subset of the C API designed to guarantee ABI +stability across Python 3 versions. +Defining the macro ``Py_LIMITED_API`` will limit the exposed API to +this subset. + +No changes that break the Stable ABI are allowed. + +The Limited API should be defined in ``Include/``, excluding the +``cpython`` and ``internal`` subdirectories. + +Guidelines for changing the Limited API +--------------------------------------- + +- Guidelines for the general :ref:`public-capi` apply. + +- New Limited API should only be defined if ``Py_LIMITED_API`` is set + to the version the API was added in or higher. + (See below for the proper ``#if`` guard.) + +- All parameter types, return values, struct members, etc. need to be part + of the Limited API. + + - Functions that deal with ``FILE*`` (or other types with ABI portability + issues) should not be added. + +- Think twice when defining macros. + + - Macros should not expose implementation details + - Functions must be exported as actual functions, not (only) + as functions-like macros. + - If possible, avoid macros. This makes the Limited API more usable in + languages that don't use the C preprocessor. + +- Please start a public discussion before expanding the Limited API + +- The Limited API and must follow standard C, not just features of currently + supported platforms. The exact C dialect is described in :pep:`7`. + + - Documentation examples (and more generally: the intended use of the API) + should also follow standard C. + - In particular, do not cast a function pointer to ``void*`` (a data pointer) + or vice versa. + +- Think about ease of use for the user. + + - In C, ease of use itself is not very important; what is useful is + reducing boilerplate code needed to use the API. Bugs like to hide in + boiler plates. + + - If a function will be often called with specific value for an argument, + consider making it default (used when ``NULL`` is passed in). + - The Limited API needs to be well documented. + +- Think about future extensions + + - If it's possible that future Python versions will need to add a new + field to your struct, make sure it can be done. + - Make as few assumptions as possible about implementation details that + might change in future CPython versions or differ across C API + implementations. The most important CPython-specific implementation + details involve: + + - The GIL + - :ref:`Garbage collection ` + - Memory layout of PyObject, lists/tuples and other structures + +If following these guidelines would hurt performance, add a fast function +(or macro) to the non-limited API and a stable equivalent to the Limited +API. + +If anything is unclear, or you have a good reason to break the guidelines, +consider discussing the change at the `capi-sig`_ mailing list. + +.. _capi-sig: https://mail.python.org/mailman3/lists/capi-sig.python.org/ + +Adding a new definition to the Limited API +------------------------------------------ + +- Add the declaration to a header file directly under ``Include/``, into a + block guarded with the following: + + .. code-block:: c + + #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03yy0000 + + with the ``yy`` corresponding to the target CPython version, e.g. + ``0x030A0000`` for Python 3.10. +- Append an entry to the Stable ABI manifest, ``Misc/stable_abi.txt``. +- Regenerate the autogenerated files using ``make regen-limited-abi``. + On platforms without ``make``, run this command directly: + + .. code-block:: shell + + ./python ./Tools/scripts/stable_abi.py --generate-all ./Misc/stable_abi.txt + +- Build Python and check the using ``make check-limited-abi``. + On platforms without ``make``, run this command directly: + + .. code-block:: shell + + ./python ./Tools/scripts/stable_abi.py --all ./Misc/stable_abi.txt diff --git a/index.rst b/index.rst index 31aa398d4..126a3428f 100644 --- a/index.rst +++ b/index.rst @@ -324,6 +324,7 @@ Full Table of Contents compiler garbage_collector extensions + c-api coverity clang buildworker From 2cbf00f29b4d740f9d75544c2043adea5798e593 Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Mon, 10 May 2021 20:35:28 +0200 Subject: [PATCH 106/871] Expand the section on renaming to the 'main' branch. (#687) Expand the section on renaming to the 'main' branch by including instructions to rename local forks on GitHub. --- gitbootcamp.rst | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/gitbootcamp.rst b/gitbootcamp.rst index 9cb76ac2f..d77a52762 100644 --- a/gitbootcamp.rst +++ b/gitbootcamp.rst @@ -157,9 +157,30 @@ You may specify more than one branch for deletion. Renaming Branch --------------- -The CPython repository's default branch was renamed from ``master`` to ``main`` -after the Python 3.10b1 release. If you had cloned the repository before this -change, you can rename your local branch as follows:: +The CPython repository's default branch was renamed from ``master`` to +``main`` after the Python 3.10b1 release. + +If you have a fork on GitHub (as described in :ref:`fork-cpython`) that was +created before the rename, you should visit the GitHub page for your fork to +rename the branch there. You only have to do this once. GitHub should +provide you with a dialog for this. If it doesn't (or the dialog was already +dismissed), you can rename the branch in your fork manually `by following +these GitHub instructions `__ + +After renaming the branch in your fork, you need to update any local clones +as well. This only has to be done once per clone:: + + git branch -m master main + git fetch origin + git branch -u origin/main main + git remote set-head origin -a + +(GitHub also provides these instructions after you rename the branch.) + +If you do not have a fork on GitHub, but rather a direct clone of the main +repo created before the branch rename, you still have to update your local +clones. This still only has to be done once per clone. In that case, you can +rename your local branch as follows:: git branch -m master main git fetch upstream From 75297e1b60c569eb4fee8c7f9b97af3ac05b949c Mon Sep 17 00:00:00 2001 From: Mariatta Wijaya Date: Mon, 10 May 2021 17:20:44 -0700 Subject: [PATCH 107/871] Use recent versions of Sphinx and python-docs-theme (#692) --- requirements.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index 2a84c8173..98de25159 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,2 +1,2 @@ -Sphinx -python-docs-theme +Sphinx==3.5.4 +python-docs-theme==2021.5 From d76a41e82327c937f03bf4fa83e376540cfbc998 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 10 May 2021 22:12:10 -0700 Subject: [PATCH 108/871] Add Irit Katriel to the developer log --- developers.csv | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/developers.csv b/developers.csv index ffee4c300..b37e5a15e 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,4 @@ +Irit Katriel,iritkatriel,2021-05-10,, Batuhan Taskaya,isidentical,2020-11-08,, Brandt Bucher,brandtbucher,2020-09-14,, Lysandros Nikolaou,lysnikolaou,2020-06-29,, @@ -24,7 +25,7 @@ Xiang Zhang,zhangyangyu,2016-11-21,, Inada Naoki,methane,2016-09-26,, Xavier de Gaye,xdegaye,2016-06-03,2018-01-25,Privileges relinquished on 2018-01-25 Davin Potts,applio,2016-03-06,, -Martin Panter,vadmium,2015-08-10,, +Martin Panter,vadmium,2015-08-10,2020-11-26, Paul Moore,pfmoore,2015-03-15,, Robert Collins,rbtcollins,2014-10-16,,To work on unittest Berker Peksağ,berkerpeksag,2014-06-26,, @@ -42,8 +43,8 @@ Peter Moody,,2012-05-20,2017-02-10,For ipaddress module; did not make GitHub tra Hynek Schlawack,hynek,2012-05-14,, Richard Oudkerk,,2012-04-29,2017-02-10,For multiprocessing module; did not make GitHub transition Andrew Svetlov,asvetlov,2012-03-13,,At PyCon sprint -Petri Lehtinen,akheron,2011-10-22,, -Meador Inge,meadori,2011-09-19,, +Petri Lehtinen,akheron,2011-10-22,2020-11-12, +Meador Inge,meadori,2011-09-19,2020-11-26, Jeremy Kloth,jkloth,2011-09-12,, Sandro Tosi,sandrotosi,2011-08-01,, Alex Gaynor,alex,2011-07-18,,For PyPy compatibility (since expanded scope) @@ -51,9 +52,9 @@ Charles-François Natali,,2011-05-19,2017-02-10,Did not make GitHub transition Nadeem Vawda,,2011-04-10,2017-02-10,Did not make GitHub transition Jason R. Coombs,jaraco,2011-03-14,,For sprinting on distutils2 Ross Lagerwall,,2011-03-13,2017-02-10,Did not make GitHub transition -Eli Bendersky,eliben,2011-01-11,, +Eli Bendersky,eliben,2011-01-11,2020-11-26, Ned Deily,ned-deily,2011-01-09,, -David Malcolm,davidmalcolm,2010-10-27,, +David Malcolm,davidmalcolm,2010-10-27,2020-11-12,relinquished privileges on 2020-11-12 Tal Einat,taleinat,2010-10-04,,For IDLE Łukasz Langa,ambv,2010-09-08,, Daniel Stutzbach,,2010-08-22,2017-02-10,Did not make GitHub transition @@ -69,16 +70,16 @@ Dino Viehland,DinoV,2010-02-23,,For IronPython compatibility Larry Hastings,larryhastings,2010-02-22,, Victor Stinner,vstinner,2010-01-30,, Stefan Krah,skrah,2010-01-05,2020-10-07,For the decimal module -Doug Hellmann,dhellmann,2009-09-20,,For documentation +Doug Hellmann,dhellmann,2009-09-20,2020-11-11,For documentation; relinquished privileges on 2020-11-11 Frank Wierzbicki,,2009-08-02,2017-02-10,For Jython compatibility; did not make GitHub transition Ezio Melotti,ezio-melotti,2009-06-07,,For documentation -Philip Jenvey,pjenvey,2009-05-07,,For Jython compatibility +Philip Jenvey,pjenvey,2009-05-07,2020-11-26,For Jython compatibility Michael Foord,voidspace,2009-04-01,,For IronPython compatibility R\. David Murray,bitdancer,2009-03-30,, Chris Withers,cjw296,2009-03-08,, Tarek Ziadé,,2008-12-21,2017-02-10,For distutils module; did not make GitHub transition Hirokazu Yamamoto,,2008-08-12,2017-02-10,For Windows build; did not make GitHub transition -Armin Ronacher,mitsuhiko,2008-07-23,,For documentation toolset and ast module +Armin Ronacher,mitsuhiko,2008-07-23,2020-11-26,For documentation toolset and ast module Antoine Pitrou,pitrou,2008-07-16,, Senthil Kumaran,orsenthil,2008-06-16,,For GSoC Jesse Noller,,2008-06-16,2017-02-10,For multiprocessing module; did not make GitHub transition @@ -86,22 +87,22 @@ Jesús Cea,jcea,2008-05-13,,For bsddb module Guilherme Polo,,2008-04-24,2017-02-10,Did not make GitHub transition Jeroen Ruigrok van der Werven,,2008-04-12,2017-02-10,For documentation; did not make GitHub transition Benjamin Peterson,benjaminp,2008-03-25,,For bug triage -David Wolever,wolever,2008-03-17,,For 2to3 module -Trent Nelson,tpn,2008-03-17,, +David Wolever,wolever,2008-03-17,2020-11-21,For 2to3 module +Trent Nelson,tpn,2008-03-17,2020-11-26, Mark Dickinson,mdickinson,2008-01-06,,For maths-related work -Amaury Forgeot d'Arc,amauryfa,2007-11-09,, +Amaury Forgeot d'Arc,amauryfa,2007-11-09,2020-11-26, Christian Heimes,tiran,2007-10-31,, Bill Janssen,,2007-08-28,2017-02-10,For ssl module; did not make GitHub transition Jeffrey Yasskin,,2007-08-09,2017-02-10,Did not make GitHub transition Mark Summerfield,,2007-08-01,2017-02-10,For documentation; did not make GitHub transition -Alexandre Vassalotti,avassalotti,2007-05-21,,For GSoC +Alexandre Vassalotti,avassalotti,2007-05-21,2020-11-12,For GSoC Travis E. Oliphant,,2007-04-17,2017-02-10,Did not make GitHub transition Eric V. Smith,ericvsmith,2007-02-28,,For PEP 3101 in a sandbox Josiah Carlson,,2007-01-06,2017-02-10,For asyncore and asynchat modules; did not make GitHub transition Collin Winter,,2007-01-05,2017-02-10,For PEP access; did not make GitHub transition Richard Jones,,2006-05-23,2017-02-10,For Need for Speed sprint; did not make GitHub transition Kristján Valur Jónsson,,2006-05-17,2017-02-10,For Need for Speed sprint; did not make GitHub transition -Jack Diederich,jackdied,2006-05-17,,For Need for Speed sprint +Jack Diederich,jackdied,2006-05-17,2020-11-26,For Need for Speed sprint Steven Bethard,,2006-04-27,2017-02-10,For PEP access and SourceForge maintenance; did not make GitHub transition Gerhard Häring,,2006-04-23,2017-02-10,Did not make the GitHub transition George Yoshida,,2006-04-17,2017-02-10,For tracker administration; did not make GitHub transition @@ -115,10 +116,10 @@ Facundo Batista,facundobatista,2004-10-16,, Sean Reifschneider,,2004-09-17,2017-02-10,Did not make GitHub transition Johannes Gijsbers,,2004-08-14,2005-07-27,Privileges relinquished on 2005-07-27 Matthias Klose,doko42,2004-08-04,, -PJ Eby,pjeby,2004-03-24,, +PJ Eby,pjeby,2004-03-24,2020-11-26, Vinay Sajip,vsajip,2004-02-20,, Hye-Shik Chang,,2003-12-10,2017-02-10,Did not make GitHub transition -Armin Rigo,,2003-10-24,2012,Privileges relinquished in 2012 +Armin Rigo,,2003-10-24,2012-06-01,Privileges relinquished in 2012 Andrew McNamara,,2003-06-09,2017-02-10,Did not make GitHub transition Samuele Pedroni,,2003-05-16,2017-02-10,Did not make GitHub transition Alex Martelli,aleaxit,2003-04-22,, @@ -145,10 +146,10 @@ Steven M. Gava,,2001-06-25,2017-02-10,Did not make GitHub transition Steve Purcell,,2001-03-22,2017-02-10,Did not make GitHub transition Jim Fulton,,2000-10-06,2017-02-10,Did not make GitHub transition Ka-Ping Yee,,2000-10-03,2017-02-10,Did not make GitHub transition -Lars Gustäbel,gustaebel,2000-09-21,,For tarfile module +Lars Gustäbel,gustaebel,2000-09-21,2020-11-26,For tarfile module Neil Schemenauer,nascheme,2000-09-15,, Martin v. Löwis,,2000-09-08,2017-02-10,Did not make GitHub transition -Thomas Heller,theller,2000-09-07,, +Thomas Heller,theller,2000-09-07,2020-11-18, Moshe Zadka,,2000-07-29,2005-04-08,Privileges relinquished on 2005-04-08 Thomas Wouters,Yhg1s,2000-07-14,, Peter Schneider-Kamp,,2000-07-10,2017-02-10,Did not make GitHub transition @@ -163,12 +164,12 @@ Eric S. Raymond,,2000-06-02,2017-02-10,Did not make GitHub transition Greg Stein,,1999-11-07,2017-02-10,Did not make GitHub transition Just van Rossum,,1999-01-22,2017-02-10,Did not make GitHub transition Greg Ward,,1998-12-18,2017-02-10,Did not make GitHub transition -Andrew Kuchling,akuchling,1998-04-09,2021-02-22,Privileges relinquished on 2021-02-22 +Andrew Kuchling,akuchling,1998-04-09,, Ken Manheimer,,1998-03-03,2005-04-08,Privileges relinquished on 2005-04-08 -Jeremy Hylton,jeremyhylton,1997-08-13,, +Jeremy Hylton,jeremyhylton,1997-08-13,2020-11-26, Roger E. Masse,,1996-12-09,2017-02-10,Did not make GitHub transition Fred Drake,freddrake,1996-07-23,, Barry Warsaw,warsaw,1994-07-25,, Jack Jansen,jackjansen,1992-08-13,, -Sjoerd Mullender,sjoerdmullender,1992-08-04,, +Sjoerd Mullender,sjoerdmullender,1992-08-04,2020-11-14, Guido van Rossum,gvanrossum,1989-12-25,, From 4a87aa0b9ec33d2f070e62df420fd4aa207ff902 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 10 May 2021 22:13:43 -0700 Subject: [PATCH 109/871] Add a link to the CoC --- coredev.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/coredev.rst b/coredev.rst index af8b8579b..195b5f559 100644 --- a/coredev.rst +++ b/coredev.rst @@ -81,7 +81,7 @@ The steps to gaining commit privileges are: https://github.com/python/voters/ - Your preferred email address to subscribe to python-committers with - - A reminder about the Code of Conduct and to report issues to the PSF + - A reminder about the `Code of Conduct`_ and to report issues to the PSF Conduct WG 6. Once you have provided the pertinent details, your various new privileges @@ -91,6 +91,8 @@ The steps to gaining commit privileges are: :ref:`developers` 9. An announcement email by the steering council member handling your new membership will be sent to python-committers + +.. _Code of Conduct: https://www.python.org/psf/conduct/ Mailing Lists From 05426d053643fd8b057e7ab272a69ea3fd99e4fa Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Tue, 11 May 2021 14:03:33 +0100 Subject: [PATCH 110/871] Added myself as traceback expert (#694) --- experts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/experts.rst b/experts.rst index 1f0c96192..11d65b982 100644 --- a/experts.rst +++ b/experts.rst @@ -236,7 +236,7 @@ tkinter gpolo, serhiy.storchaka token tokenize meador.inge trace belopolsky -traceback +traceback iritkatriel tracemalloc vstinner tty twouters* turtle gregorlingl, willingc From ad91664f51ddda8fc3ea095c0a3c9d92dd2d9b73 Mon Sep 17 00:00:00 2001 From: "Kerwin(Kaihui) Sun" Date: Tue, 11 May 2021 23:11:34 +0800 Subject: [PATCH 111/871] Remove py27 in setup step (#691) --- index.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/index.rst b/index.rst index 126a3428f..733938dbe 100644 --- a/index.rst +++ b/index.rst @@ -47,8 +47,7 @@ instructions please see the :ref:`setup guide `. ./python -m test -j3 On :ref:`most ` Mac OS X systems, replace :file:`./python` - with :file:`./python.exe`. On Windows, use :file:`python.bat`. With Python - 2.7, replace ``test`` with ``test.regrtest``. + with :file:`./python.exe`. On Windows, use :file:`python.bat`. 5. Create a new branch where your work for the issue will go, e.g.:: From d292422120918888302c6116932f873475d503c4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Wirtel?= Date: Wed, 12 May 2021 10:51:33 +0200 Subject: [PATCH 112/871] Pip updates during the installation of the virtual env (#688) --- Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/Makefile b/Makefile index b9f2a0d9c..3f3963296 100644 --- a/Makefile +++ b/Makefile @@ -43,6 +43,7 @@ clean: venv: $(PYTHON) -m venv venv + ./venv/bin/python3 -m pip install --upgrade pip ./venv/bin/python3 -m pip install -r requirements.txt html: venv From 3e2805a576bd466767137234a21c8d0a26f9b292 Mon Sep 17 00:00:00 2001 From: Tal Einat <532281+taleinat@users.noreply.github.com> Date: Sat, 15 May 2021 22:41:04 +0300 Subject: [PATCH 113/871] remove taleinat as an expert for the statistics module (#697) --- experts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/experts.rst b/experts.rst index 11d65b982..001ff642a 100644 --- a/experts.rst +++ b/experts.rst @@ -211,7 +211,7 @@ spwd sqlite3 ghaering ssl janssen, christian.heimes, dstufft, alex stat christian.heimes -statistics steven.daprano, rhettinger, taleinat +statistics steven.daprano, rhettinger string stringprep struct mark.dickinson, meador.inge From 51591ad2849eef0204149530a7b562ea897c40c4 Mon Sep 17 00:00:00 2001 From: Bonifacio de Oliveira Date: Mon, 17 May 2021 05:48:11 -0300 Subject: [PATCH 114/871] minor grammar improvement (GH-696) --- docquality.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docquality.rst b/docquality.rst index 2f26f7c55..8e20d0c7b 100644 --- a/docquality.rst +++ b/docquality.rst @@ -99,7 +99,7 @@ lives in a `separate repository`_ and bug reports should be submitted to the Our devguide workflow uses continuous integration and deployment so changes to the devguide are normally published when the pull request is merged. Changes -to CPython documentation follows the workflow of a CPython release and is +to CPython documentation follow the workflow of a CPython release and are published in the release. From 6e960efe0c28d36c7810fdf64d9d3300c22a0397 Mon Sep 17 00:00:00 2001 From: "Kerwin(Kaihui) Sun" Date: Mon, 17 May 2021 16:53:26 +0800 Subject: [PATCH 115/871] Remove py27 notes in Setup Page. (#699) --- setup.rst | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/setup.rst b/setup.rst index 385f142f2..1a6dde4df 100644 --- a/setup.rst +++ b/setup.rst @@ -266,12 +266,6 @@ Visual Studio to continue development. See the `readme`_ for more details on what other software is necessary and how to build. -.. note:: **Python 2.7** uses Microsoft Visual Studio 2008, which is most easily - obtained through an MSDN subscription. To use the build files in the - `PCbuild directory`_ you will also need Visual Studio 2010, see the `2.7 - readme`_ for more details. If you have VS 2008 but not 2010 you can use the - build files in the `PC/VS9.0 directory`_, see the `VS9 readme`_ for details. - .. note:: If you are using the Windows Subsystem for Linux (WSL), clone the repository from a native Windows terminal program like cmd.exe command prompt or PowerShell as well as use a build of git targeted for Windows, e.g., the @@ -281,11 +275,6 @@ to build. .. _this documentation: https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html .. _Visual Studio 2017: https://www.visualstudio.com/ .. _readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt -.. _PCbuild directory: https://github.com/python/cpython/tree/2.7/PCbuild/ -.. _2.7 readme: https://github.com/python/cpython/blob/2.7/PCbuild/readme.txt -.. _PC/VS9.0 directory: https://github.com/python/cpython/tree/2.7/PC/VS9.0/ -.. _VS9 readme: https://github.com/python/cpython/blob/2.7/PC/VS9.0/readme.txt - .. _build-dependencies: @@ -415,9 +404,7 @@ and ``make``:: There will sometimes be optional modules added for a new release which won't yet be identified in the OS level build dependencies. In those cases, -just ask for assistance on the core-mentorship list. If working on bug -fixes for Python 2.7, use ``python`` in place of ``python3`` in the above -commands. +just ask for assistance on the core-mentorship list. Explaining how to build optional dependencies on a UNIX based system without root access is beyond the scope of this guide. From c8f80aea8217604759360c7f13a8a484042500f5 Mon Sep 17 00:00:00 2001 From: Bonifacio de Oliveira Date: Tue, 18 May 2021 19:26:00 -0300 Subject: [PATCH 116/871] minor grammar improvement (#703) --- triaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/triaging.rst b/triaging.rst index 60025394b..4724c9f9c 100644 --- a/triaging.rst +++ b/triaging.rst @@ -57,7 +57,7 @@ Becoming a member of the Python triage team Any Python core developers are welcome to invite a Python contributor to the Python triage team. Do note that the responsibilities of a Python triager -is more elevated than a developer on bpo. For example, the Python triager +are more elevated than a developer on bpo. For example, the Python triager has access to more repositories than just CPython. Triagers will be responsible to handle not just issues, but also pull requests, and even managing backports. From d84dd7a9b0b206a139473a4f8b0ffedc4e9d8328 Mon Sep 17 00:00:00 2001 From: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Date: Tue, 25 May 2021 06:29:33 +0800 Subject: [PATCH 117/871] Bring compiler docs up to speed with Python 3.10 (#706) --- compiler.rst | 218 ++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 165 insertions(+), 53 deletions(-) diff --git a/compiler.rst b/compiler.rst index 40384cb07..9cc2a1644 100644 --- a/compiler.rst +++ b/compiler.rst @@ -139,8 +139,8 @@ that is needed to completely free all memory used by the compiler. In general, unless you are working on the critical core of the compiler, memory management can be completely ignored. But if you are working at either the very beginning of the compiler or the end, you need to care about how the arena -works. All code relating to the arena is in either :file:`Include/pyarena.h` or -:file:`Python/pyarena.c`. +works. All code relating to the arena is in either +:file:`Include/Internal/pycore_pyarena.h` or :file:`Python/pyarena.c`. ``PyArena_New()`` will create a new arena. The returned ``PyArena`` structure will store pointers to all memory given to it. This does the bookkeeping of @@ -159,49 +159,131 @@ are very rare. However, if you've allocated a PyObject, you must tell the arena about it by calling ``PyArena_AddPyObject()``. -Parse Tree to AST ------------------ +Source Code to AST +------------------ + +The AST is generated from source code using the function +``_PyParser_ASTFromString()`` or ``_PyParser_ASTFromFile()`` +(from :file:`Parser/peg_api.c`) depending on the input type. + +After some checks, a helper function in :file:`Parser/parser.c` begins applying +production rules on the source code it receives; converting source code to +tokens and matching these tokens recursively to their corresponding rule. The +rule's corresponding rule function is called on every match. These rule +functions follow the format :samp:`xx_rule`. Where *xx* is the grammar rule +that the function handles and is automatically derived from +:file:`Grammar/python.gram` by :file:`Tools/peg_generator/pegen/c_generator.py`. + +Each rule function in turn creates an AST node as it goes along. It does this +by allocating all the new nodes it needs, calling the proper AST node creation +functions for any required supporting functions and connecting them as needed. +This continues until all nonterminal symbols are replaced with terminals. If an +error occurs, the rule functions backtrack and try another rule function. If +there are no more rules, an error is set and the parsing ends. + +The AST node creation helper functions have the name :samp:`_PyAST_{xx}` +where *xx* is the AST node that the function creates. These are defined by the +ASDL grammar and contained in :file:`Python/Python-ast.c` (which is generated by +:file:`Parser/asdl_c.py` from :file:`Parser/Python.asdl`). This all leads to a +sequence of AST nodes stored in ``asdl_seq`` structs. + +To demonstrate everything explained so far, here's the +rule function responsible for a simple named import statement such as +``import sys``. Note that error-checking and debugging code has been +omitted. Removed parts are represented by ``...``. +Furthermore, some comments have been added for explanation. These comments +may not be present in the actual code. + +.. code-block:: c -The AST is generated from the parse tree (see :file:`Python/ast.c`) using the -function ``PyAST_FromNode()``. + // This is the production rule (from python.gram) the rule function + // corresponds to: + // import_name: 'import' dotted_as_names + static stmt_ty + import_name_rule(Parser *p) + { + ... + stmt_ty _res = NULL; + { // 'import' dotted_as_names + ... + Token * _keyword; + asdl_alias_seq* a; + // The tokenizing steps. + if ( + (_keyword = _PyPegen_expect_token(p, 513)) // token='import' + && + (a = dotted_as_names_rule(p)) // dotted_as_names + ) + { + ... + // Generate an AST for the import statement. + _res = _PyAST_Import ( a , ...); + ... + goto done; + } + ... + } + _res = NULL; + done: + ... + return _res; + } -The function begins a tree walk of the parse tree, creating various AST -nodes as it goes along. It does this by allocating all new nodes it -needs, calling the proper AST node creation functions for any required -supporting functions, and connecting them as needed. -Do realize that there is no automated nor symbolic connection between -the grammar specification and the nodes in the parse tree. No help is -directly provided by the parse tree as in yacc. +To improve backtracking performance, some rules (chosen by applying a +``(memo)`` flag in the grammar file) are memoized. Each rule function checks if +a memoized version exists and returns that if so, else it continues in the +manner stated in the previous paragraphs. -For instance, one must keep track of which node in the parse tree -one is working with (e.g., if you are working with an 'if' statement -you need to watch out for the ':' token to find the end of the conditional). +There are macros for creating and using ``asdl_xx_seq *`` types, where *xx* is +a type of the ASDL sequence. Three main types are defined +manually -- ``generic``, ``identifier`` and ``int``. These types are found in +:file:`Python/asdl.c` and its corresponding header file +:file:`Include/Internal/pycore_asdl.h`. Functions and macros +for creating ``asdl_xx_seq *`` types are as follows: -The functions called to generate AST nodes from the parse tree all have -the name :samp:`ast_for_{xx}` where *xx* is the grammar rule that the function -handles (``alias_for_import_name`` is the exception to this). These in turn -call the constructor functions as defined by the ASDL grammar and -contained in :file:`Python/Python-ast.c` (which was generated by -:file:`Parser/asdl_c.py`) to create the nodes of the AST. This all leads to a -sequence of AST nodes stored in ``asdl_seq`` structs. +``_Py_asdl_generic_seq_new(Py_ssize_t, PyArena *)`` + Allocate memory for an ``asdl_int_seq`` of the specified length +``_Py_asdl_identifier_seq_new(Py_ssize_t, PyArena *)`` + Allocate memory for an ``asdl_identifier_seq`` of the specified length +``_Py_asdl_int_seq_new(Py_ssize_t, PyArena *)`` + Allocate memory for an ``asdl_generic_seq`` of the specified length -Function and macros for creating and using ``asdl_seq *`` types as found -in :file:`Python/asdl.c` and :file:`Include/asdl.h` are as follows: +In addition to the three types mentioned above, some ASDL sequence types are +automatically generated by :file:`Parser/asdl_c.py` and found in +:file:`Include/Internal/pycore_ast.h`. Macros for using both manually defined +and automatically generated ASDL sequence types are as follows: -``_Py_asdl_seq_new(Py_ssize_t, PyArena *)`` - Allocate memory for an ``asdl_seq`` for the specified length -``asdl_seq_GET(asdl_seq *, int)`` +``asdl_seq_GET(asdl_xx_seq *, int)`` + Get item held at a specific position in an ``asdl_xx_seq`` +``asdl_seq_SET(asdl_xx_seq *, int, stmt_ty)`` + Set a specific index in an ``asdl_xx_seq`` to the specified value + +Untyped counterparts exist for some of the typed macros. These are useful +when a function needs to manipulate a generic ASDL sequence: + +``asdl_seq_GET_UNTYPED(asdl_seq *, int)`` Get item held at a specific position in an ``asdl_seq`` -``asdl_seq_SET(asdl_seq *, int, stmt_ty)`` +``asdl_seq_SET_UNTYPED(asdl_seq *, int, stmt_ty)`` Set a specific index in an ``asdl_seq`` to the specified value ``asdl_seq_LEN(asdl_seq *)`` - Return the length of an ``asdl_seq`` + Return the length of an ``asdl_seq`` or ``asdl_xx_seq`` + +Note that typed macros and functions are recommended over their untyped +counterparts. Typed macros carry out checks in debug mode and aid +debugging errors caused by incorrectly casting from ``void *``. If you are working with statements, you must also worry about keeping track of what line number generated the statement. Currently the line number is passed as the last parameter to each ``stmt_ty`` function. +.. versionchanged:: 3.9 + The new PEG parser generates an AST directly without creating a + parse tree. ``Python/ast.c`` is now only used to validate the AST for + debugging purposes. + +.. seealso:: :pep:`617` (PEP 617 -- New PEG parser for CPython) + Control Flow Graphs ------------------- @@ -248,13 +330,13 @@ global). With that done, the second pass essentially flattens the CFG into a list and calculates jump offsets for final output of bytecode. The conversion process is initiated by a call to the function -``PyAST_CompileObject()`` in :file:`Python/compile.c`. This function does both the +``_PyAST_Compile()`` in :file:`Python/compile.c`. This function does both the conversion of the AST to a CFG and outputting final bytecode from the CFG. The AST to CFG step is handled mostly by two functions called by -``PyAST_CompileObject()``; ``PySymtable_BuildObject()`` and ``compiler_mod()``. The former +``_PyAST_Compile()``; ``_PySymtable_Build()`` and ``compiler_mod()``. The former is in :file:`Python/symtable.c` while the latter is in :file:`Python/compile.c`. -``PySymtable_BuildObject()`` begins by entering the starting code block for the +``_PySymtable_Build()`` begins by entering the starting code block for the AST (passed-in) and then calling the proper :samp:`symtable_visit_{xx}` function (with *xx* being the AST node type). Next, the AST tree is walked with the various code blocks that delineate the reach of a local variable @@ -369,6 +451,9 @@ command before adding the new bytecode target to :file:`Python/ceval.c` will result in an error. You should only run ``make regen-importlib`` after the new bytecode target has been added. +.. note:: On Windows, running the ``./build.bat`` script will automatically + regenerate the required files without requiring additional arguments. + Finally, you need to introduce the use of the new bytecode. Altering :file:`Python/compile.c` and :file:`Python/ceval.c` will be the primary places to change. You must add the case for a new opcode into the 'switch' @@ -419,7 +504,28 @@ Important Files asdl_c.py "Generate C code from an ASDL description." Generates - :file:`Python/Python-ast.c` and :file:`Include/Python-ast.h`. + :file:`Python/Python-ast.c` and :file:`Include/Internal/pycore_ast.h`. + + parser.c + The new PEG parser introduced in Python 3.9. + Generated by :file:`Tools/peg_generator/pegen/c_generator.py` + from the grammar :file:`Grammar/python.gram`. Creates the AST from + source code. Rule functions for their corresponding production rules + are found here. + + peg_api.c + Contains high-level functions which are used by the interpreter to + create an AST from source code . + + pegen.c + Contains helper functions which are used by functions in + :file:`Parser/parser.c` to construct the AST. Also contains helper + functions which help raise better error messages when parsing source + code. + + pegen.h + Header file for the corresponding :file:`Parser/pegen.c`. Also contains + definitions of the ``Parser`` and ``Token`` structs. + Python/ @@ -437,7 +543,7 @@ Important Files identifier. Used by :file:`Python-ast.c` for marshalling AST nodes. ast.c - Converts Python's parse tree into the abstract syntax tree. + Used for validating the AST. ast_opt.c Optimizes the AST. @@ -469,31 +575,37 @@ Important Files + Include/ - Python-ast.h - Contains the actual definitions of the C structs as generated by - :file:`Python/Python-ast.c`. - "Automatically generated by :file:`Parser/asdl_c.py`". - - asdl.h - Header for the corresponding :file:`Python/ast.c`. - - ast.h - Declares ``PyAST_FromNode()`` external (from :file:`Python/ast.c`). - code.h Header file for :file:`Objects/codeobject.c`; contains definition of ``PyCodeObject``. - symtable.h - Header for :file:`Python/symtable.c`. ``struct symtable`` and - ``PySTEntryObject`` are defined here. - - pyarena.h - Header file for the corresponding :file:`Python/pyarena.c`. - opcode.h One of the files that must be modified if :file:`Lib/opcode.py` is. + + Internal/ + + pycore_ast.h + Contains the actual definitions of the C structs as generated by + :file:`Python/Python-ast.c`. + "Automatically generated by :file:`Parser/asdl_c.py`". + + pycore_asdl.h + Header for the corresponding :file:`Python/ast.c` + + pycore_ast.h + Declares ``_PyAST_Validate()`` external (from :file:`Python/ast.c`). + + pycore_symtable.h + Header for :file:`Python/symtable.c`. ``struct symtable`` and + ``PySTEntryObject`` are defined here. + + pycore_parser.h + Header for the corresponding :file:`Parser/peg_api.c`. + + pycore_pyarena.h + Header file for the corresponding :file:`Python/pyarena.c`. + + + Objects/ codeobject.c From 34b2f7e64c8b45686af54ed8aa93ba63d1fdf601 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Wed, 26 May 2021 13:42:10 +0200 Subject: [PATCH 118/871] Migrate from Freenode to LiberaChat (#708) --- buildbots.rst | 4 ++-- communication.rst | 11 +++++------ documenting.rst | 3 ++- help.rst | 4 ++-- 4 files changed, 11 insertions(+), 11 deletions(-) diff --git a/buildbots.rst b/buildbots.rst index 58535ca1e..ca342938d 100644 --- a/buildbots.rst +++ b/buildbots.rst @@ -52,8 +52,8 @@ There are three ways of visualizing recent build results: you. You can also access builder information by clicking on the builder status bubbles in the top line. -If you like IRC, having an IRC client open to the #python-dev channel on -irc.freenode.net is useful. Any time a builder changes state (last build +If you like IRC, having an IRC client open to the #python-dev-notifs channel on +irc.libera.chat is useful. Any time a builder changes state (last build passed and this one didn't, or vice versa), a message is posted to the channel. Keeping an eye on the channel after pushing a changeset is a simple way to get notified that there is something you should look in to. diff --git a/communication.rst b/communication.rst index 90b5a1027..7d39972ef 100644 --- a/communication.rst +++ b/communication.rst @@ -50,7 +50,7 @@ to any issue, subscribe to python-bugs-list_. General Python questions should go to `python-list`_ or `tutor`_ or similar resources, such as StackOverflow_ or the ``#python`` IRC channel -on Freenode_. +on Libera.Chat_. `Core-Workflow `_ mailing list is the place to discuss and work on improvements to the CPython @@ -72,7 +72,7 @@ RSS feed readers. .. _python-list: https://mail.python.org/mailman/listinfo/python-list .. _tutor: https://mail.python.org/mailman/listinfo/tutor .. _StackOverflow: https://stackoverflow.com/ -.. _Freenode: http://freenode.net/ +.. _Libera.Chat: https://libera.chat/ Zulip ----- @@ -85,10 +85,9 @@ IRC Some core developers enjoy spending time on IRC discussing various issues regarding Python's development in the ``#python-dev`` channel on -``irc.freenode.net``. This is not a place to ask for help with Python, but to -discuss issues related to Python's own development. You can use Freenode's -`Web interface `_ if you don't have an IRC -client. +``irc.libera.chat``. This is not a place to ask for help with Python, but to +discuss issues related to Python's own development. See also the +``#python-dev-notifs`` channel for bots notifications. Blogs diff --git a/documenting.rst b/documenting.rst index 5c712a001..a0521ce9e 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1736,7 +1736,8 @@ How to get help Discussions about translations occur on the `doc-sig `_ mailing list, -and there's a freenode IRC channel, ``#python-doc``. +and there's a `Libera.Chat IRC `_ channel, +``#python-doc``. Translation FAQ diff --git a/help.rst b/help.rst index b0c726cdc..dea37d0f9 100644 --- a/help.rst +++ b/help.rst @@ -41,7 +41,7 @@ Ask #python-dev --------------- If you are comfortable with IRC you can try asking on ``#python-dev`` (on -the `freenode`_ network). Typically there are a number of experienced +the `Libera.Chat`_ network). Typically there are a number of experienced developers, ranging from triagers to core developers, who can answer questions about developing for Python. As with the mailing lists, ``#python-dev`` is for questions involving the development *of* Python @@ -52,7 +52,7 @@ whereas ``#python`` is for questions concerning development *with* Python. You may not be able to access the history of this channel, so it cannot be used as a "knowledge base" of sorts. -.. _freenode: https://freenode.net/ +.. _Libera.Chat: https://libera.chat/ Zulip ----- From cc033adeb955ba0dd5fd246c459de75be2f5199b Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 12 May 2021 07:24:16 -0700 Subject: [PATCH 119/871] fix typo in status for 3.10 --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 733938dbe..7c7810ca8 100644 --- a/index.rst +++ b/index.rst @@ -99,7 +99,7 @@ Status of Python branches +==================+==============+=============+================+================+=======================+ | main | *TBD* | features | *TBD* | *TBD* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.10 | :pep:`619` | prerelase | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +| 3.10 | :pep:`619` | prerelease | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.9 | :pep:`596` | bugfix | 2020-10-05 | *TBD* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ From d6714a4c1ce1404fde158f4faa93c6fe0ab15feb Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Fri, 4 Jun 2021 18:54:24 +0100 Subject: [PATCH 120/871] Switch the theme to 'furo' (#679) --- conf.py | 18 ++++-------------- python-logo.png | Bin 0 -> 9955 bytes requirements.txt | 3 ++- 3 files changed, 6 insertions(+), 15 deletions(-) create mode 100644 python-logo.png diff --git a/conf.py b/conf.py index 407109588..468fe1f17 100644 --- a/conf.py +++ b/conf.py @@ -29,7 +29,7 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.intersphinx', 'sphinx.ext.todo'] +extensions = ['sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx_copybutton'] intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} todo_include_todos = True @@ -93,11 +93,8 @@ # -- Options for HTML output --------------------------------------------------- # Use the upstream python-docs-theme -html_theme = 'python_docs_theme' -html_theme_options = { - 'collapsiblesidebar': True, - 'issues_url': 'https://github.com/python/devguide/issues/new', -} +html_theme = 'furo' +html_theme_options = {} # The name for this set of Sphinx documents. If None, it defaults to @@ -107,13 +104,6 @@ # Path to find HTML templates. templates_path = ['tools/templates'] -# Custom sidebar templates, filenames relative to this file. -html_sidebars = { - # Defaults taken from http://www.sphinx-doc.org/en/stable/config.html#confval-html_sidebars - # Removes the quick search block - '**': ['localtoc.html', 'globaltoc.html', 'relations.html', 'customsourcelink.html'], -} - # Additional static files. html_static_path = ['tools/static'] @@ -122,7 +112,7 @@ # The name of an image file (relative to this directory) to place at the top # of the sidebar. -#html_logo = None +html_logo = "python-logo.png" # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 diff --git a/python-logo.png b/python-logo.png new file mode 100644 index 0000000000000000000000000000000000000000..49ea8f5ba9252776c0a9e0ab37d21700b42e3b2b GIT binary patch literal 9955 zcmXxK2|QH)_dkB`btf}Jj2U~>SVAHc#_rnpB^05Wq-4pyrgH5m)kGpxBTKSxg%VTQ zvZsU++7K!4B4nBW_5J+*kH^g8HLr8d>zwC#p7Xjh^SU?1-qvh8QIrS(z;+9BV+R0$ zAnXm`IWS3wwGfl+~h3B0G~ zleJZa5%Y~MEQId4;r#x z8_+@uX~FsIH%7Fe`?Syp>;Z%pj@81L7RaPwL>*REG1z^27{l9FRS3LCizwB=81}z|WykK(*=m3>8+G4I zT1bJqZx$`I7~^04Y&QF&9ed1nD{GA24>s&^d-k9a`=c$!Cws)4{oWd580$}sb5(4N zlc0Q!b2YT^hwKS^b)O7%zby8s6)oTn8>=SP60c|)(KYN*3wD>5x=$v%lV*M*D+AjY z_d*?P9RWJM&V`%B&!G&`>u7Wa&%P0b=d1&9A+iM6GES1g5Pc-RjUlWV^UKAxG~Ac( zl8h>AW5Z};EQ&=?PMMG&4Po%l5B|uycQ#EcYjw5I21044*Ao<7_mBcL1OQ7RClnk?((e3^?O9EV3=|Ls?{`4Jl;W!)?El|D7{8>K<|V z9Fa}47u*poeMo`)A>?$wce`i&hl*W;pKd*t_`p1CD7>>S(Zl30|K9Ml_8sAR!m{vz z%mfpO$IfxJp2`LmOc@>#iGtKu;pB;!>G6=l=xd<`)2pQ4-$fTULw`0-H%*rm7n3|n z%NItUS~cxVZfPEx58W2u+uJLC^ytx8#M#+wmi?Uh=DUCSvbN5+`qRq$H5E_re0oEV zY(DOciPs(xm=#I){CA>a-zks31*)|Vt;<>k&V&zZH=S+nl0Sdz{)I1lpXgrgyEd>m zcC~P9SGt+ZhPbh zXn2*xt0n6=`$u(g>9`Be0g#tM{CB0hd*Sx6f!Tb_ePR2wSd3o#^B~|6lnN=Rw7xhtWKf}11ay(sX zY2MD$2)&6|vVz~=`*T7h)^}rmdqwM)uP9o$`0hgd6CeM2(FOi-Qtg_( z=LlW9s}WIA-wc^}tS(N^ySVo_u~ZL$(}#m2t@c>9$$Z+BV+o%lVIosA*$ zPXo0(b=3MeeTSG%5W$@ohe zXrShzwesN0moJm6tr=6UffMoTBxRkZv${oz-mM?*C80Y~ak=JJh-+$X_K@MnDmP%f&-JVf+7au;AkD3=e+xDoCf3oW! zIDCLYN)bbzsvPS2a~P2xydnT)MOQLIhL~IOx&IWuK10teDr3>-vK}hS`EdBP4Xk zPOdt~cL%AMFx;9`_03sR#4h>i>rX+dS{xAZ(zUgN{^bVD>#j)WFSk`u#rc+ZVNaut zDaxC#Mfu%BNK6XJN&H<+l%WSfc$faApI}&ADD+QN`%kIPS|V9lZ_~)h{kEk%xGY|9 z?Z^Amd2s43R;s-S=T#uH?^CW%i#gQZQLVw`ZX5knZM$)c>sJ{9?K>fCUy$U%J)`>G z^AAyaa&R{{$K_cOu91MIqA8L&t1#T@XtOSTUBTY4`Z0;$j8&+ib;-#Qya4)VcfW0H zYet{tKmta4+Hi{oPb+->G?>FnP;80^@QhZxl}Y#?7Xtp7N>?zC zE_A&Cqo_=*=KQ@Z=DdhEzXgCl!$-w4?~vZj3vEqESyG!l+#d%~H#zWu0b+tG&=rtc zb3Y*~0AE%#X3rW?Z40{p{#Z5UCk?GYzk9!YBmMYg9034vAP4y7&9RLuPB;?^00l!y z5O;g*;OS48X)2J2e*b>>cW#v=^)5hSkeTGbB_E+rXN`vLZa_o;B!HIS!#bbPkGDpj z${6zlkPoE#OWH)(ESrCGO`P(~~_b%;n#u0-%{gg3%i4uV1VpF&W+#8e# zVmR=FuF*1 z{%czh&Yr+huz^Me0uxFB(txDBJq9-(IYC$R~3~|gx8*b3w9*11x>%0Y!It8F37$8jx^Z_<8f!uK@!b6_i3&6qG0WZut1jbjU zpOl%Lu!NLiCX{n_?OtlR(4tv)@@0|NLBno*pb%4Xp>^Wxs^ML?djC7Fz2%OXnZJi` zUHJ9vKdGCqiI5Dg$8#U+64pG9yf_7~FU|V8_crVX5C8Q2b587Gq~)M>_3k6S`=7Nd z30%$5(^=0*LoHDi@Eq73H6KywS-GNTXOwKU^yLeU_-ANP2A-iP@1gtn_0?bQmuz{&7P-UlaW*2!Nqb zvjlNK29cVq$wOb~UDfUq##eEYN%T8_4YUoiJxCP3lbgG~Na*&8{n^Taud+u4K`xxN zt^a1>BgoQo8Ea~Hmi}8{&luW9MQ;GQkRY)^qfRsbR*fj0E`uZ^Pasn@ zxZt z(}LP7BXJUpck6vjx%HHa{x&mvBfvDl%95@HSx*rCE|6HRtd_U%em}m~0t-vSQ|hT> zBo@XB;0AI$?bQ*ui2%`H609Joth`W{@=Rip5KLkkpyg=nEw5sNh6EZ9sKxf=B!uF) zx$XP$R&sEU0a64l5wjpP8n;HL$dSpxtP<{FS@9Ya=hH*`IO;6Mt~ zg!M#5;y$|YRB4WH0Ri+uQFw$HX--|28TtaiRYFXez|nub2HU|;!;9lm04RphtSm(0 zNvB=uf=H}`s5;04y=akXapHj@iCGhc252%W9A!9@0NjP9=6xU?xWG`$(f$K3sDtZ? zDU)PbAd!`uH6%EYtlVO1L7i8xzA^cNq_la9wIsCp}5@&3j4Igk$)dnF6ba>m`K z9r5Rp$BnHwem?aQvqXZQU1srIgDTvk4mO**LtqoG*el*7Ig;2AY;v)0%tnf?(2vWV zlAlbM7l`d|APTQmmj%4Vfw{Ob-Qr{U6FrAmopuE=PhSwh+EkqTNrlOsC&51!N8UQd zdeQcinbK8~^a7KAZo9|bNr9^UxXx{{Uq7nJL;bZMcOE7Ri`?+1)ENQ+;IGmc*UjyH zkg1+i1rY(vz=-3{cDTD!3UuQd4JRUo>&zJEWxa0-^smloj>cV?m#6a3753=UUH9ZG z5AaO(zmPMJmul=y;2O*;wk|ru;Tn%1aOv-pn`rT5$v~cnyvISSItPJ*gLRh1ZeiVi zPZPa89z0WF1r{cj7wq`2&oiv|mbIhJ1DT_7Aw{D#AD53cw2c25UmK5}_>;TbtkbX0 z>ebveetbImZoU$zm^xsqmIlbiZd8nPI+0O@$?EkBfP+SRYnD)KcjnQKEL$nC6Y4Vs z*Sr_xJHDN`jdoXDI5jb?H0iueXrTjzb@W~}@QO`}kr-UU!7|B3>3}C#mTXXLBBad! z>Aa{kw)(v=w159=9QPsZ9o&g6=<*9o;E&`0OH;a)q+A!+)X&;aM)#s&eCW{z=m(tC z?nbKkyLWOq7k9q(2uraK+7w`*9Wsa0?@2bRs1%`jwD#k8bD#uxHwW~*aM}4+aA0k4 zWcm{Wx86PpU3`Fj`v)mNoW}SWMPco@IG9e1Tt9odw%&&iI@d8yoK6F*!5%Mz=?j1} z^6P!IB79CL@Rd1a+wV!%sCbCR6!QtAh7#~5SM zg6f=S^Q{DY1Dr|;%yn9<*#*X^G@$)_xVqgngttWUfbWFM85!^lk@Z>H=;1q5Eq1iok+fJlD!#3ow#M zp`64EN4{xx>Us8XW^HT-F_?cLViWqk|1|`SW9HgDndtyGs7Scnfiy|lvIf-GiadRo#Xc1Ovk>uS-1-zu&eM zy&%2hm<+Zh>dp@*qq|KmNk}xJpMpOGtXR10Ql=heO}xcxh)mA~4mdM1lb^xd0ibd^ zPyxAKTh) z;;<|2fFPGjGr^jV${aRgE$E=qWR@A?ws)oUxAhMO!B+y1_Bm`7kJ-7)XmXCPgMH6$bM9Lx7gY$6i_-mq%e9d8A=-*v1%Mw!M zAE^1_B3VpX$lV7oDuziRkRgBIW5o$rMI{cmZt)Dzj(r4ZN+TSg;;GeYdG^LCVa0?5 zNkiQhh&kl?KKzHSAZP3$%W~GY6U!fvR0xEcyDm-uN)`sBrWg}et&Ztbv(H39HbD@*K!QE zcoR@~lYtICGrhf;I{bdw(_<%1Fc-ah4)xF_G|&^2z|kvjKE$}+NjX31&VbHgY|NP> zHdVCPtkj|=3-d==hid;md&g+5t-dg<{mA5Y*Yx!{Pw!3}q+gu|ea79yH3{F$9uEAv z(%4s#9Fe<}k^l4BkWk+fuS)%fAtSd7pAXvUEtc>$%|4Ic`!me3#J56JOsq*$SO#*3 zgF~gLidEh7&Fx30bAH7r4v8OwHYru8K^WA!Z+jzQ=2aLOzZV?+Hh-sgx$xinD`+gd z&TWG9yP^-Z_!yP{tlYk6b}Y^rvTTMR{}~6Q-vQM;5w`t|)?n1cAw!(IN&?zU0Ybtg zm@Y#vkN<0Yy4*;J+|P$wT-4#HI>rmxK8zGPabR)p zWHt}Xjgy^x&5wG1f=;g-@%IgA7hL$%%C{i5lWJLq%1=Cv7ky%CBp7%dmLpXC*k;bJ zFnOk|eFH^BCt}wK;p1fDw_=%*$a+n3G5=7+^l-GGbB4)I*III(dm? z`@$;9oX(m$A|Y`VjwE%~USH+Kg{3)ka^EAE1BnUHak0=!^R=u>zPDBi@{G*bKpw)g z(tHLZ6E*B$3SIdw8+lor&sRMcxPyareDY7eoxb#4?v*?B^3S&|sBb$Tee|pFUrPx{ zzNa~WgP&qV8rbzqI6R%v{GvxG-lmtIY{;=349g>&~hFECTUAl0D%^}qZJ?|F=Jr3Sp@C%+IJI6f_Uky3;5wZEC< zBN8MM5_DislP5{~|4}Q@8>&+mkOGwRgcQ?xFIW)}|JSb7fNLd688W?_K_h2Xsi^cY z7|G~n#!m_(elOj4^Q`h6Lqja9-y9yuOc*711qfFduKYJ!{jjHydff=LDt_h`El0;lMJ0j3;Ebg`o1Qc=20wat*~Rly zzKHMzQ5ICkqj@g#;X=C|PL3|i$s{STv?ij-X(UY2m!E5_qT&Q@*pZrJ|G&A^K5iwtDS0JEYz(g?Zzg1xC>r&V|-V=xL)+TZnaz*4#%}1k=a^pdGp{ zUv13}QPe*47$`tNO_SXr6(-!diP|lJ?T2YFm;m+`wOZxEK@- zI~A3mDk>4+yN^Hg>~gB%PQv<4507;tLw|@^I|>g9TN$~-V`+Rd_m4SfK<#*bIn@p8 zb~06Vz*i=$DDRd;fE-M)KPMIHBybrx1%28< ziFTe=QON}U{zhy6-pAWZBPuEvB_xJTsDrYx4Mc9x62^F?vb5Tk4gOQ+FhU{la>{>K zQW{J2Uh>9%FT_LVDNHF&&DPekl4G3hX{{2EZ#qe#HmHY(n2}*$m(7wDbyYG3Y&p%x z%n}|(S&?T!_M?4H;P3{Kz9qVh4j;k;Z%lqd!dsCfslL^skbS!|kPfu9+QTl+C1iT^ zo=HU{1)F1ACAUBB#w;G;2>>_o12^K&!!F&2h-kxnE>4bwxvNJJq#j(e1ie7Nt$Y=b zrL)Ma@sYOwIn?0lEQtr?|B{_Sh5u7e<{To~_u?H`m*oG)q#$Jng~Ue@2LIshVuh8H zX;+e&A~AA6?4R9Th@X()sPr%}B}Egy*#v;jDrWyjP`ed!_}nDKa&NQyA3=D61=O69 zOqb7#%RyrfUDd>YLO|lynsW5& z4kW)sVaOH8!Jw#a$9vkOZ0juvd)f|VZ5b4z?6ajDirt|uZd=5W2OW@iMcf0Aiz3d* zqXyW;&Xwrm%9-u#QGE;2%iyDeqTAqEp+e|-_IW35j3Vy0R+jG8Wd(FXU`ax~Dus!l z*TlGwVotWRD#>pw)yOOvL?VxYdn=?0PV65#BZXREUh|R?f`v{2?Cc9JTv*(PavWG`rLc%QX+0~fnh~ex`fp$=*D(gKXF~k30mjfx`$ij=6(~P&u zqyNaHn*z@dGw_2h&jt)dPl}E-B-lIDRz1#C{6QP};J)agv<5cKWzJk693A|4z53Eh;iXpl(gP*q%mQy$?*VPQZP_ZB zPhxWRu77`;nNF;tR;@4w{t|tDDyY-mWO8b=*XO^HP{V0R)R7FCAFvUlU4`=mfcNeGG~1u99y zkOwJH1zY@r6u1`~wshFArNf5*r;{YRoH574uyoZ(mN5R38U4A63W;=$b^oLXZ_Dt| zKk2Z5&sqj^B5*Rzx@zmd?SVZG&oTC>*g*;afi0!u>EvYA{0lz&}~L zGfd_>@a7trWjnsJA((`n>KEiZLMid9hUDEnVg_FXsJ>y_pAE80_Yrop+X&uJy|goC654-a7MC&dFS)% ztBStZ;r?1{0YAlYrN2=4I8Pk#RBtyw<%;Y|e}=sx4rLX4Bow7&nmK|Fh~2)H9v`NFL(!i*uMSA(JxPzh<3<#YW1THOJ9vBd z&}{}wJ0H8rFm!iDrTTSCs>V6N88_@Mz|jo@uj~=e7W~$&5@)3uYs!`6mFogs|ECit zWy=VL1Wue#1mK_U-1fJnztXj=vBOd2lUH)yKvmTGwa0{9~`hz zxY!z9%=E#QEH33^zjnUB;Ll9$i#x!V(xUn$#5`{gZ{CU9I^H`#qxF!+=4%8S${+0& zQWaU5JO9^{Csb97;zOU#EoanIlvDG5{iB)Wgr2t8cr^QM<;I&@0R}H7D{;)0X%Bi( zdtxtL0~zj^)qLpfarEhz(0SZs@W28h_jAzVi}~}0-gI}DQ`_M7*(b3@*N0Wp3h>y& zK3!YS0;F70QF(L9F{oQS9^?UO-b&dQAN=($-%)z|7;mJHv)DawxIx%MXZwJwX0d>^ zUZ3hda9FQyY~?=se(;svm*i)I_2)G;$o3{LMe18mee&=Kw~{UW*f}!aeroBmBP)!u z-RmeH*l}&36498ZZOn)fRKY^24pez85UTY*&HDa`Qg^h+f)1 zni#@^TYT-8uEj7$^e!LE87e&`v=Gbc-M;x?q{*;DI@4B*Uzyd$G{G&mUQX#XLb#4| zyt|(86wb7DUH?^lkhQ%6_OsO7-RPfs9M}o_mD(Tk0jC@IKS!PjebNg3yL9D0C%;T4 z8sQF=5b82Z&qvqYvJK{%i5hSppGfMcXltF4nbeKtX~v0-@re_KNm+*x9%sXAvmTC4 zV2rH{(tH1*?JnZIeHo}K`DI^sh`A&1=Nnz=Qj)Dh?@>N@UH2J!9uP959R4~~WUf+e zf#W6#-JQyK-aGJH`F z60uv~v7hYwc0#BBSxipcX+#^_TT5(!tgziRyQ@!nJlQN2$8FE)u8dDVciYh3{*UuO zS>qx3Dy2wsxe3kZG*<53TfSU-k@C}yGXs*%q?ewvGWy>Q~AAa^#wNI5kqZ#(dOQA^IkRD^sXD%7ryY^ zZiY{yxaom?jDw|}6k_A`PmTTUh(hAxu1`cLV z%GKYYmvTt*%(}xn*B2BE+;#ca;>WEI-W+)xAnW=Nj(>>1IF|pllnu7of9DB|Jw(AC P#sMr$Y>g`n>2d!b!VCd- literal 0 HcmV?d00001 diff --git a/requirements.txt b/requirements.txt index 98de25159..4858270e6 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,2 +1,3 @@ Sphinx==3.5.4 -python-docs-theme==2021.5 +furo +sphinx_copybutton From 63438691a414a1110e45f430563db2e0912aae30 Mon Sep 17 00:00:00 2001 From: Erlend Egeberg Aasland Date: Fri, 4 Jun 2021 20:57:25 +0200 Subject: [PATCH 121/871] Replace Zulip with Discourse (#670) --- communication.rst | 14 ++++++++++---- triaging.rst | 4 ++-- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/communication.rst b/communication.rst index 7d39972ef..90e278e65 100644 --- a/communication.rst +++ b/communication.rst @@ -74,11 +74,17 @@ RSS feed readers. .. _StackOverflow: https://stackoverflow.com/ .. _Libera.Chat: https://libera.chat/ -Zulip ------ -We have our own `zulipchat `_ instance. This should be -used to discuss the development of Python only. +Discourse +--------- + +We have our own `Discourse`_ forum for both developers and users. This forum +complements the `python-dev`_, `python-ideas`_, `python-help`_, and +`python-list`_ mailing lists. Also, voting for new core developers takes place +at `Discourse`_. + +.. _Discourse: https://discuss.python.org/ + IRC --- diff --git a/triaging.rst b/triaging.rst index 4724c9f9c..6c7468b18 100644 --- a/triaging.rst +++ b/triaging.rst @@ -66,13 +66,13 @@ They can request this to any core developer, and the core developer can pass the request to the `Python organization admin `_ on GitHub. The request -can be made confidentially via a DM in Zulip or Discourse, or publicly by opening +can be made confidentially via a DM in Discourse, or publicly by opening an `issue in the core-workflow repository `_. Any contributor who is not already a developer on b.p.o can also self-nominate to be a member of Python triage team. They can request this to any core developer, -confidentially via DM in Zulip or Discourse, or publicly by opening an issue in core-workflow. +confidentially via DM in Discourse, or publicly by opening an issue in core-workflow. If a core developer agrees and is willing to vouch for them, the core developer can pass the request to the GitHub administrator. They should also be added as developer on bpo. From df3ee1a8db42a4292c807db2e9a9e87867a1f8b1 Mon Sep 17 00:00:00 2001 From: Erlend Egeberg Aasland Date: Fri, 4 Jun 2021 20:58:10 +0200 Subject: [PATCH 122/871] Sync section 25.7 with Python/compile.c (#672) --- compiler.rst | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/compiler.rst b/compiler.rst index 9cc2a1644..baa0ae825 100644 --- a/compiler.rst +++ b/compiler.rst @@ -367,6 +367,12 @@ Emission of bytecode is handled by the following macros: ``ADDOP(struct compiler *, int)`` add a specified opcode +``ADDOP_NOLINE(struct compiler *, int)`` + like ``ADDOP`` without a line number; used for artificial opcodes without + no corresponding token in the source code +``ADDOP_IN_SCOPE(struct compiler *, int)`` + like ``ADDOP``, but also exits current scope; used for adding return value + opcodes in lambdas and closures ``ADDOP_I(struct compiler *, int, Py_ssize_t)`` add an opcode that takes an integer argument ``ADDOP_O(struct compiler *, int, PyObject *, TYPE)`` @@ -387,10 +393,14 @@ Emission of bytecode is handled by the following macros: position of the specified PyObject in the consts table. ``ADDOP_LOAD_CONST_NEW(struct compiler *, PyObject *)`` just like ``ADDOP_LOAD_CONST_NEW``, but steals a reference to PyObject -``ADDOP_JABS(struct compiler *, int, basicblock *)`` - create an absolute jump to a basic block -``ADDOP_JREL(struct compiler *, int, basicblock *)`` - create a relative jump to a basic block +``ADDOP_JUMP(struct compiler *, int, basicblock *)`` + create a jump to a basic block +``ADDOP_JUMP_NOLINE(struct compiler *, int, basicblock *)`` + like ``ADDOP_JUMP`` without a line number; used for artificial jumps + without no corresponding token in the source code. +``ADDOP_JUMP_COMPARE(struct compiler *, cmpop_ty)`` + depending on the second argument, add an ``ADDOP_I`` with either an + ``IS_OP``, ``CONTAINS_OP``, or ``COMPARE_OP`` opcode. Several helper functions that will emit bytecode and are named :samp:`compiler_{xx}()` where *xx* is what the function helps with (``list``, From c8079d7cfe38daff9e8d71a4c5313a6591875114 Mon Sep 17 00:00:00 2001 From: andrei kulakov Date: Tue, 8 Jun 2021 17:18:22 -0400 Subject: [PATCH 123/871] Remove superfluous content listing (#712) --- gitbootcamp.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/gitbootcamp.rst b/gitbootcamp.rst index d77a52762..11544f65a 100644 --- a/gitbootcamp.rst +++ b/gitbootcamp.rst @@ -26,8 +26,6 @@ relevant to CPython's workflow. get more information about that in `git documentation `_ -.. contents:: - .. _fork-cpython: Forking CPython GitHub Repository From 58f03637f180039c39c5474fc068fd18e12eb2f3 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 15 Jun 2021 22:23:30 +0100 Subject: [PATCH 124/871] Restrict the version of sphinx-copybutton to pick up bugfixes (#715) --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 4858270e6..75e94998f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ Sphinx==3.5.4 furo -sphinx_copybutton +sphinx_copybutton>=0.3.3 From 4e6c76a8deb5aa651327f7cb76ee96dd3b73586b Mon Sep 17 00:00:00 2001 From: Harry Date: Mon, 21 Jun 2021 22:16:23 +0100 Subject: [PATCH 125/871] Fix capitalization of PCbuild (GH-719) --- setup.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/setup.rst b/setup.rst index 1a6dde4df..535cdcd2f 100644 --- a/setup.rst +++ b/setup.rst @@ -258,9 +258,9 @@ are downloaded: .. code-block:: dosbatch - PCBuild\build.bat + PCbuild\build.bat -After this build succeeds, you can open the ``PCBuild\pcbuild.sln`` solution in +After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution in Visual Studio to continue development. See the `readme`_ for more details on what other software is necessary and how From dd6dba2ca552f2324345cac3350b9ccbf915d524 Mon Sep 17 00:00:00 2001 From: Marco Ippolito Date: Wed, 23 Jun 2021 16:08:25 -0300 Subject: [PATCH 126/871] Add advice on building all modules on Debian-like (GH-673) --- setup.rst | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/setup.rst b/setup.rst index 535cdcd2f..24b7c97d2 100644 --- a/setup.rst +++ b/setup.rst @@ -325,10 +325,14 @@ Then you should update the packages index:: Now you can install the build dependencies via ``apt``:: - $ sudo apt-get build-dep python3.6 + $ sudo apt-get build-dep python3 -If that package is not available for your system, try reducing the minor -version until you find a package that is available. +If you want to build all optional modules, install the following packages and +their dependencies:: + + $ sudo apt-get install build-essential gdb lcov libbz2-dev libffi-dev \ + libgdbm-dev liblzma-dev libncurses5-dev libreadline6-dev \ + libsqlite3-dev libssl-dev lzma lzma-dev tk-dev uuid-dev zlib1g-dev .. _MacOS: From 91893b0246ded3e599f4f876d149d44fe3bb58c6 Mon Sep 17 00:00:00 2001 From: Will Schlitzer Date: Thu, 24 Jun 2021 10:46:42 +0100 Subject: [PATCH 127/871] Fix minor typo in coredev.rst (GH-720) This pull request changed "requied" to "required" in the "Gaining Commit Privileges" section in coredev.rst. --- coredev.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/coredev.rst b/coredev.rst index 195b5f559..1066ae2a0 100644 --- a/coredev.rst +++ b/coredev.rst @@ -70,7 +70,7 @@ The steps to gaining commit privileges are: 2. The poll is announced on python-committers 3. Wait for the poll to close and see if the results confirm your membership - as per the voting results requied by PEP 13 + as per the voting results required by PEP 13 4. The person who nominated you emails the steering council with your email address and a request that the council either accept or reject the proposed membership From 0ae9c851419d5eba1af2623129ffd415d75be73c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 14 Jul 2021 18:22:42 -0700 Subject: [PATCH 128/871] Bump sphinx from 3.5.4 to 4.1.1 (#725) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 3.5.4 to 4.1.1. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v3.5.4...v4.1.1) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 75e94998f..60f42feeb 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ -Sphinx==3.5.4 +Sphinx==4.1.1 furo sphinx_copybutton>=0.3.3 From f8ca097b0eeb1f85863a631044ac2765aea670d2 Mon Sep 17 00:00:00 2001 From: Mariatta Wijaya Date: Thu, 15 Jul 2021 12:11:33 -0700 Subject: [PATCH 129/871] Delete travis.yml file (#726) GitHub Actions seem sufficient for our needs, we don't need both running at the same time. --- .travis.yml | 16 ---------------- 1 file changed, 16 deletions(-) delete mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 5608206df..000000000 --- a/.travis.yml +++ /dev/null @@ -1,16 +0,0 @@ -language: python -python: 3.6 -cache: pip - -install: python3 -m pip install -U pip -r requirements.txt - -jobs: - include: - - stage: build-docs - script: sphinx-build -n -W -q -b html -d _build/doctrees . _build/html - - - stage: link-check - script: sphinx-build -b linkcheck -d _build/doctrees . _build/linkcheck - - allow_failures: - - stage: link-check From 0e6f02833f0adb4a229bacf425b56a3620a50474 Mon Sep 17 00:00:00 2001 From: Edison J Abahurire <20975616+SimiCode@users.noreply.github.com> Date: Fri, 16 Jul 2021 00:20:56 +0300 Subject: [PATCH 130/871] Fixes #717: Example for Running a single test case fails (#718) The class seems to be reloaded using a different name at the bottom of `https://github.com/python/cpython/blob/main/Lib/test/test_abc.py` --- runtests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtests.rst b/runtests.rst index 74e38ff8b..2d6c3eb48 100644 --- a/runtests.rst +++ b/runtests.rst @@ -48,7 +48,7 @@ verbose mode (using ``-v``), so that individual failures are detailed:: To run a single test case, use the ``unittest`` module, providing the import path to the test case:: - ./python -m unittest -v test.test_abc.TestABC + ./python -m unittest -v test.test_abc.TestABC_Py If you have a multi-core or multi-CPU machine, you can enable parallel testing using several Python processes so as to speed up things:: From af52f4be7c9fab2ec2b658b130fc1686c3586b04 Mon Sep 17 00:00:00 2001 From: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com> Date: Fri, 16 Jul 2021 18:03:18 -0400 Subject: [PATCH 131/871] bpo-42349: Clarify basicblocks and give some examples (#714) Co-authored-by: Mariatta Wijaya --- compiler.rst | 56 +++++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/compiler.rst b/compiler.rst index baa0ae825..91fe254d2 100644 --- a/compiler.rst +++ b/compiler.rst @@ -288,26 +288,42 @@ number is passed as the last parameter to each ``stmt_ty`` function. Control Flow Graphs ------------------- -A control flow graph (often referenced by its acronym, CFG) is a -directed graph that models the flow of a program using basic blocks that -contain the intermediate representation (abbreviated "IR", and in this -case is Python bytecode) within the blocks. Basic blocks themselves are -a block of IR that has a single entry point but possibly multiple exit -points. The single entry point is the key to basic blocks; it all has -to do with jumps. An entry point is the target of something that -changes control flow (such as a function call or a jump) while exit -points are instructions that would change the flow of the program (such -as jumps and 'return' statements). What this means is that a basic -block is a chunk of code that starts at the entry point and runs to an -exit point or the end of the block. - -As an example, consider an 'if' statement with an 'else' block. The -guard on the 'if' is a basic block which is pointed to by the basic -block containing the code leading to the 'if' statement. The 'if' -statement block contains jumps (which are exit points) to the true body -of the 'if' and the 'else' body (which may be ``NULL``), each of which are -their own basic blocks. Both of those blocks in turn point to the -basic block representing the code following the entire 'if' statement. +A *control flow graph* (often referenced by its acronym, CFG) is a +directed graph that models the flow of a program. A node of a CFG is +not an individual bytecode instruction, but instead represents a +sequence of bytecode instructions that always execute sequentially. +Each node is called a *basic block* and must always execute from +start to finish, with a single entry point at the beginning and a +single exit point at the end. If some bytecode instruction *a* needs +to jump to some other bytecode instruction *b*, then *a* must occur at +the end of its basic block, and *b* must occur at the start of its +basic block. + +As an example, consider the following code snippet: + +.. code-block:: Python + + if x < 10: + f1() + f2() + else: + g() + end() + +The ``x < 10`` guard is represented by its own basic block that +compares ``x`` with ``10`` and then ends in a conditional jump based on +the result of the comparison. This conditional jump allows the block +to point to both the body of the ``if`` and the body of the ``else``. The +``if`` basic block contains the ``f1()`` and ``f2()`` calls and points to +the ``end()`` basic block. The ``else`` basic block contains the ``g()`` +call and similarly points to the ``end()`` block. + +Note that more complex code in the guard, the ``if`` body, or the ``else`` +body may be represented by multiple basic blocks. For instance, +short-circuiting boolean logic in a guard like ``if x or y:`` +will produce one basic block that tests the truth value of ``x`` +and then points both (1) to the start of the ``if`` body and (2) to +a different basic block that tests the truth value of y. CFGs are usually one step away from final code output. Code is directly generated from the basic blocks (with jump targets adjusted based on the From 37c53440bc0b504dc8a54486ca2c5fb0daed338f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Sat, 17 Jul 2021 10:26:28 +0200 Subject: [PATCH 132/871] Replace @ilevkivskyi with @Fidget-Spinner as typing expert (#728) --- experts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/experts.rst b/experts.rst index 001ff642a..dc84bdfb9 100644 --- a/experts.rst +++ b/experts.rst @@ -241,7 +241,7 @@ tracemalloc vstinner tty twouters* turtle gregorlingl, willingc types yselivanov -typing gvanrossum, levkivskyi* +typing gvanrossum, kj unicodedata lemburg, ezio.melotti unittest michael.foord*, ezio.melotti, rbcollins unittest.mock michael.foord* From 69c49e1b8d17c2f3e6fe3f4c22a0d78e6702a7f6 Mon Sep 17 00:00:00 2001 From: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Date: Fri, 30 Jul 2021 23:15:13 +0800 Subject: [PATCH 133/871] Add build.bat counterparts to to checklist (GH-729) --- grammar.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/grammar.rst b/grammar.rst index 73226e115..571d6869c 100644 --- a/grammar.rst +++ b/grammar.rst @@ -23,14 +23,16 @@ Checklist Note: sometimes things mysteriously don't work. Before giving up, try ``make clean``. * :file:`Grammar/python.gram`: The grammar, with actions that build AST nodes. After changing - it, run ``make regen-pegen``, to regenerate :file:`Parser/parser.c`. + it, run ``make regen-pegen`` (or ``build.bat --regen`` on Windows), to + regenerate :file:`Parser/parser.c`. (This runs Python's parser generator, ``Tools/peg_generator``). * :file:`Grammar/Tokens` is a place for adding new token types. After changing it, run ``make regen-token`` to regenerate :file:`Include/token.h`, :file:`Parser/token.c`, :file:`Lib/token.py` and :file:`Doc/library/token-list.inc`. If you change both ``python.gram`` and ``Tokens``, - run ``make regen-token`` before ``make regen-pegen``. + run ``make regen-token`` before ``make regen-pegen``. On Windows, + ``build.bat --regen`` will regenerate both at the same time. * :file:`Parser/Python.asdl` may need changes to match the grammar. Then run ``make regen-ast`` to regenerate :file:`Include/Python-ast.h` and :file:`Python/Python-ast.c`. From 8d97b43d87477a9c7fba439a9a0916e18be97fd8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 30 Jul 2021 17:16:12 +0200 Subject: [PATCH 134/871] Bump sphinx from 4.1.1 to 4.1.2 (GH-732) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.1.1 to 4.1.2. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.1.1...v4.1.2) Signed-off-by: dependabot[bot] --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 60f42feeb..c90e30088 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ -Sphinx==4.1.1 +Sphinx==4.1.2 furo sphinx_copybutton>=0.3.3 From 187740f715c036de61e4330a47f2ee406d06a335 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Fri, 30 Jul 2021 17:19:06 +0200 Subject: [PATCH 135/871] bpo-44777: Explicitly list the python-buildbots mailing list as a contact method (GH-733) --- buildbots.rst | 22 ++++++++++++++++++++++ pullrequest.rst | 3 +++ 2 files changed, 25 insertions(+) diff --git a/buildbots.rst b/buildbots.rst index ca342938d..47f319c30 100644 --- a/buildbots.rst +++ b/buildbots.rst @@ -25,6 +25,28 @@ build results after you push a change to the repository. It is therefore important that you get acquainted with the way these results are presented, and how various kinds of failures can be explained and diagnosed. +In case of trouble +------------------ + +Please read this page in full. If your questions aren't answered here and you +need assistance with the buildbots, a good way to get help is to either: + +* contact the ``python-buildbots@python.org`` mailing list where all buildbot + worker owners are subscribed; or +* contact the release manager of the branch you have issues with. + +Buildbot failures on Pull Requests +---------------------------------- + +The ``bedevere-bot`` on GitHub will put a message on your merged Pull Request +if building your commit on a stable buildbot worker fails. Take care to +evaluate the failure, even if it looks unrelated at first glance. + +Not all failures will generate a notification since not all builds are executed +after each commit. In particular, reference leaks builds take several hours to +complete so they are done periodically. This is why it's important for you to +be able to check the results yourself, too. + Checking results of automatic builds ------------------------------------ diff --git a/pullrequest.rst b/pullrequest.rst index 7cb194490..e6b6b767c 100644 --- a/pullrequest.rst +++ b/pullrequest.rst @@ -37,6 +37,9 @@ Here is a quick overview of how you can contribute to CPython: #. `Create Pull Request`_ on GitHub to merge a branch from your fork +#. Make sure the continuous integration checks on your Pull Request + are green (i.e. successful) + #. Review and address `comments on your Pull Request`_ #. When your changes are merged, you can :ref:`delete the PR branch From 4edf354b82aece8c6b7bc0c5fb2b082a06071f5d Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Wed, 4 Aug 2021 17:54:48 +0100 Subject: [PATCH 136/871] Add a new guide on how to understand and use the PEG parser (#734) --- compiler.rst | 1 + grammar.rst | 16 +- index.rst | 3 +- parser.rst | 915 +++++++++++++++++++++++++++++++++++++++++++++++ requirements.txt | 2 +- 5 files changed, 928 insertions(+), 9 deletions(-) create mode 100644 parser.rst diff --git a/compiler.rst b/compiler.rst index 91fe254d2..85943118b 100644 --- a/compiler.rst +++ b/compiler.rst @@ -40,6 +40,7 @@ these (see :doc:`grammar`). Abstract Syntax Trees (AST) --------------------------- +.. _compiler-ast-trees: .. sidebar:: Green Tree Snakes diff --git a/grammar.rst b/grammar.rst index 571d6869c..471d4e827 100644 --- a/grammar.rst +++ b/grammar.rst @@ -9,13 +9,15 @@ Abstract There's more to changing Python's grammar than editing :file:`Grammar/python.gram`. Here's a checklist. -NOTE: These instructions are for Python 3.9 and beyond. Earlier -versions use a different parser technology. You probably shouldn't -try to change the grammar of earlier Python versions, but if you -really want to, use GitHub to track down the earlier version of this -file in the devguide. (Python 3.9 itself actually supports both -parsers; the old parser can be invoked by passing ``-X oldparser``.) - +.. note:: + These instructions are for Python 3.9 and beyond. Earlier + versions use a different parser technology. You probably shouldn't + try to change the grammar of earlier Python versions, but if you + really want to, use GitHub to track down the earlier version of this + file in the devguide. + +For more information on how to use the new parser, check the +:ref:`section on how to use CPython's parser `. Checklist --------- diff --git a/index.rst b/index.rst index 7c7810ca8..37afb6e85 100644 --- a/index.rst +++ b/index.rst @@ -262,6 +262,7 @@ Additional Resources * Help with ... * :doc:`exploring` * :doc:`grammar` + * :doc:`parser` * :doc:`compiler` * :doc:`garbage_collector` * Tool support @@ -293,7 +294,6 @@ Full Table of Contents ---------------------- .. toctree:: - :numbered: :maxdepth: 3 setup @@ -320,6 +320,7 @@ Full Table of Contents gdb exploring grammar + parser compiler garbage_collector extensions diff --git a/parser.rst b/parser.rst new file mode 100644 index 000000000..71387b639 --- /dev/null +++ b/parser.rst @@ -0,0 +1,915 @@ +.. _parser: + +Guide of CPython's Parser +========================= + +:Author: Pablo Galindo Salgado + +.. highlight:: none + +Abstract +-------- + +The Parser in CPython is currently a `PEG (Parser Expression Grammar) +`_ parser. The first +version of the parser used to be an `LL(1) +`_ based parser that was one of the +oldest parts of CPython implemented before it was replaced by :pep:`617`. In +particular, both the current parser and the old LL(1) parser are the output of a +`parser generator `_. This +means that the way the parser is written is by feeding a description of the +Grammar of the Python language to a special program (the parser generator) which +outputs the parser. The way the Python language is changed is therefore by +modifying the grammar file and developers rarely need to interact with the +parser generator itself other than use it to generate the parser. + +How PEG Parsers Work +-------------------- + +.. _how-peg-parsers-work: + +A PEG (Parsing Expression Grammar) grammar (like the current one) differs from a +context-free grammar in that the way it is written more closely +reflects how the parser will operate when parsing it. The fundamental technical +difference is that the choice operator is ordered. This means that when writing:: + + rule: A | B | C + +a context-free-grammar parser (like an LL(1) parser) will generate constructions +that given an input string will *deduce* which alternative (``A``, ``B`` or ``C``) +must be expanded, while a PEG parser will check if the first alternative succeeds +and only if it fails, will it continue with the second or the third one in the +order in which they are written. This makes the choice operator not commutative. + +Unlike LL(1) parsers, PEG-based parsers cannot be ambiguous: if a string parses, +it has exactly one valid parse tree. This means that a PEG-based parser cannot +suffer from the ambiguity problems that can arise with LL(1) parsers and with +context-free grammars in general. + +PEG parsers are usually constructed as a recursive descent parser in which every +rule in the grammar corresponds to a function in the program implementing the +parser and the parsing expression (the "expansion" or "definition" of the rule) +represents the "code" in said function. Each parsing function conceptually takes +an input string as its argument, and yields one of the following results: + +* A "success" result. This result indicates that the expression can be parsed by + that rule and the function may optionally move forward or consume one or more + characters of the input string supplied to it. +* A "failure" result, in which case no input is consumed. + +Notice that "failure" results do not imply that the program is incorrect, nor do +they necessarily mean that the parsing has failed. Since the choice operator is +ordered, a failure very often merely indicates "try the following option". A +direct implementation of a PEG parser as a recursive descent parser will present +exponential time performance in the worst case, because PEG parsers have +infinite lookahead (this means that they can consider an arbitrary number of +tokens before deciding for a rule). Usually, PEG parsers avoid this exponential +time complexity with a technique called "packrat parsing" [1]_ which not only +loads the entire program in memory before parsing it but also allows the parser +to backtrack arbitrarily. This is made efficient by memoizing the rules already +matched for each position. The cost of the memoization cache is that the parser +will naturally use more memory than a simple LL(1) parser, which normally are +table-based. + + +Key ideas +~~~~~~~~~ + +.. important:: + Don't try to reason about a PEG grammar in the same way you would to with an EBNF + or context free grammar. PEG is optimized to describe **how** input strings will + be parsed, while context-free grammars are optimized to generate strings of the + language they describe (in EBNF, to know if a given string is in the language, you need + to do work to find out as it is not immediately obvious from the grammar). + +* Alternatives are ordered ( ``A | B`` is not the same as ``B | A`` ). +* If a rule returns a failure, it doesn't mean that the parsing has failed, + it just means "try something else". +* By default PEG parsers run in exponential time, which can be optimized to linear by + using memoization. +* If parsing fails completely (no rule succeeds in parsing all the input text), the + PEG parser doesn't have a concept of "where the :exc:`SyntaxError` is". + + +Consequences or the ordered choice operator +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _consequences-of-ordered-choice: + +Although PEG may look like EBNF, its meaning is quite different. The fact +that in PEG parsers alternatives are ordered (which is at the core of how PEG +parsers work) has deep consequences, other than removing ambiguity. + +If a rule has two alternatives and the first of them succeeds, the second one is +**not** attempted even if the caller rule fails to parse the rest of the input. +Thus the parser is said to be "eager". To illustrate this, consider +the following two rules (in these examples, a token is an individual character): :: + + first_rule: ( 'a' | 'aa' ) 'a' + second_rule: ('aa' | 'a' ) 'a' + +In a regular EBNF grammar, both rules specify the language ``{aa, aaa}`` but +in PEG, one of these two rules accepts the string ``aaa`` but not the string +``aa``. The other does the opposite -- it accepts the string the string ``aa`` +but not the string ``aaa``. The rule ``('a'|'aa')'a'`` does +not accept ``aaa`` because ``'a'|'aa'`` consumes the first ``a``, letting the +final ``a`` in the rule consume the second, and leaving out the third ``a``. +As the rule has succeeded, no attempt is ever made to go back and let +``'a'|'aa'`` try the second alternative. The expression ``('aa'|'a')'a'`` does +not accept ``aa`` because ``'aa'|'a'`` accepts all of ``aa``, leaving nothing +for the final ``a``. Again, the second alternative of ``'aa'|'a'`` is not +tried. + +.. caution:: + + The effects of ordered choice, such as the ones illustrated above, may be hidden by many levels of rules. + +For this reason, writing rules where an alternative is contained in the next one is in almost all cases a mistake, +for example: :: + + my_rule: + | 'if' expression 'then' block + | 'if' expression 'then' block 'else' block + +In this example, the second alternative will never be tried because the first one will +succeed first (even if the input string has an ``'else' block`` that follows). To correctly +write this rule you can simply alter the order: :: + + my_rule: + | 'if' expression 'then' block 'else' block + | 'if' expression 'then' block + +In this case, if the input string doesn't have an ``'else' block``, the first alternative +will fail and the second will be attempted without said part. + +Syntax +------ + +The grammar consists of a sequence of rules of the form: :: + + rule_name: expression + +Optionally, a type can be included right after the rule name, which +specifies the return type of the C or Python function corresponding to +the rule: :: + + rule_name[return_type]: expression + +If the return type is omitted, then a ``void *`` is returned in C and an +``Any`` in Python. + +Grammar Expressions +~~~~~~~~~~~~~~~~~~~ + +``# comment`` +''''''''''''' + +Python-style comments. + +``e1 e2`` +''''''''' + +Match e1, then match e2. + +:: + + rule_name: first_rule second_rule + +``e1 | e2`` +''''''''''' + +Match e1 or e2. + +The first alternative can also appear on the line after the rule name +for formatting purposes. In that case, a \| must be used before the +first alternative, like so: + +:: + + rule_name[return_type]: + | first_alt + | second_alt + +``( e )`` +''''''''' + +Match e. + +:: + + rule_name: (e) + +A slightly more complex and useful example includes using the grouping +operator together with the repeat operators: + +:: + + rule_name: (e1 e2)* + +``[ e ] or e?`` +''''''''''''''' + +Optionally match e. + +:: + + rule_name: [e] + +A more useful example includes defining that a trailing comma is +optional: + +:: + + rule_name: e (',' e)* [','] + +``e*`` +'''''' + +Match zero or more occurrences of e. + +:: + + rule_name: (e1 e2)* + +``e+`` +'''''' + +Match one or more occurrences of e. + +:: + + rule_name: (e1 e2)+ + +``s.e+`` +'''''''' + +Match one or more occurrences of e, separated by s. The generated parse +tree does not include the separator. This is otherwise identical to +``(e (s e)*)``. + +:: + + rule_name: ','.e+ + +``&e`` +'''''' + +.. _peg-positive-lookahead: + +Succeed if e can be parsed, without consuming any input. + +``!e`` +'''''' + +.. _peg-negative-lookahead: + +Fail if e can be parsed, without consuming any input. + +An example taken from the Python grammar specifies that a primary +consists of an atom, which is not followed by a ``.`` or a ``(`` or a +``[``: + +:: + + primary: atom !'.' !'(' !'[' + +``~`` +'''''' + +Commit to the current alternative, even if it fails to parse. + +:: + + rule_name: '(' ~ some_rule ')' | some_alt + +In this example, if a left parenthesis is parsed, then the other +alternative won’t be considered, even if some_rule or ‘)’ fail to be +parsed. + +Left recursion +~~~~~~~~~~~~~~ + +PEG parsers normally do not support left recursion but CPython's parser +generator implements a technique similar to the one described in Medeiros et al. +[2]_ but using the memoization cache instead of static variables. This approach +is closer to the one described in Warth et al. [3]_. This allows us to write not +only simple left-recursive rules but also more complicated rules that involve +indirect left-recursion like:: + + rule1: rule2 | 'a' + rule2: rule3 | 'b' + rule3: rule1 | 'c' + +and "hidden left-recursion" like:: + + rule: 'optional'? rule '@' some_other_rule + +Variables in the Grammar +~~~~~~~~~~~~~~~~~~~~~~~~ + +A sub-expression can be named by preceding it with an identifier and an +``=`` sign. The name can then be used in the action (see below), like this: :: + + rule_name[return_type]: '(' a=some_other_rule ')' { a } + +Grammar actions +~~~~~~~~~~~~~~~ + +.. _peg-grammar-actions: + +To avoid the intermediate steps that obscure the relationship between the +grammar and the AST generation the PEG parser allows directly generating AST +nodes for a rule via grammar actions. Grammar actions are language-specific +expressions that are evaluated when a grammar rule is successfully parsed. These +expressions can be written in Python or C depending on the desired output of the +parser generator. This means that if one would want to generate a parser in +Python and another in C, two grammar files should be written, each one with a +different set of actions, keeping everything else apart from said actions +identical in both files. As an example of a grammar with Python actions, the +piece of the parser generator that parses grammar files is bootstrapped from a +meta-grammar file with Python actions that generate the grammar tree as a result +of the parsing. + +In the specific case of the PEG grammar for Python, having actions allows +directly describing how the AST is composed in the grammar itself, making it +more clear and maintainable. This AST generation process is supported by the use +of some helper functions that factor out common AST object manipulations and +some other required operations that are not directly related to the grammar. + +To indicate these actions each alternative can be followed by the action code +inside curly-braces, which specifies the return value of the alternative:: + + rule_name[return_type]: + | first_alt1 first_alt2 { first_alt1 } + | second_alt1 second_alt2 { second_alt1 } + +If the action is omitted and C code is being generated, then there are two +different possibilities: + +1. If there’s a single name in the alternative, this gets returned. +2. If not, a dummy name object gets returned (this case should be avoided). + +If the action is ommited, a default action is generated: + +* If there's a single name in the rule in the rule, it gets returned. + +* If there is more than one name in the rule, a collection with all parsed + expressions gets returned (the type of the collection will be different + in C and Python). + +This default behaviour is primarily made for very simple situations and for +debugging pourposes. + +The full meta-grammar for the grammars supported by the PEG generator is: + +:: + + start[Grammar]: grammar ENDMARKER { grammar } + + grammar[Grammar]: + | metas rules { Grammar(rules, metas) } + | rules { Grammar(rules, []) } + + metas[MetaList]: + | meta metas { [meta] + metas } + | meta { [meta] } + + meta[MetaTuple]: + | "@" NAME NEWLINE { (name.string, None) } + | "@" a=NAME b=NAME NEWLINE { (a.string, b.string) } + | "@" NAME STRING NEWLINE { (name.string, literal_eval(string.string)) } + + rules[RuleList]: + | rule rules { [rule] + rules } + | rule { [rule] } + + rule[Rule]: + | rulename ":" alts NEWLINE INDENT more_alts DEDENT { + Rule(rulename[0], rulename[1], Rhs(alts.alts + more_alts.alts)) } + | rulename ":" NEWLINE INDENT more_alts DEDENT { Rule(rulename[0], rulename[1], more_alts) } + | rulename ":" alts NEWLINE { Rule(rulename[0], rulename[1], alts) } + + rulename[RuleName]: + | NAME '[' type=NAME '*' ']' {(name.string, type.string+"*")} + | NAME '[' type=NAME ']' {(name.string, type.string)} + | NAME {(name.string, None)} + + alts[Rhs]: + | alt "|" alts { Rhs([alt] + alts.alts)} + | alt { Rhs([alt]) } + + more_alts[Rhs]: + | "|" alts NEWLINE more_alts { Rhs(alts.alts + more_alts.alts) } + | "|" alts NEWLINE { Rhs(alts.alts) } + + alt[Alt]: + | items '$' action { Alt(items + [NamedItem(None, NameLeaf('ENDMARKER'))], action=action) } + | items '$' { Alt(items + [NamedItem(None, NameLeaf('ENDMARKER'))], action=None) } + | items action { Alt(items, action=action) } + | items { Alt(items, action=None) } + + items[NamedItemList]: + | named_item items { [named_item] + items } + | named_item { [named_item] } + + named_item[NamedItem]: + | NAME '=' ~ item {NamedItem(name.string, item)} + | item {NamedItem(None, item)} + | it=lookahead {NamedItem(None, it)} + + lookahead[LookaheadOrCut]: + | '&' ~ atom {PositiveLookahead(atom)} + | '!' ~ atom {NegativeLookahead(atom)} + | '~' {Cut()} + + item[Item]: + | '[' ~ alts ']' {Opt(alts)} + | atom '?' {Opt(atom)} + | atom '*' {Repeat0(atom)} + | atom '+' {Repeat1(atom)} + | sep=atom '.' node=atom '+' {Gather(sep, node)} + | atom {atom} + + atom[Plain]: + | '(' ~ alts ')' {Group(alts)} + | NAME {NameLeaf(name.string) } + | STRING {StringLeaf(string.string)} + + # Mini-grammar for the actions + + action[str]: "{" ~ target_atoms "}" { target_atoms } + + target_atoms[str]: + | target_atom target_atoms { target_atom + " " + target_atoms } + | target_atom { target_atom } + + target_atom[str]: + | "{" ~ target_atoms "}" { "{" + target_atoms + "}" } + | NAME { name.string } + | NUMBER { number.string } + | STRING { string.string } + | "?" { "?" } + | ":" { ":" } + +As an illustrative example this simple grammar file allows directly +generating a full parser that can parse simple arithmetic expressions and that +returns a valid C-based Python AST: + +:: + + start[mod_ty]: a=expr_stmt* ENDMARKER { _PyAST_Module(a, NULL, p->arena) } + expr_stmt[stmt_ty]: a=expr NEWLINE { _PyAST_Expr(a, EXTRA) } + + expr[expr_ty]: + | l=expr '+' r=term { _PyAST_BinOp(l, Add, r, EXTRA) } + | l=expr '-' r=term { _PyAST_BinOp(l, Sub, r, EXTRA) } + | term + + term[expr_ty]: + | l=term '*' r=factor { _PyAST_BinOp(l, Mult, r, EXTRA) } + | l=term '/' r=factor { _PyAST_BinOp(l, Div, r, EXTRA) } + | factor + + factor[expr_ty]: + | '(' e=expr ')' { e } + | atom + + atom[expr_ty]: + | NAME + | NUMBER + +Here ``EXTRA`` is a macro that expands to ``start_lineno, start_col_offset, +end_lineno, end_col_offset, p->arena``, those being variables automatically +injected by the parser; ``p`` points to an object that holds on to all state +for the parser. + +A similar grammar written to target Python AST objects: + +:: + + start[ast.Module]: a=expr_stmt* ENDMARKER { ast.Module(body=a or [] } + expr_stmt: a=expr NEWLINE { ast.Expr(value=a, EXTRA) } + + expr: + | l=expr '+' r=term { ast.BinOp(left=l, op=ast.Add(), right=r, EXTRA) } + | l=expr '-' r=term { ast.BinOp(left=l, op=ast.Sub(), right=r, EXTRA) } + | term + + term: + | l=term '*' r=factor { ast.BinOp(left=l, op=ast.Mult(), right=r, EXTRA) } + | l=term '/' r=factor { ast.BinOp(left=l, op=ast.Div(), right=r, EXTRA) } + | factor + + factor: + | '(' e=expr ')' { e } + | atom + + atom: + | NAME + | NUMBER + + +Pegen +----- + +Pegen is the parser generator used in CPython to produce the final PEG parser used by the interpreter. It is the +program that can be used to read the python grammar located in :file:`Grammar/Python.gram` and produce the final C +parser. It contains the following pieces: + +* A parser generator that can read a grammar file and produce a PEG parser written in Python or C that can parse + said grammar. The generator is located at :file:`Tools/peg_generator/pegen`. +* A PEG meta-grammar that automatically generates a Python parser that is used for the parser generator itself + (this means that there are no manually-written parsers). The meta-grammar is + located at :file:`Tools/peg_generator/pegen/metagrammar.gram`. +* A generated parser (using the parser generator) that can directly produce C and Python AST objects. + +The source code for Pegen lives at :file:`Tools/peg_generator/pegen` but normally all typical commands to interact +with the parser generator are executed from the main makefile. + +How to regenerate the parser +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once you have made the changes to the grammar files, to regenerate the ``C`` +parser (the one used by the interpreter) just execute: :: + + make regen-pegen + +using the :file:`Makefile` in the main directory. If you are on Windows you can +use the Visual Studio project files to regenerate the parser or to execute: :: + + ./PCbuild/build.bat --regen + +The generated parser file is located at :file:`Parser/parser.c`. + +How to regenerate the meta-parser +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The meta-grammar (the grammar that describes the grammar for the grammar files +themselves) is located at :file:`Tools/peg_generator/pegen/metagrammar.gram`. +Although it is very unlikely that you will ever need to modify it, if you make any modifications +to this file (in order to implement new Pegen features) you will need to regenerate +the meta-parser (the parser that parses the grammar files). To do so just execute: :: + + make regen-pegen-metaparser + +If you are on Windows you can use the Visual Studio project files +to regenerate the parser or to execute: :: + + ./PCbuild/build.bat --regen + + +Grammatical elements and rules +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Pegen has some special grammatical elements and rules: + +* Strings with single quotes (') (e.g. ``'class'``) denote KEYWORDS. +* Strings with double quotes (") (e.g. ``"match"``) denote SOFT KEYWORDS. +* Upper case names (e.g. ``NAME``) denote tokens in the :file:`Grammar/Tokens` file. +* Rule names starting with `invalid_` are used for specialized syntax errors. + + - These rules are NOT used in the first pass of the parser. + - Only if the first pass fails to parse, a second pass including the invalid + rules will be executed. + - If the parser fails in the second phase with a generic syntax error, the + location of the generic failure of the first pass will be used (this avoids + reporting incorrect locations due to the invalid rules). + - The order of the alternatives involving invalid rules matter + (like any rule in PEG). + +Tokenization +~~~~~~~~~~~~ + +It is common among PEG parser frameworks that the parser does both the parsing and the tokenization, +but this does not happen in Pegen. The reason is that the Python language needs a custom tokenizer +to handle things like indentation boundaries, some special keywords like ``ASYNC`` and ``AWAIT`` +(for compatibility purposes), backtracking errors (such as unclosed parenthesis), dealing with encoding, +interactive mode and much more. Some of these reasons are also there for historical purposes, and some +others are useful even today. + +The list of tokens (all uppercase names in the grammar) that you can use can be found in the :file:`Grammar/Tokens` +file. If you change this file to add new tokens, make sure to regenerate the files by executing: :: + + make regen-token + +If you are on Windows you can use the Visual Studio project files to regenerate the tokens or to execute: :: + + ./PCbuild/build.bat --regen + +How tokens are generated and the rules governing this is completely up to the tokenizer (:file:`Parser/tokenizer.c`) +and the parser just receives tokens from it. + +Memoization +~~~~~~~~~~~ + +As described previously, to avoid exponential time complexity in the parser, memoization is used. + +The C parser used by Python is highly optimized and memoization can be expensive both in memory and time. Although +the memory cost is obvious (the parser needs memory for storing previous results in the cache) the execution time +cost comes for continuously checking if the given rule has a cache hit or not. In many situations, just parsing it +again can be faster. Pegen **disables memoization by default** except for rules with the special marker `memo` after +the rule name (and type, if present): :: + + rule_name[typr] (memo): + ... + +By selectively turning on memoization for a handful of rules, the parser becomes faster and uses less memory. + +.. note:: + Left-recursive rules always use memoization, since the implementation of left-recursion depends on it. + +To know if a new rule needs memoization or not, benchmarking is required +(comparing execution times and memory usage of some considerably big files with +and without memoization). There is a very simple instrumentation API available +in the generated C parse code that allows to measure how much each rule uses +memoization (check the :file:`Parser/pegen.c` file for more information) but it +needs to be manually activated. + +Automatic variables +~~~~~~~~~~~~~~~~~~~ + +To make writing actions easier, Pegen injects some automatic variables in the namespace available +when writing actions. In the C parser, some of these automatic variable names are: + +* ``p``: The parser structure. +* ``EXTRA``: This is a macro that expands to ``(_start_lineno, _start_col_offset, _end_lineno, _end_col_offset, p->arena)``, + which is normally used to create AST nodes as almost all constructors need these attributes to be provided. All of the + location variables are taken from the location information of the current token. + +Hard and Soft keywords +~~~~~~~~~~~~~~~~~~~~~~ + +.. note:: + In the grammar files, keywords are defined using **single quotes** (e.g. `'class'`) while soft + keywords are defined using **double quotes** (e.g. `"match"`). + +There are two kinds of keywords allowed in pegen grammars: *hard* and *soft* +keywords. The difference between hard and soft keywords is that hard keywords +are always reserved words, even in positions where they make no sense (e.g. ``x = class + 1``), +while soft keywords only get a special meaning in context. Trying to use a hard +keyword as a variable will always fail: + +.. code-block:: + + >>> class = 3 + File "", line 1 + class = 3 + ^ + SyntaxError: invalid syntax + >>> foo(class=3) + File "", line 1 + foo(class=3) + ^^^^^ + SyntaxError: invalid syntax + +While soft keywords don't have this limitation if used in a context other the one where they +are defined as keywords: + +.. code-block:: python + + >>> match = 45 + >>> foo(match="Yeah!") + +The ``match`` and ``case`` keywords are soft keywords, so that they are recognized as +keywords at the beginning of a match statement or case block respectively, but are +allowed to be used in other places as variable or argument names. + +You can get a list of all keywords defined in the grammar from Python: + +.. code-block:: python + + >>> import keyword + >>> keyword.kwlist + ['False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', + 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', + 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', + 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield'] + +as well as soft keywords: + +.. code-block:: python + + >>> import keyword + >>> keyword.softkwlist + ['_', 'case', 'match'] + +.. caution:: + Soft keywords can be a bit challenging to manage as they can be accepted in + places you don't intend to, given how the order alternatives behave in PEG + parsers (see :ref:`consequences of ordered choice section + ` for some background on this). In general, + try to define them in places where there is not a lot of alternatives. + +Error handling +~~~~~~~~~~~~~~ + +When a pegen-generated parser detects that an exception is raised, it will +**automatically stop parsing**, no matter what the current state of the parser +is and it will unwind the stack and report the exception. This means that if a +:ref:`rule action ` raises an exception all parsing will +stop at that exact point. This is done to allow to correctly propagate any +exception set by calling Python C-API functions. This also includes :exc:`SyntaxError` +exceptions and this is the main mechanism the parser uses to report custom syntax +error messages. + +.. note:: + Tokenizer errors are normally reported by raising exceptions but some special + tokenizer errors such as unclosed parenthesis will be reported only after the + parser finishes without returning anything. + +How Syntax errors are reported +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described previously in the :ref:`how PEG parsers work section +`, PEG parsers don't have a defined concept of where +errors happened in the grammar, because a rule failure doesn't imply a +parsing failure like in context free grammars. This means that some heuristic +has to be used to report generic errors unless something is explicitly declared +as an error in the grammar. + +To report generic syntax errors, pegen uses a common heuristic in PEG parsers: +the location of *generic* syntax errors is reported in the furthest token that +was attempted to be matched but failed. This is only done if parsing has failed +(the parser returns ``NULL`` in C or ``None`` in Python) but no exception has +been raised. + +.. caution:: + Positive and negative lookaheads will try to match a token so they will affect + the location of generic syntax errors. Use them carefully at boundaries + between rules. + +As the Python grammar was primordially written as an LL(1) grammar, this heuristic +has an extremely high success rate, but some PEG features can have small effects, +such as :ref:`positive lookaheads ` and +:ref:`negative lookaheads `. + +To generate more precise syntax errors, custom rules are used. This is a common practice +also in context free grammars: the parser will try to accept some construct that is known +to be incorrect just to report a specific syntax error for that construct. In pegen grammars, +these rules start with the ``invalid_`` prefix. This is because trying to match these rules +normally has a performance impact on parsing (and can also affect the 'correct' grammar itself +in some tricky cases, depending on the ordering of the rules) so the generated parser acts in +two phases: + +1. The first phase will try to parse the input stream without taking into account rules that + start with the ``invalid_`` prefix. If the parsing succeeds it will return the generated AST + and the second phase will not be attempted. + +2. If the first phase failed, a second parsing attempt is done including the rules that start + with an ``invalid_`` prefix. By design this attempt **cannot succeed** and is only executed + to give to the invalid rules a chance to detect specific situations where custom, more precise, + syntax errors can be raised. This also allows to trade a bit of performance for precision reporting + errors: given that we know that the input text is invalid, there is no need to be fast because + the interpreter is going to stop anyway. + +.. important:: + When defining invalid rules: + + * Make sure all custom invalid rules raise :exc:`SyntaxError` exceptions (or a subclass of it). + * Make sure **all** invalid rules start with the ``invalid_`` prefix to not + impact performance of parsing correct Python code. + * Make sure the parser doesn't behave differently for regular rules when you introduce invalid rules + (see the :ref:`how PEG parsers work section ` for more information). + +You can find a collection of macros to raise specialized syntax errors in the +:file:`Parser/pegen.h` header file. These macros allow also to report ranges for +the custom errors that will be highlighted in the tracebacks that will be +displayed when the error is reported. + +.. tip:: + A good way to test if an invalid rule will be triggered when you expect is to test if introducing + a syntax error **after** valid code triggers the rule or not. For example: :: + + $ 42 + + Should trigger the syntax error in the ``$`` character. If your rule is not correctly defined this + won't happen. For example, if you try to define a rule to match Python 2 style ``print`` statements + to make a better error message and you define it as: :: + + invalid_print: "print" expression + + This will **seem** to work because the parser will correctly parse ``print(something)`` because it is valid + code and the second phase will never execute but if you try to parse ``print(something) $ 3`` the first pass + of the parser will fail (because of the ``$``) and in the second phase, the rule will match the + ``print(something)`` as ``print`` followed by the variable ``something`` between parentheses and the error + will be reported there instead of the ``$`` character. + +Generating AST objects +~~~~~~~~~~~~~~~~~~~~~~ + +The output of the C parser used by CPython that is generated by the +:file:`Grammar/Python.gram` grammar file is a Python AST object (using C +structures). This means that the actions in the grammar file generate AST objects +when they succeed. Constructing these objects can be quite cumbersome (see +the :ref:`AST compiler section ` for more information +on how these objects are constructed and how they are used by the compiler) so +special helper functions are used. These functions are declared in the +:file:`Parser/pegen.h` header file and defined in the :file:`Parser/pegen.c` +file. These functions allow you to join AST sequences, get specific elements +from them or to do extra processing on the generated tree. + +.. caution:: + Actions must **never** be used to accept or reject rules. It may be tempting + in some situations to write a very generic rule and then check the generated + AST to decide if is valid or not but this will render the `official grammar + `_ partially incorrect + (because actions are not included) and will make it more difficult for other + Python implementations to adapt the grammar to their own needs. + +As a general rule, if an action spawns multiple lines or requires something more +complicated than a single expression of C code, is normally better to create a +custom helper in :file:`Parser/pegen.c` and expose it in the +:file:`Parser/pegen.h` header file so it can be used from the grammar. + +If the parsing succeeds, the parser **must** return a **valid** AST object. + +Testing +------- + +There are three files that contain tests for the grammar and the parser: + +* `Lib/test/test_grammar.py`. +* `Lib/test/test_syntax.py`. +* `Lib/test/test_exceptions.py`. + +Check the contents of these files to know which is the best place to place new tests depending +on the nature of the new feature you are adding. + +Tests for the parser generator itself can be found in the :file:`Lib/test/test_peg_generator` directory. + + +Debugging generated parsers +--------------------------- + +Making experiments +~~~~~~~~~~~~~~~~~~ + +As the generated C parser is the one used by Python, this means that if something goes wrong when adding some +new rules to the grammar you cannot correctly compile and execute Python anymore. This makes it a bit challenging +to debug when something goes wrong, especially when making experiments. + +For this reason it is a good idea to experiment first by generating a Python parser. To do this, you can go to the +:file:`Tools/peg_generator/` directory on the CPython repository and manually call the parser generator by executing: + +.. code-block:: shell + + $ python -m pegen python ~/github/pegen/data/expr.gram + +This will generate a file called :file:`parse.py` in the same directory that you can use to parse some input: + +.. code-block:: shell + + $ python parse.py file_with_source_code_to_test.py + +As the generated :file:`parse.py` file is just Python code, you can modify it and add breakpoints to debug or +better understand some complex situations. + + +Verbose mode +~~~~~~~~~~~~ + +When Python is compiled in debug mode (by adding ``--with-pydebug`` when running the configure step in Linux or by +adding ``-d`` when calling the :file:`PCbuild/python.bat` script in Windows), is possible to activate a **very** verbose +mode in the generated parser. This is very useful to debug the generated parser and to understand how it works, but it +can be a bit hard to understand at first. + +.. note:: + + When activating verbose mode in the Python parser, it is better to not use interactive mode as it can be much harder to + understand, because interactive mode involves some special steps compared to regular parsing. + +To activate verbose mode you can add the ``-d`` flag when executing Python: + +.. code-block:: shell + + $ python -d file_to_test.py + +This will print **a lot** of output to ``stderr`` so is probably better to dump it to a file for further analysis. The output +consists of trace lines with the following structure: + + ('>'|'-'|'+'|'!') []: ... + +Every line is indented by a different amount (````) depending on how deep the call stack is. The next +character marks the type of the trace: + +* ``>`` indicates that a rule is going to be attempted to be parsed. +* ``-`` indicates that a rule has failed to be parsed. +* ``+`` indicates that a rule has been parsed correctly. +* ``!`` indicates that an exception or an error has been detected and the parser is unwinding. + +The part indicates the current index in the token array, the + part indicates what rule is being parsed and the part +indicates what alternative within that rule is being attempted. + + +References +---------- + +.. [1] Ford, Bryan + http://pdos.csail.mit.edu/~baford/packrat/thesis + +.. [2] Medeiros et al. + https://arxiv.org/pdf/1207.0443.pdf + +.. [3] Warth et al. + http://web.cs.ucla.edu/~todd/research/pepm08.pdf diff --git a/requirements.txt b/requirements.txt index c90e30088..829318d20 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ Sphinx==4.1.2 -furo +furo<=2021.7.5b38 sphinx_copybutton>=0.3.3 From d9c1441301d494c2e751479fb45043b139733cc6 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Wed, 4 Aug 2021 21:54:27 +0100 Subject: [PATCH 137/871] Minor correction on the parser document --- parser.rst | 6 ------ 1 file changed, 6 deletions(-) diff --git a/parser.rst b/parser.rst index 71387b639..6364584b1 100644 --- a/parser.rst +++ b/parser.rst @@ -343,12 +343,6 @@ inside curly-braces, which specifies the return value of the alternative:: | first_alt1 first_alt2 { first_alt1 } | second_alt1 second_alt2 { second_alt1 } -If the action is omitted and C code is being generated, then there are two -different possibilities: - -1. If there’s a single name in the alternative, this gets returned. -2. If not, a dummy name object gets returned (this case should be avoided). - If the action is ommited, a default action is generated: * If there's a single name in the rule in the rule, it gets returned. From 14e6f3ea0c2731ddebf38224566d138b6ecd2d1f Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Wed, 25 Aug 2021 11:52:23 +0100 Subject: [PATCH 138/871] Correct invocation command for testing the parser --- parser.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parser.rst b/parser.rst index 6364584b1..27ce46d62 100644 --- a/parser.rst +++ b/parser.rst @@ -847,7 +847,7 @@ For this reason it is a good idea to experiment first by generating a Python par .. code-block:: shell - $ python -m pegen python ~/github/pegen/data/expr.gram + $ python -m pegen python This will generate a file called :file:`parse.py` in the same directory that you can use to parse some input: From 03d8f1bd581319a63aeedde5654485342a572e2c Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Thu, 26 Aug 2021 11:42:02 -0700 Subject: [PATCH 139/871] Add Ammar Askar and Ken Jin to the developer log --- developers.csv | 2 ++ 1 file changed, 2 insertions(+) diff --git a/developers.csv b/developers.csv index b37e5a15e..5b4551629 100644 --- a/developers.csv +++ b/developers.csv @@ -1,3 +1,5 @@ +Ken Jin,Fidget-Spinner,2021-08-26,, +Ammar Askar,ammaraskar,2021-07-30,, Irit Katriel,iritkatriel,2021-05-10,, Batuhan Taskaya,isidentical,2020-11-08,, Brandt Bucher,brandtbucher,2020-09-14,, From dd55ff4527b8211c01131ed11db28cabe450991d Mon Sep 17 00:00:00 2001 From: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Date: Fri, 27 Aug 2021 17:23:46 +0800 Subject: [PATCH 140/871] Describe test-with-buildbots label for triagers (#740) --- triaging.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/triaging.rst b/triaging.rst index 6c7468b18..7419aaa05 100644 --- a/triaging.rst +++ b/triaging.rst @@ -170,6 +170,12 @@ type-security type-tests Used for PRs that exclusively involve changes to the tests. +test-with-buildbots + Used on PRs to test the latest commit with the buildbot fleet. Generally for + PRs with large code changes requiring more testing before merging. This + may take multiple hours to complete. Triagers can also stop a stuck build + using the web interface. + Fields in the Issue Tracker --------------------------- From 1e0275e3b2477cefbf6aed9920b7d8e9e9b85654 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Wed, 8 Sep 2021 17:44:09 +0200 Subject: [PATCH 141/871] Mention DiR role in devcycle.rst --- devcycle.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/devcycle.rst b/devcycle.rst index e8bb60934..cfb18430a 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -321,6 +321,7 @@ Current Administrators | | Maintainer of buildbot.python.org | | +-------------------+----------------------------------------------------------+-----------------+ | Łukasz Langa | Python 3.8 and 3.9 Release Manager | ambv | +| | PSF CPython Developer in Residence 2021-2022 | | +-------------------+----------------------------------------------------------+-----------------+ | Ned Deily | Python 3.6 and 3.7 Release Manager | ned-deily | +-------------------+----------------------------------------------------------+-----------------+ From eeadb62751fc9011830a637e492debc18a214ce3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Wed, 8 Sep 2021 17:44:34 +0200 Subject: [PATCH 142/871] Update planned EOL for all feature branches, add 3.11 release PEP --- index.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/index.rst b/index.rst index 37afb6e85..5f7d82632 100644 --- a/index.rst +++ b/index.rst @@ -97,11 +97,11 @@ Status of Python branches +------------------+--------------+-------------+----------------+----------------+-----------------------+ | Branch | Schedule | Status | First release | End-of-life | Release manager | +==================+==============+=============+================+================+=======================+ -| main | *TBD* | features | *TBD* | *TBD* | Pablo Galindo Salgado | +| main | :pep:`664` | features | *2022-10-03* | *2027-10* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.10 | :pep:`619` | prerelease | *2021-10-04* | *TBD* | Pablo Galindo Salgado | +| 3.10 | :pep:`619` | prerelease | *2021-10-04* | *2026-10* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.9 | :pep:`596` | bugfix | 2020-10-05 | *TBD* | Łukasz Langa | +| 3.9 | :pep:`596` | bugfix | 2020-10-05 | *2025-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.8 | :pep:`569` | security | 2019-10-14 | *2024-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ @@ -112,6 +112,8 @@ Status of Python branches .. Remember to update the end-of-life table in devcycle.rst. +Dates in *italic* are scheduled and can be adjusted. + The main branch is currently the future Python 3.11, and is the only branch that accepts new features. The latest release for each Python version can be found on the `download page `_. @@ -127,13 +129,12 @@ Status: but new source-only versions can be released :end-of-life: release cycle is frozen; no further changes can be pushed to it. -Dates in *italic* are scheduled and can be adjusted. +See also the :ref:`devcycle` page for more information about branches. By default, the end-of-life is scheduled 5 years after the first release, but can be adjusted by the release manager of each branch. All Python 2 versions have reached end-of-life. -See also the :ref:`devcycle` page for more information about branches. .. _contributing: From f5c76b915e04bdbe246dfece7033ff02e1a1304c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Wed, 8 Sep 2021 17:51:07 +0200 Subject: [PATCH 143/871] Add missing comma --- devcycle.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/devcycle.rst b/devcycle.rst index cfb18430a..35bd96ab9 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -320,7 +320,7 @@ Current Administrators | Pablo Galindo | Python 3.10 and 3.11 Release Manager, | pablogsal | | | Maintainer of buildbot.python.org | | +-------------------+----------------------------------------------------------+-----------------+ -| Łukasz Langa | Python 3.8 and 3.9 Release Manager | ambv | +| Łukasz Langa | Python 3.8 and 3.9 Release Manager, | ambv | | | PSF CPython Developer in Residence 2021-2022 | | +-------------------+----------------------------------------------------------+-----------------+ | Ned Deily | Python 3.6 and 3.7 Release Manager | ned-deily | From 54cd7c1dcd9ddc082ded55651696052fc2700c59 Mon Sep 17 00:00:00 2001 From: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Date: Fri, 10 Sep 2021 23:41:33 +0800 Subject: [PATCH 144/871] Link to configure docs in compilation guide (#743) --- setup.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/setup.rst b/setup.rst index 24b7c97d2..0c69af695 100644 --- a/setup.rst +++ b/setup.rst @@ -128,6 +128,8 @@ when you shouldn't is if you are taking performance measurements). Even when working only on pure Python code the pydebug build provides several useful checks that one should not skip. +.. seealso:: The effects of various configure and build flags are documented in + the `Python configure docs `_. .. _unix-compiling: From 7a7a663e7f0d1ad6360bb35b58dd2357d3b7bf17 Mon Sep 17 00:00:00 2001 From: Dmitriy Fishman Date: Mon, 27 Sep 2021 14:55:18 +0300 Subject: [PATCH 145/871] Typo fix in committing.rst (GH-749) --- committing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/committing.rst b/committing.rst index 213a2292e..9f1951ba7 100644 --- a/committing.rst +++ b/committing.rst @@ -102,7 +102,7 @@ These are the two exceptions: change and the original** ``NEWS`` **entry remains valid**, then no additional entry is needed. -If a change needs an entry in ``What's New in Python``, then it very +If a change needs an entry in ``What's New in Python``, then it is very likely not suitable for including in a maintenance release. ``NEWS`` entries go into the ``Misc/NEWS.d`` directory as individual files. The From b1efd6a9ee74fad66d853ad13856f815a2822f8f Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Mon, 4 Oct 2021 01:05:39 +0100 Subject: [PATCH 146/871] Add instructions on running autoreconf with pkg-config (#750) --- setup.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/setup.rst b/setup.rst index 0c69af695..42cbe1758 100644 --- a/setup.rst +++ b/setup.rst @@ -447,7 +447,14 @@ example, ``autoconf`` by itself will not regenerate ``pyconfig.h.in``. appropriate. Python's ``configure.ac`` script typically requires a specific version of -Autoconf. At the moment, this reads: ``AC_PREREQ(2.69)``. +Autoconf. At the moment, this reads: ``AC_PREREQ(2.69)``. It also requires +to have the ``autoconf-archive`` and ``pkg-config`` utilities installed in +the system and the ``pkg.m4`` macro file located in the appropriate ``alocal`` +location. You can easily check if this is correctly configured by running: + +.. code-block:: bash + + ls $(aclocal --print-ac-dir) | grep pkg.m4 If the system copy of Autoconf does not match this version, you will need to install your own copy of Autoconf. From ebc5777ac4332f4507f73331057f670715e44f47 Mon Sep 17 00:00:00 2001 From: Ned Deily Date: Mon, 4 Oct 2021 16:09:38 -0400 Subject: [PATCH 147/871] update for 3.10 release --- index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.rst b/index.rst index 5f7d82632..3e311ae4a 100644 --- a/index.rst +++ b/index.rst @@ -99,7 +99,7 @@ Status of Python branches +==================+==============+=============+================+================+=======================+ | main | :pep:`664` | features | *2022-10-03* | *2027-10* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ -| 3.10 | :pep:`619` | prerelease | *2021-10-04* | *2026-10* | Pablo Galindo Salgado | +| 3.10 | :pep:`619` | bugfix | 2021-10-04 | *2026-10* | Pablo Galindo Salgado | +------------------+--------------+-------------+----------------+----------------+-----------------------+ | 3.9 | :pep:`596` | bugfix | 2020-10-05 | *2025-10* | Łukasz Langa | +------------------+--------------+-------------+----------------+----------------+-----------------------+ From 0dc9c92bafb58e61cde43bf57b3f31b65d78fd46 Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Tue, 19 Oct 2021 09:20:33 -0400 Subject: [PATCH 148/871] update Python org owners (#753) --- devcycle.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/devcycle.rst b/devcycle.rst index 35bd96ab9..e6e20b4fe 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -260,7 +260,7 @@ This role is paramount to the security of the Python Language, Community, and Infrastructure. The Executive Director of the Python Software Foundation delegates authority on -GitHub Organization Owner Status to Ee W. Durbin III - Python Software +GitHub Organization Owner Status to Ee Durbin - Python Software Foundation Director of Infrastructure. Common reasons for this role are: Infrastructure Staff Membership, Python Software Foundation General Counsel, and Python Software Foundation Staff as fallback. @@ -285,10 +285,14 @@ Current Owners +----------------------+--------------------------------+-----------------+ | Ewa Jodlowska | PSF Executive Director | ejodlowska | +----------------------+--------------------------------+-----------------+ -| Ee W. Durbin III | PSF Director of Infrastructure | ewdurbin | +| Ee Durbin | PSF Director of Infrastructure | ewdurbin | +----------------------+--------------------------------+-----------------+ | Van Lindberg | PSF General Counsel | VanL | +----------------------+--------------------------------+-----------------+ +| Ezio Melotti | roundup -> github migration | ezio-melotti | ++----------------------+--------------------------------+-----------------+ +| Łukasz Langa | CPython Developr in Residence | ambv | ++----------------------+--------------------------------+-----------------+ Repository Administrator Role Policy ------------------------------------ From f34e3442c3e80683df4c738352e8139992fb08b0 Mon Sep 17 00:00:00 2001 From: Zachary Ware Date: Tue, 19 Oct 2021 11:08:03 -0500 Subject: [PATCH 149/871] Typo fix --- devcycle.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/devcycle.rst b/devcycle.rst index e6e20b4fe..5c1b4b53e 100644 --- a/devcycle.rst +++ b/devcycle.rst @@ -291,7 +291,7 @@ Current Owners +----------------------+--------------------------------+-----------------+ | Ezio Melotti | roundup -> github migration | ezio-melotti | +----------------------+--------------------------------+-----------------+ -| Łukasz Langa | CPython Developr in Residence | ambv | +| Łukasz Langa | CPython Developer in Residence | ambv | +----------------------+--------------------------------+-----------------+ Repository Administrator Role Policy From c4f3c7300c59b19a91cc66f3363b8f5985635f12 Mon Sep 17 00:00:00 2001 From: Wey-Han Liaw Date: Sun, 24 Oct 2021 02:44:19 -0700 Subject: [PATCH 150/871] Change the coordinator of the zh-tw translation (#752) --- documenting.rst | 7 ++++--- experts.rst | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/documenting.rst b/documenting.rst index a0521ce9e..6b91652ca 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1641,9 +1641,9 @@ in production, other are work in progress: +-----------------+-------------------------------+----------------------------+ | Spanish (es) | Raúl Cumplido | `GitHub `_ | +-----------------+-------------------------------+----------------------------+ -| Traditional | 廖偉涵 Adrian Liaw | `GitHub `_ | -| Chinese | | `Transifex `_ | -| (zh-tw) | | `Doc `_ | +| Traditional | `王威翔 Matt Wang | `GitHub `_ | +| Chinese | `_, | `Transifex `_ | +| (zh-tw) | Josix Wang | `Doc `_ | +-----------------+-------------------------------+----------------------------+ | Turkish (tr) | | `GitHub `_ | +-----------------+-------------------------------+----------------------------+ @@ -1655,6 +1655,7 @@ in production, other are work in progress: .. _bpo_mdk: https://bugs.python.org/user23063 .. _bpo_oonid: https://bugs.python.org/user32660 .. _bpo_zhsj: https://bugs.python.org/user24811 +.. _bpo_mattwang44: https://bugs.python.org/user39654 .. _chat_pt_br: https://t.me/pybr_i18n .. _doc_ja: https://docs.python.org/ja/ .. _doc_ko: https://docs.python.org/ko/ diff --git a/experts.rst b/experts.rst index dc84bdfb9..f1495ca6c 100644 --- a/experts.rst +++ b/experts.rst @@ -368,6 +368,6 @@ Korean flowdas Bengali India kushal.das Hungarian gbtami Portuguese rougeth -Chinese (TW) adrianliaw +Chinese (TW) mattwang44, josix Chinese (CN) zhsj ============= ============ From 2e6b3279711d4d528ee75533c46acb87519d62c0 Mon Sep 17 00:00:00 2001 From: Christian Heimes Date: Thu, 28 Oct 2021 12:24:45 +0300 Subject: [PATCH 151/871] Update Debian build requirements (GH-755) --- setup.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/setup.rst b/setup.rst index 42cbe1758..782a498a5 100644 --- a/setup.rst +++ b/setup.rst @@ -328,13 +328,15 @@ Then you should update the packages index:: Now you can install the build dependencies via ``apt``:: $ sudo apt-get build-dep python3 + $ sudo apt-get install pkg-config If you want to build all optional modules, install the following packages and their dependencies:: - $ sudo apt-get install build-essential gdb lcov libbz2-dev libffi-dev \ - libgdbm-dev liblzma-dev libncurses5-dev libreadline6-dev \ - libsqlite3-dev libssl-dev lzma lzma-dev tk-dev uuid-dev zlib1g-dev + $ sudo apt-get install build-essential gdb lcov pkg-config \ + libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \ + libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \ + lzma lzma-dev tk-dev uuid-dev zlib1g-dev .. _MacOS: From 8d13a975939b9d9b93f1cff28b0ade3be7744695 Mon Sep 17 00:00:00 2001 From: Itamar Ostricher Date: Tue, 2 Nov 2021 22:14:35 -0700 Subject: [PATCH 152/871] Minor typo fix in Issue Tracking page (#758) --- tracker.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tracker.rst b/tracker.rst index 9f28aa3c6..367306dee 100644 --- a/tracker.rst +++ b/tracker.rst @@ -30,7 +30,7 @@ already been reported. Checking if the problem is an existing issue will: * determine if additional information, such as how to replicate the issue, is needed -To do see if the issue already exists, search the bug database using the +To see if an issue already exists, search the bug database using the search box on the top of the issue tracker page. An `advanced search`_ is also available by clicking on "Search" in the sidebar. From 5db50fc080e102b194a8e7ca46bc6f94cff3bd0d Mon Sep 17 00:00:00 2001 From: Abdur-Rahmaan Janhangeer Date: Mon, 8 Nov 2021 00:08:40 +0400 Subject: [PATCH 153/871] Update Arabic coordinator (#760) --- documenting.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/documenting.rst b/documenting.rst index 6b91652ca..4649bab3d 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1595,7 +1595,8 @@ in production, other are work in progress: +-----------------+-------------------------------+----------------------------+ | Language | Contact | Links | +=================+===============================+============================+ -| Arabic (ar) | Ibrahim Elbouhissi | `GitHub `_ | +| Arabic (ar) | `Abdur-Rahmaan Janhangeer | `GitHub `_ | +| | `_ | | +-----------------+-------------------------------+----------------------------+ | Bengali as | `Kushal Das `_ | `GitHub `_ | | spoken in | | | @@ -1654,6 +1655,7 @@ in production, other are work in progress: .. _bpo_kushal: https://bugs.python.org/user16382 .. _bpo_mdk: https://bugs.python.org/user23063 .. _bpo_oonid: https://bugs.python.org/user32660 +.. _bpo_osdotsystem: https://bugs.python.org/user28057 .. _bpo_zhsj: https://bugs.python.org/user24811 .. _bpo_mattwang44: https://bugs.python.org/user39654 .. _chat_pt_br: https://t.me/pybr_i18n From 75e0e7ece4e967cac88d4a707a5c52c98d9b33c6 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sun, 7 Nov 2021 23:21:36 +0000 Subject: [PATCH 154/871] Fix grammar in the title of the Parser guide (#762) --- parser.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parser.rst b/parser.rst index 27ce46d62..491074022 100644 --- a/parser.rst +++ b/parser.rst @@ -1,6 +1,6 @@ .. _parser: -Guide of CPython's Parser +Guide to CPython's Parser ========================= :Author: Pablo Galindo Salgado From 46590a51d1e1224e9b987cb4c33b313937fd5ecd Mon Sep 17 00:00:00 2001 From: Itamar Ostricher Date: Sun, 14 Nov 2021 10:29:40 -0800 Subject: [PATCH 155/871] Fix typo in garbage_collector (#768) Minor fix / nit --- garbage_collector.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/garbage_collector.rst b/garbage_collector.rst index 0dbac59fc..7e15a4ea2 100644 --- a/garbage_collector.rst +++ b/garbage_collector.rst @@ -303,7 +303,7 @@ In order to limit the time each garbage collection takes, the GC uses a popular optimization: generations. The main idea behind this concept is the assumption that most objects have a very short lifespan and can thus be collected shortly after their creation. This has proven to be very close to the reality of many Python programs as -many temporarily objects are created and destroyed very fast. The older an object is +many temporary objects are created and destroyed very fast. The older an object is the less likely it is that it will become unreachable. To take advantage of this fact, all container objects are segregated into From 3036184234b4821b380bec0dd4e2bbf81ae812b1 Mon Sep 17 00:00:00 2001 From: Itamar Ostricher Date: Sun, 14 Nov 2021 10:29:58 -0800 Subject: [PATCH 156/871] Update reference to the parser from LL(1) to PEG (#767) --- langchanges.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/langchanges.rst b/langchanges.rst index f7984d608..3dbaf6d47 100644 --- a/langchanges.rst +++ b/langchanges.rst @@ -46,9 +46,8 @@ the change. This process is the Python Enhancement Proposal (PEP) process. You will first need a PEP that you will present to python-ideas. You may be a little hazy on the technical details as various core developers can help with that, but do realize that if you do not present your idea to python-ideas or -python-list ahead of time you may find out it is technically not possible -(e.g., Python's parser will not support the grammar change as it is an LL(1) -parser). Expect extensive comments on the PEP, some of which will be negative. +python-list ahead of time you may find out it is technically not possible. +Expect extensive comments on the PEP, some of which will be negative. Once your PEP has been modified to be of proper quality and to take into account comments made on python-ideas, it may proceed to python-dev. There it From 90bd4308657ea61ef740c648ac87e49dde819fa3 Mon Sep 17 00:00:00 2001 From: Dino Viehland Date: Mon, 15 Nov 2021 13:29:39 -0800 Subject: [PATCH 157/871] Update motivations.rst for Dino Way out of date... --- motivations.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/motivations.rst b/motivations.rst index f1a1a41cc..27361311d 100644 --- a/motivations.rst +++ b/motivations.rst @@ -244,16 +244,16 @@ participating in the CPython core development process: .. topic:: Dino Viehland (United States) - * Microsoft: ``_ (Software Engineer) - * Email address: dinov@microsoft.com + * Meta (Software Engineer) + * Email address: dinoviehland@gmail.com Dino started working with Python in 2005 by working on IronPython, an implementation of Python running on .NET. He was one of the primary developers on the project for 6 years. After that he started the Python Tools for Visual Studio project focusing on providing advanced code completion and debugging features for Python. Today he works on - `Azure Notebooks `_ bringing the Python based - Jupyter notebook as a hosted on-line service. + `Cinder `_ improving Python + performance for Instagram. .. topic:: Carol Willing (United States) From 3f372f9a1c5416df0f3efaa529c23c0a2fc4aac3 Mon Sep 17 00:00:00 2001 From: Arthur Milchior Date: Tue, 16 Nov 2021 16:06:47 +0100 Subject: [PATCH 158/871] Replacing "country" by "translation" (GH-757) While it is easy to understand what was meant by "country", this is inaccurate. Some countries have multiple languages, each having their translation (e.g. Canada with English and French) and some languages are spoken in many countries, and the regional variation are small enough that there will probably never be multiple translations. (While Canadian French and France French are so different that it's sometimes hard for French native to understand French Canadian speaking, I doubt anybody will want to create a ca-fr translation, given that the technical language will probably be almost the same.) --- documenting.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documenting.rst b/documenting.rst index 4649bab3d..852b16237 100644 --- a/documenting.rst +++ b/documenting.rst @@ -1771,7 +1771,7 @@ Here's what's we're using: How a coordinator is elected? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -There is no election, each country have to sort this out. Here are some suggestions. +There is no election, each translation have to sort this out. Here are some suggestions. - Coordinator requests are to be public on doc-sig mailing list. - If the given language have a native core dev, the core dev have its From 266f1c0eb60cc2165f833da0d6d1d6f4772ea2fd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 16 Nov 2021 09:56:22 -0800 Subject: [PATCH 159/871] Update furo requirement from <=2021.7.5b38 to <2021.11.16 (#770) Updates the requirements on [furo](https://github.com/pradyunsg/furo) to permit the latest version. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](https://github.com/pradyunsg/furo/compare/2020.08.14.beta5...2021.11.15) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 829318d20..bd66e003c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ Sphinx==4.1.2 -furo<=2021.7.5b38 +furo<2021.11.16 sphinx_copybutton>=0.3.3 From 1ff3dc740e946aace6b62ff85821eafd4b98d0a4 Mon Sep 17 00:00:00 2001 From: Itamar Ostricher Date: Fri, 19 Nov 2021 00:10:26 -0800 Subject: [PATCH 160/871] Four fixes in parser page (#772) Two duplications, a typo, and minor grammar fix. --- parser.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/parser.rst b/parser.rst index 491074022..06434fc57 100644 --- a/parser.rst +++ b/parser.rst @@ -110,7 +110,7 @@ the following two rules (in these examples, a token is an individual character): In a regular EBNF grammar, both rules specify the language ``{aa, aaa}`` but in PEG, one of these two rules accepts the string ``aaa`` but not the string -``aa``. The other does the opposite -- it accepts the string the string ``aa`` +``aa``. The other does the opposite -- it accepts the string ``aa`` but not the string ``aaa``. The rule ``('a'|'aa')'a'`` does not accept ``aaa`` because ``'a'|'aa'`` consumes the first ``a``, letting the final ``a`` in the rule consume the second, and leaving out the third ``a``. @@ -345,14 +345,14 @@ inside curly-braces, which specifies the return value of the alternative:: If the action is ommited, a default action is generated: -* If there's a single name in the rule in the rule, it gets returned. +* If there's a single name in the rule, it gets returned. * If there is more than one name in the rule, a collection with all parsed expressions gets returned (the type of the collection will be different in C and Python). This default behaviour is primarily made for very simple situations and for -debugging pourposes. +debugging purposes. The full meta-grammar for the grammars supported by the PEG generator is: @@ -863,7 +863,7 @@ Verbose mode ~~~~~~~~~~~~ When Python is compiled in debug mode (by adding ``--with-pydebug`` when running the configure step in Linux or by -adding ``-d`` when calling the :file:`PCbuild/python.bat` script in Windows), is possible to activate a **very** verbose +adding ``-d`` when calling the :file:`PCbuild/python.bat` script in Windows), it is possible to activate a **very** verbose mode in the generated parser. This is very useful to debug the generated parser and to understand how it works, but it can be a bit hard to understand at first. From 9d880b6f54aa2ee193fccdb1242a52923df8c8aa Mon Sep 17 00:00:00 2001 From: Carl Friedrich Bolz-Tereick Date: Fri, 19 Nov 2021 22:16:43 +0100 Subject: [PATCH 161/871] =?UTF-8?q?Improvements=20to=20"Guide=20to=20CPyth?= =?UTF-8?q?on=E2=80=99s=20Parser"=20(#763)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * put references to names from rules into ``backticks`` * two typos * mention that ~ is called "the cut" (The grammar gives that name, and I as a prolog programmer was looking for the terminology ;-)) --- parser.rst | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/parser.rst b/parser.rst index 06434fc57..6001051a6 100644 --- a/parser.rst +++ b/parser.rst @@ -169,7 +169,7 @@ Python-style comments. ``e1 e2`` ''''''''' -Match e1, then match e2. +Match ``e1``, then match ``e2``. :: @@ -178,7 +178,7 @@ Match e1, then match e2. ``e1 | e2`` ''''''''''' -Match e1 or e2. +Match ``e1`` or ``e2``. The first alternative can also appear on the line after the rule name for formatting purposes. In that case, a \| must be used before the @@ -193,7 +193,7 @@ first alternative, like so: ``( e )`` ''''''''' -Match e. +Match ``e``. :: @@ -209,7 +209,7 @@ operator together with the repeat operators: ``[ e ] or e?`` ''''''''''''''' -Optionally match e. +Optionally match ``e``. :: @@ -225,7 +225,7 @@ optional: ``e*`` '''''' -Match zero or more occurrences of e. +Match zero or more occurrences of ``e``. :: @@ -234,7 +234,7 @@ Match zero or more occurrences of e. ``e+`` '''''' -Match one or more occurrences of e. +Match one or more occurrences of ``e``. :: @@ -243,7 +243,7 @@ Match one or more occurrences of e. ``s.e+`` '''''''' -Match one or more occurrences of e, separated by s. The generated parse +Match one or more occurrences of ``e``, separated by ``s``. The generated parse tree does not include the separator. This is otherwise identical to ``(e (s e)*)``. @@ -256,14 +256,14 @@ tree does not include the separator. This is otherwise identical to .. _peg-positive-lookahead: -Succeed if e can be parsed, without consuming any input. +Succeed if ``e`` can be parsed, without consuming any input. ``!e`` '''''' .. _peg-negative-lookahead: -Fail if e can be parsed, without consuming any input. +Fail if ``e`` can be parsed, without consuming any input. An example taken from the Python grammar specifies that a primary consists of an atom, which is not followed by a ``.`` or a ``(`` or a @@ -276,14 +276,15 @@ consists of an atom, which is not followed by a ``.`` or a ``(`` or a ``~`` '''''' -Commit to the current alternative, even if it fails to parse. +Commit to the current alternative, even if it fails to parse (this is called +the "cut"). :: rule_name: '(' ~ some_rule ')' | some_alt In this example, if a left parenthesis is parsed, then the other -alternative won’t be considered, even if some_rule or ‘)’ fail to be +alternative won’t be considered, even if some_rule or ``)`` fail to be parsed. Left recursion @@ -343,7 +344,7 @@ inside curly-braces, which specifies the return value of the alternative:: | first_alt1 first_alt2 { first_alt1 } | second_alt1 second_alt2 { second_alt1 } -If the action is ommited, a default action is generated: +If the action is omitted, a default action is generated: * If there's a single name in the rule, it gets returned. From ed2af4311a8db23b0d6267fea45090c22a3d8ce7 Mon Sep 17 00:00:00 2001 From: Carl Friedrich Bolz-Tereick Date: Fri, 19 Nov 2021 22:16:59 +0100 Subject: [PATCH 162/871] add warning about actions mutating things (#766) --- parser.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/parser.rst b/parser.rst index 6001051a6..c773b00c7 100644 --- a/parser.rst +++ b/parser.rst @@ -355,6 +355,15 @@ If the action is omitted, a default action is generated: This default behaviour is primarily made for very simple situations and for debugging purposes. +.. warning:: + + It's important that the actions don't mutate any AST nodes that are passed + into them via variables referring to other rules. The reason for mutation + being not allowed is that the AST nodes are cached by memoization and could + potentially be reused in a different context, where the mutation would be + invalid. If an action needs to change an AST node, it should instead make a + new copy of the node and change that. + The full meta-grammar for the grammars supported by the PEG generator is: :: From f3ecfb8a525a971662413c1ecdc0c8d8fc540393 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Sun, 21 Nov 2021 20:41:08 +0000 Subject: [PATCH 163/871] Update AST actions helper location in the PEG parser (#773) --- parser.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/parser.rst b/parser.rst index c773b00c7..1bc795dec 100644 --- a/parser.rst +++ b/parser.rst @@ -808,7 +808,7 @@ when they succeed. Constructing these objects can be quite cumbersome (see the :ref:`AST compiler section ` for more information on how these objects are constructed and how they are used by the compiler) so special helper functions are used. These functions are declared in the -:file:`Parser/pegen.h` header file and defined in the :file:`Parser/pegen.c` +:file:`Parser/pegen.h` header file and defined in the :file:`Parser/action_helpers.c` file. These functions allow you to join AST sequences, get specific elements from them or to do extra processing on the generated tree. @@ -822,7 +822,7 @@ from them or to do extra processing on the generated tree. As a general rule, if an action spawns multiple lines or requires something more complicated than a single expression of C code, is normally better to create a -custom helper in :file:`Parser/pegen.c` and expose it in the +custom helper in :file:`Parser/action_helpers.c` and expose it in the :file:`Parser/pegen.h` header file so it can be used from the grammar. If the parsing succeeds, the parser **must** return a **valid** AST object. From 0b95382d21c77eed3f496585844d683f43603aa9 Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Thu, 2 Dec 2021 13:40:16 -0500 Subject: [PATCH 164/871] rename default branch to main (#776) also covers a handful of external references who have similarly made this change --- .github/workflows/ci.yml | 2 +- runtests.rst | 2 +- setup.rst | 2 +- tools/templates/customsourcelink.html | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 09cd927e7..a01ee9f37 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -3,7 +3,7 @@ name: Tests on: pull_request: push: - branches: master + branches: main jobs: test: diff --git a/runtests.rst b/runtests.rst index 2d6c3eb48..e018faa25 100644 --- a/runtests.rst +++ b/runtests.rst @@ -132,4 +132,4 @@ Benchmarking is useful to test that a change does not degrade performance. `The Python Benchmark Suite `_ has a collection of benchmarks for all Python implementations. Documentation about running the benchmarks is in the `README.txt -`_ of the repo. +`_ of the repo. diff --git a/setup.rst b/setup.rst index 782a498a5..7ef2bc460 100644 --- a/setup.rst +++ b/setup.rst @@ -13,7 +13,7 @@ directory structure of the CPython source code. Alternatively, if you have `Docker `_ installed you might want to use `our official images -`_. These +`_. These contain the latest releases of several Python versions, along with git head, and are provided for development and testing purposes only. diff --git a/tools/templates/customsourcelink.html b/tools/templates/customsourcelink.html index 2487a0b04..a50734f18 100644 --- a/tools/templates/customsourcelink.html +++ b/tools/templates/customsourcelink.html @@ -3,7 +3,7 @@

      {{ _('This Page') }}