cpython/Lib
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
..
asyncio bpo-42140: Improve asyncio.wait function (GH-22938) 2020-11-10 14:12:52 -08:00
collections
concurrent
ctypes bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
curses
dbm
distutils bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
email
encodings
ensurepip
html
http bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
idlelib bpo-40511: Stop unwanted flashing of IDLE calltips (GH-20910) 2020-11-01 22:59:52 -05:00
importlib Bump magic number. (GH-23245) 2020-11-12 10:42:44 +00:00
json
lib2to3
logging bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053) 2020-11-02 23:17:46 +01:00
msilib
multiprocessing
pydoc_data Python 3.10.0a2 2020-11-03 00:02:08 +00:00
site-packages
sqlite3 bpo-42264: Deprecate sqlite3.OptimizedUnicode (GH-23163) 2020-11-17 17:55:12 +02:00
test bpo-42398: Fix "make regen-all" race condition (GH-23362) 2020-11-18 15:36:27 +01:00
tkinter
turtledemo
unittest bpo-41877 Check for asert, aseert, assrt in mocks (GH-23165) 2020-11-05 09:04:38 -08:00
urllib bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
venv
wsgiref
xml bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147) 2020-11-09 08:02:39 +01:00
xmlrpc
zoneinfo
__future__.py
__phello__.foo.py
_aix_support.py
_bootsubprocess.py
_collections_abc.py Fix: Docstrings hidden by slots. (GH-23352) 2020-11-17 13:50:23 -08:00
_compat_pickle.py
_compression.py
_markupbase.py
_osx_support.py bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
_py_abc.py
_pydecimal.py
_pyio.py
_sitebuiltins.py
_strptime.py
_threading_local.py
_weakrefset.py
abc.py
aifc.py
antigravity.py
argparse.py
ast.py
asynchat.py
asyncore.py
base64.py
bdb.py
binhex.py bpo-29566: binhex.binhex now consitently writes MacOS 9 line endings. (GH-23059) 2020-11-01 01:08:48 -08:00
bisect.py
bz2.py
calendar.py
cgi.py
cgitb.py
chunk.py
cmd.py
code.py
codecs.py
codeop.py
colorsys.py
compileall.py
configparser.py
contextlib.py bpo-42395: Add aclosing to __all__ (GH-23356) 2020-11-17 15:18:05 -08:00
contextvars.py
copy.py
copyreg.py
cProfile.py
crypt.py
csv.py
dataclasses.py
datetime.py
decimal.py
difflib.py
dis.py bpo-42246: Partial implementation of PEP 626. (GH-23113) 2020-11-12 09:43:29 +00:00
doctest.py bpo-42133: update parts of the stdlib to fall back to __spec__.loader when __loader__ is missing (#22929) 2020-11-06 18:45:56 -08:00
enum.py
filecmp.py
fileinput.py
fnmatch.py
formatter.py
fractions.py
ftplib.py
functools.py
genericpath.py
getopt.py
getpass.py
gettext.py
glob.py
graphlib.py
gzip.py
hashlib.py
heapq.py
hmac.py
imaplib.py
imghdr.py
imp.py
inspect.py bpo-42133: update parts of the stdlib to fall back to __spec__.loader when __loader__ is missing (#22929) 2020-11-06 18:45:56 -08:00
io.py
ipaddress.py
keyword.py
linecache.py bpo-42133: update parts of the stdlib to fall back to __spec__.loader when __loader__ is missing (#22929) 2020-11-06 18:45:56 -08:00
locale.py bpo-42208: Add _locale._get_locale_encoding() (GH-23052) 2020-10-31 01:32:11 +01:00
lzma.py
mailbox.py
mailcap.py
mimetypes.py
modulefinder.py
netrc.py
nntplib.py
ntpath.py
nturl2path.py
numbers.py
opcode.py
operator.py
optparse.py
os.py
pathlib.py
pdb.py
pickle.py
pickletools.py
pipes.py
pkgutil.py
platform.py bpo-42029: Remove IRIX code (GH-23023) 2020-10-29 15:16:23 +01:00
plistlib.py bpo-42249: Fix writing binary Plist files larger than 4 GiB. (GH-23121) 2020-11-03 16:15:56 +02:00
poplib.py
posixpath.py
pprint.py
profile.py
pstats.py
pty.py
py_compile.py
pyclbr.py bpo-39411: pyclbr rewrite on AST (#18103) 2020-11-11 10:14:12 +03:00
pydoc.py
queue.py
quopri.py
random.py
re.py
reprlib.py
rlcompleter.py
runpy.py
sched.py
secrets.py
selectors.py
shelve.py bpo-34204: Use pickle.DEFAULT_PROTOCOL in shelve (GH-19639) 2020-10-29 02:44:35 -07:00
shlex.py
shutil.py bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585) 2020-11-10 08:06:02 -08:00
signal.py
site.py bpo-42133: update parts of the stdlib to fall back to __spec__.loader when __loader__ is missing (#22929) 2020-11-06 18:45:56 -08:00
smtpd.py
smtplib.py
sndhdr.py
socket.py
socketserver.py Revert "bpo-37193: remove thread objects which finished process its request (GH-13893)" (GH-23107) 2020-11-02 16:48:56 +00:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py
stat.py
statistics.py
string.py
stringprep.py
struct.py
subprocess.py
sunau.py
symtable.py
sysconfig.py
tabnanny.py
tarfile.py
telnetlib.py
tempfile.py tempfile: Use random.choises() instead of choise() (GH-23068) 2020-11-01 20:02:03 +09:00
textwrap.py
this.py
threading.py bpo-42350: Fix Thread._reset_internal_locks() (GH-23268) 2020-11-16 15:20:34 +01:00
timeit.py
token.py
tokenize.py
trace.py
traceback.py bpo-26389: Allow passing an exception object in the traceback module (GH-22610) 2020-11-05 22:18:44 +00:00
tracemalloc.py
tty.py
turtle.py
types.py
typing.py bpo-42345: Fix three issues with typing.Literal parameters (GH-23294) 2020-11-16 18:23:19 -08:00
uu.py
uuid.py
warnings.py
wave.py
weakref.py
webbrowser.py bpo-41754: Ignore NotADirectoryError in invocation of xdg-settings (GH-23075) 2020-11-08 10:46:55 +01:00
xdrlib.py
zipapp.py
zipfile.py
zipimport.py bpo-42131: Add PEP 451-related methods to zipimport (GH-23187) 2020-11-13 15:14:58 -08:00