cpython/Parser/tokenizer
Pablo Galindo Salgado f892b11f22
gh-153569: Simplify tokenizer source views and reader ownership (#158849)
Move source access and logical spans into the tokenizer source API, and keep input and encoding ownership in the reader. Export the incremental decoder helper for parser generator extensions.
2026-10-13 13:01:17 +00:00
..
api.c gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
decoder.c gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
helpers.c gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
helpers.h gh-153569: consolidate tokenizer input, layout, and diagnostics (#157055) 2026-09-24 16:26:47 +00:00
reader.c gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
reader.h gh-153569: consolidate tokenizer input, layout, and diagnostics (#157055) 2026-09-24 16:26:47 +00:00
reader_internal.h gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
source.c gh-153569: Simplify tokenizer state (#158124) 2026-10-05 14:28:51 +01:00
source.h gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
tokenizer.h gh-153569: Simplify tokenizer source views and reader ownership (#158849) 2026-10-13 13:01:17 +00:00
types.h gh-153569: Simplify tokenizer state (#158124) 2026-10-05 14:28:51 +01:00