-
-
Notifications
You must be signed in to change notification settings - Fork 8.3k
py/objcode: Implement PEP626 to add co_lines to code objects on settrace builds. #16989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #16989 +/- ##
==========================================
+ Coverage 98.54% 98.56% +0.02%
==========================================
Files 169 169
Lines 21877 21945 +68
==========================================
+ Hits 21558 21631 +73
+ Misses 319 314 -5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Code size report:
|
Related to #16797? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this looks like a nice addition!
I think we should then deprecate co_lnotab
following CPython.
And it would be good to add another test to this PR, to test the actual numeric values from co_lines()
, to make sure they match the expected bytecode offsets for a simple test function (that test would need a corresponding .py.exp
file because you can't use CPython to work out the right answer).
iter->bc = 0; | ||
iter->source_line = 1; | ||
iter->ci = self->rc->prelude.line_info; | ||
code_colines_next(MP_OBJ_FROM_PTR(iter)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to discard the first line-no entry, is that intended?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is intentional; this actually works around a bit of weirdness the test case helped me discover. For whatever reason, the first entry in this table always has a zero-length bytecode segment attributed to line 1.
This might be something to fix in the bytecode emitter, though that could also have knock-on effects for the other consumers of that table.
Skipping that first entry results in co_lines
behavior behavior that matches CPython 3.10, where it only enumerates the body lines of a function --- though either way that's a diff with CPython 3.12, where co_lines
includes the function's declaration line.
Hmm, actually, looking at the compilation result, it looks like |
There we are, having it return the new line_info value instead of an increment value for it manages to get it fully elided: https://godbolt.org/z/bEY6ffTK8 Though, that makes it much less clear what's going on --- and there's similar differences in the other contexts with how incrementing (Also, clang gets the exact same compilation result regardless of whether or which way I refactor it.) I'll leave the PR as-is unless we really feel like it's important to turn the screws as tight as possible just to win back 2 opcodes here. |
The Was initially concerned if there was some sort of overflow making |
py/bc.h
Outdated
size_t info_increment; | ||
} mp_code_lineinfo_t; | ||
|
||
static inline mp_code_lineinfo_t mp_bytecode_decode_lineinfo(const byte *line_info) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code size increases with this bit of logic factored out into this inline function.
I had a little play, and it seems that code size can be reclaimed by passing in a pointer to line_info
, updating it here as needed (adding 1 or 2) and not returning info_increment
. Ie:
static inline mp_code_lineinfo_t mp_bytecode_decode_lineinfo(const byte **line_info) {
mp_code_lineinfo_t result;
size_t c = **line_info;
if (...) {
...
*line_info += 1;
} else {
...
*line_info += 2;
}
return result;
}
I suggest making that change to try and get the code size diff to be +0.
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
0acb0e6
to
5c31dc4
Compare
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
5c31dc4
to
c27a6ad
Compare
I've been playing around with these tests; at the moment the test I added also covers the cases where a correct The current outputs from co_lines are still meaningful and correct in some sense, though --- is that summing behavior actually something this needs to implement? Or can these extra coverage cases just be dropped (or made into cpydiffs or something)? Also, should I add coverage for a case where the table needs to skip more than 2048 lines at once? (e.g. to overflow the 11-bit line number increment in the 2-byte format, and require a multi-entry encoding for it like |
Summary
This PR implements PEP 626 – Precise line numbers for debugging and other tools..
This adds one method,
co_lines
, that returns an iterator over(begin, end, line_no)
tuples of line numbers for applicable bytecode ranges.Testing
This PR includes automated tests of all of the currently-implemented code object attributes, including the existing attributes that previously had no test coverage.
The actual values in
co_lines
can't be compared directly to CPython's, but all of the invariants that the method is meant to observe are checked.In addition to the github workflow checks, I've also manually verified that all tests pass when run against a unix
MICROPY_PY_SYS_SETTRACE
build. (In particular, I've checked to make sure they're not skipped).Trade-offs and Alternatives
As in CPython, this is implemented as its own distinct iterator object. Potentially, this function could be functionally replicated by instead eagerly building a list of the tuples and returning an iterator over that list, at a slightly smaller code size; but the iterator approach allows for a tighter behavioral match with CPython (
fun_code_full.py:25
), and code size is not at nearly the same premium in settrace builds.