Skip to content

tools/gen-cpydiff: Fail CPython difference generation if output matches. #17060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 7, 2025

Conversation

projectgus
Copy link
Contributor

@projectgus projectgus commented Apr 1, 2025

Summary

MicroPython uses a set of automated test cases (tests/cpydiff) to generate documentation of differences between CPython and MicroPython. If a MicroPython improvement made something CPython compatible, the generation tool would print a note but would keep running as normal including generating the "difference" (with the docs containing identical generated output for CPython vs MicroPython).

This PR adjusts the generation tool to fail if any differences are non-existent (which should also fail CI), and also removes one fixed difference (str.endswith() was made compatible as of #16812 🎉).

Testing

  • Ran the docs build and generation tool locally, verified the new failure due to a non-difference and non-failure after removing.

Trade-offs and Alternatives

  • We could make this behaviour configurable in gen-cpydiff.py, but this tool is only ever run to generate this output for the docs and almost all of the other behaviour is hard-coded for this purpose.

@projectgus projectgus added the docs label Apr 1, 2025
Copy link

codecov bot commented Apr 1, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.54%. Comparing base (57f1e60) to head (74a5bf9).
Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #17060   +/-   ##
=======================================
  Coverage   98.54%   98.54%           
=======================================
  Files         169      169           
  Lines       21890    21890           
=======================================
  Hits        21571    21571           
  Misses        319      319           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@projectgus projectgus marked this pull request as draft April 1, 2025 01:13
Copy link

github-actions bot commented Apr 1, 2025

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@AJMansfield
Copy link
Contributor

How will this affect port-specific differences, e.g. modules_time_time_returns_int.py from #16984? This probably needs a "SKIP"-like mechanism, so platform-dependent cpydiffs for issues not present in unix don't create spurious CI failures.

@projectgus
Copy link
Contributor Author

How will this affect port-specific differences,

Thanks for pointing this out, @AJMansfield. I've left a reply on the linked PR: https://github.com/micropython/micropython/pull/16984/files#r2008260448

MicroPython support for this behaviour was added in eb45d97.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Previously this information was recorded in a "status" field of the result,
but nothing ever parsed this result which led to non-differences not being
removed.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
@dpgeorge dpgeorge force-pushed the doc/cpydiff_fail_same branch from 6e6a0d0 to 74a5bf9 Compare April 7, 2025 06:10
Copy link
Member

@dpgeorge dpgeorge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a good improvement!

@dpgeorge dpgeorge merged commit 74a5bf9 into micropython:master Apr 7, 2025
65 checks passed
@projectgus projectgus deleted the doc/cpydiff_fail_same branch April 7, 2025 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants