Skip to content

Update FileIO comments, documentation to match implementation #129011

Open
@cmaloney

Description

@cmaloney

Documentation

Discuss thread: https://discuss.python.org/t/change-open-write-to-guarantee-all-bytes-will-be-written-or-an-exception-will-be-thrown/71082

  1. FileIO behaves differently than its code comments around read and readall. Update the code comments to match current behavior
    • PEP 475, reads are retried in some cases
    • .readall() makes multiple system calls by design
  2. Add documentation to open() builtin that buffering=0, which currently just says "disables buffering") changes how .write() behaves and may result in data loss as a result of a partial write (ex. Corrupt .pyc files stay on disk after failed writes #126606).
    • TextIO and BufferedIO (which are gotten via commonly used open('README.rst'), open('README.rst', 'rb')) retry partial writes providing a user a guarantee either all bytes will be written or an exception will be thrown.
    • FileIO is "Raw I/O" and by PEP-3116 design, as documented, and currently implemented does not retry partial writes.
    • Most the time, buffering=0 currently speeds up writing a file, but it can also result in corrupted files, ex. Corrupt .pyc files stay on disk after failed writes #126606 from using FileIO directly
    • I would like to try and change the behavior of buffering=0 to use BufferedIO but with a 0 sized buffer, and that is on my roadmap but will be a while (people use the flag for a reason / get benefit!). This will mean open() always returns an object which implements "Write all or throw exception" behavior.
    • Add warning / document existing behavior on open() in the meantime.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    docsDocumentation in the Doc dir

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions