Skip to content

Possible memory leak in pybind11 migration #28866

Closed
@tacaswell

Description

@tacaswell

Hi @QuLogic, I think I might have found an issue with this commit. I'm not sure if this is the right forum (or if I should create an issue?). I suspect this commit is causing a memory usage growth, even after running the garbage collector. I've only tested it on Windows 10 with python3.11. Can you reproduce this behavior? Details below.

memory_usage_test.py

import gc

from matplotlib.figure import Figure

from numpy.random import rand

import psutil



def plot(num_points):

    fig = Figure()

    ax = fig.add_subplot()

    ax.plot(rand(num_points))

    fig.clear()



def delta_mb_after_plot_and_gc(num_points):

    proc = psutil.Process()

    before_mb = proc.memory_info().rss / 1e6

    plot(num_points)

    gc.collect()

    after_mb = proc.memory_info().rss / 1e6

    delta_mb = after_mb - before_mb

    print(f"plotting {num_points} points leaks {delta_mb:.3f} MB")



def main():

    gc.disable()

    delta_mb_after_plot_and_gc(100_000)

    delta_mb_after_plot_and_gc(1_000_000)

    delta_mb_after_plot_and_gc(10_000_000)



if __name__ == "__main__":

    main()

Running it against this commit and the previous commit (I found this commit via git bisect)


(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> git checkout 597554db667344c4c4ad026ec1c6dd5f4c688d8f

HEAD is now at 597554db66 Add a pybind11 type caster for GCAgg and its requirements



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> python ..\memory_usage_test.py

plotting 100000 points leaks 1.978 MB

plotting 1000000 points leaks 16.077 MB

plotting 10000000 points leaks 159.719 MB



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> python ..\memory_usage_test.py

plotting 100000 points leaks 1.970 MB

plotting 1000000 points leaks 16.081 MB

plotting 10000000 points leaks 159.625 MB



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> git checkout a47e26bd8583f0fcacc1dd83c3f2ac39b1f7a091

Previous HEAD position was 597554db66 Add a pybind11 type caster for GCAgg and its requirements

HEAD is now at a47e26bd85 Add a pybind11 type caster for agg::rgba



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> python ..\memory_usage_test.py

plotting 100000 points leaks 0.303 MB

plotting 1000000 points leaks 0.041 MB

plotting 10000000 points leaks -0.279 MB



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> python ..\memory_usage_test.py

plotting 100000 points leaks 0.197 MB

plotting 1000000 points leaks 0.041 MB

plotting 10000000 points leaks -0.311 MB

Environment Details


(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> python --version

Python 3.11.9



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> [System.Environment]::OSVersion.Version

Major  Minor  Build  Revision

-----  -----  -----  --------

10     0      19045  0



(mpl_venv) PS C:\Users\justi\projects\matplotlib_ref_cycle\matplotlib> pip freeze | findstr matplotlib

-e git+https://github.com/justinjhendrick/matplotlib.git@a47e26bd8583f0fcacc1dd83c3f2ac39b1f7a091#egg=matplotlib

matplotlib-inline==0.1.7

Originally posted by @justinjhendrick in 597554d

Metadata

Metadata

Assignees

No one assigned

    Labels

    Release criticalFor bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions