gh-85403: Make wraps retain type annotations #21392

david-caro · 2020-07-08T09:15:59Z

When decorating a function with wraps, retain the type annotations
of the wrapper if there's any, and use the wrapped function ones to
complete the annotations by default.

This changes the update_wrapper function to use the wrapper function
annotations if any, and fall back to the wrapped function annotations if the
wrapper one did not have them. For the return type annotations, and the
parameter types annotations independently.

The annotations overriding behavior was introduced in bpo-8814.

https://bugs.python.org/issue41231

Issue: Type annotations lost when using wraps by default #85403

When decorating a function with ``wraps``, retain the type annotations of the wrapper if there's any, and use the wrapped function ones to complete the annotations by default. This changes the ``update_wrapper`` function to use the wrapper function annotations if any, and fall back to the wrapped function annotations if the wrapper one did not have them. For the return type annotations, and the parameter types annotations independently. The annotations overriding behavior was introduced in bpo-8814.

Signed-off-by: David Caro <david@dcaro.es>

terryjreedy · 2022-07-11T16:40:42Z

Conflict was trivial. Do a simple update merge of main into the branch rather than a rebase.

hmc-cs-mdrissi · 2022-07-12T04:37:16Z

Hmm, how does this interact with pep 612?

For toy example,

from functools import wraps

def log_info(func: Callable[P, R]) -> Callable[P, R]:
  @wraps
  def _log_func(*args: P.args, **kwargs: P.kwargs) -> R:
    print("stuff")
    return func(*args, **kwargs)

  return _log_func

@log_info
def foo(x: int) -> str:
  ...

The current behavior of wrap is very different here than new behavior. Paramspecs are intended to preserve original function signature and not overrwrite it. I know that code that uses decorators with beam library's runtime time introspection would be affected by this change in a way that can cause it to fail to work. beam inspects function signature types and doesn't handle paramspecs. If both original function (foo) and inner wrapper function (_log_func) have all type hints available still at runtime then I think this is possible to solve, but a very easy edge case bug for any runtime type library.

gvanrossum

I was quite confused by the docs (which make things much more complicated than the code).

gvanrossum · 2022-07-12T17:02:40Z

Doc/library/functools.rst

+   ``assigned`` list parameter), it will be populated only if missing some
+   annotations as follows:
+
+   * If the wrapper function defines it's own return type and parameter


Suggested change

* If the wrapper function defines it's own return type and parameter

* If the wrapper function defines its own return type and parameter

gvanrossum · 2022-07-12T17:03:33Z

Doc/library/functools.rst

+     annotations, those will be preserved.
+
+   * If the wrapper function does not annotate the return type, it will be
+     copied from the wrapped if any.


Suggested change

copied from the wrapped if any.

copied from the wrapped function, if it specifies a return type.

gvanrossum · 2022-07-12T17:04:31Z

Doc/library/functools.rst

+   When assigning the ``__annotations__`` attribute (when it's part of the
+   ``assigned`` list parameter), it will be populated only if missing some


Is it not part of WRAPPER_ASSIGNMENTS?

gvanrossum · 2022-07-12T17:05:20Z

Doc/library/functools.rst

+     copied from the wrapped if any.
+
+   * If the wrapped function does not annotate any parameters, the annotations
+     for them will be copied over from the wrapped if any.


Suggested change

for them will be copied over from the wrapped if any.

for them will be copied over from the wrapped function, if it has any.

gvanrossum · 2022-07-12T17:07:44Z

Doc/library/functools.rst

+   * If the wrapper function defines it's own return type and parameter
+     annotations, those will be preserved.


It is not very clear from this paragraph what this actually does.

Suppose the wrapper has annotations for 'x' and 'y' and the wrapped function has annotations for 'z'. Does 'z' get added to the wrapper's annotations?

Suppose the wrapper has no annotations, so it receives annotations for 'x', 'y' and 'z' from the wrapped function. What happens if the wrapper doesn't have arguments by those names? That is going to be confusing in some cases of further introspection.

It almost sounds like __annotations__ should be listed under updated instead of assigned, there are so many special cases.

gvanrossum · 2022-07-12T17:09:45Z

Doc/library/functools.rst

+   That ensures that any custom annotations from the wrapper will not be
+   overridden by the wrapped, allowing to change the annotated types of
+   the wrapped function.
+
   .. versionadded:: 3.2


Clearly you need a versionadded:: 3.12 block for the new treatment of __annotations__.

gvanrossum · 2022-07-12T17:11:36Z

Lib/functools.py

+
+    if '__annotations__' in assigned:
+        try:
+            # Issue #41231: copy the annotations from wrapped, only if they are


Update the bug number using the GH-NNN or bpo-NNN format.

gvanrossum · 2022-07-12T17:14:25Z

Lib/functools.py

+            # if the wrapper does not annotate the return type, copy the
+            # annotation from the wrapped
+            elif ('return' not in wrapper.__annotations__
+                  and 'return' in wrapped.__annotations__):
+                wrapper.__annotations__['return'] = wrapped.__annotations__['return']


Oh this is quite different from what I had understood from the docs (which I read before reading the code). I recommend changing the docs to more clearly describe what this does.

gvanrossum · 2022-07-12T17:20:37Z

Hmm, how does this interact with pep 612?

That's a really good point. I'll leave it to the patch author to respond, but without a good answer (and backwards compatible behavior, given that ParamSpec is in 3.10 and 3.11) this PR can't go through.

hauntsaninja

Thanks for the PR! Perhaps you could address the feedback / chime in on the PEP 612 thoughts?

Pros:

It seems to me that the behaviour of this PR for PEP 612 functions might actually be desirable. PEP 612 has Concatenate, which can change the signature of the function:

from typing import *
from functools import wraps

P = ParamSpec("P")
R = TypeVar("R")

def log_info(func: Callable[P, R]) -> Callable[Concatenate[str, P], R]:
  @wraps(func)
  def _log_func(x: str, /, *args: P.args, **kwargs: P.kwargs) -> R:
    print(f"stuff {x}")
    return func(*args, **kwargs)

  return _log_func

@log_info
def foo(x: int) -> str:
  ...

print(foo.__annotations__)
print(foo.__wrapped__.__annotations__)

import inspect
print(inspect.signature(foo, follow_wrapped=False))
print(inspect.signature(foo, follow_wrapped=True))

Output on main:

{'x': <class 'int'>, 'return': <class 'str'>}
{'x': <class 'int'>, 'return': <class 'str'>}
(x: int, /, *args, **kwargs) -> str
(x: int) -> str

Output with this PR:

{'x': <class 'str'>, 'args': P.args, 'kwargs': P.kwargs, 'return': ~R}
{'x': <class 'int'>, 'return': <class 'str'>}
(x: str, /, *args: P.args, **kwargs: P.kwargs) -> ~R
(x: int) -> str

I like that this PR makes runtime annotations less lossy (users still have access to the original annotations via __wrapped__).
This fixes the behaviour of inspect.signature(foo, follow_wrapped=False) in the above example

Cons:

It may be hard for runtime type checkers to make sense of this, since they can't see where type variables in the resulting signature are coming from
(weak) The default behaviour of inspect.signature is follow_wrapped=True, which will no longer match up with __annotations__
This is a change in behaviour, so there are some backward compatibility concerns

I'll note that there's some confusion in the original issue from OP regarding static vs dynamic typing. This PR will not change mypy's behaviour and the example of asynccontextmanager will not change in behaviour because asynccontextmanager does not have annotations at runtime.

Overall I lean in favour of this PR.

bedevere-bot · 2023-01-07T23:19:38Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

encukou · 2024-04-10T14:25:50Z

@david-caro, do you want to continue working on this PR?

david-caro · 2024-04-10T15:12:18Z

I might yes, absolutely missed the comment and forgot the pr existed, sorry

Happy to hand over if you are interested though

python-cla-bot · 2025-04-18T09:50:49Z

All commit authors signed the Contributor License Agreement.

JelleZijlstra · 2025-04-18T13:27:53Z

With PEP-649 implemented, it is much harder to partially copy the annotations as this PR does. If we want to make a change here, we'll have to start over, so I'm going to close this for now.

david-caro requested review from gvanrossum, ilevkivskyi and rhettinger as code owners July 8, 2020 09:16

the-knights-who-say-ni added the CLA not signed label Jul 8, 2020

bedevere-bot added the awaiting review label Jul 8, 2020

the-knights-who-say-ni added CLA signed and removed CLA not signed labels Jul 11, 2020

david-caro force-pushed the fix-issue-41231 branch from c5c1acd to 48a2155 Compare August 8, 2020 12:42

Improved the comments and docs

a020a97

Signed-off-by: David Caro <david@dcaro.es>

david-caro force-pushed the fix-issue-41231 branch from 48a2155 to a020a97 Compare August 8, 2020 13:07

rhettinger removed their request for review May 3, 2022 05:59

david-caro mentioned this pull request Jul 11, 2022

Type annotations lost when using wraps by default #85403

Open

python deleted a comment from the-knights-who-say-ni Jul 11, 2022

terryjreedy changed the title ~~bpo-41231: Make wraps retain type annotations~~ gh-85403: Make wraps retain type annotations Jul 11, 2022

Merge branch 'main' into fix-issue-41231

62381ba

terryjreedy requested review from Fidget-Spinner and JelleZijlstra as code owners July 11, 2022 16:38

gvanrossum reviewed Jul 12, 2022

View reviewed changes

ezio-melotti removed the CLA signed label Jul 13, 2022

JelleZijlstra removed their request for review October 3, 2022 20:20

Merge branch 'main' into fix-issue-41231

4097407

hauntsaninja requested a review from AlexWaygood as a code owner January 7, 2023 22:32

hauntsaninja requested changes Jan 7, 2023

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting review labels Jan 7, 2023

hauntsaninja added the topic-typing label Jan 7, 2023

JelleZijlstra closed this Apr 18, 2025

region-spotteR mentioned this pull request Jun 25, 2025

functools.wraps Any in wrapped function python/mypy#18204

Open

	* If the wrapper function defines it's own return type and parameter
	* If the wrapper function defines its own return type and parameter

	copied from the wrapped if any.
	copied from the wrapped function, if it specifies a return type.

		When assigning the ``__annotations__`` attribute (when it's part of the
		``assigned`` list parameter), it will be populated only if missing some

	for them will be copied over from the wrapped if any.
	for them will be copied over from the wrapped function, if it has any.

		* If the wrapper function defines it's own return type and parameter
		annotations, those will be preserved.

Uh oh!

gh-85403: Make wraps retain type annotations #21392

gh-85403: Make wraps retain type annotations #21392

Uh oh!

Conversation

david-caro commented Jul 8, 2020 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

terryjreedy commented Jul 11, 2022

Uh oh!

hmc-cs-mdrissi commented Jul 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gvanrossum commented Jul 12, 2022

Uh oh!

hauntsaninja left a comment • edited by AlexWaygood Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Jan 7, 2023

Uh oh!

encukou commented Apr 10, 2024

Uh oh!

david-caro commented Apr 10, 2024

Uh oh!

python-cla-bot bot commented Apr 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JelleZijlstra commented Apr 18, 2025

Uh oh!

Uh oh!

david-caro commented Jul 8, 2020 •

edited by bedevere-bot

Loading

hmc-cs-mdrissi commented Jul 12, 2022 •

edited

Loading

hauntsaninja left a comment •

edited by AlexWaygood

Loading

python-cla-bot bot commented Apr 18, 2025 •

edited

Loading