Skip to content

Simplifying the implementation of signature-overloaded functions #7966

Closed
@anntzer

Description

@anntzer

There are many functions in matplotlib that have "interesting" call signatures, e.g. that could be called with 1, 2 or 3 arguments with different semantics (i.e. not just binding arguments in order and having defaults for the later ones). In this case, the binding of arguments is typically written on an ad-hoc basis, with some bugs (e.g. the one I fixed in https://github.com/matplotlib/matplotlib/pull/7859/files#diff-84224cb1c8cd1f13b7adc5930ee2fc8fR365) or difficult to read code (e.g. https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/quiver.py#L374 (quiver._parse_args)).

Moreover, there are also cases where a function argument should be renamed, but cannot be due to backwards compatibility considerations (e.g. #7954).

I propose to fix both issues using a signature-overloading decorator (see below for prototype implementation). Basically, the idea would be to write something like

@signature_dispatch
def func(<inner_signature>): ... # inner function definition

@func.overload
def func(<signature_1>):
    # <play with args until they match inner_signature>
    return func.__wrapped__(<new_args>)  # Refers to the "inner" function

@func.overload
def func(<signature_2>):
    # <play with args until they match inner_signature>
    return func.__wrapped__(<new_args>)  # Refers to the "inner" function

where the first overload that can bind the arguments is the one used.

In order to support changes in signature due to argument renaming, an overload with the previous signature that raises a DeprecationWarning before forwarding the argument to the "inner" function can be used.

Thoughts?

Protoype implementation:

"""A signature dispatch decorator.

Decorate a function using::

    @signature_dispatch
    def func(...):
        ...

and provide signature overloads using::

    @func.overload
    def func(...): # Note the use of the same name.
        ...
        # Refer to the "original" function as ``func.__wrapped__``

Calling the function will try binding the arguments passed to each overload in
turn, until one binding succeeds; that overload will be called and its return
value (or raised Exception) be used for the original function call.

Overloads can define keyword-only arguments in trailing position in a Py2
compatible manner by having a marker argument named ``__kw_only__``; that
argument behaves like "*" in Py3, i.e., later arguments become keyword-only
(and the ``__kw_only__`` argument itself is always bound to None).

Overloads can define positional arguments in leading position (as defined in
https://docs.python.org/3/library/inspect.html#inspect.Parameter.kind) by
having a marker argument named ``__pos_only__``; earlier arguments become
positional-only (and the ``__pos_only__`` argument iself is always bound to
None).

This implementation should be compatible with Py2 as long as a backport of the
signature object (e.g. funcsigs) is used instead.
"""


from collections import OrderedDict
from functools import wraps
from inspect import signature


def signature_dispatch(func):

    def overload(impl):
        sig = signature(impl)
        params = list(sig.parameters.values())
        try:
            idx = next(idx for idx, param in enumerate(params)
                       if param.name == "__pos_only__")
        except ValueError:
            pass
        else:
            # Make earlier parameters positional only, skip __pos_only__ marker.
            params = ([param.replace(kind=param.POSITIONAL_ONLY)
                       for param in params[:idx]]
                      + params[idx + 1:])
        try:
            idx = next(idx for idx, param in enumerate(params)
                       if param.name == "__kw_only__")
        except ValueError:
            pass
        else:
            # Make later parameters positional only, skip __kw_only__ marker.
            params = (params[:idx]
                      + [param.replace(kind=param.KEYWORD_ONLY)
                         for param in params[idx + 1:]]

        sig = sig.replace(parameters=params)
        impls_sigs.append((impl, sig))
        return wrapper

    @wraps(func)
    def wrapper(*args, **kwargs):
        for impl, sig in impls_sigs:
            try:
                ba = sig.bind(*args, **kwargs)
            except TypeError:
                continue
            else:
                if "__pos_only__" in signature(impl).parameters:
                    ba.arguments["__pos_only__"] = None
                return impl(**ba.arguments)
        raise TypeError("No matching signature")

    impls_sigs = []
    wrapper.overload = overload
    return wrapper


@signature_dispatch
def slice_like(x, y, z):
    return slice(x, y, z)

@slice_like.overload
def slice_like(x, __pos_only__):
    return slice_like.__wrapped__(None, x, None)

@slice_like.overload
def slice_like(x, y, __pos_only__):
    return slice_like.__wrapped__(x, y, None)

@slice_like.overload
def slice_like(x, y, z, __pos_only__):
    return slice_like.__wrapped__(x, y, z)

assert slice_like(10) == slice(10)
assert slice_like(10, 20) == slice(10, 20)
assert slice_like(10, 20, 30) == slice(10, 20, 30)
try: slice_like(x=10)
except TypeError: pass
else: assert False

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions