Repository navigation
Restore read-only properties on builtins.type - #16674
Draft
AlexWaygood wants to merge 2 commits into
Draft
AlexWaygood wants to merge 2 commits into
AlexWaygood wants to merge 2 commits into
Conversation
This comment has been minimized.
This comment has been minimized.
Contributor
|
According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#16549 changed
type.__mro__from a getter-onlypropertyto aFinalattribute, and #16554 made the same change for seven other attributes oftype. All eight are read-only data descriptors on CPython. If a classCdefines__mro__ = "instance value"in its body,C().__mro__reads the string, whileC.__mro__returns the MRO tuple because the descriptor on its metaclass,type, takes precedence.This PR restores the getter-only properties so the stubs express that lookup rule. I think the motivating examples in #16549 (comment) should instead be reported as bugs to pyright; I'm happy to do that if this is merged.
Rationale
Runtime behaviour of the eight descriptors
On CPython,
__mro__and__text_signature__are get-set descriptors. The other six are read-only member descriptors. Looking intype.__dict__retrieves the descriptors themselves, before attribute access invokes their getters. CPython’s get-set definitions · Read-only member definitionsClass and instance lookup, and why restoring the getters permits compatible metaclass overrides
Consider a class that puts its own string named
__mro__in its namespace. The same attribute name reads two different values depending on the receiver.For
C().__mro__, Python finds the string onC. ForC.__mro__, the receiver isCitself, whose metaclass istype. Python finds the data descriptor ontypeand calls its getter withCas the instance, before it ever consultsvars(C). A getter-onlypropertyhas the same precedence. It implements__set__even when there is no setter, in which case assignment raisesAttributeError. That also matches what happens if you try to reassignC.__mro__. CPython’s class attribute lookupWith
__mro__: Final[tuple[type, ...]], ty infersstrfor both reads and acceptsC.__mro__.upper(), which fails on a tuple at runtime. Restoring the getter-onlypropertylets ty infer a tuple forC.__mro__and a string forC().__mro__. This PR restores that spelling for all eight descriptors.Defining an attribute on
C, as above, leaves the descriptor ontypeintact. Defining apropertyon a subclass oftypecan override the descriptor for classes created by that metaclass. CPython allows the following, as do Pyright and ty with the restoredproperty. With theFinalstub, both checkers reject the definition onM.Here the override returns
42, while the built-in descriptor returns the interpreter’s flags forX(22044in this session). #16554 deliberately discouraged overrides that make these values differ. Restoring the getter also restores the previous policy of allowing a compatible override. Discussion of the override policyWhy declaring
type.__mro__asFinalmakes Pyright accept unpackingThe original report in #16549 concerns
type.__mro__itself. The same lookup rule applies becausetypeis its own metaclass.With the property stub, Pyright 1.1.414 infers a property and rejects the unpacking.
Pyright incorrectly skips its metaclass data-descriptor check when the metaclass is the built-in
typeor when the class and metaclass are the same. Both guards apply here. Pyright consequently finds the property in the class namespace and leaves it unbound. With theFinalstub, looking in that same namespace yields the declared tuple type, so unpacking works. The lookup order is still observable in theCexample. Pyright infersstrforC.__mro__with either stub, and acceptsC.__mro__.upper(), which raisesAttributeErrorat runtime. Pyright’s lookup implementationBoth
Dand its metaclassMetadefinefieldin the following example. Pyright gives the property onMetaprecedence.Here
MetaandDare distinct, and the metaclass ofDisMeta. Neither guard skips the descriptor check. Pyright, mypy, and ty all correctly inferintforD.fieldandstrforD().field. Pyrefly incorrectly infersstrfor both.ty's latest typeshed-sync PR exposed two conformance regressions. For both
cls: type[Any]andcls: typing.Type[Any], the suite expectscls.__mro__to have typetuple[type, ...]. WithFinal, ty infers plainAnyand fails both assertions. Restoring the property fixes both.The two conformance assertions and the types inferred by ty
The ty PR syncing these stubs introduced two new false positives. The typing conformance suite checks
__mro__through two equivalent annotations. ty typeshed-sync PR · CI conformance report · Conformance testWith the
propertystub, ty knows that a same-named class attribute cannot hide the getter inherited fromtype(due to the fact that the getter appears on the metaclass oftype, which is itselftype). The precise metaclass of an object of typetype[Any]is unknown and could override the getter, so ty inferstuple[type, ...] & Any, which is "close enough" to the results expected by the conformance suite for ty to pass these assertions.With the
Finalstub, however, ty sees an ordinary value ontype's metaclass (which istype) that a value in the unknown class namespace could shadow. It infers plainAnyfor both reads and reports thatAnydoes not match the asserted tuple type. Restoring the properties removes exactly those two new diagnostics. ty’s rationale for the intersection.Comparison across type checkers
Full results for Pyright, mypy, Pyrefly, and ty, with the examples used in the table
The first two rows use a class whose body defines
__mro__as a string, with this definition:C().__mro__reads the string fromC.__dict__. The classCitself is an instance of the built-intype, whose data descriptor takes precedence forC.__mro__. A checker that infersstrforC.__mro__has picked the class-body string over the descriptor.Each checker targeted Python 3.12 with the same typeshed revision. An arrow shows how its result changes from the current
Finalstub to the proposed@propertystub. A single result means both stubs gave that result. In the table,tupleabbreviatestuple[type, ...](displayed by Pyrefly astuple[type[Any], ...]).C.__mro__strstrstr→ tupleC().__mro__strstrstrstrstrtype.__mro__property(*type.__mro__,)cls.__mro__(cls: type[Any])Any→tuple & AnyThe last row comes from the conformance test, where
clsis a function parameter annotated astype[Any]. ty passes the assertion thatcls.__mro__has typetuple[type, ...]when it inferstuple[type, ...] & Any. It fails the same assertion when it infers plainAny.For the first two rows, CPython 3.14.9 returns the following values.
Proposed polices regarding type-checker bugs
In #8403, Shantanu argued for preserving a checker’s correct behavior when a workaround for another checker causes a significant regression. Because
typeis its own metaclass,type.__mro__returns a tuple and unpacking it is valid. Pyright, mypy, and Pyrefly accept(*type.__mro__,)withFinaland reject it with the property. The cost of that workaround is that ty also acceptsC.__mro__.upper(), which raisesAttributeErrorat runtime.The policy proposed in #16070 generally favors fixing bugs in the checker, while allowing workarounds for bugs that affect many users or alternative spellings that are equally accurate and benefit many users. Here, only the getter expresses why
C.__mro__still returns a tuple whenCdefines its own__mro__. The mypy_primer run on #16549 found no change in its sample, so how often users encounter the unpacking error remains unclear. I think typeshed should restore the getters because they accurately describe how these attributes are descriptors at runtime, which is already fully modeled by ty. Pyright, mypy, and Pyrefly can then fix the unpacking error by applying the descriptor totypeitself when resolvingtype.__mro__.