Post by Ethan FurmanPost by Andreas MaierPost by Raymond HettingerPersonally, I see no need to make the same mistake by removing
the identity-implies-equality rule from the built-in containers.
There's no need to upset the apple cart for nearly zero benefit.
Containers delegate the equal comparison on the container to their
elements; they do not apply identity-based comparison
to their elements. At least that is the externally visible behavior.
If that were true, then [NaN] == [NaN] would be False, and it is not.
Python 3.5.0a0 (default:34881ee3eec5, Jun 16 2014, 11:31:20)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
--> NaN = float('nan')
--> NaN == NaN
False
--> [NaN] == [NaN]
True
Ouch, that hurts ;-)
First, the delegation of sequence equality to element equality is not
something I have come up with during my doc patch. It has always been in
5.9 Comparisons of the Language Reference (copied from Python 3.4):
"Tuples and lists are compared lexicographically using comparison of
corresponding elements. This means that to compare equal, each element
must compare equal and the two sequences must be of the same type and
have the same length."
Second, if not by delegation to equality of its elements, how would the
equality of sequences defined otherwise?
But your test is definitely worth having a closer look at. I have
broadened the test somewhat and that brings up further questions. Here
is the test output, and a discussion of the results (test program
try_eq.py and its output test_eq.out are attached to issue #12067):
Test #1: Different equal int objects:
obj1: type=<class 'int'>, str=257, id=39305936
obj2: type=<class 'int'>, str=257, id=39306160
a) obj1 is obj2: False
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
Discussion:
Case 1.c) can be interpreted that the list delegates its == to the == on
its elements. It cannot be interpreted to delegate to identity
comparison. That is consistent with how everyone (I hope ;-) would
expect int objects to behave, or lists or dicts of them.
The motivation for case f) is explained further down, it has to do with
caching.
Test #2: Same int object:
obj1: type=<class 'int'>, str=257, id=39305936
obj2: type=<class 'int'>, str=257, id=39305936
a) obj1 is obj2: True
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
-> No surprises (I hope).
Test #3: Different equal float objects:
obj1: type=<class 'float'>, str=257.0, id=5734664
obj2: type=<class 'float'>, str=257.0, id=5734640
a) obj1 is obj2: False
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
Discussion:
I added this test only to show that float NaN is a special case, and
that this test for float objects - that are not NaN - behaves like test
#1 for int objects.
Test #4: Same float object:
obj1: type=<class 'float'>, str=257.0, id=5734664
obj2: type=<class 'float'>, str=257.0, id=5734664
a) obj1 is obj2: True
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
-> Same as test #2, hopefully no surprises.
Test #5: Different float NaN objects:
obj1: type=<class 'float'>, str=nan, id=5734784
obj2: type=<class 'float'>, str=nan, id=5734976
a) obj1 is obj2: False
b) obj1 == obj2: False
c) [obj1] == [obj2]: False
d) {obj1:'v'} == {obj2:'v'}: False
e) {'k':obj1} == {'k':obj2}: False
f) obj1 == obj2: False
Discussion:
Here, the list behaves as I would expect under the rule that it
delegates equality to its elements. Case c) allows that interpretation.
However, an interpretation based on identity would also be possible.
Test #6: Same float NaN object:
obj1: type=<class 'float'>, str=nan, id=5734784
obj2: type=<class 'float'>, str=nan, id=5734784
a) obj1 is obj2: True
b) obj1 == obj2: False
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: False
Discussion (this is Ethan's example):
Case 6.b) shows the special behavior of float NaN that is documented: a
float NaN object is the same as itself but unequal to itself.
Case 6.c) is the surprising case. It could be interpreted in two ways
(at least that's what I found):
1) The comparison is based on identity of the float objects. But that is
inconsistent with test #4. And why would the list special-case NaN
comparison in such a way that it ends up being inconsistent with the
special definition of NaN (outside of the list)?
2) The list does not always delegate to element equality, but attempts
to optimize if the objects are the same (same identity). We will see
later that that happens. Further, when comparing float NaNs of the same
identity, the list implementation forgot to special-case NaNs. Which
would be a bug, IMHO. I did not analyze the C implementation, so this is
all speculation based upon external visible behavior.
Test #7: Different objects (with equal x) of class C
(C.__eq__() implemented with equality of x,
C.__ne__() returning NotImplemented):
obj1: type=<class '__main__.C'>, str=C(256), id=39406504
obj2: type=<class '__main__.C'>, str=C(256), id=39406616
a) obj1 is obj2: False
C.__eq__(): self=39406504, other=39406616, returning True
b) obj1 == obj2: True
C.__eq__(): self=39406504, other=39406616, returning True
c) [obj1] == [obj2]: True
C.__eq__(): self=39406616, other=39406504, returning True
d) {obj1:'v'} == {obj2:'v'}: True
C.__eq__(): self=39406504, other=39406616, returning True
e) {'k':obj1} == {'k':obj2}: True
C.__eq__(): self=39406504, other=39406616, returning True
f) obj1 == obj2: True
The __eq__() and __ne__() implementations each print a debug message.
The __ne__() is only defined to verify that it is not invoked, and that
the inherited default __ne__() does not chime in.
Discussion:
Here we see that the list equality comparison does invoke the element
equality. However, the picture becomes more complex further down.
Test #8: Same object of class C
(C.__eq__() implemented with equality of x,
C.__ne__() returning NotImplemented):
obj1: type=<class '__main__.C'>, str=C(256), id=39406504
obj2: type=<class '__main__.C'>, str=C(256), id=39406504
a) obj1 is obj2: True
C.__eq__(): self=39406504, other=39406504, returning True
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
C.__eq__(): self=39406504, other=39406504, returning True
f) obj1 == obj2: True
Discussion:
The == on the class C objects in case 8.b) invokes __eq__(), even though
the objects are the same object. This can be explained by the desire in
Python that classes should be able not to be reflexive, if needed. Like
float NaN, for example.
Now, the list equality in case 8.c) is interesting. The list equality
does not invoke element equality. Even though object equality in case
8.b) did not assume reflexivity and invoked the __eq__() method, the
list seems to assume reflexivity and seems to go by object identity.
The only other potential explanation (that I found) would be that some
aspects of the comparison behavior are cached. That's why I added the
cases f), which show that caching for comparison results does not happen
(the __eq__() method is invoked again).
So we are back to discussing why element equality does not assume
reflexivity, but list equality does. IMHO, that is another bug, or maybe
the same one.
Test #9: Different objects (with equal x) of class D
(D.__eq__() implemented with inequality of x,
D.__ne__() returning NotImplemented):
obj1: type=<class '__main__.D'>, str=C(256), id=39407064
obj2: type=<class '__main__.D'>, str=C(256), id=39406952
a) obj1 is obj2: False
D.__eq__(): self=39407064, other=39406952, returning False
b) obj1 == obj2: False
D.__eq__(): self=39407064, other=39406952, returning False
c) [obj1] == [obj2]: False
D.__eq__(): self=39406952, other=39407064, returning False
d) {obj1:'v'} == {obj2:'v'}: False
D.__eq__(): self=39407064, other=39406952, returning False
e) {'k':obj1} == {'k':obj2}: False
D.__eq__(): self=39407064, other=39406952, returning False
f) obj1 == obj2: False
Discussion:
Class D implements __eq__() by != on the data attribute. This test does
not really show any surprises, and is consistent with the theory that
list comparison delegates to element comparison. This is really just a
preparation for the next test, that uses the same object of this class.
Test #10: Same object of class D
(D.__eq__() implemented with inequality of x,
D.__ne__() returning NotImplemented):
obj1: type=<class '__main__.D'>, str=C(256), id=39407064
obj2: type=<class '__main__.D'>, str=C(256), id=39407064
a) obj1 is obj2: True
D.__eq__(): self=39407064, other=39407064, returning False
b) obj1 == obj2: False
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
D.__eq__(): self=39407064, other=39407064, returning False
f) obj1 == obj2: False
Discussion:
The inequality-based implementation of __eq__() explains case 10.b). It
is surprising (to me) that the list comparison in case 10.c) returns
True. If one compares that to case 9.c), one could believe that the
identities of the objects are used for both cases. But why would the
list not respect the result of __eq__() if it is implemented?
This behavior seems at least to be consistent with surprise of case 6.c)
In order to not just rely on the external behavior, I started digging
into the C implementation. For list equality comparison, I started at
list_richcompare() which uses PyObject_RichCompareBool(), which
shortcuts its result based on identity comparison, and thus enforces
reflexitivity.
The comment on line 714 in object.c in PyObject_RichCompareBool() also
confirms that:
/* Quick result when objects are the same.
Guarantees that identity implies equality. */
IMHO, we need to discuss whether we are serious with the direction that
was claimed earlier in this thread, that reflexivity (i.e. identity
implies equality) should be decided upon by the classes and not by the
Python language. As I see it, we have some pieces of code that enforce
reflexivity, and some that don't.
Andy