cvsps: use a different tiebreaker to avoid flaky test
After adding some sneaky debug printing[0], I determined that this
test flaked when a CVS commit containing two files starts too close to
the end of a second, thus putting file "a" in one second and "b/c" in
the following second. The secondary sort key meant that these changes
sorted in a different order when the timestamps were different than
they did when they matched. As far as I can tell, CVS walks through
the files in a stable order, so by sorting on the filenames in cvsps
we'll get stable output. It's fine for us to switch from sorting on
the branchpoint as a secondary key because this was already the point
when we didn't care, and we're just trying to break ties in a stable
way. It's unclear to be if having the branchpoint present matters
anymore, but it doesn't really hurt to leave it.
With this change in place, I was able to run test-convert-cvs over 650
times in a row without a failure. test-convert-cvcs-synthetic.t
appears to still be flaky, but I don't think it's *worse* than it was
before - just not better (I observed one flaky failure in 200 runs on
that test).
0: The helpful debug hack ended up being this, in case it's useful to
future flaky test assassins:
--- a/hgext/convert/cvsps.py
+++ b/hgext/convert/cvsps.py
@@ -854,6 +854,8 @@ def debugcvsps(ui, *args, **opts):
ui.write(('Branch: %s\n' % (cs.branch or 'HEAD')))
ui.write(('Tag%s: %s \n' % (['', 's'][len(cs.tags) > 1],
','.join(cs.tags) or '(none)')))
+ if cs.comment == 'ci1' and (cs.id == 6) == bool(cs.branchpoints):
+ ui.write('raw timestamp %r\n' % (cs.date,))
if cs.branchpoints:
ui.write(('Branchpoints: %s \n') %
', '.join(sorted(cs.branchpoints)))
--- a/hgext/convert/cvsps.py Fri Mar 13 17:55:04 2015 -0500
+++ b/hgext/convert/cvsps.py Fri Mar 13 14:20:13 2015 -0400
@@ -634,14 +634,21 @@
# By this point, the changesets are sufficiently compared that
# we don't really care about ordering. However, this leaves
# some race conditions in the tests, so we compare on the
- # number of files modified and the number of branchpoints in
- # each changeset to ensure test output remains stable.
+ # number of files modified, the files contained in each
+ # changeset, and the branchpoints in the change to ensure test
+ # output remains stable.
# recommended replacement for cmp from
# https://docs.python.org/3.0/whatsnew/3.0.html
c = lambda x, y: (x > y) - (x < y)
+ # Sort bigger changes first.
if not d:
d = c(len(l.entries), len(r.entries))
+ # Try sorting by filename in the change.
+ if not d:
+ d = c([e.file for e in l.entries], [e.file for e in r.entries])
+ # Try and put changes without a branch point before ones with
+ # a branch point.
if not d:
d = c(len(l.branchpoints), len(r.branchpoints))
return d
--- a/tests/test-convert-cvs.t Fri Mar 13 17:55:04 2015 -0500
+++ b/tests/test-convert-cvs.t Fri Mar 13 14:20:13 2015 -0400
@@ -397,11 +397,12 @@
Author: * (glob)
Branch: HEAD
Tag: (none)
+ Branchpoints: branch
Log:
ci1
Members:
- b/c:1.2->1.3
+ a:1.1->1.2
---------------------
PatchSet 6
@@ -409,12 +410,11 @@
Author: * (glob)
Branch: HEAD
Tag: (none)
- Branchpoints: branch
Log:
ci1
Members:
- a:1.1->1.2
+ b/c:1.2->1.3
---------------------
PatchSet 7