Mercurial > hg
changeset 24305:867c3649be5d
cvsps: use a different tiebreaker to avoid flaky test
After adding some sneaky debug printing[0], I determined that this
test flaked when a CVS commit containing two files starts too close to
the end of a second, thus putting file "a" in one second and "b/c" in
the following second. The secondary sort key meant that these changes
sorted in a different order when the timestamps were different than
they did when they matched. As far as I can tell, CVS walks through
the files in a stable order, so by sorting on the filenames in cvsps
we'll get stable output. It's fine for us to switch from sorting on
the branchpoint as a secondary key because this was already the point
when we didn't care, and we're just trying to break ties in a stable
way. It's unclear to be if having the branchpoint present matters
anymore, but it doesn't really hurt to leave it.
With this change in place, I was able to run test-convert-cvs over 650
times in a row without a failure. test-convert-cvcs-synthetic.t
appears to still be flaky, but I don't think it's *worse* than it was
before - just not better (I observed one flaky failure in 200 runs on
that test).
0: The helpful debug hack ended up being this, in case it's useful to
future flaky test assassins:
--- a/hgext/convert/cvsps.py
+++ b/hgext/convert/cvsps.py
@@ -854,6 +854,8 @@ def debugcvsps(ui, *args, **opts):
ui.write(('Branch: %s\n' % (cs.branch or 'HEAD')))
ui.write(('Tag%s: %s \n' % (['', 's'][len(cs.tags) > 1],
','.join(cs.tags) or '(none)')))
+ if cs.comment == 'ci1' and (cs.id == 6) == bool(cs.branchpoints):
+ ui.write('raw timestamp %r\n' % (cs.date,))
if cs.branchpoints:
ui.write(('Branchpoints: %s \n') %
', '.join(sorted(cs.branchpoints)))
author | Augie Fackler <raf@durin42.com> |
---|---|
date | Fri, 13 Mar 2015 14:20:13 -0400 |
parents | 15afda349b11 |
children | 6ddc86eedc3b |
files | hgext/convert/cvsps.py tests/test-convert-cvs.t |
diffstat | 2 files changed, 12 insertions(+), 5 deletions(-) [+] |
line wrap: on
line diff
--- a/hgext/convert/cvsps.py Fri Mar 13 17:55:04 2015 -0500 +++ b/hgext/convert/cvsps.py Fri Mar 13 14:20:13 2015 -0400 @@ -634,14 +634,21 @@ # By this point, the changesets are sufficiently compared that # we don't really care about ordering. However, this leaves # some race conditions in the tests, so we compare on the - # number of files modified and the number of branchpoints in - # each changeset to ensure test output remains stable. + # number of files modified, the files contained in each + # changeset, and the branchpoints in the change to ensure test + # output remains stable. # recommended replacement for cmp from # https://docs.python.org/3.0/whatsnew/3.0.html c = lambda x, y: (x > y) - (x < y) + # Sort bigger changes first. if not d: d = c(len(l.entries), len(r.entries)) + # Try sorting by filename in the change. + if not d: + d = c([e.file for e in l.entries], [e.file for e in r.entries]) + # Try and put changes without a branch point before ones with + # a branch point. if not d: d = c(len(l.branchpoints), len(r.branchpoints)) return d
--- a/tests/test-convert-cvs.t Fri Mar 13 17:55:04 2015 -0500 +++ b/tests/test-convert-cvs.t Fri Mar 13 14:20:13 2015 -0400 @@ -397,11 +397,12 @@ Author: * (glob) Branch: HEAD Tag: (none) + Branchpoints: branch Log: ci1 Members: - b/c:1.2->1.3 + a:1.1->1.2 --------------------- PatchSet 6 @@ -409,12 +410,11 @@ Author: * (glob) Branch: HEAD Tag: (none) - Branchpoints: branch Log: ci1 Members: - a:1.1->1.2 + b/c:1.2->1.3 --------------------- PatchSet 7