Mercurial > hg
annotate contrib/dirstatenonnormalcheck.py @ 36367:043e77f3be09
sshpeer: return framed file object when needed
Currently, wireproto.wirepeer has a default implementation of
_submitbatch() and sshv1peer has a very similar implementation.
The main difference is that sshv1peer is aware of the total amount
of bytes it can read whereas the default implementation reads the
stream until no more data is returned. The default implementation
works for HTTP, since there is a known end to HTTP responses (either
Content-Length or 0 sized chunk).
This commit teaches sshv1peer to use our just-introduced "cappedreader"
class for wrapping a file object to limit the number of bytes that
can be read. We do this by introducing an argument to specify whether
the response is framed. If set, we returned a cappedreader instance
instead of the raw pipe.
_call() always has framed responses. So we set this argument
unconditionally and then .read() the entirety of the result.
Strictly speaking, we don't need to use cappedreader in this case
and can inline frame decoding/read logic. But I like when things
are consistent. The overhead should be negligible.
_callstream() and _callcompressable() are special: whether framing
is used depends on the specific command. So, we define a set
of commands that have framed response. It currently only
contains "batch."
As a result of this change, the one-off implementation of
_submitbatch() in sshv1peer can be removed since it is now
safe to .read() the response's file object until end of stream.
cappedreader takes care of not overrunning the frame.
Differential Revision: https://phab.mercurial-scm.org/D2380
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 21 Feb 2018 08:35:48 -0800 |
parents | 6e7fae8f1c6c |
children | 2372284d9457 |
rev | line source |
---|---|
27591
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
1 # dirstatenonnormalcheck.py - extension to check the consistency of the |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
2 # dirstate's non-normal map |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
3 # |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
4 # For most operations on dirstate, this extensions checks that the nonnormalset |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
5 # contains the right entries. |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
6 # It compares the nonnormal file to a nonnormalset built from the map of all |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
7 # the files in the dirstate to check that they contain the same files. |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
8 |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
9 from __future__ import absolute_import |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
10 |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
11 from mercurial import ( |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
12 dirstate, |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
13 extensions, |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
14 ) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
15 |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
16 def nonnormalentries(dmap): |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
17 """Compute nonnormal entries from dirstate's dmap""" |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
18 res = set() |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
19 for f, e in dmap.iteritems(): |
35878
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
20 if e[0] != b'n' or e[3] == -1: |
27591
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
21 res.add(f) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
22 return res |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
23 |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
24 def checkconsistency(ui, orig, dmap, _nonnormalset, label): |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
25 """Compute nonnormalset from dmap, check that it matches _nonnormalset""" |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
26 nonnormalcomputedmap = nonnormalentries(dmap) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
27 if _nonnormalset != nonnormalcomputedmap: |
35878
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
28 ui.develwarn(b"%s call to %s\n" % (label, orig), config=b'dirstate') |
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
29 ui.develwarn(b"inconsistency in nonnormalset\n", config=b'dirstate') |
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
30 ui.develwarn(b"[nonnormalset] %s\n" % _nonnormalset, config=b'dirstate') |
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
31 ui.develwarn(b"[map] %s\n" % nonnormalcomputedmap, config=b'dirstate') |
27591
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
32 |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
33 def _checkdirstate(orig, self, arg): |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
34 """Check nonnormal set consistency before and after the call to orig""" |
34674
60927b19ed65
dirstate: move nonnormal and otherparent sets to dirstatemap
Durham Goode <durham@fb.com>
parents:
29100
diff
changeset
|
35 checkconsistency(self._ui, orig, self._map, self._map.nonnormalset, |
35878
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
36 b"before") |
27637
b502138f5faa
cleanup: remove superfluous space after space after equals (python)
timeless <timeless@mozdev.org>
parents:
27591
diff
changeset
|
37 r = orig(self, arg) |
35878
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
38 checkconsistency(self._ui, orig, self._map, self._map.nonnormalset, |
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
39 b"after") |
27591
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
40 return r |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
41 |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
42 def extsetup(ui): |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
43 """Wrap functions modifying dirstate to check nonnormalset consistency""" |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
44 dirstatecl = dirstate.dirstate |
35878
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
45 devel = ui.configbool(b'devel', b'all-warnings') |
6e7fae8f1c6c
contrib: fix dirstatenonnormalcheck to work in Python 3
Augie Fackler <augie@google.com>
parents:
34674
diff
changeset
|
46 paranoid = ui.configbool(b'experimental', b'nonnormalparanoidcheck') |
27591
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
47 if devel: |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
48 extensions.wrapfunction(dirstatecl, '_writedirstate', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
49 if paranoid: |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
50 # We don't do all these checks when paranoid is disable as it would |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
51 # make the extension run very slowly on large repos |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
52 extensions.wrapfunction(dirstatecl, 'normallookup', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
53 extensions.wrapfunction(dirstatecl, 'otherparent', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
54 extensions.wrapfunction(dirstatecl, 'normal', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
55 extensions.wrapfunction(dirstatecl, 'write', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
56 extensions.wrapfunction(dirstatecl, 'add', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
57 extensions.wrapfunction(dirstatecl, 'remove', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
58 extensions.wrapfunction(dirstatecl, 'merge', _checkdirstate) |
127cc7f78475
dirstate: add test for non-normal set consistency
Laurent Charignon <lcharignon@fb.com>
parents:
diff
changeset
|
59 extensions.wrapfunction(dirstatecl, 'drop', _checkdirstate) |