Mercurial > hg
annotate mercurial/node.py @ 40626:87a872555e90
revlog: detect incomplete revlog reads
_readsegment() is supposed to return N bytes of revlog revision
data starting at a file offset. Surprisingly, its behavior before
this patch never verified that it actually read and returned N
bytes! Instead, it would perform the read(), then return whatever
data was available. And even more surprisingly, nothing in the
call chain appears to have been validating that it received all
the data it was expecting.
This behavior could lead to partial or incomplete revision chunks
being operated on. This could result in e.g. cached deltas being
applied against incomplete base revisions. The delta application
process would happily perform this operation. Only hash
verification would detect the corruption and save us.
This commit changes the behavior of raw revlog reading to validate
that we actually read() the number of bytes that were requested.
We will raise a more specific error faster, rather than possibly
have it go undetected or manifest later in the call stack, at
delta application or hash verification.
Differential Revision: https://phab.mercurial-scm.org/D5266
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Nov 2018 12:30:59 -0800 |
parents | 1e7a462cb946 |
children | 57875cf423c9 |
rev | line source |
---|---|
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
1 # node.py - basic nodeid manipulation for mercurial |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
2 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
3 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com> |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
4 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
10263 | 6 # GNU General Public License version 2 or any later version. |
1089 | 7 |
25962
738314da6c75
node: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25737
diff
changeset
|
8 from __future__ import absolute_import |
738314da6c75
node: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25737
diff
changeset
|
9 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
3578
diff
changeset
|
10 import binascii |
1089 | 11 |
26980
18f50b8cbf1e
node: add 'nullhex', hex-encoded nullid
Siddharth Agarwal <sid0@fb.com>
parents:
25962
diff
changeset
|
12 # This ugly style has a noticeable effect in manifest parsing |
18f50b8cbf1e
node: add 'nullhex', hex-encoded nullid
Siddharth Agarwal <sid0@fb.com>
parents:
25962
diff
changeset
|
13 hex = binascii.hexlify |
36238
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
14 # Adapt to Python 3 API changes. If this ends up showing up in |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
15 # profiles, we can use this version only on Python 3, and forward |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
16 # binascii.unhexlify like we used to on Python 2. |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
17 def bin(s): |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
18 try: |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
19 return binascii.unhexlify(s) |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
20 except binascii.Error as e: |
f574cc00831a
node: make bin() be a wrapper instead of just an alias
Augie Fackler <augie@google.com>
parents:
32684
diff
changeset
|
21 raise TypeError(e) |
26980
18f50b8cbf1e
node: add 'nullhex', hex-encoded nullid
Siddharth Agarwal <sid0@fb.com>
parents:
25962
diff
changeset
|
22 |
3578
3b4e00cba57a
Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
2859
diff
changeset
|
23 nullrev = -1 |
39145
b623c7b23695
nodes: expand/comment the magic nodes so they are more easily searchable
Kyle Lippincott <spectral@google.com>
parents:
37448
diff
changeset
|
24 # In hex, this is '0000000000000000000000000000000000000000' |
28585
a3f3fdac8433
node: use byte literals to construct nullid and wdirid
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26980
diff
changeset
|
25 nullid = b"\0" * 20 |
26980
18f50b8cbf1e
node: add 'nullhex', hex-encoded nullid
Siddharth Agarwal <sid0@fb.com>
parents:
25962
diff
changeset
|
26 nullhex = hex(nullid) |
1089 | 27 |
30360
0298a07f64d9
dirstate: change placeholder hash length to 20 bytes
Durham Goode <durham@fb.com>
parents:
28585
diff
changeset
|
28 # Phony node value to stand-in for new files in some uses of |
0298a07f64d9
dirstate: change placeholder hash length to 20 bytes
Durham Goode <durham@fb.com>
parents:
28585
diff
changeset
|
29 # manifests. |
39145
b623c7b23695
nodes: expand/comment the magic nodes so they are more easily searchable
Kyle Lippincott <spectral@google.com>
parents:
37448
diff
changeset
|
30 # In hex, this is '2121212121212121212121212121212121212121' |
b623c7b23695
nodes: expand/comment the magic nodes so they are more easily searchable
Kyle Lippincott <spectral@google.com>
parents:
37448
diff
changeset
|
31 newnodeid = '!!!!!!!!!!!!!!!!!!!!' |
39195
1e7a462cb946
node: correct hex representation of pseudo node ids
Yuya Nishihara <yuya@tcha.org>
parents:
39145
diff
changeset
|
32 # In hex, this is '3030303030303030303030303030306164646564' |
39145
b623c7b23695
nodes: expand/comment the magic nodes so they are more easily searchable
Kyle Lippincott <spectral@google.com>
parents:
37448
diff
changeset
|
33 addednodeid = '000000000000000added' |
39195
1e7a462cb946
node: correct hex representation of pseudo node ids
Yuya Nishihara <yuya@tcha.org>
parents:
39145
diff
changeset
|
34 # In hex, this is '3030303030303030303030306d6f646966696564' |
39145
b623c7b23695
nodes: expand/comment the magic nodes so they are more easily searchable
Kyle Lippincott <spectral@google.com>
parents:
37448
diff
changeset
|
35 modifiednodeid = '000000000000modified' |
30360
0298a07f64d9
dirstate: change placeholder hash length to 20 bytes
Durham Goode <durham@fb.com>
parents:
28585
diff
changeset
|
36 |
37448
d7114f883505
node: rename wdirnodes to clarify they are for manifest/filelogs
Yuya Nishihara <yuya@tcha.org>
parents:
36238
diff
changeset
|
37 wdirfilenodeids = {newnodeid, addednodeid, modifiednodeid} |
30360
0298a07f64d9
dirstate: change placeholder hash length to 20 bytes
Durham Goode <durham@fb.com>
parents:
28585
diff
changeset
|
38 |
25737
1a5211f2f87f
node: define experimental identifiers for working directory
Yuya Nishihara <yuya@tcha.org>
parents:
10263
diff
changeset
|
39 # pseudo identifiers for working directory |
1a5211f2f87f
node: define experimental identifiers for working directory
Yuya Nishihara <yuya@tcha.org>
parents:
10263
diff
changeset
|
40 # (they are experimental, so don't add too many dependencies on them) |
1a5211f2f87f
node: define experimental identifiers for working directory
Yuya Nishihara <yuya@tcha.org>
parents:
10263
diff
changeset
|
41 wdirrev = 0x7fffffff |
39145
b623c7b23695
nodes: expand/comment the magic nodes so they are more easily searchable
Kyle Lippincott <spectral@google.com>
parents:
37448
diff
changeset
|
42 # In hex, this is 'ffffffffffffffffffffffffffffffffffffffff' |
28585
a3f3fdac8433
node: use byte literals to construct nullid and wdirid
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26980
diff
changeset
|
43 wdirid = b"\xff" * 20 |
32684
af854b1b36f8
revlog: add support for partial matching of wdir node id
Yuya Nishihara <yuya@tcha.org>
parents:
32291
diff
changeset
|
44 wdirhex = hex(wdirid) |
25737
1a5211f2f87f
node: define experimental identifiers for working directory
Yuya Nishihara <yuya@tcha.org>
parents:
10263
diff
changeset
|
45 |
1089 | 46 def short(node): |
47 return hex(node[:6]) |