Mercurial > hg
annotate mercurial/state.py @ 40626:87a872555e90
revlog: detect incomplete revlog reads
_readsegment() is supposed to return N bytes of revlog revision
data starting at a file offset. Surprisingly, its behavior before
this patch never verified that it actually read and returned N
bytes! Instead, it would perform the read(), then return whatever
data was available. And even more surprisingly, nothing in the
call chain appears to have been validating that it received all
the data it was expecting.
This behavior could lead to partial or incomplete revision chunks
being operated on. This could result in e.g. cached deltas being
applied against incomplete base revisions. The delta application
process would happily perform this operation. Only hash
verification would detect the corruption and save us.
This commit changes the behavior of raw revlog reading to validate
that we actually read() the number of bytes that were requested.
We will raise a more specific error faster, rather than possibly
have it go undetected or manifest later in the call stack, at
delta application or hash verification.
Differential Revision: https://phab.mercurial-scm.org/D5266
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Nov 2018 12:30:59 -0800 |
parents | 5bfab9400daf |
children | 050ea8eb42a5 |
rev | line source |
---|---|
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
1 # state.py - writing and reading state files in Mercurial |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
2 # |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2018 Pulkit Goyal <pulkitmgoyal@gmail.com> |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
4 # |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
7 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
8 """ |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
9 This file contains class to wrap the state for commands and other |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
10 related logic. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
11 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
12 All the data related to the command state is stored as dictionary in the object. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
13 The class has methods using which the data can be stored to disk in a file under |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
14 .hg/ directory. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
15 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
16 We store the data on disk in cbor, for which we use the third party cbor library |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
17 to serialize and deserialize data. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
18 """ |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
19 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
20 from __future__ import absolute_import |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
21 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
22 from . import ( |
38106
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
23 error, |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
24 util, |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
25 ) |
39451
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
26 from .utils import ( |
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
27 cborutil, |
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
28 ) |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
29 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
30 class cmdstate(object): |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
31 """a wrapper class to store the state of commands like `rebase`, `graft`, |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
32 `histedit`, `shelve` etc. Extensions can also use this to write state files. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
33 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
34 All the data for the state is stored in the form of key-value pairs in a |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
35 dictionary. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
36 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
37 The class object can write all the data to a file in .hg/ directory and |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
38 can populate the object data reading that file. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
39 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
40 Uses cbor to serialize and deserialize data while writing and reading from |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
41 disk. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
42 """ |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
43 |
38145
6f67bfe4b82f
state: removing remaining instances of opts class variable
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38133
diff
changeset
|
44 def __init__(self, repo, fname): |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
45 """ repo is the repo object |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
46 fname is the file name in which data should be stored in .hg directory |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
47 """ |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
48 self._repo = repo |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
49 self.fname = fname |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
50 |
38104
36a5a1239a15
state: don't have a dict like interface for cmdstate class
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38103
diff
changeset
|
51 def read(self): |
36a5a1239a15
state: don't have a dict like interface for cmdstate class
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38103
diff
changeset
|
52 """read the existing state file and return a dict of data stored""" |
36a5a1239a15
state: don't have a dict like interface for cmdstate class
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38103
diff
changeset
|
53 return self._read() |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
54 |
38106
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
55 def save(self, version, data): |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
56 """write all the state data stored to .hg/<filename> file |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
57 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
58 we use third-party library cbor to serialize data to write in the file. |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
59 """ |
38106
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
60 if not isinstance(version, int): |
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
61 raise error.ProgrammingError("version of state file should be" |
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
62 " an integer") |
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
63 |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
64 with self._repo.vfs(self.fname, 'wb', atomictemp=True) as fp: |
38126
bdc4079ceb16
state: fix usage of an unassigned variable
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38106
diff
changeset
|
65 fp.write('%d\n' % version) |
39451
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
66 for chunk in cborutil.streamencode(data): |
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
67 fp.write(chunk) |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
68 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
69 def _read(self): |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
70 """reads the state file and returns a dictionary which contain |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
71 data in the same format as it was before storing""" |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
72 with self._repo.vfs(self.fname, 'rb') as fp: |
38106
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
73 try: |
38127
b7e5c53a779e
state: temporary silence pyflakes warning by removing variable assignment
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38126
diff
changeset
|
74 int(fp.readline()) |
38106
a0e4d654bceb
state: write the version number in plain text on top of state files
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38105
diff
changeset
|
75 except ValueError: |
38133
dce718404ce6
state: raise CorruptedState error isntead of ProgrammingError
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38127
diff
changeset
|
76 raise error.CorruptedState("unknown version of state file" |
dce718404ce6
state: raise CorruptedState error isntead of ProgrammingError
Pulkit Goyal <7895pulkit@gmail.com>
parents:
38127
diff
changeset
|
77 " found") |
39451
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
78 |
5bfab9400daf
state: use our CBOR module
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38145
diff
changeset
|
79 return cborutil.decodeall(fp.read())[0] |
38103
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
80 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
81 def delete(self): |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
82 """drop the state file if exists""" |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
83 util.unlinkpath(self._repo.vfs.join(self.fname), ignoremissing=True) |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
84 |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
85 def exists(self): |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
86 """check whether the state file exists or not""" |
a2f83661f721
state: import the file to write state files from evolve extension
Pulkit Goyal <7895pulkit@gmail.com>
parents:
diff
changeset
|
87 return self._repo.vfs.exists(self.fname) |