Mercurial > hg
annotate mercurial/bitmanipulation.h @ 40626:87a872555e90
revlog: detect incomplete revlog reads
_readsegment() is supposed to return N bytes of revlog revision
data starting at a file offset. Surprisingly, its behavior before
this patch never verified that it actually read and returned N
bytes! Instead, it would perform the read(), then return whatever
data was available. And even more surprisingly, nothing in the
call chain appears to have been validating that it received all
the data it was expecting.
This behavior could lead to partial or incomplete revision chunks
being operated on. This could result in e.g. cached deltas being
applied against incomplete base revisions. The delta application
process would happily perform this operation. Only hash
verification would detect the corruption and save us.
This commit changes the behavior of raw revlog reading to validate
that we actually read() the number of bytes that were requested.
We will raise a more specific error faster, rather than possibly
have it go undetected or manifest later in the call stack, at
delta application or hash verification.
Differential Revision: https://phab.mercurial-scm.org/D5266
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 13 Nov 2018 12:30:59 -0800 |
parents | 1fb2510cf8c8 |
children | eed42f1c22d6 |
rev | line source |
---|---|
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
1 #ifndef _HG_BITMANIPULATION_H_ |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
2 #define _HG_BITMANIPULATION_H_ |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
3 |
32646
b4356d1cf3e4
bitmanipulation: add missing include of string.h
Martin von Zweigbergk <martinvonz@google.com>
parents:
29444
diff
changeset
|
4 #include <string.h> |
b4356d1cf3e4
bitmanipulation: add missing include of string.h
Martin von Zweigbergk <martinvonz@google.com>
parents:
29444
diff
changeset
|
5 |
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
6 #include "compat.h" |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
7 |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
8 static inline uint32_t getbe32(const char *c) |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
9 { |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
10 const unsigned char *d = (const unsigned char *)c; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
11 |
38303
1fb2510cf8c8
bitmanipulation: fix undefined behavior in bit shift in getbe32
Augie Fackler <augie@google.com>
parents:
34697
diff
changeset
|
12 return ((((uint32_t)d[0]) << 24) | (((uint32_t)d[1]) << 16) | |
1fb2510cf8c8
bitmanipulation: fix undefined behavior in bit shift in getbe32
Augie Fackler <augie@google.com>
parents:
34697
diff
changeset
|
13 (((uint32_t)d[2]) << 8) | (d[3])); |
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
14 } |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
15 |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
16 static inline int16_t getbeint16(const char *c) |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
17 { |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
18 const unsigned char *d = (const unsigned char *)c; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
19 |
34697
ce77b0563228
bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents:
32646
diff
changeset
|
20 return ((d[0] << 8) | (d[1])); |
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
21 } |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
22 |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
23 static inline uint16_t getbeuint16(const char *c) |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
24 { |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
25 const unsigned char *d = (const unsigned char *)c; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
26 |
34697
ce77b0563228
bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents:
32646
diff
changeset
|
27 return ((d[0] << 8) | (d[1])); |
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
28 } |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
29 |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
30 static inline void putbe32(uint32_t x, char *c) |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
31 { |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
32 c[0] = (x >> 24) & 0xff; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
33 c[1] = (x >> 16) & 0xff; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
34 c[2] = (x >> 8) & 0xff; |
34697
ce77b0563228
bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents:
32646
diff
changeset
|
35 c[3] = (x)&0xff; |
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
36 } |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
37 |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
38 static inline double getbefloat64(const char *c) |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
39 { |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
40 const unsigned char *d = (const unsigned char *)c; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
41 double ret; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
42 int i; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
43 uint64_t t = 0; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
44 for (i = 0; i < 8; i++) { |
34697
ce77b0563228
bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents:
32646
diff
changeset
|
45 t = (t << 8) + d[i]; |
29444
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
46 } |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
47 memcpy(&ret, &t, sizeof(t)); |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
48 return ret; |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
49 } |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
50 |
284d742e5611
internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff
changeset
|
51 #endif |