annotate mercurial/bitmanipulation.h @ 40626:87a872555e90

revlog: detect incomplete revlog reads _readsegment() is supposed to return N bytes of revlog revision data starting at a file offset. Surprisingly, its behavior before this patch never verified that it actually read and returned N bytes! Instead, it would perform the read(), then return whatever data was available. And even more surprisingly, nothing in the call chain appears to have been validating that it received all the data it was expecting. This behavior could lead to partial or incomplete revision chunks being operated on. This could result in e.g. cached deltas being applied against incomplete base revisions. The delta application process would happily perform this operation. Only hash verification would detect the corruption and save us. This commit changes the behavior of raw revlog reading to validate that we actually read() the number of bytes that were requested. We will raise a more specific error faster, rather than possibly have it go undetected or manifest later in the call stack, at delta application or hash verification. Differential Revision: https://phab.mercurial-scm.org/D5266
author Gregory Szorc <gregory.szorc@gmail.com>
date Tue, 13 Nov 2018 12:30:59 -0800
parents 1fb2510cf8c8
children eed42f1c22d6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
1 #ifndef _HG_BITMANIPULATION_H_
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
2 #define _HG_BITMANIPULATION_H_
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
3
32646
b4356d1cf3e4 bitmanipulation: add missing include of string.h
Martin von Zweigbergk <martinvonz@google.com>
parents: 29444
diff changeset
4 #include <string.h>
b4356d1cf3e4 bitmanipulation: add missing include of string.h
Martin von Zweigbergk <martinvonz@google.com>
parents: 29444
diff changeset
5
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
6 #include "compat.h"
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
7
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
8 static inline uint32_t getbe32(const char *c)
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
9 {
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
10 const unsigned char *d = (const unsigned char *)c;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
11
38303
1fb2510cf8c8 bitmanipulation: fix undefined behavior in bit shift in getbe32
Augie Fackler <augie@google.com>
parents: 34697
diff changeset
12 return ((((uint32_t)d[0]) << 24) | (((uint32_t)d[1]) << 16) |
1fb2510cf8c8 bitmanipulation: fix undefined behavior in bit shift in getbe32
Augie Fackler <augie@google.com>
parents: 34697
diff changeset
13 (((uint32_t)d[2]) << 8) | (d[3]));
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
14 }
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
15
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
16 static inline int16_t getbeint16(const char *c)
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
17 {
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
18 const unsigned char *d = (const unsigned char *)c;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
19
34697
ce77b0563228 bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents: 32646
diff changeset
20 return ((d[0] << 8) | (d[1]));
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
21 }
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
22
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
23 static inline uint16_t getbeuint16(const char *c)
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
24 {
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
25 const unsigned char *d = (const unsigned char *)c;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
26
34697
ce77b0563228 bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents: 32646
diff changeset
27 return ((d[0] << 8) | (d[1]));
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
28 }
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
29
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
30 static inline void putbe32(uint32_t x, char *c)
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
31 {
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
32 c[0] = (x >> 24) & 0xff;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
33 c[1] = (x >> 16) & 0xff;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
34 c[2] = (x >> 8) & 0xff;
34697
ce77b0563228 bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents: 32646
diff changeset
35 c[3] = (x)&0xff;
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
36 }
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
37
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
38 static inline double getbefloat64(const char *c)
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
39 {
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
40 const unsigned char *d = (const unsigned char *)c;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
41 double ret;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
42 int i;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
43 uint64_t t = 0;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
44 for (i = 0; i < 8; i++) {
34697
ce77b0563228 bitmanipulation: reformat with clang-format
Augie Fackler <augie@google.com>
parents: 32646
diff changeset
45 t = (t << 8) + d[i];
29444
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
46 }
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
47 memcpy(&ret, &t, sizeof(t));
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
48 return ret;
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
49 }
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
50
284d742e5611 internals: move the bitmanipulation routines into its own file
Maciej Fijalkowski <fijall@gmail.com>
parents:
diff changeset
51 #endif