Mercurial > hg
view .hgtags @ 41978:a56487081109
store: don't read the whole fncache in memory
In large repositories with lot of files, the fncache grows more than 100 MB and
reading that whole thing into memory slows things down. Let's not read the whole
thing into memory.
This patch changes fncache loading code to read 1 MB at once. Loading 1 MB at
once saves ~1 sec on perffncacheload for our internal repository. I tried
various values such as 0.5 MB, 5 MB, 10 MB but best results were produced using
1 MB as the chunksize.
On a narrow clone with fncache around 40 MB, this patch saves ~0.04 seconds on
average on perffncacheload.
To test the code, I have coded an extension in test-fncache.t which set
chunksize to 1 byte, and the test passes with that.
Differential Revision: https://phab.mercurial-scm.org/D5296
| author | Pulkit Goyal <pulkit@yandex-team.ru> |
|---|---|
| date | Thu, 22 Nov 2018 15:14:24 +0300 |
| parents | 5a4d2ab7b55c |
| children | 55834ace6f2d |
line wrap: on
line source
d40cc5aacc31ed673d9b5b24f98bee78c283062c 0.4f 1c590d34bf61e2ea12c71738e5a746cd74586157 0.4e 7eca4cfa8aad5fce9a04f7d8acadcd0452e2f34e 0.4d b4d0c3786ad3e47beacf8412157326a32b6d25a4 0.4c f40273b0ad7b3a6d3012fd37736d0611f41ecf54 0.5 0a28dfe59f8fab54a5118c5be4f40da34a53cdb7 0.5b 12e0fdbc57a0be78f0e817fd1d170a3615cd35da 0.6 4ccf3de52989b14c3d84e1097f59e39a992e00bd 0.6b eac9c8efcd9bd8244e72fb6821f769f450457a32 0.6c 979c049974485125e1f9357f6bbe9c1b548a64c3 0.7 3a56574f329a368d645853e0f9e09472aee62349 0.8 6a03cff2b0f5d30281e6addefe96b993582f2eac 0.8.1 35fb62a3a673d5322f6274a44ba6456e5e4b3b37 0.9 2be3001847cb18a23c403439d9e7d0ace30804e9 0.9.1 36a957364b1b89c150f2d0e60a99befe0ee08bd3 0.9.2 27230c29bfec36d5540fbe1c976810aefecfd1d2 0.9.3 fb4b6d5fe100b0886f8bc3d6731ec0e5ed5c4694 0.9.4 23889160905a1b09fffe1c07378e9fc1827606eb 0.9.5 bae2e9c838e90a393bae3973a7850280413e091a 1.0 d5cbbe2c49cee22a9fbeb9ea41daa0ac4e26b846 1.0.1 d2375bbee6d47e62ba8e415c86e83a465dc4dce9 1.0.2 2a67430f92f15ea5159c26b09ec4839a0c549a26 1.1 3773e510d433969e277b1863c317b674cbee2065 1.1.1 11a4eb81fb4f4742451591489e2797dc47903277 1.1.2 11efa41037e280d08cfb07c09ad485df30fb0ea8 1.2 02981000012e3adf40c4849bd7b3d5618f9ce82d 1.2.1 196d40e7c885fa6e95f89134809b3ec7bdbca34b 1.3 3ef6c14a1e8e83a31226f5881b7fe6095bbfa6f6 1.3.1 31ec469f9b556f11819937cf68ee53f2be927ebf 1.4 439d7ea6fe3aa4ab9ec274a68846779153789de9 1.4.1 296a0b14a68621f6990c54fdba0083f6f20935bf 1.4.2 4aa619c4c2c09907034d9824ebb1dd0e878206eb 1.4.3 ff2704a8ded37fbebd8b6eb5ec733731d725da8a 1.5 2b01dab594167bc0dd33331dbaa6dca3dca1b3aa 1.5.1 39f725929f0c48c5fb3b90c071fc3066012456ca 1.5.2 fdcf80f26604f233dc4d8f0a5ef9d7470e317e8a 1.5.3 24fe2629c6fd0c74c90bd066e77387c2b02e8437 1.5.4 f786fc4b8764cd2a5526d259cf2f94d8a66924d9 1.6 bf1774d95bde614af3956d92b20e2a0c68c5fec7 1.6.1 c00f03a4982e467fb6b6bd45908767db6df4771d 1.6.2 ff5cec76b1c5b6be9c3bb923aae8c3c6d079d6b9 1.6.3 93d8bff78c96fe7e33237b257558ee97290048a4 1.6.4 333421b9e0f96c7bc788e5667c146a58a9440a55 1.7 4438875ec01bd0fc32be92b0872eb6daeed4d44f 1.7.1 6aff4f144ad356311318b0011df0bb21f2c97429 1.7.2 e3bf16703e2601de99e563cdb3a5d50b64e6d320 1.7.3 a6c855c32ea081da3c3b8ff628f1847ff271482f 1.7.4 2b2155623ee2559caf288fd333f30475966c4525 1.7.5 2616325766e3504c8ae7c84bd15ee610901fe91d 1.8 aa1f3be38ab127280761889d2dca906ca465b5f4 1.8.1 b032bec2c0a651ca0ddecb65714bfe6770f67d70 1.8.2 3cb1e95676ad089596bd81d0937cad37d6e3b7fb 1.8.3 733af5d9f6b22387913e1d11350fb8cb7c1487dd 1.8.4 de9eb6b1da4fc522b1cab16d86ca166204c24f25 1.9 4a43e23b8c55b4566b8200bf69fe2158485a2634 1.9.1 d629f1e89021103f1753addcef6b310e4435b184 1.9.2 351a9292e430e35766c552066ed3e87c557b803b 1.9.3 384082750f2c51dc917d85a7145748330fa6ef4d 2.0-rc 41453d55b481ddfcc1dacb445179649e24ca861d 2.0 195dbd1cef0c2f9f8bcf4ea303238105f716bda3 2.0.1 6344043924497cd06d781d9014c66802285072e4 2.0.2 db33555eafeaf9df1e18950e29439eaa706d399b 2.1-rc 2aa5b51f310fb3befd26bed99c02267f5c12c734 2.1 53e2cd303ecf8ca7c7eeebd785c34e5ed6b0f4a4 2.1.1 b9bd95e61b49c221c4cca24e6da7c946fc02f992 2.1.2 d9e2f09d5488c395ae9ddbb320ceacd24757e055 2.2-rc 00182b3d087909e3c3ae44761efecdde8f319ef3 2.2 5983de86462c5a9f42a3ad0f5e90ce5b1d221d25 2.2.1 85a358df5bbbe404ca25730c9c459b34263441dc 2.2.2 b013baa3898e117959984fc64c29d8c784d2f28b 2.2.3 a06e2681dd1786e2354d84a5fa9c1c88dd4fa3e0 2.3-rc 7f5094bb3f423fc799e471aac2aee81a7ce57a0b 2.3 072209ae4ddb654eb2d5fd35bff358c738414432 2.3.1 b3f0f9a39c4e1d0250048cd803ab03542d6f140a 2.3.2 d118a4f4fd16d9b558ec3f3e87bfee772861d2b7 2.4-rc 195ad823b5d58c68903a6153a25e3fb4ed25239d 2.4 0c10cf8191469e7c3c8844922e17e71a176cb7cb 2.4.1 a4765077b65e6ae29ba42bab7834717b5072d5ba 2.4.2 f5fbe15ca7449f2c9a3cf817c86d0ae68b307214 2.5-rc a6088c05e43a8aee0472ca3a4f6f8d7dd914ebbf 2.5 7511d4df752e61fe7ae4f3682e0a0008573b0402 2.5.1 5b7175377babacce80a6c1e12366d8032a6d4340 2.5.2 50c922c1b5145dab8baefefb0437d363b6a6c21c 2.5.3 8a7bd2dccd44ed571afe7424cd7f95594f27c092 2.5.4 292cd385856d98bacb2c3086f8897bc660c2beea 2.6-rc 23f785b38af38d2fca6b8f3db56b8007a84cd73a 2.6 ddc7a6be20212d18f3e27d9d7e6f079a66d96f21 2.6.1 cceaf7af4c9e9e6fa2dbfdcfe9856c5da69c4ffd 2.6.2 009794acc6e37a650f0fae37872e733382ac1c0c 2.6.3 f0d7721d7322dcfb5af33599c2543f27335334bb 2.7-rc f37b5a17e6a0ee17afde2cdde5393dd74715fb58 2.7 335a558f81dc73afeab4d7be63617392b130117f 2.7.1 e7fa36d2ad3a7944a52dca126458d6f482db3524 2.7.2 1596f2d8f2421314b1ddead8f7d0c91009358994 2.8-rc d825e4025e39d1c39db943cdc89818abd0a87c27 2.8 209e04a06467e2969c0cc6501335be0406d46ef0 2.8.1 ca387377df7a3a67dbb90b6336b781cdadc3ef41 2.8.2 8862469e16f9236208581b20de5f96bd13cc039d 2.9-rc 3cec5134e9c4bceab6a00c60f52a4f80677a78f2 2.9 b96cb15ec9e04d8ac5ee08b34fcbbe4200588965 2.9.1 3f83fc5cfe715d292069ee8417c83804f6c6c1e4 2.9.2 564f55b251224f16508dd1311452db7780dafe2b 3.0-rc 2195ac506c6ababe86985b932f4948837c0891b5 3.0 269c80ee5b3cb3684fa8edc61501b3506d02eb10 3.0.1 2d8cd3d0e83c7336c0cb45a9f88638363f993848 3.0.2 6c36dc6cd61a0e1b563f1d51e55bdf4dacf12162 3.1-rc 3178e49892020336491cdc6945885c4de26ffa8b 3.1 5dc91146f35369949ea56b40172308158b59063a 3.1.1 f768c888aaa68d12dd7f509dcc7f01c9584357d0 3.1.2 7f8d16af8cae246fa5a48e723d48d58b015aed94 3.2-rc ced632394371a36953ce4d394f86278ae51a2aae 3.2 643c58303fb0ec020907af28b9e486be299ba043 3.2.1 902554884335e5ca3661d63be9978eb4aec3f68a 3.2.2 6dad422ecc5adb63d9fa649eeb8e05a5f9bc4900 3.2.3 1265a3a71d75396f5d4cf6935ae7d9ba5407a547 3.2.4 db8e3f7948b1fdeb9ad12d448fc3525759908b9f 3.3-rc fbdd5195528fae4f41feebc1838215c110b25d6a 3.3 5b4ed033390bf6e2879c8f5c28c84e1ee3b87231 3.3.1 07a92bbd02e5e3a625e0820389b47786b02b2cea 3.3.2 2e2e9a0750f91a6fe0ad88e4de34f8efefdcab08 3.3.3 e89f909edffad558b56f4affa8239e4832f88de0 3.4-rc 8cc6036bca532e06681c5a8fa37efaa812de67b5 3.4 ed18f4acf435a2824c6f49fba40f42b9df5da7ad 3.4.1 540cd0ddac49c1125b2e013aa2ff18ecbd4dd954 3.4.2 96a38d44ba093bd1d1ecfd34119e94056030278b 3.5-rc 21aa1c313b05b1a85f8ffa1120d51579ddf6bf24 3.5 1a45e49a6bed023deb229102a8903234d18054d3 3.5.1 9a466b9f9792e3ad7ae3fc6c43c3ff2e136b718d 3.5.2 b66e3ca0b90c3095ea28dfd39aa24247bebf5c20 3.6-rc 47dd34f2e7272be9e3b2a5a83cd0d20be44293f4 3.6 1aa5083cbebbe7575c88f3402ab377539b484897 3.6.1 2d437a0f3355834a9485bbbeb30a52a052c98f19 3.6.2 ea389970c08449440587712117f178d33bab3f1e 3.6.3 158bdc8965720ca4061f8f8d806563cfc7cdb62e 3.7-rc 2408645de650d8a29a6ce9e7dce601d8dd0d1474 3.7 b698abf971e7377d9b7ec7fc8c52df45255b0329 3.7.1 d493d64757eb45ada99fcb3693e479a51b7782da 3.7.2 ae279d4a19e9683214cbd1fe8298cf0b50571432 3.7.3 740156eedf2c450aee58b1a90b0e826f47c5da64 3.8-rc f85de28eae32e7d3064b1a1321309071bbaaa069 3.8 a56296f55a5e1038ea5016dace2076b693c28a56 3.8.1 aaabed77791a75968a12b8c43ad263631a23ee81 3.8.2 a9764ab80e11bcf6a37255db7dd079011f767c6c 3.8.3 26a5d605b8683a292bb89aea11f37a81b06ac016 3.8.4 519bb4f9d3a47a6e83c2b414d58811ed38f503c2 3.9-rc 299546f84e68dbb9bd026f0f3a974ce4bdb93686 3.9 ccd436f7db6d5d7b9af89715179b911d031d44f1 3.9.1 149433e68974eb5c63ccb03f794d8b57339a80c4 3.9.2 438173c415874f6ac653efc1099dec9c9150e90f 4.0-rc eab27446995210c334c3d06f1a659e3b9b5da769 4.0 b3b1ae98f6a0e14c1e1ba806a6c18e193b6dae5c 4.0.1 e69874dc1f4e142746ff3df91e678a09c6fc208c 4.0.2 a1dd2c0c479e0550040542e392e87bc91262517e 4.1-rc e1526da1e6d84e03146151c9b6e6950fe9a83d7d 4.1 25703b624d27e3917d978af56d6ad59331e0464a 4.1.1 ed5b25874d998ababb181a939dd37a16ea644435 4.1.2 77eaf9539499a1b8be259ffe7ada787d07857f80 4.1.3 616e788321cc4ae9975b7f0c54c849f36d82182b 4.2-rc bb96d4a497432722623ae60d9bc734a1e360179e 4.2 c850f0ed54c1d42f9aa079ad528f8127e5775217 4.2.1 26c49ed51a698ec016d2b4c6b44ca3c3f73cc788 4.2.2 857876ebaed4e315f63157bd157d6ce553c7ab73 4.3-rc 5544af8622863796a0027566f6b646e10d522c4c 4.3 943c91326b23954e6e1c6960d0239511f9530258 4.2.3 3fee7f7d2da04226914c2258cc2884dc27384fd7 4.3.1 920977f72c7b70acfdaf56ab35360584d7845827 4.3.2 2f427b57bf9019c6dc3750baa539dc22c1be50f6 4.3.3 1e2454b60e5936f5e77498cab2648db469504487 4.4-rc 0ccb43d4cf01d013ae05917ec4f305509f851b2d 4.4 cabc840ffdee8a72f3689fb77dd74d04fdc2bc04 4.4.1 a92b9f8e11ba330614cdfd6af0e03b15c1ff3797 4.4.2 27b6df1b5adbdf647cf5c6675b40575e1b197c60 4.5-rc d334afc585e29577f271c5eda03378736a16ca6b 4.5 369aadf7a3264b03c8b09efce715bc41e6ab4a9b 4.5.1 8bba684efde7f45add05f737952093bb2aa07155 4.5.2 7de7bd407251af2bc98e5b809c8598ee95830daf 4.5.3 ed5448edcbfa747b9154099e18630e49024fd47b 4.6rc0 1ec874717d8a93b19e0d50628443e0ee5efab3a9 4.6rc1 6614cac550aea66d19c601e45efd1b7bd08d7c40 4.6 9c5ced5276d6e7d54f7c3dadf5247b7ee98ec79c 4.6.1 0b63a6743010dfdbf8a8154186e119949bdaa1cc 4.6.2 e90130af47ce8dd53a3109aed9d15876b3e7dee8 4.7rc0 33ac6a72308a215e6086fbced347ec10aa963b0a 4.7 ede3bf31fe63677fdf5bd8db687977d4e3d792ed 4.7.1 5405cb1a79010ac50c58cd84e6f50c4556bf2a4c 4.7.2 956ec6f1320df26f3133ec40f3de866ea0695fd7 4.8rc0 a91a2837150bdcb27ae76b3646e6c93cd6a15904 4.8 1c8c54cf97256f4468da2eb4dbee24f7f3888e71 4.8.1 197f092b2cd9691e2a55d198f717b231af9be6f9 4.8.2 593718ff5844cad7a27ee3eb5adad89ac8550949 4.9rc0 83377b4b4ae0e9a6b8e579f7b0a693b8cf5c3b10 4.9