annotate hgext/remotefilelog/repack.py @ 43104:74802979dd9d

py3: define and use pycompat.itervalues() .itervalues() only exists on Python 2. Python 3's equivalent is .values(). But we don't want to blindly use .values() everywhere because on Python 2, it will create a list, which will have performance implications. This commit introduces pycompat.itervalues() which will call the appropriate method on the passed object. We update all callers of obj.itervalues() to pycompat.itervalues(obj) instead. With this commit, the only source tranforming remaining is for iteritems(). Victory is near... Differential Revision: https://phab.mercurial-scm.org/D7013
author Gregory Szorc <gregory.szorc@gmail.com>
date Sun, 06 Oct 2019 17:59:15 -0400
parents 687b865b95ad
children 649d3ac37a12
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
1 from __future__ import absolute_import
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
2
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
3 import os
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
4 import time
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
5
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
6 from mercurial.i18n import _
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
7 from mercurial.node import (
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
8 nullid,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
9 short,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
10 )
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
11 from mercurial import (
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
12 encoding,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
13 error,
42943
5fadf6103790 remotefilelog: replace repack lock to solve race condition
Boris Feld <boris.feld@octobus.net>
parents: 42842
diff changeset
14 lock as lockmod,
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
15 mdiff,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
16 policy,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
17 pycompat,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
18 scmutil,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
19 util,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
20 vfs,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
21 )
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
22 from mercurial.utils import procutil
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
23 from . import (
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
24 constants,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
25 contentstore,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
26 datapack,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
27 historypack,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
28 metadatastore,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
29 shallowutil,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
30 )
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
31
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
32 osutil = policy.importmod(r'osutil')
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
33
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
34
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
35 class RepackAlreadyRunning(error.Abort):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
36 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
37
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
38
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
39 def backgroundrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
40 repo, incremental=True, packsonly=False, ensurestart=False
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
41 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
42 cmd = [procutil.hgexecutable(), b'-R', repo.origroot, b'repack']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
43 msg = _(b"(running background repack)\n")
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
44 if incremental:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
45 cmd.append(b'--incremental')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
46 msg = _(b"(running background incremental repack)\n")
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
47 if packsonly:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
48 cmd.append(b'--packsonly')
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
49 repo.ui.warn(msg)
42497
df1419c5756a remotefilelog: tell runbgcommand to not block on child process startup
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
50 # We know this command will find a binary, so don't block on it starting.
42842
2c74337e6483 remotefilelog: reduce probability of race-condition in remotefilelog tests
Boris Feld <boris.feld@octobus.net>
parents: 42497
diff changeset
51 procutil.runbgcommand(cmd, encoding.environ, ensurestart=ensurestart)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
52
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
53
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
54 def fullrepack(repo, options=None):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
55 """If ``packsonly`` is True, stores creating only loose objects are skipped.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
56 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
57 if util.safehasattr(repo, b'shareddatastores'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
58 datasource = contentstore.unioncontentstore(*repo.shareddatastores)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
59 historysource = metadatastore.unionmetadatastore(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
60 *repo.sharedhistorystores, allowincomplete=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
61 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
62
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
63 packpath = shallowutil.getcachepackpath(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
64 repo, constants.FILEPACK_CATEGORY
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
65 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
66 _runrepack(
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
67 repo,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
68 datasource,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
69 historysource,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
70 packpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
71 constants.FILEPACK_CATEGORY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
72 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
73 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
74
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
75 if util.safehasattr(repo.manifestlog, b'datastore'):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
76 localdata, shareddata = _getmanifeststores(repo)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
77 lpackpath, ldstores, lhstores = localdata
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
78 spackpath, sdstores, shstores = shareddata
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
79
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
80 # Repack the shared manifest store
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
81 datasource = contentstore.unioncontentstore(*sdstores)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
82 historysource = metadatastore.unionmetadatastore(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
83 *shstores, allowincomplete=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
84 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
85 _runrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
86 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
87 datasource,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
88 historysource,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
89 spackpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
90 constants.TREEPACK_CATEGORY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
91 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
92 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
93
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
94 # Repack the local manifest store
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
95 datasource = contentstore.unioncontentstore(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
96 *ldstores, allowincomplete=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
97 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
98 historysource = metadatastore.unionmetadatastore(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
99 *lhstores, allowincomplete=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
100 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
101 _runrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
102 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
103 datasource,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
104 historysource,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
105 lpackpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
106 constants.TREEPACK_CATEGORY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
107 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
108 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
109
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
110
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
111 def incrementalrepack(repo, options=None):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
112 """This repacks the repo by looking at the distribution of pack files in the
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
113 repo and performing the most minimal repack to keep the repo in good shape.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
114 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
115 if util.safehasattr(repo, b'shareddatastores'):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
116 packpath = shallowutil.getcachepackpath(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
117 repo, constants.FILEPACK_CATEGORY
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
118 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
119 _incrementalrepack(
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
120 repo,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
121 repo.shareddatastores,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
122 repo.sharedhistorystores,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
123 packpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
124 constants.FILEPACK_CATEGORY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
125 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
126 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
127
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
128 if util.safehasattr(repo.manifestlog, b'datastore'):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
129 localdata, shareddata = _getmanifeststores(repo)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
130 lpackpath, ldstores, lhstores = localdata
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
131 spackpath, sdstores, shstores = shareddata
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
132
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
133 # Repack the shared manifest store
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
134 _incrementalrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
135 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
136 sdstores,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
137 shstores,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
138 spackpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
139 constants.TREEPACK_CATEGORY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
140 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
141 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
142
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
143 # Repack the local manifest store
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
144 _incrementalrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
145 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
146 ldstores,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
147 lhstores,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
148 lpackpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
149 constants.TREEPACK_CATEGORY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
150 allowincompletedata=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
151 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
152 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
153
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
154
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
155 def _getmanifeststores(repo):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
156 shareddatastores = repo.manifestlog.shareddatastores
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
157 localdatastores = repo.manifestlog.localdatastores
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
158 sharedhistorystores = repo.manifestlog.sharedhistorystores
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
159 localhistorystores = repo.manifestlog.localhistorystores
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
160
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
161 sharedpackpath = shallowutil.getcachepackpath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
162 repo, constants.TREEPACK_CATEGORY
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
163 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
164 localpackpath = shallowutil.getlocalpackpath(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
165 repo.svfs.vfs.base, constants.TREEPACK_CATEGORY
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
166 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
167
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
168 return (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
169 (localpackpath, localdatastores, localhistorystores),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
170 (sharedpackpath, shareddatastores, sharedhistorystores),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
171 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
172
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
173
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
174 def _topacks(packpath, files, constructor):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
175 paths = list(os.path.join(packpath, p) for p in files)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
176 packs = list(constructor(p) for p in paths)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
177 return packs
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
178
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
179
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
180 def _deletebigpacks(repo, folder, files):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
181 """Deletes packfiles that are bigger than ``packs.maxpacksize``.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
182
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
183 Returns ``files` with the removed files omitted."""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
184 maxsize = repo.ui.configbytes(b"packs", b"maxpacksize")
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
185 if maxsize <= 0:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
186 return files
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
187
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
188 # This only considers datapacks today, but we could broaden it to include
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
189 # historypacks.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
190 VALIDEXTS = [b".datapack", b".dataidx"]
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
191
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
192 # Either an oversize index or datapack will trigger cleanup of the whole
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
193 # pack:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
194 oversized = {
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
195 os.path.splitext(path)[0]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
196 for path, ftype, stat in files
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
197 if (stat.st_size > maxsize and (os.path.splitext(path)[1] in VALIDEXTS))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
198 }
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
199
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
200 for rootfname in oversized:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
201 rootpath = os.path.join(folder, rootfname)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
202 for ext in VALIDEXTS:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
203 path = rootpath + ext
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
204 repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
205 b'removing oversize packfile %s (%s)\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
206 % (path, util.bytecount(os.stat(path).st_size))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
207 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
208 os.unlink(path)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
209 return [row for row in files if os.path.basename(row[0]) not in oversized]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
210
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
211
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
212 def _incrementalrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
213 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
214 datastore,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
215 historystore,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
216 packpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
217 category,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
218 allowincompletedata=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
219 options=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
220 ):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
221 shallowutil.mkstickygroupdir(repo.ui, packpath)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
222
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
223 files = osutil.listdir(packpath, stat=True)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
224 files = _deletebigpacks(repo, packpath, files)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
225 datapacks = _topacks(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
226 packpath, _computeincrementaldatapack(repo.ui, files), datapack.datapack
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
227 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
228 datapacks.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
229 s for s in datastore if not isinstance(s, datapack.datapackstore)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
230 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
231
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
232 historypacks = _topacks(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
233 packpath,
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
234 _computeincrementalhistorypack(repo.ui, files),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
235 historypack.historypack,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
236 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
237 historypacks.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
238 s
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
239 for s in historystore
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
240 if not isinstance(s, historypack.historypackstore)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
241 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
242
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
243 # ``allhistory{files,packs}`` contains all known history packs, even ones we
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
244 # don't plan to repack. They are used during the datapack repack to ensure
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
245 # good ordering of nodes.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
246 allhistoryfiles = _allpackfileswithsuffix(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
247 files, historypack.PACKSUFFIX, historypack.INDEXSUFFIX
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
248 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
249 allhistorypacks = _topacks(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
250 packpath,
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
251 (f for f, mode, stat in allhistoryfiles),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
252 historypack.historypack,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
253 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
254 allhistorypacks.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
255 s
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
256 for s in historystore
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
257 if not isinstance(s, historypack.historypackstore)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
258 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
259 _runrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
260 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
261 contentstore.unioncontentstore(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
262 *datapacks, allowincomplete=allowincompletedata
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
263 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
264 metadatastore.unionmetadatastore(*historypacks, allowincomplete=True),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
265 packpath,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
266 category,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
267 fullhistory=metadatastore.unionmetadatastore(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
268 *allhistorypacks, allowincomplete=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
269 ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
270 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
271 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
272
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
273
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
274 def _computeincrementaldatapack(ui, files):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
275 opts = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
276 b'gencountlimit': ui.configint(b'remotefilelog', b'data.gencountlimit'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
277 b'generations': ui.configlist(b'remotefilelog', b'data.generations'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
278 b'maxrepackpacks': ui.configint(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
279 b'remotefilelog', b'data.maxrepackpacks'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
280 ),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
281 b'repackmaxpacksize': ui.configbytes(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
282 b'remotefilelog', b'data.repackmaxpacksize'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
283 ),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
284 b'repacksizelimit': ui.configbytes(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
285 b'remotefilelog', b'data.repacksizelimit'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
286 ),
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
287 }
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
288
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
289 packfiles = _allpackfileswithsuffix(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
290 files, datapack.PACKSUFFIX, datapack.INDEXSUFFIX
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
291 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
292 return _computeincrementalpack(packfiles, opts)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
293
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
294
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
295 def _computeincrementalhistorypack(ui, files):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
296 opts = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
297 b'gencountlimit': ui.configint(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
298 b'remotefilelog', b'history.gencountlimit'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
299 ),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
300 b'generations': ui.configlist(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
301 b'remotefilelog', b'history.generations', [b'100MB']
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
302 ),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
303 b'maxrepackpacks': ui.configint(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
304 b'remotefilelog', b'history.maxrepackpacks'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
305 ),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
306 b'repackmaxpacksize': ui.configbytes(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
307 b'remotefilelog', b'history.repackmaxpacksize', b'400MB'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
308 ),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
309 b'repacksizelimit': ui.configbytes(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
310 b'remotefilelog', b'history.repacksizelimit'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
311 ),
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
312 }
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
313
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
314 packfiles = _allpackfileswithsuffix(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
315 files, historypack.PACKSUFFIX, historypack.INDEXSUFFIX
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
316 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
317 return _computeincrementalpack(packfiles, opts)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
318
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
319
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
320 def _allpackfileswithsuffix(files, packsuffix, indexsuffix):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
321 result = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
322 fileset = set(fn for fn, mode, stat in files)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
323 for filename, mode, stat in files:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
324 if not filename.endswith(packsuffix):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
325 continue
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
326
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
327 prefix = filename[: -len(packsuffix)]
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
328
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
329 # Don't process a pack if it doesn't have an index.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
330 if (prefix + indexsuffix) not in fileset:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
331 continue
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
332 result.append((prefix, mode, stat))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
333
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
334 return result
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
335
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
336
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
337 def _computeincrementalpack(files, opts):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
338 """Given a set of pack files along with the configuration options, this
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
339 function computes the list of files that should be packed as part of an
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
340 incremental repack.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
341
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
342 It tries to strike a balance between keeping incremental repacks cheap (i.e.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
343 packing small things when possible, and rolling the packs up to the big ones
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
344 over time).
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
345 """
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
346
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
347 limits = list(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
348 sorted((util.sizetoint(s) for s in opts[b'generations']), reverse=True)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
349 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
350 limits.append(0)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
351
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
352 # Group the packs by generation (i.e. by size)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
353 generations = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
354 for i in pycompat.xrange(len(limits)):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
355 generations.append([])
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
356
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
357 sizes = {}
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
358 for prefix, mode, stat in files:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
359 size = stat.st_size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
360 if size > opts[b'repackmaxpacksize']:
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
361 continue
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
362
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
363 sizes[prefix] = size
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
364 for i, limit in enumerate(limits):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
365 if size > limit:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
366 generations[i].append(prefix)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
367 break
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
368
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
369 # Steps for picking what packs to repack:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
370 # 1. Pick the largest generation with > gencountlimit pack files.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
371 # 2. Take the smallest three packs.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
372 # 3. While total-size-of-packs < repacksizelimit: add another pack
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
373
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
374 # Find the largest generation with more than gencountlimit packs
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
375 genpacks = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
376 for i, limit in enumerate(limits):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
377 if len(generations[i]) > opts[b'gencountlimit']:
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
378 # Sort to be smallest last, for easy popping later
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
379 genpacks.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
380 sorted(generations[i], reverse=True, key=lambda x: sizes[x])
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
381 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
382 break
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
383
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
384 # Take as many packs from the generation as we can
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
385 chosenpacks = genpacks[-3:]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
386 genpacks = genpacks[:-3]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
387 repacksize = sum(sizes[n] for n in chosenpacks)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
388 while (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
389 repacksize < opts[b'repacksizelimit']
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
390 and genpacks
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
391 and len(chosenpacks) < opts[b'maxrepackpacks']
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
392 ):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
393 chosenpacks.append(genpacks.pop())
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
394 repacksize += sizes[chosenpacks[-1]]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
395
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
396 return chosenpacks
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
397
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
398
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
399 def _runrepack(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
400 repo, data, history, packpath, category, fullhistory=None, options=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
401 ):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
402 shallowutil.mkstickygroupdir(repo.ui, packpath)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
403
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
404 def isold(repo, filename, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
405 """Check if the file node is older than a limit.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
406 Unless a limit is specified in the config the default limit is taken.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
407 """
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
408 filectx = repo.filectx(filename, fileid=node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
409 filetime = repo[filectx.linkrev()].date()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
410
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
411 ttl = repo.ui.configint(b'remotefilelog', b'nodettl')
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
412
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
413 limit = time.time() - ttl
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
414 return filetime[0] < limit
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
415
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
416 garbagecollect = repo.ui.configbool(b'remotefilelog', b'gcrepack')
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
417 if not fullhistory:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
418 fullhistory = history
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
419 packer = repacker(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
420 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
421 data,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
422 history,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
423 fullhistory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
424 category,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
425 gc=garbagecollect,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
426 isold=isold,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
427 options=options,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
428 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
429
41806
118c1ec4f31b remotefilelog: do not specify an explicit version for repack
Kyle Lippincott <spectral@google.com>
parents: 41365
diff changeset
430 with datapack.mutabledatapack(repo.ui, packpath) as dpack:
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
431 with historypack.mutablehistorypack(repo.ui, packpath) as hpack:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
432 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
433 packer.run(dpack, hpack)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
434 except error.LockHeld:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
435 raise RepackAlreadyRunning(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
436 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
437 b"skipping repack - another repack "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
438 b"is already running"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
439 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
440 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
441
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
442
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
443 def keepset(repo, keyfn, lastkeepkeys=None):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
444 """Computes a keepset which is not garbage collected.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
445 'keyfn' is a function that maps filename, node to a unique key.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
446 'lastkeepkeys' is an optional argument and if provided the keepset
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
447 function updates lastkeepkeys with more keys and returns the result.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
448 """
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
449 if not lastkeepkeys:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
450 keepkeys = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
451 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
452 keepkeys = lastkeepkeys
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
453
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
454 # We want to keep:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
455 # 1. Working copy parent
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
456 # 2. Draft commits
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
457 # 3. Parents of draft commits
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
458 # 4. Pullprefetch and bgprefetchrevs revsets if specified
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
459 revs = [b'.', b'draft()', b'parents(draft())']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
460 prefetchrevs = repo.ui.config(b'remotefilelog', b'pullprefetch', None)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
461 if prefetchrevs:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
462 revs.append(b'(%s)' % prefetchrevs)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
463 prefetchrevs = repo.ui.config(b'remotefilelog', b'bgprefetchrevs', None)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
464 if prefetchrevs:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
465 revs.append(b'(%s)' % prefetchrevs)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
466 revs = b'+'.join(revs)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
467
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
468 revs = [b'sort((%s), "topo")' % revs]
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
469 keep = scmutil.revrange(repo, revs)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
470
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
471 processed = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
472 lastmanifest = None
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
473
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
474 # process the commits in toposorted order starting from the oldest
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
475 for r in reversed(keep._list):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
476 if repo[r].p1().rev() in processed:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
477 # if the direct parent has already been processed
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
478 # then we only need to process the delta
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
479 m = repo[r].manifestctx().readdelta()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
480 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
481 # otherwise take the manifest and diff it
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
482 # with the previous manifest if one exists
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
483 if lastmanifest:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
484 m = repo[r].manifest().diff(lastmanifest)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
485 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
486 m = repo[r].manifest()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
487 lastmanifest = repo[r].manifest()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
488 processed.add(r)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
489
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
490 # populate keepkeys with keys from the current manifest
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
491 if type(m) is dict:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
492 # m is a result of diff of two manifests and is a dictionary that
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
493 # maps filename to ((newnode, newflag), (oldnode, oldflag)) tuple
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
494 for filename, diff in m.iteritems():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
495 if diff[0][0] is not None:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
496 keepkeys.add(keyfn(filename, diff[0][0]))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
497 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
498 # m is a manifest object
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
499 for filename, filenode in m.iteritems():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
500 keepkeys.add(keyfn(filename, filenode))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
501
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
502 return keepkeys
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
503
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
504
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
505 class repacker(object):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
506 """Class for orchestrating the repack of data and history information into a
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
507 new format.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
508 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
509
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
510 def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
511 self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
512 repo,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
513 data,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
514 history,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
515 fullhistory,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
516 category,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
517 gc=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
518 isold=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
519 options=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
520 ):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
521 self.repo = repo
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
522 self.data = data
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
523 self.history = history
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
524 self.fullhistory = fullhistory
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
525 self.unit = constants.getunits(category)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
526 self.garbagecollect = gc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
527 self.options = options
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
528 if self.garbagecollect:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
529 if not isold:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
530 raise ValueError(b"Function 'isold' is not properly specified")
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
531 # use (filename, node) tuple as a keepset key
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
532 self.keepkeys = keepset(repo, lambda f, n: (f, n))
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
533 self.isold = isold
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
534
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
535 def run(self, targetdata, targethistory):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
536 ledger = repackledger()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
537
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
538 with lockmod.lock(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
539 repacklockvfs(self.repo), b"repacklock", desc=None, timeout=0
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
540 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
541 self.repo.hook(b'prerepack')
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
542
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
543 # Populate ledger from source
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
544 self.data.markledger(ledger, options=self.options)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
545 self.history.markledger(ledger, options=self.options)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
546
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
547 # Run repack
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
548 self.repackdata(ledger, targetdata)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
549 self.repackhistory(ledger, targethistory)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
550
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
551 # Call cleanup on each source
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
552 for source in ledger.sources:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
553 source.cleanup(ledger)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
554
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
555 def _chainorphans(self, ui, filename, nodes, orphans, deltabases):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
556 """Reorderes ``orphans`` into a single chain inside ``nodes`` and
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
557 ``deltabases``.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
558
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
559 We often have orphan entries (nodes without a base that aren't
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
560 referenced by other nodes -- i.e., part of a chain) due to gaps in
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
561 history. Rather than store them as individual fulltexts, we prefer to
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
562 insert them as one chain sorted by size.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
563 """
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
564 if not orphans:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
565 return nodes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
566
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
567 def getsize(node, default=0):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
568 meta = self.data.getmeta(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
569 if constants.METAKEYSIZE in meta:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
570 return meta[constants.METAKEYSIZE]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
571 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
572 return default
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
573
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
574 # Sort orphans by size; biggest first is preferred, since it's more
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
575 # likely to be the newest version assuming files grow over time.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
576 # (Sort by node first to ensure the sort is stable.)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
577 orphans = sorted(orphans)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
578 orphans = list(sorted(orphans, key=getsize, reverse=True))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
579 if ui.debugflag:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
580 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
581 b"%s: orphan chain: %s\n"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
582 % (filename, b", ".join([short(s) for s in orphans]))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
583 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
584
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
585 # Create one contiguous chain and reassign deltabases.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
586 for i, node in enumerate(orphans):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
587 if i == 0:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
588 deltabases[node] = (nullid, 0)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
589 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
590 parent = orphans[i - 1]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
591 deltabases[node] = (parent, deltabases[parent][1] + 1)
41255
60b3edccf2f7 remotefilelog: use list comprehension instead of filter for py3 portability
Augie Fackler <augie@google.com>
parents: 40843
diff changeset
592 nodes = [n for n in nodes if n not in orphans]
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
593 nodes += orphans
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
594 return nodes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
595
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
596 def repackdata(self, ledger, target):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
597 ui = self.repo.ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
598 maxchainlen = ui.configint(b'packs', b'maxchainlen', 1000)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
599
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
600 byfile = {}
43104
74802979dd9d py3: define and use pycompat.itervalues()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
601 for entry in pycompat.itervalues(ledger.entries):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
602 if entry.datasource:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
603 byfile.setdefault(entry.filename, {})[entry.node] = entry
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
604
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
605 count = 0
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
606 repackprogress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
607 _(b"repacking data"), unit=self.unit, total=len(byfile)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
608 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
609 for filename, entries in sorted(byfile.iteritems()):
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
610 repackprogress.update(count)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
611
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
612 ancestors = {}
40613
9769e0f6ffe0 py3: don't use dict.iterkeys()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40526
diff changeset
613 nodes = list(node for node in entries)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
614 nohistory = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
615 buildprogress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
616 _(b"building history"), unit=b'nodes', total=len(nodes)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
617 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
618 for i, node in enumerate(nodes):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
619 if node in ancestors:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
620 continue
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
621 buildprogress.update(i)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
622 try:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
623 ancestors.update(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
624 self.fullhistory.getancestors(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
625 filename, node, known=ancestors
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
626 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
627 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
628 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
629 # Since we're packing data entries, we may not have the
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
630 # corresponding history entries for them. It's not a big
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
631 # deal, but the entries won't be delta'd perfectly.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
632 nohistory.append(node)
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
633 buildprogress.complete()
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
634
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
635 # Order the nodes children first, so we can produce reverse deltas
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
636 orderednodes = list(reversed(self._toposort(ancestors)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
637 if len(nohistory) > 0:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
638 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
639 b'repackdata: %d nodes without history\n' % len(nohistory)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
640 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
641 orderednodes.extend(sorted(nohistory))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
642
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
643 # Filter orderednodes to just the nodes we want to serialize (it
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
644 # currently also has the edge nodes' ancestors).
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
645 orderednodes = list(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
646 filter(lambda node: node in nodes, orderednodes)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
647 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
648
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
649 # Garbage collect old nodes:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
650 if self.garbagecollect:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
651 neworderednodes = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
652 for node in orderednodes:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
653 # If the node is old and is not in the keepset, we skip it,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
654 # and mark as garbage collected
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
655 if (filename, node) not in self.keepkeys and self.isold(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
656 self.repo, filename, node
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
657 ):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
658 entries[node].gced = True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
659 continue
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
660 neworderednodes.append(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
661 orderednodes = neworderednodes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
662
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
663 # Compute delta bases for nodes:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
664 deltabases = {}
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
665 nobase = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
666 referenced = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
667 nodes = set(nodes)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
668 processprogress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
669 _(b"processing nodes"), unit=b'nodes', total=len(orderednodes)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
670 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
671 for i, node in enumerate(orderednodes):
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
672 processprogress.update(i)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
673 # Find delta base
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
674 # TODO: allow delta'ing against most recent descendant instead
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
675 # of immediate child
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
676 deltatuple = deltabases.get(node, None)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
677 if deltatuple is None:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
678 deltabase, chainlen = nullid, 0
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
679 deltabases[node] = (nullid, 0)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
680 nobase.add(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
681 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
682 deltabase, chainlen = deltatuple
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
683 referenced.add(deltabase)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
684
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
685 # Use available ancestor information to inform our delta choices
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
686 ancestorinfo = ancestors.get(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
687 if ancestorinfo:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
688 p1, p2, linknode, copyfrom = ancestorinfo
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
689
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
690 # The presence of copyfrom means we're at a point where the
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
691 # file was copied from elsewhere. So don't attempt to do any
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
692 # deltas with the other file.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
693 if copyfrom:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
694 p1 = nullid
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
695
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
696 if chainlen < maxchainlen:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
697 # Record this child as the delta base for its parents.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
698 # This may be non optimal, since the parents may have
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
699 # many children, and this will only choose the last one.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
700 # TODO: record all children and try all deltas to find
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
701 # best
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
702 if p1 != nullid:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
703 deltabases[p1] = (node, chainlen + 1)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
704 if p2 != nullid:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
705 deltabases[p2] = (node, chainlen + 1)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
706
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
707 # experimental config: repack.chainorphansbysize
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
708 if ui.configbool(b'repack', b'chainorphansbysize'):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
709 orphans = nobase - referenced
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
710 orderednodes = self._chainorphans(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
711 ui, filename, orderednodes, orphans, deltabases
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
712 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
713
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
714 # Compute deltas and write to the pack
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
715 for i, node in enumerate(orderednodes):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
716 deltabase, chainlen = deltabases[node]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
717 # Compute delta
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
718 # TODO: Optimize the deltachain fetching. Since we're
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
719 # iterating over the different version of the file, we may
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
720 # be fetching the same deltachain over and over again.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
721 if deltabase != nullid:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
722 deltaentry = self.data.getdelta(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
723 delta, deltabasename, origdeltabase, meta = deltaentry
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
724 size = meta.get(constants.METAKEYSIZE)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
725 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
726 deltabasename != filename
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
727 or origdeltabase != deltabase
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
728 or size is None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
729 ):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
730 deltabasetext = self.data.get(filename, deltabase)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
731 original = self.data.get(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
732 size = len(original)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
733 delta = mdiff.textdiff(deltabasetext, original)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
734 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
735 delta = self.data.get(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
736 size = len(delta)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
737 meta = self.data.getmeta(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
738
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
739 # TODO: don't use the delta if it's larger than the fulltext
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
740 if constants.METAKEYSIZE not in meta:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
741 meta[constants.METAKEYSIZE] = size
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
742 target.add(filename, node, deltabase, delta, meta)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
743
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
744 entries[node].datarepacked = True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
745
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
746 processprogress.complete()
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
747 count += 1
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
748
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
749 repackprogress.complete()
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
750 target.close(ledger=ledger)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
751
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
752 def repackhistory(self, ledger, target):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
753 ui = self.repo.ui
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
754
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
755 byfile = {}
43104
74802979dd9d py3: define and use pycompat.itervalues()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
756 for entry in pycompat.itervalues(ledger.entries):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
757 if entry.historysource:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
758 byfile.setdefault(entry.filename, {})[entry.node] = entry
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
759
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
760 progress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
761 _(b"repacking history"), unit=self.unit, total=len(byfile)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
762 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
763 for filename, entries in sorted(byfile.iteritems()):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
764 ancestors = {}
40613
9769e0f6ffe0 py3: don't use dict.iterkeys()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40526
diff changeset
765 nodes = list(node for node in entries)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
766
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
767 for node in nodes:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
768 if node in ancestors:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
769 continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
770 ancestors.update(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
771 self.history.getancestors(filename, node, known=ancestors)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
772 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
773
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
774 # Order the nodes children first
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
775 orderednodes = reversed(self._toposort(ancestors))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
776
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
777 # Write to the pack
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
778 dontprocess = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
779 for node in orderednodes:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
780 p1, p2, linknode, copyfrom = ancestors[node]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
781
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
782 # If the node is marked dontprocess, but it's also in the
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
783 # explicit entries set, that means the node exists both in this
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
784 # file and in another file that was copied to this file.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
785 # Usually this happens if the file was copied to another file,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
786 # then the copy was deleted, then reintroduced without copy
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
787 # metadata. The original add and the new add have the same hash
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
788 # since the content is identical and the parents are null.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
789 if node in dontprocess and node not in entries:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
790 # If copyfrom == filename, it means the copy history
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
791 # went to come other file, then came back to this one, so we
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
792 # should continue processing it.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
793 if p1 != nullid and copyfrom != filename:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
794 dontprocess.add(p1)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
795 if p2 != nullid:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
796 dontprocess.add(p2)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
797 continue
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
798
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
799 if copyfrom:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
800 dontprocess.add(p1)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
801
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
802 target.add(filename, node, p1, p2, linknode, copyfrom)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
803
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
804 if node in entries:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
805 entries[node].historyrepacked = True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
806
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
807 progress.increment()
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
808
40843
b80915b52476 remotefilelog: use progress helper in repack
Martin von Zweigbergk <martinvonz@google.com>
parents: 40804
diff changeset
809 progress.complete()
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
810 target.close(ledger=ledger)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
811
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
812 def _toposort(self, ancestors):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
813 def parentfunc(node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
814 p1, p2, linknode, copyfrom = ancestors[node]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
815 parents = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
816 if p1 != nullid:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
817 parents.append(p1)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
818 if p2 != nullid:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
819 parents.append(p2)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
820 return parents
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
821
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
822 sortednodes = shallowutil.sortnodes(ancestors.keys(), parentfunc)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
823 return sortednodes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
824
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
825
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
826 class repackledger(object):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
827 """Storage for all the bookkeeping that happens during a repack. It contains
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
828 the list of revisions being repacked, what happened to each revision, and
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
829 which source store contained which revision originally (for later cleanup).
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
830 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
831
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
832 def __init__(self):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
833 self.entries = {}
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
834 self.sources = {}
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
835 self.created = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
836
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
837 def markdataentry(self, source, filename, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
838 """Mark the given filename+node revision as having a data rev in the
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
839 given source.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
840 """
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
841 entry = self._getorcreateentry(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
842 entry.datasource = True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
843 entries = self.sources.get(source)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
844 if not entries:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
845 entries = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
846 self.sources[source] = entries
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
847 entries.add(entry)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
848
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
849 def markhistoryentry(self, source, filename, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
850 """Mark the given filename+node revision as having a history rev in the
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
851 given source.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
852 """
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
853 entry = self._getorcreateentry(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
854 entry.historysource = True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
855 entries = self.sources.get(source)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
856 if not entries:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
857 entries = set()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
858 self.sources[source] = entries
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
859 entries.add(entry)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
860
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
861 def _getorcreateentry(self, filename, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
862 key = (filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
863 value = self.entries.get(key)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
864 if not value:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
865 value = repackentry(filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
866 self.entries[key] = value
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
867
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
868 return value
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
869
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
870 def addcreated(self, value):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
871 self.created.add(value)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
872
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
873
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
874 class repackentry(object):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
875 """Simple class representing a single revision entry in the repackledger.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
876 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
877
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
878 __slots__ = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
879 r'filename',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
880 r'node',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
881 r'datasource',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
882 r'historysource',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
883 r'datarepacked',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
884 r'historyrepacked',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
885 r'gced',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
886 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
887
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
888 def __init__(self, filename, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
889 self.filename = filename
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
890 self.node = node
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
891 # If the revision has a data entry in the source
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
892 self.datasource = False
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
893 # If the revision has a history entry in the source
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
894 self.historysource = False
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
895 # If the revision's data entry was repacked into the repack target
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
896 self.datarepacked = False
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
897 # If the revision's history entry was repacked into the repack target
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
898 self.historyrepacked = False
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
899 # If garbage collected
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
900 self.gced = False
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
901
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
902
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
903 def repacklockvfs(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
904 if util.safehasattr(repo, b'name'):
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
905 # Lock in the shared cache so repacks across multiple copies of the same
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
906 # repo are coordinated.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
907 sharedcachepath = shallowutil.getcachepackpath(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
908 repo, constants.FILEPACK_CATEGORY
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42943
diff changeset
909 )
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
910 return vfs.vfs(sharedcachepath)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
911 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
912 return repo.svfs