annotate mercurial/store.py @ 34968:3649c3f2cd90

revert: do not reverse hunks in interactive when REV is not parent (issue5096) And introduce a new "apply" operation verb for this case as suggested in issue5096. This replaces the no longer used "revert" operation. In interactive revert, when reverting to something else that the parent revision, display an "apply this change" message with a diff that is not reversed. The rationale is that `hg revert -i -r REV` will show hunks of the diff from the working directory to REV and prompt the user to select them for applying (to working directory). This contradicts dcc56e10c23b in which it was decided to have the "direction" of prompted hunks reversed. Later on [1], there was a broad consensus (but no decision) towards the "as to be applied direction". Now that --interactive is no longer experimental (5910db5d1913), it's time to switch and thus we drop no longer used "experimental.revertalternateinteractivemode" configuration option. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-November/090142.html .. feature:: When interactive revert is run against a revision other than the working directory parent, the diff shown is the diff to *apply* to the working directory, rather than the diff to *discard* from the working copy. This is in line with related user experiences with `git` and appears to be less confusing with `ui.interface=curses`.
author Denis Laxalde <denis.laxalde@logilab.fr>
date Fri, 03 Nov 2017 14:47:37 +0100
parents b4abc438a8c9
children 72b91f905065
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
1 # store.py - repository store handling for Mercurial
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
2 #
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
3 # Copyright 2008 Matt Mackall <mpm@selenic.com>
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8210
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9133
diff changeset
6 # GNU General Public License version 2 or any later version.
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
7
27480
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
8 from __future__ import absolute_import
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
9
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
10 import errno
29338
81c38cb9c1a1 store: use hashlib.sha1 directly instead of through util
Augie Fackler <raf@durin42.com>
parents: 29071
diff changeset
11 import hashlib
27480
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
12 import os
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
13 import stat
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
14
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
15 from .i18n import _
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
16 from . import (
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
17 error,
32372
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 31362
diff changeset
18 policy,
30077
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
19 pycompat,
27480
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
20 util,
31234
9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31219
diff changeset
21 vfs as vfsmod,
27480
509159675cdb store: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26778
diff changeset
22 )
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
23
32372
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 31362
diff changeset
24 parsers = policy.importmod(r'parsers')
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 31362
diff changeset
25
8531
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
26 # This avoids a collision between a file named foo and a dir named
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
27 # foo.i or foo.d
17607
cc58dc47cb5e store: use fast C implementation of encodedir() if it's available
Adrian Buehlmann <adrian@cadifra.com>
parents: 17605
diff changeset
28 def _encodedir(path):
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
29 '''
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
30 >>> _encodedir(b'data/foo.i')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
31 'data/foo.i'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
32 >>> _encodedir(b'data/foo.i/bla.i')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
33 'data/foo.i.hg/bla.i'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
34 >>> _encodedir(b'data/foo.i.hg/bla.i')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
35 'data/foo.i.hg.hg/bla.i'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
36 >>> _encodedir(b'data/foo.i\\ndata/foo.i/bla.i\\ndata/foo.i.hg/bla.i\\n')
17605
e9cc29be3305 store: add multiline doctest case for encodedir()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17604
diff changeset
37 'data/foo.i\\ndata/foo.i.hg/bla.i\\ndata/foo.i.hg.hg/bla.i\\n'
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
38 '''
8531
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
39 return (path
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
40 .replace(".hg/", ".hg.hg/")
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
41 .replace(".i/", ".i.hg/")
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
42 .replace(".d/", ".d.hg/"))
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
43
17607
cc58dc47cb5e store: use fast C implementation of encodedir() if it's available
Adrian Buehlmann <adrian@cadifra.com>
parents: 17605
diff changeset
44 encodedir = getattr(parsers, 'encodedir', _encodedir)
cc58dc47cb5e store: use fast C implementation of encodedir() if it's available
Adrian Buehlmann <adrian@cadifra.com>
parents: 17605
diff changeset
45
8531
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
46 def decodedir(path):
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
47 '''
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
48 >>> decodedir(b'data/foo.i')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
49 'data/foo.i'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
50 >>> decodedir(b'data/foo.i.hg/bla.i')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
51 'data/foo.i/bla.i'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
52 >>> decodedir(b'data/foo.i.hg.hg/bla.i')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
53 'data/foo.i.hg/bla.i'
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
54 '''
17586
2f1475da1940 store: remove uneeded startswith('data/') checks in encodedir() and decodedir()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17585
diff changeset
55 if ".hg/" not in path:
8531
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
56 return path
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
57 return (path
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
58 .replace(".d.hg/", ".d/")
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
59 .replace(".i.hg/", ".i/")
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
60 .replace(".hg.hg/", ".hg/"))
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
61
29071
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
62 def _reserved():
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
63 ''' characters that are problematic for filesystems
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
64
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
65 * ascii escapes (0..31)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
66 * ascii hi (126..255)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
67 * windows specials
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
68
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
69 these characters will be escaped by encodefunctions
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
70 '''
30076
400dfded8a29 py3: make the string unicode so its iterable in py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 29338
diff changeset
71 winreserved = [ord(x) for x in u'\\:*?"<>|']
29071
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
72 for x in range(32):
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
73 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
74 for x in range(126, 256):
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
75 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
76 for x in winreserved:
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
77 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
78
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
79 def _buildencodefun():
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
80 '''
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
81 >>> enc, dec = _buildencodefun()
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
82
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
83 >>> enc(b'nothing/special.txt')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
84 'nothing/special.txt'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
85 >>> dec(b'nothing/special.txt')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
86 'nothing/special.txt'
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
87
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
88 >>> enc(b'HELLO')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
89 '_h_e_l_l_o'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
90 >>> dec(b'_h_e_l_l_o')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
91 'HELLO'
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
92
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
93 >>> enc(b'hello:world?')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
94 'hello~3aworld~3f'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
95 >>> dec(b'hello~3aworld~3f')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
96 'hello:world?'
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
97
34136
414a3513c2bd doctest: do not embed non-ascii characters in docstring
Yuya Nishihara <yuya@tcha.org>
parents: 34131
diff changeset
98 >>> enc(b'the\\x07quick\\xADshot')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
99 'the~07quick~adshot'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
100 >>> dec(b'the~07quick~adshot')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
101 'the\\x07quick\\xadshot'
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
102 '''
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
103 e = '_'
31253
64596338ba10 py3: factor out bytechr() function
Yuya Nishihara <yuya@tcha.org>
parents: 31234
diff changeset
104 xchr = pycompat.bytechr
64596338ba10 py3: factor out bytechr() function
Yuya Nishihara <yuya@tcha.org>
parents: 31234
diff changeset
105 asciistr = list(map(xchr, range(127)))
30077
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
106 capitals = list(range(ord("A"), ord("Z") + 1))
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
107
30108
e1e7190457cf store: py26 compat, don't use a dict comprehension
Martijn Pieters <mjpieters@fb.com>
parents: 30077
diff changeset
108 cmap = dict((x, x) for x in asciistr)
29071
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
109 for x in _reserved():
30077
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
110 cmap[xchr(x)] = "~%02x" % x
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
111 for x in capitals + [ord(e)]:
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
112 cmap[xchr(x)] = e + xchr(x).lower()
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k
Mateusz Kwapich <mitrandir@fb.com>
parents: 30076
diff changeset
113
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
114 dmap = {}
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
115 for k, v in cmap.iteritems():
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
116 dmap[v] = k
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
117 def decode(s):
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
118 i = 0
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
119 while i < len(s):
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
120 for l in xrange(1, 4):
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
121 try:
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
122 yield dmap[s[i:i + l]]
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
123 i += l
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
124 break
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
125 except KeyError:
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
126 pass
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
127 else:
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
128 raise KeyError
31219
f8e06821c118 store: slice over a bytestring to get characters instead of ascii values
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30893
diff changeset
129 return (lambda s: ''.join([cmap[s[c:c + 1]] for c in xrange(len(s))]),
17608
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
130 lambda s: ''.join(list(decode(s))))
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
131
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
132 _encodefname, _decodefname = _buildencodefun()
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
133
17608
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
134 def encodefilename(s):
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
135 '''
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
136 >>> encodefilename(b'foo.i/bar.d/bla.hg/hi:world?/HELLO')
17608
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
137 'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o'
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
138 '''
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
139 return _encodefname(encodedir(s))
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
140
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
141 def decodefilename(s):
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
142 '''
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
143 >>> decodefilename(b'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o')
17608
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
144 'foo.i/bar.d/bla.hg/hi:world?/HELLO'
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
145 '''
776240123525 store: extract functions _encodefname and _decodefname
Adrian Buehlmann <adrian@cadifra.com>
parents: 17607
diff changeset
146 return decodedir(_decodefname(s))
6839
01db3e101362 move filename encoding functions from util.py to new store.py
Adrian Buehlmann <adrian@cadifra.com>
parents:
diff changeset
147
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
148 def _buildlowerencodefun():
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
149 '''
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
150 >>> f = _buildlowerencodefun()
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
151 >>> f(b'nothing/special.txt')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
152 'nothing/special.txt'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
153 >>> f(b'HELLO')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
154 'hello'
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
155 >>> f(b'hello:world?')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
156 'hello~3aworld~3f'
34136
414a3513c2bd doctest: do not embed non-ascii characters in docstring
Yuya Nishihara <yuya@tcha.org>
parents: 34131
diff changeset
157 >>> f(b'the\\x07quick\\xADshot')
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
158 'the~07quick~adshot'
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
159 '''
34214
7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun()
Yuya Nishihara <yuya@tcha.org>
parents: 34213
diff changeset
160 xchr = pycompat.bytechr
7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun()
Yuya Nishihara <yuya@tcha.org>
parents: 34213
diff changeset
161 cmap = dict([(xchr(x), xchr(x)) for x in xrange(127)])
29071
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat
timeless <timeless@mozdev.org>
parents: 28007
diff changeset
162 for x in _reserved():
34214
7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun()
Yuya Nishihara <yuya@tcha.org>
parents: 34213
diff changeset
163 cmap[xchr(x)] = "~%02x" % x
18054
b35e3364f94a check-code: there must also be whitespace between ')' and operator
Mads Kiilerich <madski@unity3d.com>
parents: 17845
diff changeset
164 for x in range(ord("A"), ord("Z") + 1):
34214
7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun()
Yuya Nishihara <yuya@tcha.org>
parents: 34213
diff changeset
165 cmap[xchr(x)] = xchr(x).lower()
34213
96808804b68f store: give name to lowerencode function
Yuya Nishihara <yuya@tcha.org>
parents: 34136
diff changeset
166 def lowerencode(s):
34215
b4abc438a8c9 py3: iterate bytes as a byte string in store.lowerencode()
Yuya Nishihara <yuya@tcha.org>
parents: 34214
diff changeset
167 return "".join([cmap[c] for c in pycompat.iterbytestr(s)])
34213
96808804b68f store: give name to lowerencode function
Yuya Nishihara <yuya@tcha.org>
parents: 34136
diff changeset
168 return lowerencode
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
169
18430
0459c6555f69 store: implement lowerencode in C
Bryan O'Sullivan <bryano@fb.com>
parents: 18054
diff changeset
170 lowerencode = getattr(parsers, 'lowerencode', None) or _buildlowerencodefun()
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
171
17570
f53a7b256ca6 store: optimze _auxencode() a bit by grouping the reserved names by length
Adrian Buehlmann <adrian@cadifra.com>
parents: 17569
diff changeset
172 # Windows reserved names: con, prn, aux, nul, com1..com9, lpt1..lpt9
f53a7b256ca6 store: optimze _auxencode() a bit by grouping the reserved names by length
Adrian Buehlmann <adrian@cadifra.com>
parents: 17569
diff changeset
173 _winres3 = ('aux', 'con', 'prn', 'nul') # length 3
f53a7b256ca6 store: optimze _auxencode() a bit by grouping the reserved names by length
Adrian Buehlmann <adrian@cadifra.com>
parents: 17569
diff changeset
174 _winres4 = ('com', 'lpt') # length 4 (with trailing 1..9)
12687
34d8247a4595 store: encode first period or space in filenames (issue1713)
Adrian Buehlmann <adrian@cadifra.com>
parents: 12171
diff changeset
175 def _auxencode(path, dotencode):
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
176 '''
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
177 Encodes filenames containing names reserved by Windows or which end in
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
178 period or space. Does not touch other single reserved characters c.
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
179 Specifically, c in '\\:*?"<>|' or ord(c) <= 31 are *not* encoded here.
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
180 Additionally encodes space or period at the beginning, if dotencode is
17569
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17568
diff changeset
181 True. Parameter path is assumed to be all lowercase.
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17568
diff changeset
182 A segment only needs encoding if a reserved name appears as a
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17568
diff changeset
183 basename (e.g. "aux", "aux.foo"). A directory or file named "foo.aux"
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17568
diff changeset
184 doesn't need encoding.
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
185
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
186 >>> s = b'.foo/aux.txt/txt.aux/con/prn/nul/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
187 >>> _auxencode(s.split(b'/'), True)
17574
81a033bb29bc store: let _auxencode() return the list of path segments
Adrian Buehlmann <adrian@cadifra.com>
parents: 17573
diff changeset
188 ['~2efoo', 'au~78.txt', 'txt.aux', 'co~6e', 'pr~6e', 'nu~6c', 'foo~2e']
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
189 >>> s = b'.com1com2/lpt9.lpt4.lpt1/conprn/com0/lpt0/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
190 >>> _auxencode(s.split(b'/'), False)
17574
81a033bb29bc store: let _auxencode() return the list of path segments
Adrian Buehlmann <adrian@cadifra.com>
parents: 17573
diff changeset
191 ['.com1com2', 'lp~749.lpt4.lpt1', 'conprn', 'com0', 'lpt0', 'foo~2e']
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
192 >>> _auxencode([b'foo. '], True)
17574
81a033bb29bc store: let _auxencode() return the list of path segments
Adrian Buehlmann <adrian@cadifra.com>
parents: 17573
diff changeset
193 ['foo.~20']
34131
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3
Yuya Nishihara <yuya@tcha.org>
parents: 33412
diff changeset
194 >>> _auxencode([b' .foo'], True)
17574
81a033bb29bc store: let _auxencode() return the list of path segments
Adrian Buehlmann <adrian@cadifra.com>
parents: 17573
diff changeset
195 ['~20.foo']
13949
ba43aa1e173c store: add some doctests
Adrian Buehlmann <adrian@cadifra.com>
parents: 13426
diff changeset
196 '''
17589
b11024849db6 store: parameter path of _auxencode is now a list of strings
Adrian Buehlmann <adrian@cadifra.com>
parents: 17588
diff changeset
197 for i, n in enumerate(path):
17572
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
198 if not n:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
199 continue
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
200 if dotencode and n[0] in '. ':
31362
50cd81346ad4 store: fix many single-byte ops to use slicing in _auxencode
Augie Fackler <augie@google.com>
parents: 31253
diff changeset
201 n = "~%02x" % ord(n[0:1]) + n[1:]
17589
b11024849db6 store: parameter path of _auxencode is now a list of strings
Adrian Buehlmann <adrian@cadifra.com>
parents: 17588
diff changeset
202 path[i] = n
17572
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
203 else:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
204 l = n.find('.')
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
205 if l == -1:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
206 l = len(n)
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
207 if ((l == 3 and n[:3] in _winres3) or
31362
50cd81346ad4 store: fix many single-byte ops to use slicing in _auxencode
Augie Fackler <augie@google.com>
parents: 31253
diff changeset
208 (l == 4 and n[3:4] <= '9' and n[3:4] >= '1'
17572
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
209 and n[:3] in _winres4)):
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
210 # encode third letter ('aux' -> 'au~78')
31362
50cd81346ad4 store: fix many single-byte ops to use slicing in _auxencode
Augie Fackler <augie@google.com>
parents: 31253
diff changeset
211 ec = "~%02x" % ord(n[2:3])
17572
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
212 n = n[0:2] + ec + n[3:]
17589
b11024849db6 store: parameter path of _auxencode is now a list of strings
Adrian Buehlmann <adrian@cadifra.com>
parents: 17588
diff changeset
213 path[i] = n
17572
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
214 if n[-1] in '. ':
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode()
Adrian Buehlmann <adrian@cadifra.com>
parents: 17571
diff changeset
215 # encode last period or space ('foo...' -> 'foo..~2e')
31362
50cd81346ad4 store: fix many single-byte ops to use slicing in _auxencode
Augie Fackler <augie@google.com>
parents: 31253
diff changeset
216 path[i] = n[:-1] + "~%02x" % ord(n[-1:])
17589
b11024849db6 store: parameter path of _auxencode is now a list of strings
Adrian Buehlmann <adrian@cadifra.com>
parents: 17588
diff changeset
217 return path
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
218
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
219 _maxstorepathlen = 120
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
220 _dirprefixlen = 8
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
221 _maxshortdirslen = 8 * (_dirprefixlen + 1) - 4
17610
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
222
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
223 def _hashencode(path, dotencode):
29338
81c38cb9c1a1 store: use hashlib.sha1 directly instead of through util
Augie Fackler <raf@durin42.com>
parents: 29071
diff changeset
224 digest = hashlib.sha1(path).hexdigest()
25091
b5052fc73300 treemanifest: store submanifest revlog per directory
Martin von Zweigbergk <martinvonz@google.com>
parents: 25062
diff changeset
225 le = lowerencode(path[5:]).split('/') # skips prefix 'data/' or 'meta/'
17610
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
226 parts = _auxencode(le, dotencode)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
227 basename = parts[-1]
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
228 _root, ext = os.path.splitext(basename)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
229 sdirs = []
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
230 sdirslen = 0
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
231 for p in parts[:-1]:
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
232 d = p[:_dirprefixlen]
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
233 if d[-1] in '. ':
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
234 # Windows can't access dirs ending in period or space
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
235 d = d[:-1] + '_'
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
236 if sdirslen == 0:
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
237 t = len(d)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
238 else:
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
239 t = sdirslen + 1 + len(d)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
240 if t > _maxshortdirslen:
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
241 break
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
242 sdirs.append(d)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
243 sdirslen = t
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
244 dirs = '/'.join(sdirs)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
245 if len(dirs) > 0:
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
246 dirs += '/'
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
247 res = 'dh/' + dirs + digest + ext
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
248 spaceleft = _maxstorepathlen - len(res)
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
249 if spaceleft > 0:
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
250 filler = basename[:spaceleft]
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
251 res = 'dh/' + dirs + filler + digest + ext
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
252 return res
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
253
17590
eb0884680f5c store: eliminate one level of lambda functions on _hybridencode
Adrian Buehlmann <adrian@cadifra.com>
parents: 17589
diff changeset
254 def _hybridencode(path, dotencode):
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
255 '''encodes path with a length limit
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
256
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
257 Encodes all paths that begin with 'data/', according to the following.
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
258
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
259 Default encoding (reversible):
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
260
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
261 Encodes all uppercase letters 'X' as '_x'. All reserved or illegal
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
262 characters are encoded as '~xx', where xx is the two digit hex code
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
263 of the character (see encodefilename).
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
264 Relevant path components consisting of Windows reserved filenames are
17738
b8424c92ba2b spelling: fix minor spell checker issues
Mads Kiilerich <mads@kiilerich.com>
parents: 17731
diff changeset
265 masked by encoding the third character ('aux' -> 'au~78', see _auxencode).
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
266
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
267 Hashed encoding (not reversible):
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
268
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
269 If the default-encoded path is longer than _maxstorepathlen, a
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
270 non-reversible hybrid hashing of the path is done instead.
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
271 This encoding uses up to _dirprefixlen characters of all directory
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
272 levels of the lowerencoded path, but not more levels than can fit into
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
273 _maxshortdirslen.
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
274 Then follows the filler followed by the sha digest of the full path.
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
275 The filler is the beginning of the basename of the lowerencoded path
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
276 (the basename is everything after the last path separator). The filler
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
277 is as long as possible, filling in characters from the basename until
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
278 the encoded path has _maxstorepathlen characters (or all chars of the
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
279 basename have been taken).
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
280 The extension (e.g. '.i' or '.d') is preserved.
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
281
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
282 The string 'data/' at the beginning is replaced with 'dh/', if the hashed
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
283 encoding was used.
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
284 '''
17609
cbc180cfd60b store: reuse direncoded path in _hybridencode
Adrian Buehlmann <adrian@cadifra.com>
parents: 17608
diff changeset
285 path = encodedir(path)
cbc180cfd60b store: reuse direncoded path in _hybridencode
Adrian Buehlmann <adrian@cadifra.com>
parents: 17608
diff changeset
286 ef = _encodefname(path).split('/')
17590
eb0884680f5c store: eliminate one level of lambda functions on _hybridencode
Adrian Buehlmann <adrian@cadifra.com>
parents: 17589
diff changeset
287 res = '/'.join(_auxencode(ef, dotencode))
14288
00a0ab08f986 store: change names to comply with project coding standards
Adrian Buehlmann <adrian@cadifra.com>
parents: 14194
diff changeset
288 if len(res) > _maxstorepathlen:
17610
d0afa149e059 store: refactor hashed encoding into its own function
Bryan O'Sullivan <bryano@fb.com>
parents: 17609
diff changeset
289 res = _hashencode(path, dotencode)
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
290 return res
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
291
17624
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
292 def _pathencode(path):
18435
8c019d2fd7c0 store: switch to C-based hashed path encoding
Bryan O'Sullivan <bryano@fb.com>
parents: 18430
diff changeset
293 de = encodedir(path)
17693
0c6de45e1212 store: optimize _pathencode by checking the length of the unencoded path
Adrian Buehlmann <adrian@cadifra.com>
parents: 17653
diff changeset
294 if len(path) > _maxstorepathlen:
18435
8c019d2fd7c0 store: switch to C-based hashed path encoding
Bryan O'Sullivan <bryano@fb.com>
parents: 18430
diff changeset
295 return _hashencode(de, True)
8c019d2fd7c0 store: switch to C-based hashed path encoding
Bryan O'Sullivan <bryano@fb.com>
parents: 18430
diff changeset
296 ef = _encodefname(de).split('/')
17624
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
297 res = '/'.join(_auxencode(ef, True))
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
298 if len(res) > _maxstorepathlen:
18435
8c019d2fd7c0 store: switch to C-based hashed path encoding
Bryan O'Sullivan <bryano@fb.com>
parents: 18430
diff changeset
299 return _hashencode(de, True)
17624
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
300 return res
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
301
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
302 _pathencode = getattr(parsers, 'pathencode', _pathencode)
ae103510f6aa store: add a fallback _pathencode Python function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17623
diff changeset
303
17623
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file
Adrian Buehlmann <adrian@cadifra.com>
parents: 17621
diff changeset
304 def _plainhybridencode(f):
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file
Adrian Buehlmann <adrian@cadifra.com>
parents: 17621
diff changeset
305 return _hybridencode(f, False)
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file
Adrian Buehlmann <adrian@cadifra.com>
parents: 17621
diff changeset
306
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
307 def _calcmode(vfs):
6898
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
308 try:
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
309 # files in .hg/ will be created using this mode
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
310 mode = vfs.stat().st_mode
6898
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
311 # avoid some useless chmods
25658
e93036747902 global: mass rewrite to use modern octal syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25091
diff changeset
312 if (0o777 & ~util.umask) == (0o777 & mode):
6898
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
313 mode = None
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
314 except OSError:
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
315 mode = None
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
316 return mode
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
317
28006
bb45190a54b7 treemanifests: fix local clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 27480
diff changeset
318 _data = ('data meta 00manifest.d 00manifest.i 00changelog.d 00changelog.i'
17249
7d4747c711a9 clone: copy obsolete markers during local clone
Pierre-Yves.David@ens-lyon.org
parents: 17154
diff changeset
319 ' phaseroots obsstore')
6903
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
320
8778
c5f36402daad use new style classes
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8633
diff changeset
321 class basicstore(object):
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
322 '''base class for local repository stores'''
17651
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
323 def __init__(self, path, vfstype):
17724
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17722
diff changeset
324 vfs = vfstype(path)
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17722
diff changeset
325 self.path = vfs.base
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
326 self.createmode = _calcmode(vfs)
17652
2c6f7231becc store: rename "op" variables to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17651
diff changeset
327 vfs.createmode = self.createmode
17728
004bd533880d store: invoke "os.path.isdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17727
diff changeset
328 self.rawvfs = vfs
31234
9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31219
diff changeset
329 self.vfs = vfsmod.filtervfs(vfs, encodedir)
17653
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17652
diff changeset
330 self.opener = self.vfs
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
331
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
332 def join(self, f):
13426
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
333 return self.path + '/' + encodedir(f)
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
334
6899
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
335 def _walk(self, relpath, recurse):
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
336 '''yields (unencoded, encoded, size)'''
13426
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
337 path = self.path
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
338 if relpath:
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
339 path += '/' + relpath
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
340 striplen = len(self.path) + 1
6899
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
341 l = []
17728
004bd533880d store: invoke "os.path.isdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17727
diff changeset
342 if self.rawvfs.isdir(path):
6899
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
343 visit = [path]
17747
aad3bce98f76 store: invoke "osutil.listdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17745
diff changeset
344 readdir = self.rawvfs.readdir
6899
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
345 while visit:
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
346 p = visit.pop()
17747
aad3bce98f76 store: invoke "osutil.listdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17745
diff changeset
347 for f, kind, st in readdir(p, stat=True):
13426
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
348 fp = p + '/' + f
6899
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
349 if kind == stat.S_IFREG and f[-2:] in ('.d', '.i'):
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
350 n = util.pconvert(fp[striplen:])
8531
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
351 l.append((decodedir(n), n, st.st_size))
6899
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
352 elif kind == stat.S_IFDIR and recurse:
56a7a54e074f store: simplify walking
Matt Mackall <mpm@selenic.com>
parents: 6898
diff changeset
353 visit.append(fp)
17054
125ff5654b72 store: sort filenames in place
Bryan O'Sullivan <bryano@fb.com>
parents: 16404
diff changeset
354 l.sort()
125ff5654b72 store: sort filenames in place
Bryan O'Sullivan <bryano@fb.com>
parents: 16404
diff changeset
355 return l
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
356
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
357 def datafiles(self):
28007
fb92927f9775 treemanifests: fix streaming clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 28006
diff changeset
358 return self._walk('data', True) + self._walk('meta', True)
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
359
19177
1e104aaa4c44 store: move top file walk to a separate function
Durham Goode <durham@fb.com>
parents: 18435
diff changeset
360 def topfiles(self):
1e104aaa4c44 store: move top file walk to a separate function
Durham Goode <durham@fb.com>
parents: 18435
diff changeset
361 # yield manifest before changelog
1e104aaa4c44 store: move top file walk to a separate function
Durham Goode <durham@fb.com>
parents: 18435
diff changeset
362 return reversed(self._walk('', False))
1e104aaa4c44 store: move top file walk to a separate function
Durham Goode <durham@fb.com>
parents: 18435
diff changeset
363
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
364 def walk(self):
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
365 '''yields (unencoded, encoded, size)'''
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
366 # yield data files first
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6890
diff changeset
367 for x in self.datafiles():
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
368 yield x
19177
1e104aaa4c44 store: move top file walk to a separate function
Durham Goode <durham@fb.com>
parents: 18435
diff changeset
369 for x in self.topfiles():
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
370 yield x
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
371
6903
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
372 def copylist(self):
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
373 return ['requires'] + _data.split()
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
374
20883
cd443c7589cc fncache: move fncache writing to be in a transaction
Durham Goode <durham@fb.com>
parents: 20879
diff changeset
375 def write(self, tr):
13391
d00bbff8600e fncachestore: defer updating the fncache file to a single file open
Adrian Buehlmann <adrian@cadifra.com>
parents: 13169
diff changeset
376 pass
d00bbff8600e fncachestore: defer updating the fncache file to a single file open
Adrian Buehlmann <adrian@cadifra.com>
parents: 13169
diff changeset
377
20884
2efdd186925d caches: invalidate store caches when lock is taken
Durham Goode <durham@fb.com>
parents: 20883
diff changeset
378 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken
Durham Goode <durham@fb.com>
parents: 20883
diff changeset
379 pass
2efdd186925d caches: invalidate store caches when lock is taken
Durham Goode <durham@fb.com>
parents: 20883
diff changeset
380
20885
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
381 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
382 pass
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
383
17744
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
384 def __contains__(self, path):
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
385 '''Checks if the store contains path'''
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
386 path = "/".join(("data", path))
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
387 # file?
19903
ca875b271ac3 store: use "vfs.exists()" instead of "os.path.exists()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19177
diff changeset
388 if self.vfs.exists(path + ".i"):
17744
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
389 return True
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
390 # dir?
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
391 if not path.endswith("/"):
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
392 path = path + "/"
19903
ca875b271ac3 store: use "vfs.exists()" instead of "os.path.exists()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19177
diff changeset
393 return self.vfs.exists(path)
17744
09d5b2055295 store: add a contains method to basicstore
smuralid
parents: 17738
diff changeset
394
6898
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
395 class encodedstore(basicstore):
17651
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
396 def __init__(self, path, vfstype):
17724
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17722
diff changeset
397 vfs = vfstype(path + '/store')
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17722
diff changeset
398 self.path = vfs.base
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
399 self.createmode = _calcmode(vfs)
17652
2c6f7231becc store: rename "op" variables to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17651
diff changeset
400 vfs.createmode = self.createmode
17728
004bd533880d store: invoke "os.path.isdir()" via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17727
diff changeset
401 self.rawvfs = vfs
31234
9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31219
diff changeset
402 self.vfs = vfsmod.filtervfs(vfs, encodefilename)
17653
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17652
diff changeset
403 self.opener = self.vfs
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
404
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
405 def datafiles(self):
28007
fb92927f9775 treemanifests: fix streaming clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 28006
diff changeset
406 for a, b, size in super(encodedstore, self).datafiles():
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6890
diff changeset
407 try:
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
408 a = decodefilename(a)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6890
diff changeset
409 except KeyError:
6900
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
410 a = None
def492d1b592 store: change handling of decoding errors
Matt Mackall <mpm@selenic.com>
parents: 6899
diff changeset
411 yield a, b, size
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
412
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
413 def join(self, f):
13426
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
414 return self.path + '/' + encodefilename(f)
6840
80e51429cb9a introduce store classes
Adrian Buehlmann <adrian@cadifra.com>
parents: 6839
diff changeset
415
6903
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
416 def copylist(self):
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
417 return (['requires', '00changelog.i'] +
13426
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
418 ['store/' + f for f in _data.split()])
6903
0642d9d7ec80 clone: get a list of files to clone from store
Matt Mackall <mpm@selenic.com>
parents: 6902
diff changeset
419
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
420 class fncache(object):
8531
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
421 # the filename used to be partially encoded
810387f59696 filelog encoding: move the encoding/decoding into store
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8530
diff changeset
422 # hence the encodedir/decodedir dance
17722
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17721
diff changeset
423 def __init__(self, vfs):
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17721
diff changeset
424 self.vfs = vfs
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
425 self.entries = None
13391
d00bbff8600e fncachestore: defer updating the fncache file to a single file open
Adrian Buehlmann <adrian@cadifra.com>
parents: 13169
diff changeset
426 self._dirty = False
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
427
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
428 def _load(self):
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
429 '''fill the entries from the fncache file'''
13391
d00bbff8600e fncachestore: defer updating the fncache file to a single file open
Adrian Buehlmann <adrian@cadifra.com>
parents: 13169
diff changeset
430 self._dirty = False
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
431 try:
17722
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17721
diff changeset
432 fp = self.vfs('fncache', mode='rb')
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
433 except IOError:
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
434 # skip nonexistent file
16404
9fca5b056c0a store: speed up read and write of large fncache files
Bryan O'Sullivan <bryano@fb.com>
parents: 15742
diff changeset
435 self.entries = set()
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
436 return
17604
bf9894889d82 store: optimize fncache._load a bit by dirdecoding the contents in one go
Adrian Buehlmann <adrian@cadifra.com>
parents: 17592
diff changeset
437 self.entries = set(decodedir(fp.read()).splitlines())
16404
9fca5b056c0a store: speed up read and write of large fncache files
Bryan O'Sullivan <bryano@fb.com>
parents: 15742
diff changeset
438 if '' in self.entries:
9fca5b056c0a store: speed up read and write of large fncache files
Bryan O'Sullivan <bryano@fb.com>
parents: 15742
diff changeset
439 fp.seek(0)
30398
b63bef41a7b6 store: migrate to util.iterfile
Jun Wu <quark@fb.com>
parents: 30108
diff changeset
440 for n, line in enumerate(util.iterfile(fp)):
16404
9fca5b056c0a store: speed up read and write of large fncache files
Bryan O'Sullivan <bryano@fb.com>
parents: 15742
diff changeset
441 if not line.rstrip('\n'):
26778
a95c975f42e3 l10n: use %d instead of %s for numbers
timeless@mozdev.org
parents: 26587
diff changeset
442 t = _('invalid entry in fncache, line %d') % (n + 1)
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
443 raise error.Abort(t)
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
444 fp.close()
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
445
20883
cd443c7589cc fncache: move fncache writing to be in a transaction
Durham Goode <durham@fb.com>
parents: 20879
diff changeset
446 def write(self, tr):
16404
9fca5b056c0a store: speed up read and write of large fncache files
Bryan O'Sullivan <bryano@fb.com>
parents: 15742
diff changeset
447 if self._dirty:
20883
cd443c7589cc fncache: move fncache writing to be in a transaction
Durham Goode <durham@fb.com>
parents: 20879
diff changeset
448 tr.addbackup('fncache')
20879
cd03854a2e06 fncache: remove the rewriting logic
Durham Goode <durham@fb.com>
parents: 19903
diff changeset
449 fp = self.vfs('fncache', mode='wb', atomictemp=True)
cd03854a2e06 fncache: remove the rewriting logic
Durham Goode <durham@fb.com>
parents: 19903
diff changeset
450 if self.entries:
cd03854a2e06 fncache: remove the rewriting logic
Durham Goode <durham@fb.com>
parents: 19903
diff changeset
451 fp.write(encodedir('\n'.join(self.entries) + '\n'))
cd03854a2e06 fncache: remove the rewriting logic
Durham Goode <durham@fb.com>
parents: 19903
diff changeset
452 fp.close()
cd03854a2e06 fncache: remove the rewriting logic
Durham Goode <durham@fb.com>
parents: 19903
diff changeset
453 self._dirty = False
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
454
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
455 def add(self, fn):
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
456 if self.entries is None:
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
457 self._load()
10577
d5bd1beff794 store: only add new entries to the fncache file
Adrian Buehlmann <adrian@cadifra.com>
parents: 10339
diff changeset
458 if fn not in self.entries:
13391
d00bbff8600e fncachestore: defer updating the fncache file to a single file open
Adrian Buehlmann <adrian@cadifra.com>
parents: 13169
diff changeset
459 self._dirty = True
10577
d5bd1beff794 store: only add new entries to the fncache file
Adrian Buehlmann <adrian@cadifra.com>
parents: 10339
diff changeset
460 self.entries.add(fn)
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
461
20885
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
462 def remove(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
463 if self.entries is None:
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
464 self._load()
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
465 try:
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
466 self.entries.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
467 self._dirty = True
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
468 except KeyError:
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
469 pass
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
470
17782
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
471 def __contains__(self, fn):
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
472 if self.entries is None:
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
473 self._load()
17782
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
474 return fn in self.entries
8530
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
475
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
476 def __iter__(self):
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
477 if self.entries is None:
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
478 self._load()
03196ac9a8b9 store: refactor the fncache handling
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8480
diff changeset
479 return iter(self.entries)
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
480
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 32372
diff changeset
481 class _fncachevfs(vfsmod.abstractvfs, vfsmod.proxyvfs):
17721
cf236e3501c3 store: rename argument name from "op"(ener) to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17693
diff changeset
482 def __init__(self, vfs, fnc, encode):
33412
a42369e04aee vfs: rename auditvfs to proxyvfs
Yuya Nishihara <yuya@tcha.org>
parents: 32372
diff changeset
483 vfsmod.proxyvfs.__init__(self, vfs)
14194
3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a
Adrian Buehlmann <adrian@cadifra.com>
parents: 14166
diff changeset
484 self.fncache = fnc
3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a
Adrian Buehlmann <adrian@cadifra.com>
parents: 14166
diff changeset
485 self.encode = encode
3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a
Adrian Buehlmann <adrian@cadifra.com>
parents: 14166
diff changeset
486
3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a
Adrian Buehlmann <adrian@cadifra.com>
parents: 14166
diff changeset
487 def __call__(self, path, mode='r', *args, **kw):
28007
fb92927f9775 treemanifests: fix streaming clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 28006
diff changeset
488 if mode not in ('r', 'rb') and (path.startswith('data/') or
fb92927f9775 treemanifests: fix streaming clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 28006
diff changeset
489 path.startswith('meta/')):
14194
3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a
Adrian Buehlmann <adrian@cadifra.com>
parents: 14166
diff changeset
490 self.fncache.add(path)
17722
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17721
diff changeset
491 return self.vfs(self.encode(path), mode, *args, **kw)
14194
3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a
Adrian Buehlmann <adrian@cadifra.com>
parents: 14166
diff changeset
492
17725
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17724
diff changeset
493 def join(self, path):
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17724
diff changeset
494 if path:
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17724
diff changeset
495 return self.vfs.join(self.encode(path))
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17724
diff changeset
496 else:
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17724
diff changeset
497 return self.vfs.join(path)
ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17724
diff changeset
498
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
499 class fncachestore(basicstore):
17651
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
500 def __init__(self, path, vfstype, dotencode):
17591
9a5c2ecd1158 store: move encode lambda logic into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17590
diff changeset
501 if dotencode:
18435
8c019d2fd7c0 store: switch to C-based hashed path encoding
Bryan O'Sullivan <bryano@fb.com>
parents: 18430
diff changeset
502 encode = _pathencode
17591
9a5c2ecd1158 store: move encode lambda logic into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17590
diff changeset
503 else:
9a5c2ecd1158 store: move encode lambda logic into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17590
diff changeset
504 encode = _plainhybridencode
12687
34d8247a4595 store: encode first period or space in filenames (issue1713)
Adrian Buehlmann <adrian@cadifra.com>
parents: 12171
diff changeset
505 self.encode = encode
17724
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17722
diff changeset
506 vfs = vfstype(path + '/store')
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17722
diff changeset
507 self.path = vfs.base
17562
b42b0729744d store: reduce string concatenation when joining
Bryan O'Sullivan <bryano@fb.com>
parents: 17555
diff changeset
508 self.pathsep = self.path + '/'
17726
7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17725
diff changeset
509 self.createmode = _calcmode(vfs)
17652
2c6f7231becc store: rename "op" variables to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17651
diff changeset
510 vfs.createmode = self.createmode
17727
6492b39a44d5 store: replace invocation of "getsize()" by "vfs.stat()"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17726
diff changeset
511 self.rawvfs = vfs
17652
2c6f7231becc store: rename "op" variables to "vfs"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17651
diff changeset
512 fnc = fncache(vfs)
9133
996c1cd8f530 store: eliminate reference cycle in fncachestore
Simon Heimberg <simohe@besonet.ch>
parents: 8778
diff changeset
513 self.fncache = fnc
17653
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17652
diff changeset
514 self.vfs = _fncachevfs(vfs, fnc, encode)
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17652
diff changeset
515 self.opener = self.vfs
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
516
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
517 def join(self, f):
17562
b42b0729744d store: reduce string concatenation when joining
Bryan O'Sullivan <bryano@fb.com>
parents: 17555
diff changeset
518 return self.pathsep + self.encode(f)
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
519
17731
c85dbae29684 store: restore getsize method
Matt Mackall <mpm@selenic.com>
parents: 17728
diff changeset
520 def getsize(self, path):
c85dbae29684 store: restore getsize method
Matt Mackall <mpm@selenic.com>
parents: 17728
diff changeset
521 return self.rawvfs.stat(path).st_size
c85dbae29684 store: restore getsize method
Matt Mackall <mpm@selenic.com>
parents: 17728
diff changeset
522
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
523 def datafiles(self):
17373
4cbb1137941d store: sort the results of fncachestore.datafiles()
Bryan O'Sullivan <bryano@fb.com>
parents: 17249
diff changeset
524 for f in sorted(self.fncache):
12687
34d8247a4595 store: encode first period or space in filenames (issue1713)
Adrian Buehlmann <adrian@cadifra.com>
parents: 12171
diff changeset
525 ef = self.encode(f)
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
526 try:
17731
c85dbae29684 store: restore getsize method
Matt Mackall <mpm@selenic.com>
parents: 17728
diff changeset
527 yield f, ef, self.getsize(ef)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25658
diff changeset
528 except OSError as err:
17374
0cec762790ed store: only one kind of OSError means "nonexistent entry"
Bryan O'Sullivan <bryano@fb.com>
parents: 17373
diff changeset
529 if err.errno != errno.ENOENT:
0cec762790ed store: only one kind of OSError means "nonexistent entry"
Bryan O'Sullivan <bryano@fb.com>
parents: 17373
diff changeset
530 raise
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
531
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
532 def copylist(self):
28006
bb45190a54b7 treemanifests: fix local clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 27480
diff changeset
533 d = ('data meta dh fncache phaseroots obsstore'
15742
65df60a3f96b phases: prevent rebase to rebase immutable changeset.
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 15740
diff changeset
534 ' 00manifest.d 00manifest.i 00changelog.d 00changelog.i')
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
535 return (['requires', '00changelog.i'] +
13426
643b8212813e store: remove pointless pathjoiner parameter
Adrian Buehlmann <adrian@cadifra.com>
parents: 13391
diff changeset
536 ['store/' + f for f in d.split()])
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
537
20883
cd443c7589cc fncache: move fncache writing to be in a transaction
Durham Goode <durham@fb.com>
parents: 20879
diff changeset
538 def write(self, tr):
cd443c7589cc fncache: move fncache writing to be in a transaction
Durham Goode <durham@fb.com>
parents: 20879
diff changeset
539 self.fncache.write(tr)
13391
d00bbff8600e fncachestore: defer updating the fncache file to a single file open
Adrian Buehlmann <adrian@cadifra.com>
parents: 13169
diff changeset
540
20884
2efdd186925d caches: invalidate store caches when lock is taken
Durham Goode <durham@fb.com>
parents: 20883
diff changeset
541 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken
Durham Goode <durham@fb.com>
parents: 20883
diff changeset
542 self.fncache.entries = None
2efdd186925d caches: invalidate store caches when lock is taken
Durham Goode <durham@fb.com>
parents: 20883
diff changeset
543
20885
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
544 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
545 self.fncache.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips
Durham Goode <durham@fb.com>
parents: 20884
diff changeset
546
17783
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
547 def _exists(self, f):
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
548 ef = self.encode(f)
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
549 try:
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
550 self.getsize(ef)
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
551 return True
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25658
diff changeset
552 except OSError as err:
17783
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
553 if err.errno != errno.ENOENT:
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
554 raise
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
555 # nonexistent entry
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
556 return False
df55ce6854c3 store: add new _exists helper function on fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17782
diff changeset
557
17745
b9a56b816ff2 store: add a contains method to fncachestore
smuralid
parents: 17744
diff changeset
558 def __contains__(self, path):
b9a56b816ff2 store: add a contains method to fncachestore
smuralid
parents: 17744
diff changeset
559 '''Checks if the store contains path'''
b9a56b816ff2 store: add a contains method to fncachestore
smuralid
parents: 17744
diff changeset
560 path = "/".join(("data", path))
17782
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
561 # check for files (exact match)
17784
73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2)
Adrian Buehlmann <adrian@cadifra.com>
parents: 17783
diff changeset
562 e = path + '.i'
73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2)
Adrian Buehlmann <adrian@cadifra.com>
parents: 17783
diff changeset
563 if e in self.fncache and self._exists(e):
17782
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
564 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
565 # now check for directories (prefix match)
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
566 if not path.endswith('/'):
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
567 path += '/'
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
568 for e in self.fncache:
17784
73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2)
Adrian Buehlmann <adrian@cadifra.com>
parents: 17783
diff changeset
569 if e.startswith(path) and self._exists(e):
17782
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
570 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore
Adrian Buehlmann <adrian@cadifra.com>
parents: 17747
diff changeset
571 return False
17745
b9a56b816ff2 store: add a contains method to fncachestore
smuralid
parents: 17744
diff changeset
572
17651
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
573 def store(requirements, path, vfstype):
6898
69aeaaaf6e07 store: simplify class hierarchy
Matt Mackall <mpm@selenic.com>
parents: 6897
diff changeset
574 if 'store' in requirements:
7229
7946503ec76e introduce fncache repository layout
Adrian Buehlmann <adrian@cadifra.com>
parents: 6989
diff changeset
575 if 'fncache' in requirements:
17651
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
576 return fncachestore(path, vfstype, 'dotencode' in requirements)
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
577 return encodedstore(path, vfstype)
3b49c28658f6 store: rename "openertype" argument to "vfstype"
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 17649
diff changeset
578 return basicstore(path, vfstype)