Mercurial > hg
annotate mercurial/obsolete.py @ 39627:ee7ee0c516ca
narrow: intersect provided matcher with narrowmatcher in `hg diff`
This provides significant speedups when running diff, and no change in behavior
that I'm aware of (or that the tests found). I tested with a repo that I started
using narrow in after it was created and attempted to run `hg diff -c .` and
similar commands in it on a commit that had files not in the narrowspec.
Timing numbers below, using a similar setup as my previous commits.
before=9db85644, m-u is mozilla-unified at eb39298e432d (flatmanifest) and
0553b7f29eaf (treemanifest). l-d-r is a repo simulating a situation I've
encountered where there's one directory with 30k+ subdirectories. N means
narrow, T means treemanifest. The narrowspec is pretty small when in use, and
importantly the narrowspec is applied *after* doing the initial checkout
(without narrowing), so all of these files exist in the filesystem, which is not
normally the case if someone has been using narrow for the entire life of the
clone.
Anything less than a 5% difference in performance is most likely noise.
diff --git:
repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
------+---+---+------------------------+-----------------------+------------
m-u | | | 1.292 s +- 0.009 s | 1.295 s +- 0.010 s | 100.2%
m-u | | x | 1.296 s +- 0.042 s | 1.299 s +- 0.026 s | 100.2%
m-u | x | | 1.292 s +- 0.010 s | 1.297 s +- 0.021 s | 100.4%
m-u | x | x | 84.2 ms +- 1.2 ms | 83.6 ms +- 0.2 ms | 99.3%
l-d-r | | | 188.7 ms +- 2.7 ms | 188.8 ms +- 2.0 ms | 100.1%
l-d-r | | x | 189.9 ms +- 1.5 ms | 189.4 ms +- 1.2 ms | 99.7%
l-d-r | x | | 97.1 ms +- 1.0 ms | 87.1 ms +- 1.0 ms | 89.7% <--
l-d-r | x | x | 96.9 ms +- 0.8 ms | 87.2 ms +- 0.7 ms | 90.0% <--
diff -c . --git:
repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
------+---+---+------------------------+-----------------------+------------
m-u | | | 231.6 ms +- 3.1 ms | 228.9 ms +- 1.6 ms | 98.8%
m-u | | x | 150.5 ms +- 1.7 ms | 150.7 ms +- 1.4 ms | 100.1%
m-u | x | | 233.7 ms +- 2.4 ms | 232.2 ms +- 1.9 ms | 99.4%
m-u | x | x | 126.1 ms +- 1.2 ms | 126.8 ms +- 1.2 ms | 100.6%
l-d-r | | | 82.1 ms +- 2.0 ms | 81.8 ms +- 1.4 ms | 99.6%
l-d-r | | x | 3.732 s +- 0.020 s | 3.746 s +- 0.027 s | 100.4%
l-d-r | x | | 83.1 ms +- 0.8 ms | 107.6 ms +- 2.4 ms | 129.5% <--
l-d-r | x | x | 758.2 ms +- 38.8 ms | 188.5 ms +- 1.8 ms | 24.9% <--
rebase -r . --keep -d .^^:
repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
------+---+---+------------------------+-----------------------+------------
m-u | | | 5.532 s +- 0.087 s | 5.496 s +- 0.016 s | 99.3%
m-u | | x | 5.554 s +- 0.061 s | 5.532 s +- 0.013 s | 99.6%
m-u | x | | 5.602 s +- 0.134 s | 5.508 s +- 0.035 s | 98.3%
m-u | x | x | 582.2 ms +- 15.2 ms | 572.9 ms +- 12.0 ms | 98.4%
l-d-r | | | 629.5 ms +- 12.3 ms | 622.5 ms +- 7.3 ms | 98.9%
l-d-r | | x | 6.173 s +- 0.062 s | 6.185 s +- 0.076 s | 100.2%
l-d-r | x | | 274.5 ms +- 10.0 ms | 272.1 ms +- 6.2 ms | 99.1%
l-d-r | x | x | 4.835 s +- 0.056 s | 4.826 s +- 0.034 s | 99.8%
status --change . --copies:
repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
------+---+---+------------------------+-----------------------+------------
m-u | | | 214.4 ms +- 1.4 ms | 212.2 ms +- 1.7 ms | 99.0%
m-u | | x | 130.9 ms +- 1.2 ms | 131.7 ms +- 1.1 ms | 100.6%
m-u | x | | 215.0 ms +- 2.1 ms | 214.9 ms +- 2.7 ms | 100.0%
m-u | x | x | 109.5 ms +- 2.3 ms | 107.8 ms +- 0.9 ms | 98.4%
l-d-r | | | 79.6 ms +- 0.9 ms | 79.8 ms +- 1.6 ms | 100.3%
l-d-r | | x | 3.799 s +- 0.037 s | 3.928 s +- 0.021 s | 103.4% <--?
l-d-r | x | | 82.7 ms +- 0.7 ms | 83.2 ms +- 1.0 ms | 100.6%
l-d-r | x | x | 746.8 ms +- 6.1 ms | 739.0 ms +- 4.2 ms | 99.0%
status --copies:
repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
------+---+---+------------------------+-----------------------+------------
m-u | | | 1.884 s +- 0.012 s | 1.885 s +- 0.013 s | 100.1%
m-u | | x | 1.897 s +- 0.027 s | 1.909 s +- 0.077 s | 100.6%
m-u | x | | 1.886 s +- 0.021 s | 1.891 s +- 0.030 s | 100.3%
m-u | x | x | 92.0 ms +- 0.7 ms | 92.4 ms +- 0.4 ms | 100.4%
l-d-r | | | 570.3 ms +- 18.7 ms | 552.2 ms +- 4.5 ms | 96.8%
l-d-r | | x | 568.9 ms +- 16.1 ms | 567.2 ms +- 11.9 ms | 99.7%
l-d-r | x | | 171.1 ms +- 2.5 ms | 170.4 ms +- 1.2 ms | 99.6%
l-d-r | x | x | 171.6 ms +- 3.4 ms | 171.5 ms +- 1.7 ms | 99.9%
update $rev^; ~/src/hg/hg{hg}/hg update $rev:
repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
------+---+---+------------------------+-----------------------+------------
m-u | | | 3.107 s +- 0.017 s | 3.116 s +- 0.012 s | 100.3%
m-u | | x | 2.943 s +- 0.010 s | 2.945 s +- 0.019 s | 100.1%
m-u | x | | 3.116 s +- 0.033 s | 3.118 s +- 0.027 s | 100.1%
m-u | x | x | 318.5 ms +- 2.7 ms | 320.8 ms +- 4.8 ms | 100.7%
l-d-r | | | 428.9 ms +- 4.4 ms | 429.5 ms +- 4.0 ms | 100.1%
l-d-r | | x | 9.593 s +- 0.081 s | 9.869 s +- 0.043 s | 102.9%
l-d-r | x | | 253.2 ms +- 3.6 ms | 254.0 ms +- 2.8 ms | 100.3%
l-d-r | x | x | 1.613 s +- 0.009 s | 1.630 s +- 0.017 s | 101.1%
Differential Revision: https://phab.mercurial-scm.org/D4587
author | spectral <spectral@google.com> |
---|---|
date | Thu, 13 Sep 2018 22:48:27 -0700 |
parents | e7aa113b14f7 |
children | bae6f1418a95 |
rev | line source |
---|---|
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
1 # obsolete.py - obsolete markers handling |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
2 # |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
3 # Copyright 2012 Pierre-Yves David <pierre-yves.david@ens-lyon.org> |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
4 # Logilab SA <contact@logilab.fr> |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
5 # |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
6 # This software may be used and distributed according to the terms of the |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
7 # GNU General Public License version 2 or any later version. |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
8 |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
9 """Obsolete marker handling |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
10 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
11 An obsolete marker maps an old changeset to a list of new |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
12 changesets. If the list of new changesets is empty, the old changeset |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
13 is said to be "killed". Otherwise, the old changeset is being |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
14 "replaced" by the new changesets. |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
15 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
16 Obsolete markers can be used to record and distribute changeset graph |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
17 transformations performed by history rewrite operations, and help |
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
18 building new tools to reconcile conflicting rewrite actions. To |
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
19 facilitate conflict resolution, markers include various annotations |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
20 besides old and news changeset identifiers, such as creation date or |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
21 author name. |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
22 |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
23 The old obsoleted changeset is called a "predecessor" and possible |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
24 replacements are called "successors". Markers that used changeset X as |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
25 a predecessor are called "successor markers of X" because they hold |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
26 information about the successors of X. Markers that use changeset Y as |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
27 a successors are call "predecessor markers of Y" because they hold |
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
28 information about the predecessors of Y. |
17776
072812e9f570
obsolete: flip `obstore.successors` and `obsolete.precursors`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17775
diff
changeset
|
29 |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
30 Examples: |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
31 |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
32 - When changeset A is replaced by changeset A', one marker is stored: |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
33 |
21166
bf2891877378
obsolete: fix one-element tuple in module docstring
Martin Geisler <martin@geisler.net>
parents:
21165
diff
changeset
|
34 (A, (A',)) |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
35 |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
36 - When changesets A and B are folded into a new changeset C, two markers are |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
37 stored: |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
38 |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
39 (A, (C,)) and (B, (C,)) |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
40 |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
41 - When changeset A is simply "pruned" from the graph, a marker is created: |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
42 |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
43 (A, ()) |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
44 |
29894
02c150850e16
obsolete: fix minor module documentation issues
liscju <piotr.listkiewicz@gmail.com>
parents:
29389
diff
changeset
|
45 - When changeset A is split into B and C, a single marker is used: |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
46 |
29894
02c150850e16
obsolete: fix minor module documentation issues
liscju <piotr.listkiewicz@gmail.com>
parents:
29389
diff
changeset
|
47 (A, (B, C)) |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
48 |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
49 We use a single marker to distinguish the "split" case from the "divergence" |
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
50 case. If two independent operations rewrite the same changeset A in to A' and |
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
51 A'', we have an error case: divergent rewriting. We can detect it because |
17775
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
52 two markers will be created independently: |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
53 |
13744acc4ad7
obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17774
diff
changeset
|
54 (A, (B,)) and (A, (C,)) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
55 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
56 Format |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
57 ------ |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
58 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
59 Markers are stored in an append-only file stored in |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
60 '.hg/store/obsstore'. |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
61 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
62 The file starts with a version header: |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
63 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
64 - 1 unsigned byte: version number, starting at zero. |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
65 |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
66 The header is followed by the markers. Marker format depend of the version. See |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
67 comment associated with each format for details. |
21164
2efcef493aa2
obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents:
21098
diff
changeset
|
68 |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
69 """ |
27332
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
70 from __future__ import absolute_import |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
71 |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
72 import errno |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
73 import struct |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
74 |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
75 from .i18n import _ |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
76 from . import ( |
38707
6b5ca1d0aa1e
obsolete: store user name and note in UTF-8 (issue5754) (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
38157
diff
changeset
|
77 encoding, |
27332
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
78 error, |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
79 node, |
33142
4f49810a1011
obsutil: move 'successorssets' to the new modules
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33130
diff
changeset
|
80 obsutil, |
27332
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
81 phases, |
32372
df448de7cf3b
parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents:
32354
diff
changeset
|
82 policy, |
38708
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
83 pycompat, |
27332
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
84 util, |
04f346b8768f
obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26684
diff
changeset
|
85 ) |
36607
c6061cadb400
util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents:
36423
diff
changeset
|
86 from .utils import dateutil |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
87 |
32372
df448de7cf3b
parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents:
32354
diff
changeset
|
88 parsers = policy.importmod(r'parsers') |
df448de7cf3b
parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents:
32354
diff
changeset
|
89 |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
90 _pack = struct.pack |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
91 _unpack = struct.unpack |
23498
ac910b1f5658
obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23497
diff
changeset
|
92 _calcsize = struct.calcsize |
24046
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
93 propertycache = util.propertycache |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
94 |
17424
e7cfe3587ea4
fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents:
17405
diff
changeset
|
95 # the obsolete feature is not mature enough to be enabled by default. |
17296
a1f8869f2eee
obsolete: introduce an `_enabled` switch to disable the feature by default
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17295
diff
changeset
|
96 # you have to rely on third party extension extension to enable this. |
a1f8869f2eee
obsolete: introduce an `_enabled` switch to disable the feature by default
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17295
diff
changeset
|
97 _enabled = False |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
98 |
22951
6c86c673dde6
obsolete: add createmarkers option
Durham Goode <durham@fb.com>
parents:
22950
diff
changeset
|
99 # Options for obsolescence |
6c86c673dde6
obsolete: add createmarkers option
Durham Goode <durham@fb.com>
parents:
22950
diff
changeset
|
100 createmarkersopt = 'createmarkers' |
22952
8792ac090e3b
obsolete: add allowunstable option
Durham Goode <durham@fb.com>
parents:
22951
diff
changeset
|
101 allowunstableopt = 'allowunstable' |
22953
b1d694d3975e
obsolete: add exchange option
Durham Goode <durham@fb.com>
parents:
22952
diff
changeset
|
102 exchangeopt = 'exchange' |
22951
6c86c673dde6
obsolete: add createmarkers option
Durham Goode <durham@fb.com>
parents:
22950
diff
changeset
|
103 |
34864
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
104 def _getoptionvalue(repo, option): |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
105 """Returns True if the given repository has the given obsolete option |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
106 enabled. |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
107 """ |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
108 configkey = 'evolution.%s' % option |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
109 newconfig = repo.ui.configbool('experimental', configkey) |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
110 |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
111 # Return the value only if defined |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
112 if newconfig is not None: |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
113 return newconfig |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
114 |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
115 # Fallback on generic option |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
116 try: |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
117 return repo.ui.configbool('experimental', 'evolution') |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
118 except (error.ConfigError, AttributeError): |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
119 # Fallback on old-fashion config |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
120 # inconsistent config: experimental.evolution |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
121 result = set(repo.ui.configlist('experimental', 'evolution')) |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
122 |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
123 if 'all' in result: |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
124 return True |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
125 |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
126 # For migration purposes, temporarily return true if the config hasn't |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
127 # been set but _enabled is true. |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
128 if len(result) == 0 and _enabled: |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
129 return True |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
130 |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
131 # Temporary hack for next check |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
132 newconfig = repo.ui.config('experimental', 'evolution.createmarkers') |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
133 if newconfig: |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
134 result.add('createmarkers') |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
135 |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
136 return option in result |
fec79a3f250f
config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents:
34863
diff
changeset
|
137 |
37131
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
138 def getoptions(repo): |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
139 """Returns dicts showing state of obsolescence features.""" |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
140 |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
141 createmarkersvalue = _getoptionvalue(repo, createmarkersopt) |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
142 unstablevalue = _getoptionvalue(repo, allowunstableopt) |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
143 exchangevalue = _getoptionvalue(repo, exchangeopt) |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
144 |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
145 # createmarkers must be enabled if other options are enabled |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
146 if ((unstablevalue or exchangevalue) and not createmarkersvalue): |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
147 raise error.Abort(_("'createmarkers' obsolete option must be enabled " |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
148 "if other obsolete options are enabled")) |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
149 |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
150 return { |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
151 createmarkersopt: createmarkersvalue, |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
152 allowunstableopt: unstablevalue, |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
153 exchangeopt: exchangevalue, |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
154 } |
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
155 |
32333
566cfe9cbbb9
obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32327
diff
changeset
|
156 def isenabled(repo, option): |
566cfe9cbbb9
obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32327
diff
changeset
|
157 """Returns True if the given repository has the given obsolete option |
566cfe9cbbb9
obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32327
diff
changeset
|
158 enabled. |
566cfe9cbbb9
obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32327
diff
changeset
|
159 """ |
37131
d30810d09d6f
obsolete: refactor function for getting obsolete options
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37017
diff
changeset
|
160 return getoptions(repo)[option] |
32333
566cfe9cbbb9
obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32327
diff
changeset
|
161 |
37017
98c14e857e71
obsolete: add a note that explains creating aliases for marker flags
Anton Shestakov <av6@dwimlabs.net>
parents:
36953
diff
changeset
|
162 # Creating aliases for marker flags because evolve extension looks for |
98c14e857e71
obsolete: add a note that explains creating aliases for marker flags
Anton Shestakov <av6@dwimlabs.net>
parents:
36953
diff
changeset
|
163 # bumpedfix in obsolete.py |
36953
b9bbcf9ffac1
obsolete: move marker flags to obsutil
Anton Shestakov <av6@dwimlabs.net>
parents:
36607
diff
changeset
|
164 bumpedfix = obsutil.bumpedfix |
b9bbcf9ffac1
obsolete: move marker flags to obsutil
Anton Shestakov <av6@dwimlabs.net>
parents:
36607
diff
changeset
|
165 usingsha256 = obsutil.usingsha256 |
17831
70b08df24fef
obsolete: add a flag that allows fixing "bumped" changeset
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17828
diff
changeset
|
166 |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
167 ## Parsing and writing of version "0" |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
168 # |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
169 # The header is followed by the markers. Each marker is made of: |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
170 # |
22849
fd759142c6e5
obsolete: use uint## in the format documention
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22848
diff
changeset
|
171 # - 1 uint8 : number of new changesets "N", can be zero. |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
172 # |
22849
fd759142c6e5
obsolete: use uint## in the format documention
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22848
diff
changeset
|
173 # - 1 uint32: metadata size "M" in bytes. |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
174 # |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
175 # - 1 byte: a bit field. It is reserved for flags used in common |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
176 # obsolete marker operations, to avoid repeated decoding of metadata |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
177 # entries. |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
178 # |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
179 # - 20 bytes: obsoleted changeset identifier. |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
180 # |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
181 # - N*20 bytes: new changesets identifiers. |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
182 # |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
183 # - M bytes: metadata as a sequence of nul-terminated strings. Each |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
184 # string contains a key and a value, separated by a colon ':', without |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
185 # additional encoding. Keys cannot contain '\0' or ':' and values |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
186 # cannot contain '\0'. |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
187 _fm0version = 0 |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
188 _fm0fixed = '>BIB20s' |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
189 _fm0node = '20s' |
23498
ac910b1f5658
obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23497
diff
changeset
|
190 _fm0fsize = _calcsize(_fm0fixed) |
ac910b1f5658
obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23497
diff
changeset
|
191 _fm0fnodesize = _calcsize(_fm0node) |
22334
f2c3cfc30e9c
obsolete: move _encodemarkers next to _readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22333
diff
changeset
|
192 |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
193 def _fm0readmarkers(data, off, stop): |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
194 # Loop on markers |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
195 while off < stop: |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
196 # read fixed part |
22327
f737631a9f0a
obsolete: rename all _fm to _fm0
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22326
diff
changeset
|
197 cur = data[off:off + _fm0fsize] |
f737631a9f0a
obsolete: rename all _fm to _fm0
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22326
diff
changeset
|
198 off += _fm0fsize |
22685
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
199 numsuc, mdsize, flags, pre = _unpack(_fm0fixed, cur) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
200 # read replacement |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
201 sucs = () |
22685
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
202 if numsuc: |
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
203 s = (_fm0fnodesize * numsuc) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
204 cur = data[off:off + s] |
22685
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
205 sucs = _unpack(_fm0node * numsuc, cur) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
206 off += s |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
207 # read metadata |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
208 # (metadata will be decoded on demand) |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
209 metadata = data[off:off + mdsize] |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
210 if len(metadata) != mdsize: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26310
diff
changeset
|
211 raise error.Abort(_('parsing obsolete marker: metadata is too ' |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
212 'short, %d bytes expected, got %d') |
17253
67f56ff5afcd
obsolete: fix decoding error message arguments
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17220
diff
changeset
|
213 % (mdsize, len(metadata))) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
214 off += mdsize |
22847
37460ee2003c
obsolete: _rename decodemeta to _fm0decodemeta
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22846
diff
changeset
|
215 metadata = _fm0decodemeta(metadata) |
22222
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
216 try: |
22845
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
217 when, offset = metadata.pop('date', '0 0').split(' ') |
22309
a65697c3f20e
obsolete: avoid slow, generic date parsing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
22274
diff
changeset
|
218 date = float(when), int(offset) |
a65697c3f20e
obsolete: avoid slow, generic date parsing
Gregory Szorc <gregory.szorc@gmail.com>
parents:
22274
diff
changeset
|
219 except ValueError: |
22222
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
220 date = (0., 0) |
22258
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
221 parents = None |
22845
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
222 if 'p2' in metadata: |
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
223 parents = (metadata.pop('p1', None), metadata.pop('p2', None)) |
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
224 elif 'p1' in metadata: |
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
225 parents = (metadata.pop('p1', None),) |
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
226 elif 'p0' in metadata: |
22258
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
227 parents = () |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
228 if parents is not None: |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
229 try: |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
230 parents = tuple(node.bin(p) for p in parents) |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
231 # if parent content is not a nodeid, drop the data |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
232 for p in parents: |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
233 if len(p) != 20: |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
234 parents = None |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
235 break |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
236 except TypeError: |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
237 # if content cannot be translated to nodeid drop the data. |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
238 parents = None |
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
239 |
22845
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
240 metadata = tuple(sorted(metadata.iteritems())) |
22222
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
241 |
22258
cf414981978a
obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22257
diff
changeset
|
242 yield (pre, sucs, flags, metadata, date, parents) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
243 |
22330
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
244 def _fm0encodeonemarker(marker): |
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
245 pre, sucs, flags, metadata, date, parents = marker |
22850
b078e4dc9f9a
obsstore: add a flag for sha256 hashes
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22849
diff
changeset
|
246 if flags & usingsha256: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26310
diff
changeset
|
247 raise error.Abort(_('cannot handle sha256 with old obsstore format')) |
22845
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
248 metadata = dict(metadata) |
23002
2920a96f5839
obsstore: record data as floating point in fm0 format
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22954
diff
changeset
|
249 time, tz = date |
2920a96f5839
obsstore: record data as floating point in fm0 format
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22954
diff
changeset
|
250 metadata['date'] = '%r %i' % (time, tz) |
22330
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
251 if parents is not None: |
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
252 if not parents: |
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
253 # mark that we explicitly recorded no parents |
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
254 metadata['p0'] = '' |
32278
7c3ef55dedbe
obsolete: use 2 argument form of enumerate()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32201
diff
changeset
|
255 for i, p in enumerate(parents, 1): |
7c3ef55dedbe
obsolete: use 2 argument form of enumerate()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32201
diff
changeset
|
256 metadata['p%i' % i] = node.hex(p) |
22846
b1efc4893da4
obsolete: _rename encodemeta to _fm0encodemeta
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22845
diff
changeset
|
257 metadata = _fm0encodemeta(metadata) |
22685
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
258 numsuc = len(sucs) |
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
259 format = _fm0fixed + (_fm0node * numsuc) |
753515d9e274
obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22614
diff
changeset
|
260 data = [numsuc, len(metadata), flags, pre] |
22330
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
261 data.extend(sucs) |
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
262 return _pack(format, *data) + metadata |
e74f8a65252d
obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22329
diff
changeset
|
263 |
22848
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
264 def _fm0encodemeta(meta): |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
265 """Return encoded metadata string to string mapping. |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
266 |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
267 Assume no ':' in key and no '\0' in both key and value.""" |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
268 for key, value in meta.iteritems(): |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
269 if ':' in key or '\0' in key: |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
270 raise ValueError("':' and '\0' are forbidden in metadata key'") |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
271 if '\0' in value: |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
272 raise ValueError("':' is forbidden in metadata value'") |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
273 return '\0'.join(['%s:%s' % (k, meta[k]) for k in sorted(meta)]) |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
274 |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
275 def _fm0decodemeta(data): |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
276 """Return string to string dictionary from encoded version.""" |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
277 d = {} |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
278 for l in data.split('\0'): |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
279 if l: |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
280 key, value = l.split(':') |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
281 d[key] = value |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
282 return d |
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
283 |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
284 ## Parsing and writing of version "1" |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
285 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
286 # The header is followed by the markers. Each marker is made of: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
287 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
288 # - uint32: total size of the marker (including this field) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
289 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
290 # - float64: date in seconds since epoch |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
291 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
292 # - int16: timezone offset in minutes |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
293 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
294 # - uint16: a bit field. It is reserved for flags used in common |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
295 # obsolete marker operations, to avoid repeated decoding of metadata |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
296 # entries. |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
297 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
298 # - uint8: number of successors "N", can be zero. |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
299 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
300 # - uint8: number of parents "P", can be zero. |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
301 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
302 # 0: parents data stored but no parent, |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
303 # 1: one parent stored, |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
304 # 2: two parents stored, |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
305 # 3: no parent data stored |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
306 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
307 # - uint8: number of metadata entries M |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
308 # |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
309 # - 20 or 32 bytes: predecessor changeset identifier. |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
310 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
311 # - N*(20 or 32) bytes: successors changesets identifiers. |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
312 # |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
313 # - P*(20 or 32) bytes: parents of the predecessors changesets. |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
314 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
315 # - M*(uint8, uint8): size of all metadata entries (key and value) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
316 # |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
317 # - remaining bytes: the metadata, each (key, value) pair after the other. |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
318 _fm1version = 1 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
319 _fm1fixed = '>IdhHBBB20s' |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
320 _fm1nodesha1 = '20s' |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
321 _fm1nodesha256 = '32s' |
23499
b46876c94a93
obsstore: cache size computation for fm1 node
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23498
diff
changeset
|
322 _fm1nodesha1size = _calcsize(_fm1nodesha1) |
b46876c94a93
obsstore: cache size computation for fm1 node
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23498
diff
changeset
|
323 _fm1nodesha256size = _calcsize(_fm1nodesha256) |
23498
ac910b1f5658
obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23497
diff
changeset
|
324 _fm1fsize = _calcsize(_fm1fixed) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
325 _fm1parentnone = 3 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
326 _fm1parentshift = 14 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
327 _fm1parentmask = (_fm1parentnone << _fm1parentshift) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
328 _fm1metapair = 'BB' |
33670
e5f8c7baa729
obsolete: reuse _fm1metapair for computing _fm1metapairsize
Augie Fackler <augie@google.com>
parents:
33504
diff
changeset
|
329 _fm1metapairsize = _calcsize(_fm1metapair) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
330 |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
331 def _fm1purereadmarkers(data, off, stop): |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
332 # make some global constants local for performance |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
333 noneflag = _fm1parentnone |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
334 sha2flag = usingsha256 |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
335 sha1size = _fm1nodesha1size |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
336 sha2size = _fm1nodesha256size |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
337 sha1fmt = _fm1nodesha1 |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
338 sha2fmt = _fm1nodesha256 |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
339 metasize = _fm1metapairsize |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
340 metafmt = _fm1metapair |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
341 fsize = _fm1fsize |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
342 unpack = _unpack |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
343 |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
344 # Loop on markers |
25211
22f4ce49044d
util: drop the 'unpacker' helper
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25149
diff
changeset
|
345 ufixed = struct.Struct(_fm1fixed).unpack |
24018
26d6a6a78c1d
obsolete: use parsers.fm1readmarker if it exists for a ~38% perf win
Augie Fackler <augie@google.com>
parents:
24014
diff
changeset
|
346 |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
347 while off < stop: |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
348 # read fixed part |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
349 o1 = off + fsize |
23800
83f361a21d31
readmarkers: drop a temporary
Matt Mackall <mpm@selenic.com>
parents:
23799
diff
changeset
|
350 t, secs, tz, flags, numsuc, numpar, nummeta, prec = ufixed(data[off:o1]) |
23792
db3bc2bcb689
readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents:
23791
diff
changeset
|
351 |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
352 if flags & sha2flag: |
23805
345042e024dc
readmarkers: add a SHA256 fixme note
Matt Mackall <mpm@selenic.com>
parents:
23804
diff
changeset
|
353 # FIXME: prec was read as a SHA1, needs to be amended |
345042e024dc
readmarkers: add a SHA256 fixme note
Matt Mackall <mpm@selenic.com>
parents:
23804
diff
changeset
|
354 |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
355 # read 0 or more successors |
23804
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
356 if numsuc == 1: |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
357 o2 = o1 + sha2size |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
358 sucs = (data[o1:o2],) |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
359 else: |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
360 o2 = o1 + sha2size * numsuc |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
361 sucs = unpack(sha2fmt * numsuc, data[o1:o2]) |
23792
db3bc2bcb689
readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents:
23791
diff
changeset
|
362 |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
363 # read parents |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
364 if numpar == noneflag: |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
365 o3 = o2 |
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
366 parents = None |
23804
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
367 elif numpar == 1: |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
368 o3 = o2 + sha2size |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
369 parents = (data[o2:o3],) |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
370 else: |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
371 o3 = o2 + sha2size * numpar |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
372 parents = unpack(sha2fmt * numpar, data[o2:o3]) |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
373 else: |
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
374 # read 0 or more successors |
23804
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
375 if numsuc == 1: |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
376 o2 = o1 + sha1size |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
377 sucs = (data[o1:o2],) |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
378 else: |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
379 o2 = o1 + sha1size * numsuc |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
380 sucs = unpack(sha1fmt * numsuc, data[o1:o2]) |
23792
db3bc2bcb689
readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents:
23791
diff
changeset
|
381 |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
382 # read parents |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
383 if numpar == noneflag: |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
384 o3 = o2 |
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
385 parents = None |
23804
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
386 elif numpar == 1: |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
387 o3 = o2 + sha1size |
e880036719e3
readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents:
23803
diff
changeset
|
388 parents = (data[o2:o3],) |
23801
20b5701cd90a
readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents:
23800
diff
changeset
|
389 else: |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
390 o3 = o2 + sha1size * numpar |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
391 parents = unpack(sha1fmt * numpar, data[o2:o3]) |
23792
db3bc2bcb689
readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents:
23791
diff
changeset
|
392 |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
393 # read metadata |
23803
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
394 off = o3 + metasize * nummeta |
2763edaba857
readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents:
23802
diff
changeset
|
395 metapairsize = unpack('>' + (metafmt * nummeta), data[o3:off]) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
396 metadata = [] |
38783
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38708
diff
changeset
|
397 for idx in pycompat.xrange(0, len(metapairsize), 2): |
23798
86d2a0c41f44
readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents:
23797
diff
changeset
|
398 o1 = off + metapairsize[idx] |
86d2a0c41f44
readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents:
23797
diff
changeset
|
399 o2 = o1 + metapairsize[idx + 1] |
86d2a0c41f44
readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents:
23797
diff
changeset
|
400 metadata.append((data[off:o1], data[o1:o2])) |
86d2a0c41f44
readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents:
23797
diff
changeset
|
401 off = o2 |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
402 |
23800
83f361a21d31
readmarkers: drop a temporary
Matt Mackall <mpm@selenic.com>
parents:
23799
diff
changeset
|
403 yield (prec, sucs, flags, tuple(metadata), (secs, tz * 60), parents) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
404 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
405 def _fm1encodeonemarker(marker): |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
406 pre, sucs, flags, metadata, date, parents = marker |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
407 # determine node size |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
408 _fm1node = _fm1nodesha1 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
409 if flags & usingsha256: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
410 _fm1node = _fm1nodesha256 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
411 numsuc = len(sucs) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
412 numextranodes = numsuc |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
413 if parents is None: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
414 numpar = _fm1parentnone |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
415 else: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
416 numpar = len(parents) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
417 numextranodes += numpar |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
418 formatnodes = _fm1node * numextranodes |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
419 formatmeta = _fm1metapair * len(metadata) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
420 format = _fm1fixed + formatnodes + formatmeta |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
421 # tz is stored in minutes so we divide by 60 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
422 tz = date[1]//60 |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
423 data = [None, date[0], tz, flags, numsuc, numpar, len(metadata), pre] |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
424 data.extend(sucs) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
425 if parents is not None: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
426 data.extend(parents) |
23498
ac910b1f5658
obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23497
diff
changeset
|
427 totalsize = _calcsize(format) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
428 for key, value in metadata: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
429 lk = len(key) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
430 lv = len(value) |
34407
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
431 if lk > 255: |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
432 msg = ('obsstore metadata key cannot be longer than 255 bytes' |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
433 ' (key "%s" is %u bytes)') % (key, lk) |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
434 raise error.ProgrammingError(msg) |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
435 if lv > 255: |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
436 msg = ('obsstore metadata value cannot be longer than 255 bytes' |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
437 ' (value "%s" for key "%s" is %u bytes)') % (value, key, lv) |
b6692ba7d5b0
obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents:
34388
diff
changeset
|
438 raise error.ProgrammingError(msg) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
439 data.append(lk) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
440 data.append(lv) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
441 totalsize += lk + lv |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
442 data[0] = totalsize |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
443 data = [_pack(format, *data)] |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
444 for key, value in metadata: |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
445 data.append(key) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
446 data.append(value) |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
447 return ''.join(data) |
22848
72100c1d53d4
obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22847
diff
changeset
|
448 |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
449 def _fm1readmarkers(data, off, stop): |
24019
26fbf07482b2
_fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents:
24018
diff
changeset
|
450 native = getattr(parsers, 'fm1readmarkers', None) |
26fbf07482b2
_fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents:
24018
diff
changeset
|
451 if not native: |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
452 return _fm1purereadmarkers(data, off, stop) |
24019
26fbf07482b2
_fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents:
24018
diff
changeset
|
453 return native(data, off, stop) |
26fbf07482b2
_fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents:
24018
diff
changeset
|
454 |
22331
b130b241718e
obsolete: support for any known obsstore format when reading or writing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22330
diff
changeset
|
455 # mapping to read/write various marker formats |
b130b241718e
obsolete: support for any known obsstore format when reading or writing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22330
diff
changeset
|
456 # <version> -> (decoder, encoder) |
22851
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
457 formats = {_fm0version: (_fm0readmarkers, _fm0encodeonemarker), |
974389427e5f
obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22850
diff
changeset
|
458 _fm1version: (_fm1readmarkers, _fm1encodeonemarker)} |
22331
b130b241718e
obsolete: support for any known obsstore format when reading or writing
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22330
diff
changeset
|
459 |
32689
9933410bf763
obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents:
32688
diff
changeset
|
460 def _readmarkerversion(data): |
9933410bf763
obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents:
32688
diff
changeset
|
461 return _unpack('>B', data[0:1])[0] |
9933410bf763
obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents:
32688
diff
changeset
|
462 |
23497
5817f71c2336
obsstore: disable garbage collection during initialization (issue4456)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23002
diff
changeset
|
463 @util.nogc |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
464 def _readmarkers(data, off=None, stop=None): |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
465 """Read and enumerate markers from raw data""" |
32689
9933410bf763
obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents:
32688
diff
changeset
|
466 diskversion = _readmarkerversion(data) |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
467 if not off: |
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
468 off = 1 # skip 1 byte version number |
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
469 if stop is None: |
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
470 stop = len(data) |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
471 if diskversion not in formats: |
32596
19df975eb555
obsolete: raise richer exception on unknown version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32488
diff
changeset
|
472 msg = _('parsing obsolete marker: unknown version %r') % diskversion |
19df975eb555
obsolete: raise richer exception on unknown version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32488
diff
changeset
|
473 raise error.UnknownVersion(msg, version=diskversion) |
33504
5d3ba4395288
obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents:
33499
diff
changeset
|
474 return diskversion, formats[diskversion][0](data, off, stop) |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
475 |
32692
9576974a3b6d
obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents:
32691
diff
changeset
|
476 def encodeheader(version=_fm0version): |
9576974a3b6d
obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents:
32691
diff
changeset
|
477 return _pack('>B', version) |
9576974a3b6d
obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents:
32691
diff
changeset
|
478 |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
479 def encodemarkers(markers, addheader=False, version=_fm0version): |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
480 # Kept separate from flushmarkers(), it will be reused for |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
481 # markers exchange. |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
482 encodeone = formats[version][1] |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
483 if addheader: |
32692
9576974a3b6d
obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents:
32691
diff
changeset
|
484 yield encodeheader(version) |
22612
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
485 for marker in markers: |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
486 yield encodeone(marker) |
fdfa40ee75cf
obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22507
diff
changeset
|
487 |
24044
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
488 @util.nogc |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
489 def _addsuccessors(successors, markers): |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
490 for mark in markers: |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
491 successors.setdefault(mark[0], set()).add(mark) |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
492 |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
493 @util.nogc |
33732
32d4f815c4f4
obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents:
33719
diff
changeset
|
494 def _addpredecessors(predecessors, markers): |
24044
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
495 for mark in markers: |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
496 for suc in mark[1]: |
33732
32d4f815c4f4
obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents:
33719
diff
changeset
|
497 predecessors.setdefault(suc, set()).add(mark) |
24044
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
498 |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
499 @util.nogc |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
500 def _addchildren(children, markers): |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
501 for mark in markers: |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
502 parents = mark[5] |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
503 if parents is not None: |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
504 for p in parents: |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
505 children.setdefault(p, set()).add(mark) |
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
506 |
24045
43061f96e40e
obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24044
diff
changeset
|
507 def _checkinvalidmarkers(markers): |
23973
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
508 """search for marker with invalid data and raise error if needed |
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
509 |
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
510 Exist as a separated function to allow the evolve extension for a more |
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
511 subtle handling. |
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
512 """ |
24045
43061f96e40e
obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24044
diff
changeset
|
513 for mark in markers: |
43061f96e40e
obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24044
diff
changeset
|
514 if node.nullid in mark[1]: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26310
diff
changeset
|
515 raise error.Abort(_('bad obsolescence marker detected: ' |
24045
43061f96e40e
obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24044
diff
changeset
|
516 'invalid successors nullid')) |
23973
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
517 |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
518 class obsstore(object): |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
519 """Store obsolete markers |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
520 |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
521 Markers can be accessed with two mappings: |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
522 - predecessors[x] -> set(markers on predecessors edges of x) |
17776
072812e9f570
obsolete: flip `obstore.successors` and `obsolete.precursors`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17775
diff
changeset
|
523 - successors[x] -> set(markers on successors edges of x) |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
524 - children[x] -> set(markers on predecessors edges of children(x) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
525 """ |
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
526 |
22254
b8a0e8176693
obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22253
diff
changeset
|
527 fields = ('prec', 'succs', 'flag', 'meta', 'date', 'parents') |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
528 # prec: nodeid, predecessors changesets |
22254
b8a0e8176693
obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22253
diff
changeset
|
529 # succs: tuple of nodeid, successor changesets (0-N length) |
b8a0e8176693
obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22253
diff
changeset
|
530 # flag: integer, flag field carrying modifier for the markers (see doc) |
38707
6b5ca1d0aa1e
obsolete: store user name and note in UTF-8 (issue5754) (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
38157
diff
changeset
|
531 # meta: binary blob in UTF-8, encoded metadata dictionary |
22254
b8a0e8176693
obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22253
diff
changeset
|
532 # date: (float, int) tuple, date of marker creation |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
533 # parents: (tuple of nodeid) or None, parents of predecessors |
22254
b8a0e8176693
obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22253
diff
changeset
|
534 # None is used when no data has been recorded |
22221
e75b1a3c1dbc
obsstore: add fields attribute to track each field in a marker
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22219
diff
changeset
|
535 |
25669
8d94861842e5
obsolete: replace references to 'sopener' with 'svfs'
Siddharth Agarwal <sid0@fb.com>
parents:
25412
diff
changeset
|
536 def __init__(self, svfs, defaultformat=_fm1version, readonly=False): |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
537 # caches for various obsolescence related cache |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
538 self.caches = {} |
25669
8d94861842e5
obsolete: replace references to 'sopener' with 'svfs'
Siddharth Agarwal <sid0@fb.com>
parents:
25412
diff
changeset
|
539 self.svfs = svfs |
32691
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
540 self._defaultformat = defaultformat |
22950
bb8278b289ee
obsolete: add readonly flag to obstore constructor
Durham Goode <durham@fb.com>
parents:
22949
diff
changeset
|
541 self._readonly = readonly |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
542 |
17073
3a79a5682af1
obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents:
17072
diff
changeset
|
543 def __iter__(self): |
3a79a5682af1
obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents:
17072
diff
changeset
|
544 return iter(self._all) |
3a79a5682af1
obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents:
17072
diff
changeset
|
545 |
20585
f3c8db3d6d66
obsstore: add a __len__ method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20584
diff
changeset
|
546 def __len__(self): |
f3c8db3d6d66
obsstore: add a __len__ method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20584
diff
changeset
|
547 return len(self._all) |
f3c8db3d6d66
obsstore: add a __len__ method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20584
diff
changeset
|
548 |
17075
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
549 def __nonzero__(self): |
35837
d8f891ec5d7d
obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents:
35574
diff
changeset
|
550 if not self._cached(r'_all'): |
26310
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
551 try: |
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
552 return self.svfs.stat('obsstore').st_size > 1 |
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
553 except OSError as inst: |
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
554 if inst.errno != errno.ENOENT: |
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
555 raise |
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
556 # just build an empty _all list if no obsstore exists, which |
61efe9ef6ad4
obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents:
26309
diff
changeset
|
557 # avoids further stat() syscalls |
17075
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
558 return bool(self._all) |
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
559 |
31476
413b44003462
py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents:
31018
diff
changeset
|
560 __bool__ = __nonzero__ |
413b44003462
py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents:
31018
diff
changeset
|
561 |
26684
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
562 @property |
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
563 def readonly(self): |
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
564 """True if marker creation is disabled |
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
565 |
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
566 Remove me in the future when obsolete marker is always on.""" |
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
567 return self._readonly |
74ff350c208c
obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
26587
diff
changeset
|
568 |
22255
adb3798dce49
obsstore: add a `parents` argument to obsstore.create
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22254
diff
changeset
|
569 def create(self, transaction, prec, succs=(), flag=0, parents=None, |
32411
08d02c1d7e67
devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents:
32387
diff
changeset
|
570 date=None, metadata=None, ui=None): |
17071
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
571 """obsolete: add a new obsolete marker |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
572 |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
573 * ensuring it is hashable |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
574 * check mandatory metadata |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
575 * encode metadata |
20516
3af218cf2007
obsstore: update create docstring to point to the coder friendly function
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20476
diff
changeset
|
576 |
3af218cf2007
obsstore: update create docstring to point to the coder friendly function
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20476
diff
changeset
|
577 If you are a human writing code creating marker you want to use the |
3af218cf2007
obsstore: update create docstring to point to the coder friendly function
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20476
diff
changeset
|
578 `createmarkers` function in this module instead. |
20584
224a058f7cd1
obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20549
diff
changeset
|
579 |
224a058f7cd1
obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20549
diff
changeset
|
580 return True if a new marker have been added, False if the markers |
224a058f7cd1
obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20549
diff
changeset
|
581 already existed (no op). |
17071
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
582 """ |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
583 if metadata is None: |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
584 metadata = {} |
22222
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
585 if date is None: |
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
586 if 'date' in metadata: |
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
587 # as a courtesy for out-of-tree extensions |
36607
c6061cadb400
util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents:
36423
diff
changeset
|
588 date = dateutil.parsedate(metadata.pop('date')) |
32411
08d02c1d7e67
devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents:
32387
diff
changeset
|
589 elif ui is not None: |
08d02c1d7e67
devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents:
32387
diff
changeset
|
590 date = ui.configdate('devel', 'default-date') |
08d02c1d7e67
devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents:
32387
diff
changeset
|
591 if date is None: |
36607
c6061cadb400
util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents:
36423
diff
changeset
|
592 date = dateutil.makedate() |
22222
883e8b6bd461
obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22221
diff
changeset
|
593 else: |
36607
c6061cadb400
util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents:
36423
diff
changeset
|
594 date = dateutil.makedate() |
17071
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
595 if len(prec) != 20: |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
596 raise ValueError(prec) |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
597 for succ in succs: |
11f26e2669aa
command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents:
17070
diff
changeset
|
598 if len(succ) != 20: |
17117
217bfb10e6db
obsolete: fix error message at marker creation
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17076
diff
changeset
|
599 raise ValueError(succ) |
22177
a56038e6a3c9
obsstore.create: add a simple safeguard against cyclic markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21166
diff
changeset
|
600 if prec in succs: |
a56038e6a3c9
obsstore.create: add a simple safeguard against cyclic markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
21166
diff
changeset
|
601 raise ValueError(_('in-marker cycle with %s') % node.hex(prec)) |
22845
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
602 |
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
603 metadata = tuple(sorted(metadata.iteritems())) |
38708
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
604 for k, v in metadata: |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
605 try: |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
606 # might be better to reject non-ASCII keys |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
607 k.decode('utf-8') |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
608 v.decode('utf-8') |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
609 except UnicodeDecodeError: |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
610 raise error.ProgrammingError( |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
611 'obsstore metadata must be valid UTF-8 sequence ' |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
612 '(key = %r, value = %r)' |
ff1182d166a2
obsolete: explode if metadata contains invalid UTF-8 sequence (API)
Yuya Nishihara <yuya@tcha.org>
parents:
38707
diff
changeset
|
613 % (pycompat.bytestr(k), pycompat.bytestr(v))) |
22845
ef880e28e56a
obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22685
diff
changeset
|
614 |
33719
388901a15bfa
obsolete: use bytes() instead of str() so the node is bytes on py3
Augie Fackler <augie@google.com>
parents:
33670
diff
changeset
|
615 marker = (bytes(prec), tuple(succs), int(flag), metadata, date, parents) |
20584
224a058f7cd1
obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20549
diff
changeset
|
616 return bool(self.add(transaction, [marker])) |
17220
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
617 |
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
618 def add(self, transaction, markers): |
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
619 """Add new markers to the store |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
620 |
17220
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
621 Take care of filtering duplicate. |
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
622 Return the number of new marker.""" |
22950
bb8278b289ee
obsolete: add readonly flag to obstore constructor
Durham Goode <durham@fb.com>
parents:
22949
diff
changeset
|
623 if self._readonly: |
29389
98e8313dcd9e
i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents:
29281
diff
changeset
|
624 raise error.Abort(_('creating obsolete markers is not enabled on ' |
98e8313dcd9e
i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents:
29281
diff
changeset
|
625 'this repo')) |
32774
5ffb138d7b1a
obsstore: do not load all markers to detect duplication
Jun Wu <quark@fb.com>
parents:
32729
diff
changeset
|
626 known = set() |
5ffb138d7b1a
obsstore: do not load all markers to detect duplication
Jun Wu <quark@fb.com>
parents:
32729
diff
changeset
|
627 getsuccessors = self.successors.get |
20030
5931489b65e0
obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20028
diff
changeset
|
628 new = [] |
5931489b65e0
obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20028
diff
changeset
|
629 for m in markers: |
32774
5ffb138d7b1a
obsstore: do not load all markers to detect duplication
Jun Wu <quark@fb.com>
parents:
32729
diff
changeset
|
630 if m not in getsuccessors(m[0], ()) and m not in known: |
20030
5931489b65e0
obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20028
diff
changeset
|
631 known.add(m) |
5931489b65e0
obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20028
diff
changeset
|
632 new.append(m) |
17220
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
633 if new: |
25669
8d94861842e5
obsolete: replace references to 'sopener' with 'svfs'
Siddharth Agarwal <sid0@fb.com>
parents:
25412
diff
changeset
|
634 f = self.svfs('obsstore', 'ab') |
17124
f1b7683f3f95
obsolete: move obsolete markers read/write logic to obsstore object
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17117
diff
changeset
|
635 try: |
17126
8fa8717b47b6
obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17125
diff
changeset
|
636 offset = f.tell() |
8fa8717b47b6
obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17125
diff
changeset
|
637 transaction.add('obsstore', offset) |
17219
494a970f68de
obsolete: refactor writemarkers to only encode them
Pierre-Yves.David@ens-lyon.org
parents:
17213
diff
changeset
|
638 # offset == 0: new file - add the version header |
33479
8b48dad66be4
obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents:
33273
diff
changeset
|
639 data = b''.join(encodemarkers(new, offset == 0, self._version)) |
8b48dad66be4
obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents:
33273
diff
changeset
|
640 f.write(data) |
17126
8fa8717b47b6
obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17125
diff
changeset
|
641 finally: |
8fa8717b47b6
obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17125
diff
changeset
|
642 # XXX: f.close() == filecache invalidation == obsstore rebuilt. |
8fa8717b47b6
obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17125
diff
changeset
|
643 # call 'filecacheentry.refresh()' here |
17124
f1b7683f3f95
obsolete: move obsolete markers read/write logic to obsstore object
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17117
diff
changeset
|
644 f.close() |
33251
a5cb2e4460de
transaction: track new obsmarkers in the 'changes' mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33244
diff
changeset
|
645 addedmarkers = transaction.changes.get('obsmarkers') |
a5cb2e4460de
transaction: track new obsmarkers in the 'changes' mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33244
diff
changeset
|
646 if addedmarkers is not None: |
a5cb2e4460de
transaction: track new obsmarkers in the 'changes' mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33244
diff
changeset
|
647 addedmarkers.update(new) |
33479
8b48dad66be4
obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents:
33273
diff
changeset
|
648 self._addmarkers(new, data) |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
649 # new marker *may* have changed several set. invalidate the cache. |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
650 self.caches.clear() |
22339
9680da73cfe0
obsmarker: record the number of new markers in the transaction
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22335
diff
changeset
|
651 # records the number of new markers for the transaction hooks |
9680da73cfe0
obsmarker: record the number of new markers in the transaction
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22335
diff
changeset
|
652 previous = int(transaction.hookargs.get('new_obsmarkers', '0')) |
36423
2831d918e1b4
py3: convert known-int values to bytes using %d
Augie Fackler <augie@google.com>
parents:
35891
diff
changeset
|
653 transaction.hookargs['new_obsmarkers'] = '%d' % (previous + len(new)) |
17220
bdac214a4705
obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents:
17219
diff
changeset
|
654 return len(new) |
17126
8fa8717b47b6
obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17125
diff
changeset
|
655 |
17524 | 656 def mergemarkers(self, transaction, data): |
22325
3363f2d36015
obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22309
diff
changeset
|
657 """merge a binary stream of markers inside the obsstore |
3363f2d36015
obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22309
diff
changeset
|
658 |
3363f2d36015
obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22309
diff
changeset
|
659 Returns the number of new markers added.""" |
22332
13e22358e9d2
obsolete: have _readmarkers return the format version
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22331
diff
changeset
|
660 version, markers = _readmarkers(data) |
22325
3363f2d36015
obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22309
diff
changeset
|
661 return self.add(transaction, markers) |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
662 |
24046
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
663 @propertycache |
32690
3fad59faba8a
obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents:
32689
diff
changeset
|
664 def _data(self): |
3fad59faba8a
obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents:
32689
diff
changeset
|
665 return self.svfs.tryread('obsstore') |
3fad59faba8a
obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents:
32689
diff
changeset
|
666 |
3fad59faba8a
obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents:
32689
diff
changeset
|
667 @propertycache |
32691
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
668 def _version(self): |
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
669 if len(self._data) >= 1: |
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
670 return _readmarkerversion(self._data) |
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
671 else: |
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
672 return self._defaultformat |
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
673 |
5d8dd6c90af9
obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents:
32690
diff
changeset
|
674 @propertycache |
26309
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
675 def _all(self): |
32690
3fad59faba8a
obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents:
32689
diff
changeset
|
676 data = self._data |
26309
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
677 if not data: |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
678 return [] |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
679 self._version, markers = _readmarkers(data) |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
680 markers = list(markers) |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
681 _checkinvalidmarkers(markers) |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
682 return markers |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
683 |
44918682093f
obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents:
26308
diff
changeset
|
684 @propertycache |
24046
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
685 def successors(self): |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
686 successors = {} |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
687 _addsuccessors(successors, self._all) |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
688 return successors |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
689 |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
690 @propertycache |
33733
d5acd967f95a
obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents:
33732
diff
changeset
|
691 def predecessors(self): |
33732
32d4f815c4f4
obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents:
33719
diff
changeset
|
692 predecessors = {} |
32d4f815c4f4
obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents:
33719
diff
changeset
|
693 _addpredecessors(predecessors, self._all) |
32d4f815c4f4
obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents:
33719
diff
changeset
|
694 return predecessors |
24046
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
695 |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
696 @propertycache |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
697 def children(self): |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
698 children = {} |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
699 _addchildren(children, self._all) |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
700 return children |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
701 |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
702 def _cached(self, attr): |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
703 return attr in self.__dict__ |
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
704 |
33479
8b48dad66be4
obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents:
33273
diff
changeset
|
705 def _addmarkers(self, markers, rawdata): |
24044
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
706 markers = list(markers) # to allow repeated iteration |
33479
8b48dad66be4
obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents:
33273
diff
changeset
|
707 self._data = self._data + rawdata |
24044
e0e6dd806b23
obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24019
diff
changeset
|
708 self._all.extend(markers) |
35837
d8f891ec5d7d
obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents:
35574
diff
changeset
|
709 if self._cached(r'successors'): |
24046
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
710 _addsuccessors(self.successors, markers) |
35837
d8f891ec5d7d
obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents:
35574
diff
changeset
|
711 if self._cached(r'predecessors'): |
33733
d5acd967f95a
obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents:
33732
diff
changeset
|
712 _addpredecessors(self.predecessors, markers) |
35837
d8f891ec5d7d
obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents:
35574
diff
changeset
|
713 if self._cached(r'children'): |
24046
6e1d9f9932a9
obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents:
24045
diff
changeset
|
714 _addchildren(self.children, markers) |
24045
43061f96e40e
obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents:
24044
diff
changeset
|
715 _checkinvalidmarkers(markers) |
23973
18d431147cc1
obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23805
diff
changeset
|
716 |
22271
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
717 def relevantmarkers(self, nodes): |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
718 """return a set of all obsolescence markers relevant to a set of nodes. |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
719 |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
720 "relevant" to a set of nodes mean: |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
721 |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
722 - marker that use this changeset as successor |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
723 - prune marker of direct children on this changeset |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
724 - recursive application of the two rules on predecessors of these |
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
725 markers |
22271
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
726 |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
727 It is a set so you cannot rely on order.""" |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
728 |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
729 pendingnodes = set(nodes) |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
730 seenmarkers = set() |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
731 seennodes = set(pendingnodes) |
33733
d5acd967f95a
obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents:
33732
diff
changeset
|
732 precursorsmarkers = self.predecessors |
32488
176d1a0ce385
obsolete: fix relevant-obsmarkers computation on pruned changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32411
diff
changeset
|
733 succsmarkers = self.successors |
22271
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
734 children = self.children |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
735 while pendingnodes: |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
736 direct = set() |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
737 for current in pendingnodes: |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
738 direct.update(precursorsmarkers.get(current, ())) |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
739 pruned = [m for m in children.get(current, ()) if not m[1]] |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
740 direct.update(pruned) |
32488
176d1a0ce385
obsolete: fix relevant-obsmarkers computation on pruned changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32411
diff
changeset
|
741 pruned = [m for m in succsmarkers.get(current, ()) if not m[1]] |
176d1a0ce385
obsolete: fix relevant-obsmarkers computation on pruned changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32411
diff
changeset
|
742 direct.update(pruned) |
22271
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
743 direct -= seenmarkers |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
744 pendingnodes = set([m[0] for m in direct]) |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
745 seenmarkers |= direct |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
746 pendingnodes -= seennodes |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
747 seennodes |= pendingnodes |
8c69262df82d
obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22270
diff
changeset
|
748 return seenmarkers |
17070
ad0d6c2b3279
obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff
changeset
|
749 |
32729
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
750 def makestore(ui, repo): |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
751 """Create an obsstore instance from a repo.""" |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
752 # read default format for new obsstore. |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
753 # developer config: format.obsstore-version |
33244
fd50788a2d4f
configitems: register the 'format.obsstore-version' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33149
diff
changeset
|
754 defaultformat = ui.configint('format', 'obsstore-version') |
32729
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
755 # rely on obsstore class default when possible. |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
756 kwargs = {} |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
757 if defaultformat is not None: |
35365
7354fda50e8b
py3: handle keyword arguments correctly in obsolete.py
Pulkit Goyal <7895pulkit@gmail.com>
parents:
35137
diff
changeset
|
758 kwargs[r'defaultformat'] = defaultformat |
32729
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
759 readonly = not isenabled(repo, createmarkersopt) |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
760 store = obsstore(repo.svfs, readonly=readonly, **kwargs) |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
761 if store and readonly: |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
762 ui.warn(_('obsolete feature not enabled but %i markers found!\n') |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
763 % len(list(store))) |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
764 return store |
c8177792fef6
obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32692
diff
changeset
|
765 |
22345
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
766 def commonversion(versions): |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
767 """Return the newest version listed in both versions and our local formats. |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
768 |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
769 Returns None if no common version exists. |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
770 """ |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
771 versions.sort(reverse=True) |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
772 # search for highest version known on both side |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
773 for v in versions: |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
774 if v in formats: |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
775 return v |
cf7014c5f087
obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22339
diff
changeset
|
776 return None |
17295
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
777 |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
778 # arbitrary picked to fit into 8K limit from HTTP server |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
779 # you have to take in account: |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
780 # - the version header |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
781 # - the base85 encoding |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
782 _maxpayload = 5300 |
17075
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
783 |
20599
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
784 def _pushkeyescape(markers): |
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
785 """encode markers into a dict suitable for pushkey exchange |
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
786 |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
20599
diff
changeset
|
787 - binary data is base85 encoded |
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
20599
diff
changeset
|
788 - split in chunks smaller than 5300 bytes""" |
17295
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
789 keys = {} |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
790 parts = [] |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
791 currentlen = _maxpayload * 2 # ensure we create a new part |
20599
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
792 for marker in markers: |
22329
ed37aa74d208
obsolete: rename _encodeonemarker to _fm0encodeonemarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22328
diff
changeset
|
793 nextdata = _fm0encodeonemarker(marker) |
17295
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
794 if (len(nextdata) + currentlen > _maxpayload): |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
795 currentpart = [] |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
796 currentlen = 0 |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
797 parts.append(currentpart) |
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
798 currentpart.append(nextdata) |
17304
0368fc55d572
obsolete: properly increment currentlen when building pushkey payload
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17298
diff
changeset
|
799 currentlen += len(nextdata) |
17295
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
800 for idx, part in enumerate(reversed(parts)): |
22327
f737631a9f0a
obsolete: rename all _fm to _fm0
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22326
diff
changeset
|
801 data = ''.join([_pack('>B', _fm0version)] + part) |
32201
4462a981e8df
base85: proxy through util module
Yuya Nishihara <yuya@tcha.org>
parents:
31476
diff
changeset
|
802 keys['dump%i' % idx] = util.b85encode(data) |
17295
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
803 return keys |
17073
3a79a5682af1
obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents:
17072
diff
changeset
|
804 |
20599
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
805 def listmarkers(repo): |
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
806 """List markers over pushkey""" |
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
807 if not repo.obsstore: |
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
808 return {} |
25118
e632a2429982
obsolete: sort obsmarkers during exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24928
diff
changeset
|
809 return _pushkeyescape(sorted(repo.obsstore)) |
20599
dad29624b056
obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20585
diff
changeset
|
810 |
17075
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
811 def pushmarker(repo, key, old, new): |
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
812 """Push markers over pushkey""" |
17295
1f08ecc7febb
pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17253
diff
changeset
|
813 if not key.startswith('dump'): |
17075
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
814 repo.ui.warn(_('unknown key: %r') % key) |
32822
e65ff29dbeb0
pushkey: use False/True for return values from push functions
Martin von Zweigbergk <martinvonz@google.com>
parents:
32774
diff
changeset
|
815 return False |
17075
28ed1c4511ce
obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents:
17073
diff
changeset
|
816 if old: |
21098
399d7770eef2
obsolete: add '%' specifier to the format string to avoid TypeError at runtime
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
21024
diff
changeset
|
817 repo.ui.warn(_('unexpected old value for %r') % key) |
32822
e65ff29dbeb0
pushkey: use False/True for return values from push functions
Martin von Zweigbergk <martinvonz@google.com>
parents:
32774
diff
changeset
|
818 return False |
32201
4462a981e8df
base85: proxy through util module
Yuya Nishihara <yuya@tcha.org>
parents:
31476
diff
changeset
|
819 data = util.b85decode(new) |
35574
09285733ad71
obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents:
35573
diff
changeset
|
820 with repo.lock(), repo.transaction('pushkey: obsolete markers') as tr: |
09285733ad71
obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents:
35573
diff
changeset
|
821 repo.obsstore.mergemarkers(tr, data) |
09285733ad71
obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents:
35573
diff
changeset
|
822 repo.invalidatevolatilesets() |
09285733ad71
obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents:
35573
diff
changeset
|
823 return True |
17073
3a79a5682af1
obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents:
17072
diff
changeset
|
824 |
17828
9495be4126ef
obsolete: add the detection of bumped changeset.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17827
diff
changeset
|
825 # mapping of 'set-name' -> <function to compute this set> |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
826 cachefuncs = {} |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
827 def cachefor(name): |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
828 """Decorator to register a function as computing the cache for a set""" |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
829 def decorator(func): |
32884
9d472b219fb0
obsolete: use ProgrammingError over assert for volatile set registration
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
32822
diff
changeset
|
830 if name in cachefuncs: |
9d472b219fb0
obsolete: use ProgrammingError over assert for volatile set registration
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
32822
diff
changeset
|
831 msg = "duplicated registration for volatileset '%s' (existing: %r)" |
9d472b219fb0
obsolete: use ProgrammingError over assert for volatile set registration
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
32822
diff
changeset
|
832 raise error.ProgrammingError(msg % (name, cachefuncs[name])) |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
833 cachefuncs[name] = func |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
834 return func |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
835 return decorator |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
836 |
17825
3cc06457f15e
obsolete: rename `getobscache` into `getrevs`
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17776
diff
changeset
|
837 def getrevs(repo, name): |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
838 """Return the set of revision that belong to the <name> set |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
839 |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
840 Such access may compute the set and cache it for future use""" |
18001
e02feadd15ea
clfilter: unfilter computation of obsolescence related computation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
17831
diff
changeset
|
841 repo = repo.unfiltered() |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
842 if not repo.obsstore: |
22507
5c00c5298f98
obsolete: ensure that `getrevs` always return a set
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22394
diff
changeset
|
843 return frozenset() |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
844 if name not in repo.obsstore.caches: |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
845 repo.obsstore.caches[name] = cachefuncs[name](repo) |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
846 return repo.obsstore.caches[name] |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
847 |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
848 # To be simple we need to invalidate obsolescence cache when: |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
849 # |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
850 # - new changeset is added: |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
851 # - public phase is changed |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
852 # - obsolescence marker are added |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
853 # - strip is used a repo |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
854 def clearobscaches(repo): |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
855 """Remove all obsolescence related cache from a repo |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
856 |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
857 This remove all cache in obsstore is the obsstore already exist on the |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
858 repo. |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
859 |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
860 (We could be smarter here given the exact event that trigger the cache |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
861 clearing)""" |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
862 # only clear cache is there is obsstore data in this repo |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
863 if 'obsstore' in repo._filecache: |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
864 repo.obsstore.caches.clear() |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
865 |
33129
765c6ab07a88
obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32884
diff
changeset
|
866 def _mutablerevs(repo): |
765c6ab07a88
obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32884
diff
changeset
|
867 """the set of mutable revision in the repository""" |
38157
02f992ac26e9
phases: define an official tuple of mutable phases
Boris Feld <boris.feld@octobus.net>
parents:
37131
diff
changeset
|
868 return repo._phasecache.getrevset(repo, phases.mutablephases) |
33129
765c6ab07a88
obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32884
diff
changeset
|
869 |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
870 @cachefor('obsolete') |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
871 def _computeobsoleteset(repo): |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
872 """the set of obsolete revisions""" |
27784
432242f41d9f
obsolete: make _computeobsoleteset much faster
Laurent Charignon <lc2817@columbia.edu>
parents:
27332
diff
changeset
|
873 getnode = repo.changelog.node |
33129
765c6ab07a88
obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32884
diff
changeset
|
874 notpublic = _mutablerevs(repo) |
32688
2c1400d43fd2
obsstore: minor optimization for the obsolete revset
Jun Wu <quark@fb.com>
parents:
32627
diff
changeset
|
875 isobs = repo.obsstore.successors.__contains__ |
2c1400d43fd2
obsstore: minor optimization for the obsolete revset
Jun Wu <quark@fb.com>
parents:
32627
diff
changeset
|
876 obs = set(r for r in notpublic if isobs(getnode(r))) |
18271
67872e939945
performance: speedup computation of obsolete revisions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18101
diff
changeset
|
877 return obs |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
878 |
33777
d4b7496f7d0b
obsolete: rename unstable volatile set into orphan volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33772
diff
changeset
|
879 @cachefor('orphan') |
d4b7496f7d0b
obsolete: rename unstable volatile set into orphan volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33772
diff
changeset
|
880 def _computeorphanset(repo): |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
881 """the set of non obsolete revisions with obsolete parents""" |
33130
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
882 pfunc = repo.changelog.parentrevs |
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
883 mutable = _mutablerevs(repo) |
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
884 obsolete = getrevs(repo, 'obsolete') |
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
885 others = mutable - obsolete |
24928
876a2ebfbf4f
obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents:
24927
diff
changeset
|
886 unstable = set() |
33130
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
887 for r in sorted(others): |
24928
876a2ebfbf4f
obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents:
24927
diff
changeset
|
888 # A rev is unstable if one of its parent is obsolete or unstable |
876a2ebfbf4f
obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents:
24927
diff
changeset
|
889 # this works since we traverse following growing rev order |
33130
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
890 for p in pfunc(r): |
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
891 if p in obsolete or p in unstable: |
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
892 unstable.add(r) |
31ab1912678a
obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
33129
diff
changeset
|
893 break |
24928
876a2ebfbf4f
obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents:
24927
diff
changeset
|
894 return unstable |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
895 |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
896 @cachefor('suspended') |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
897 def _computesuspendedset(repo): |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
898 """the set of obsolete parents with non obsolete descendants""" |
33777
d4b7496f7d0b
obsolete: rename unstable volatile set into orphan volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33772
diff
changeset
|
899 suspended = repo.changelog.ancestors(getrevs(repo, 'orphan')) |
18276
834ef7e70d0f
performance: speedup computation of suspended revisions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18275
diff
changeset
|
900 return set(r for r in getrevs(repo, 'obsolete') if r in suspended) |
17469
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
901 |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
902 @cachefor('extinct') |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
903 def _computeextinctset(repo): |
fb72eec7efd8
obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17429
diff
changeset
|
904 """the set of obsolete parents without non obsolete descendants""" |
18277
a58260bc101f
performance: speedup computation of extinct revisions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18276
diff
changeset
|
905 return getrevs(repo, 'obsolete') - getrevs(repo, 'suspended') |
17474
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
906 |
33779
9fa874fb34e1
obsolete: rename bumped volatile set into phasedivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33778
diff
changeset
|
907 @cachefor('phasedivergent') |
9fa874fb34e1
obsolete: rename bumped volatile set into phasedivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33778
diff
changeset
|
908 def _computephasedivergentset(repo): |
17828
9495be4126ef
obsolete: add the detection of bumped changeset.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17827
diff
changeset
|
909 """the set of revs trying to obsolete public revisions""" |
20207
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
910 bumped = set() |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
20599
diff
changeset
|
911 # util function (avoid attribute lookup in the loop) |
20207
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
912 phase = repo._phasecache.phase # would be faster to grab the full list |
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
913 public = phases.public |
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
914 cl = repo.changelog |
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
915 torev = cl.nodemap.get |
35136
82680919d75e
obsolete: drop usage of changectx in '_computephasedivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
34902
diff
changeset
|
916 tonode = cl.node |
82680919d75e
obsolete: drop usage of changectx in '_computephasedivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
34902
diff
changeset
|
917 for rev in repo.revs('(not public()) and (not obsolete())'): |
20207
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
918 # We only evaluate mutable, non-obsolete revision |
35136
82680919d75e
obsolete: drop usage of changectx in '_computephasedivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
34902
diff
changeset
|
919 node = tonode(rev) |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
920 # (future) A cache of predecessors may worth if split is very common |
33735
e6d8ee3c9ec3
obsutil: rename allprecursors into allpredecessors
Boris Feld <boris.feld@octobus.net>
parents:
33734
diff
changeset
|
921 for pnode in obsutil.allpredecessors(repo.obsstore, [node], |
24927
cd0068232ec0
obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents:
24393
diff
changeset
|
922 ignoreflags=bumpedfix): |
cd0068232ec0
obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents:
24393
diff
changeset
|
923 prev = torev(pnode) # unfiltered! but so is phasecache |
cd0068232ec0
obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents:
24393
diff
changeset
|
924 if (prev is not None) and (phase(repo, prev) <= public): |
33734
0c3112f17610
obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents:
33733
diff
changeset
|
925 # we have a public predecessor |
24927
cd0068232ec0
obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents:
24393
diff
changeset
|
926 bumped.add(rev) |
cd0068232ec0
obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents:
24393
diff
changeset
|
927 break # Next draft! |
20207
cd62532c62a1
obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
20206
diff
changeset
|
928 return bumped |
17828
9495be4126ef
obsolete: add the detection of bumped changeset.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17827
diff
changeset
|
929 |
33778
f3f06c260e9e
obsolete: rename divergent volatile set into contentdivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33777
diff
changeset
|
930 @cachefor('contentdivergent') |
f3f06c260e9e
obsolete: rename divergent volatile set into contentdivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents:
33777
diff
changeset
|
931 def _computecontentdivergentset(repo): |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
932 """the set of rev that compete to be the final successors of some revision. |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
933 """ |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
934 divergent = set() |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
935 obsstore = repo.obsstore |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
936 newermap = {} |
35137
5cd6682b5a90
obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
35136
diff
changeset
|
937 tonode = repo.changelog.node |
5cd6682b5a90
obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
35136
diff
changeset
|
938 for rev in repo.revs('(not public()) - obsolete()'): |
5cd6682b5a90
obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
35136
diff
changeset
|
939 node = tonode(rev) |
5cd6682b5a90
obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
35136
diff
changeset
|
940 mark = obsstore.predecessors.get(node, ()) |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
941 toprocess = set(mark) |
24393
77eace2a63cb
obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24335
diff
changeset
|
942 seen = set() |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
943 while toprocess: |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
944 prec = toprocess.pop()[0] |
24393
77eace2a63cb
obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24335
diff
changeset
|
945 if prec in seen: |
77eace2a63cb
obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24335
diff
changeset
|
946 continue # emergency cycle hanging prevention |
77eace2a63cb
obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
24335
diff
changeset
|
947 seen.add(prec) |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
948 if prec not in newermap: |
33273
5724aaa99dd6
obsolete: pass cache argument of successors set explicitly
Boris Feld <boris.feld@octobus.net>
parents:
33251
diff
changeset
|
949 obsutil.successorssets(repo, prec, cache=newermap) |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
950 newer = [n for n in newermap[prec] if n] |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
951 if len(newer) > 1: |
35137
5cd6682b5a90
obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents:
35136
diff
changeset
|
952 divergent.add(rev) |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
953 break |
33733
d5acd967f95a
obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents:
33732
diff
changeset
|
954 toprocess.update(obsstore.predecessors.get(prec, ())) |
18070
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
955 return divergent |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
956 |
af632936d3d9
obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
18069
diff
changeset
|
957 |
32327
3546a771e376
obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Durham Goode <durham@fb.com>
parents:
32278
diff
changeset
|
958 def createmarkers(repo, relations, flag=0, date=None, metadata=None, |
3546a771e376
obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Durham Goode <durham@fb.com>
parents:
32278
diff
changeset
|
959 operation=None): |
17474
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
960 """Add obsolete markers between changesets in a repo |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
961 |
20517
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
962 <relations> must be an iterable of (<old>, (<new>, ...)[,{metadata}]) |
21024
7731a2281cf0
spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents:
20599
diff
changeset
|
963 tuple. `old` and `news` are changectx. metadata is an optional dictionary |
20517
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
964 containing metadata for this marker only. It is merged with the global |
38707
6b5ca1d0aa1e
obsolete: store user name and note in UTF-8 (issue5754) (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
38157
diff
changeset
|
965 metadata specified through the `metadata` argument of this function. |
6b5ca1d0aa1e
obsolete: store user name and note in UTF-8 (issue5754) (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
38157
diff
changeset
|
966 Any string values in metadata must be UTF-8 bytes. |
17474
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
967 |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
968 Trying to obsolete a public changeset will raise an exception. |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
969 |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
970 Current user and date are used except if specified otherwise in the |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
971 metadata attribute. |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
972 |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
973 This function operates within a transaction of its own, but does |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
974 not take any lock on the repo. |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
975 """ |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
976 # prepare metadata |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
977 if metadata is None: |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
978 metadata = {} |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
979 if 'user' not in metadata: |
38707
6b5ca1d0aa1e
obsolete: store user name and note in UTF-8 (issue5754) (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
38157
diff
changeset
|
980 luser = repo.ui.config('devel', 'user.obsmarker') or repo.ui.username() |
6b5ca1d0aa1e
obsolete: store user name and note in UTF-8 (issue5754) (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
38157
diff
changeset
|
981 metadata['user'] = encoding.fromlocal(luser) |
34388
ddcef6d0b0ef
obsolete: clean createmarkers part about operation
Boris Feld <boris.feld@octobus.net>
parents:
34379
diff
changeset
|
982 |
ddcef6d0b0ef
obsolete: clean createmarkers part about operation
Boris Feld <boris.feld@octobus.net>
parents:
34379
diff
changeset
|
983 # Operation metadata handling |
32354
f432897a9f49
obsmarker: add an experimental flag controlling "operation" recording
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32333
diff
changeset
|
984 useoperation = repo.ui.configbool('experimental', |
34863
b1e3f609bf45
config: invert evolution-related configuration aliases
Boris Feld <boris.feld@octobus.net>
parents:
34827
diff
changeset
|
985 'evolution.track-operation') |
32354
f432897a9f49
obsmarker: add an experimental flag controlling "operation" recording
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32333
diff
changeset
|
986 if useoperation and operation: |
32327
3546a771e376
obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Durham Goode <durham@fb.com>
parents:
32278
diff
changeset
|
987 metadata['operation'] = operation |
34388
ddcef6d0b0ef
obsolete: clean createmarkers part about operation
Boris Feld <boris.feld@octobus.net>
parents:
34379
diff
changeset
|
988 |
34413
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
989 # Effect flag metadata handling |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
990 saveeffectflag = repo.ui.configbool('experimental', |
34902
cc977ec0b8b9
config: also gather effect-flags on experimental.evolution
Boris Feld <boris.feld@octobus.net>
parents:
34864
diff
changeset
|
991 'evolution.effect-flags') |
34413
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
992 |
35572
8b729856ca03
obsolete: use context manager for transaction in createmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents:
35365
diff
changeset
|
993 with repo.transaction('add-obsolescence-marker') as tr: |
27984
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
994 markerargs = [] |
20517
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
995 for rel in relations: |
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
996 prec = rel[0] |
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
997 sucs = rel[1] |
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
998 localmetadata = metadata.copy() |
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
999 if 2 < len(rel): |
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
1000 localmetadata.update(rel[2]) |
2158e8f3cbd2
createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
20516
diff
changeset
|
1001 |
17474
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
1002 if not prec.mutable(): |
29389
98e8313dcd9e
i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents:
29281
diff
changeset
|
1003 raise error.Abort(_("cannot obsolete public changeset: %s") |
25412
443d4635e630
phases: add `hg help phases` hint to failures to edit public commits
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
25411
diff
changeset
|
1004 % prec, |
29976
42d031560294
obsolete: use single quotes in use warning
timeless <timeless@mozdev.org>
parents:
29894
diff
changeset
|
1005 hint="see 'hg help phases' for details") |
17474
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
1006 nprec = prec.node() |
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
1007 nsucs = tuple(s.node() for s in sucs) |
22256
3ae6cc6173e3
createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22255
diff
changeset
|
1008 npare = None |
3ae6cc6173e3
createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22255
diff
changeset
|
1009 if not nsucs: |
3ae6cc6173e3
createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22255
diff
changeset
|
1010 npare = tuple(p.node() for p in prec.parents()) |
17474
f85816af6294
obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents:
17469
diff
changeset
|
1011 if nprec in nsucs: |
29389
98e8313dcd9e
i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents:
29281
diff
changeset
|
1012 raise error.Abort(_("changeset %s cannot obsolete itself") |
98e8313dcd9e
i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents:
29281
diff
changeset
|
1013 % prec) |
27984
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1014 |
34413
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1015 # Effect flag can be different by relation |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1016 if saveeffectflag: |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1017 # The effect flag is saved in a versioned field name for future |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1018 # evolution |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1019 effectflag = obsutil.geteffectflag(rel) |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1020 localmetadata[obsutil.EFFECTFLAGFIELD] = "%d" % effectflag |
014d467f9d08
effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents:
34407
diff
changeset
|
1021 |
27984
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1022 # Creating the marker causes the hidden cache to become invalid, |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1023 # which causes recomputation when we ask for prec.parents() above. |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1024 # Resulting in n^2 behavior. So let's prepare all of the args |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1025 # first, then create the markers. |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1026 markerargs.append((nprec, nsucs, npare, localmetadata)) |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1027 |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1028 for args in markerargs: |
e60e13a86529
obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents:
27784
diff
changeset
|
1029 nprec, nsucs, npare, localmetadata = args |
22256
3ae6cc6173e3
createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
22255
diff
changeset
|
1030 repo.obsstore.create(tr, nprec, nsucs, flag, parents=npare, |
32411
08d02c1d7e67
devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents:
32387
diff
changeset
|
1031 date=date, metadata=localmetadata, |
08d02c1d7e67
devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents:
32387
diff
changeset
|
1032 ui=repo.ui) |
18101
a464deecc9dd
clfilter: add a cache on repo for set of revision to filter for a given set.
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
18070
diff
changeset
|
1033 repo.filteredrevcache.clear() |