annotate mercurial/minifileset.py @ 46800:186c0f6fbc16

tests: ask any chg instance to terminate before looking at sqlite dbs There are spurious errors in CI where the database is still locked, so force the daemon to quit to get deterministic behavior. Since the kill command itself is racy, also sleep 2s to give the server time to wake up and exit. Differential Revision: https://phab.mercurial-scm.org/D10244
author Joerg Sonnenberger <joerg@bec.de>
date Sat, 20 Mar 2021 00:37:57 +0100
parents 687b865b95ad
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
1 # minifileset.py - a simple language to select files
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
2 #
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
3 # Copyright 2017 Facebook, Inc.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
4 #
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
7
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
8 from __future__ import absolute_import
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
9
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
10 from .i18n import _
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
11 from . import (
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
12 error,
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
13 fileset,
38805
b9162ea1b815 fileset: extract language processing part to new module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 38804
diff changeset
14 filesetlang,
37876
9c98cb30f4de minifileset: fix on Python 3
Augie Fackler <augie@google.com>
parents: 35800
diff changeset
15 pycompat,
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
16 )
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
17
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
18
38687
1500cbe22d53 fileset: parse argument of size() by predicate function
Yuya Nishihara <yuya@tcha.org>
parents: 37876
diff changeset
19 def _sizep(x):
1500cbe22d53 fileset: parse argument of size() by predicate function
Yuya Nishihara <yuya@tcha.org>
parents: 37876
diff changeset
20 # i18n: "size" is a keyword
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
21 expr = filesetlang.getstring(x, _(b"size requires an expression"))
38687
1500cbe22d53 fileset: parse argument of size() by predicate function
Yuya Nishihara <yuya@tcha.org>
parents: 37876
diff changeset
22 return fileset.sizematcher(expr)
1500cbe22d53 fileset: parse argument of size() by predicate function
Yuya Nishihara <yuya@tcha.org>
parents: 37876
diff changeset
23
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
24
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
25 def _compile(tree):
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
26 if not tree:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
27 raise error.ParseError(_(b"missing argument"))
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
28 op = tree[0]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
29 if op == b'withstatus':
38879
e79a69af1593 fileset: insert hints where status should be computed
Yuya Nishihara <yuya@tcha.org>
parents: 38865
diff changeset
30 return _compile(tree[1])
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
31 elif op in {b'symbol', b'string', b'kindpat'}:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
32 name = filesetlang.getpattern(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
33 tree, {b'path'}, _(b'invalid file pattern')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
34 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
35 if name.startswith(b'**'): # file extension test, ex. "**.tar.gz"
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
36 ext = name[2:]
37876
9c98cb30f4de minifileset: fix on Python 3
Augie Fackler <augie@google.com>
parents: 35800
diff changeset
37 for c in pycompat.bytestr(ext):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
38 if c in b'*{}[]?/\\':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
39 raise error.ParseError(_(b'reserved character: %s') % c)
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
40 return lambda n, s: n.endswith(ext)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
41 elif name.startswith(b'path:'): # directory or full path test
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
42 p = name[5:] # prefix
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
43 pl = len(p)
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
44 f = lambda n, s: n.startswith(p) and (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
45 len(n) == pl or n[pl : pl + 1] == b'/'
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
46 )
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
47 return f
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
48 raise error.ParseError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
49 _(b"unsupported file pattern: %s") % name,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
50 hint=_(b'paths must be prefixed with "path:"'),
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
51 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
52 elif op in {b'or', b'patterns'}:
38804
d82c4d42b615 fileset: flatten 'or' nodes to unnest unionmatchers
Yuya Nishihara <yuya@tcha.org>
parents: 38687
diff changeset
53 funcs = [_compile(x) for x in tree[1:]]
d82c4d42b615 fileset: flatten 'or' nodes to unnest unionmatchers
Yuya Nishihara <yuya@tcha.org>
parents: 38687
diff changeset
54 return lambda n, s: any(f(n, s) for f in funcs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
55 elif op == b'and':
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
56 func1 = _compile(tree[1])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
57 func2 = _compile(tree[2])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
58 return lambda n, s: func1(n, s) and func2(n, s)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
59 elif op == b'not':
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
60 return lambda n, s: not _compile(tree[1])(n, s)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
61 elif op == b'func':
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
62 symbols = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
63 b'all': lambda n, s: True,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
64 b'none': lambda n, s: False,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
65 b'size': lambda n, s: _sizep(tree[2])(s),
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
66 }
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
67
38805
b9162ea1b815 fileset: extract language processing part to new module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 38804
diff changeset
68 name = filesetlang.getsymbol(tree[1])
35691
735f47b41521 fileset: make it robust for bad function calls
Yuya Nishihara <yuya@tcha.org>
parents: 35616
diff changeset
69 if name in symbols:
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
70 return symbols[name]
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
71
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
72 raise error.UnknownIdentifier(name, symbols.keys())
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
73 elif op == b'minus': # equivalent to 'x and not y'
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
74 func1 = _compile(tree[1])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
75 func2 = _compile(tree[2])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
76 return lambda n, s: func1(n, s) and not func2(n, s)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
77 elif op == b'list':
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
78 raise error.ParseError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
79 _(b"can't use a list in this context"),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
80 hint=_(b'see \'hg help "filesets.x or y"\''),
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
81 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
82 raise error.ProgrammingError(b'illegal tree: %r' % (tree,))
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
83
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 38879
diff changeset
84
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
85 def compile(text):
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
86 """generate a function (path, size) -> bool from filter specification.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
87
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
88 "text" could contain the operators defined by the fileset language for
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
89 common logic operations, and parenthesis for grouping. The supported path
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
90 tests are '**.extname' for file extension test, and '"path:dir/subdir"'
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
91 for prefix test. The ``size()`` predicate is borrowed from filesets to test
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
92 file size. The predicates ``all()`` and ``none()`` are also supported.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
93
35741
73432eee0ac4 fileset: add kind:pat operator
Yuya Nishihara <yuya@tcha.org>
parents: 35740
diff changeset
94 '(**.php & size(">10MB")) | **.zip | (path:bin & !path:bin/README)' for
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
95 example, will catch all php files whose size is greater than 10 MB, all
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
96 files whose name ends with ".zip", and all files under "bin" in the repo
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
97 root except for "bin/README".
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
98 """
38805
b9162ea1b815 fileset: extract language processing part to new module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 38804
diff changeset
99 tree = filesetlang.parse(text)
38826
6371ab78c3b3 fileset: add phase to transform parsed tree
Yuya Nishihara <yuya@tcha.org>
parents: 38810
diff changeset
100 tree = filesetlang.analyze(tree)
38829
7e7e2b2ff284 fileset: add stub for weight-based optimization
Yuya Nishihara <yuya@tcha.org>
parents: 38828
diff changeset
101 tree = filesetlang.optimize(tree)
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
102 return _compile(tree)