record: add message when starting record's curses interface We are adding this log message to reduce a confusion when a command prints something just before starting the curses interface. Since the interactive mode is taking over the entire screen, starts with no delay and does wait for a key press, the user believes that messages printed before opening the interactive mode were actually printed after using interactive mode, not before. The fix adds the line "Starting interactive mode" helping the user separate the messages that were printed before and after the start of the interactive mode. One particular example where this was a problem is the revert command where we first print the list of changes to be considered for revert, then opens the curses interface right away without letting the user see the messages. The user then selects the changes, validates and then see the messages from before opening the interactive mode and is confused.

subrepo: add include/exclude support for diffing git subrepos Previously, git subrepos were ignored if any type of path selection was done. This can be solved by using subrepo status and filtering matching files.

revset: undocument wdir() until its command outputs get stable wdir() implementation is still incomplete and shouldn't be advertised to users. This patch will be backed out when - template values such as {rev} and {node} are settled - major commands and revsets work without crashing

linkrev: fix issue with annotate of working copy The introrev was appearing as None in new annotate tests, which the code from the stable branch wasn't expecting.

merge with stable

histedit: delete all non-actionclass related code Now that all actions have been migrated to the class format, we can delete all the unnecessary code that supported the old function format.

histedit: improve roll action integration with fold Previously the fold action would inspect it's class to figure out if it was a rollup or not. This was hacky. Now that finishfold is inside the fold class, let's modify it to check a function (which roll can override) to determine if it should be prompting for a commit message.

histedit: move finishfold into fold class Let's move the finishfold function into the fold class so we can modify it from the roll class more easily.

histedit: convert fold/roll actions into a class This converts the fold/roll actions into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness. The tests changed for two reasons: 1) We get a new 'empty changeset' warning because we now warn more consistently between normal histedit and --continue about commits disappearing. 2) Previously we were not putting the histedit-source extra field on the temporary fold commit during normal runs, but we were on --continue runs. By unifying these code paths we now consistently put histedit-source on the temporary fold commit, which changes some of the hashes in the backup bundles.

histedit: convert edit action into a class This converts the edit action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert message action into a class This converts the message action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert drop action into a class This converts the drop action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert pick action into a class This converts the pick action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness. The test output changed because previously pick would only report the commit disappearing if there were no merge conflicts. Now that we've unified the normal and the --continue flows, they act the same and we get the warning even after --continue.

histedit: integrate action class into flow This takes the newly added histeditaction class and integrates it into the histedit run and bootstrapcontinue logic. No actions implement the class yet, but they will be add in upcoming patches.

histedit: add a new histeditaction class This adds a new class called histeditaction. It represents a single action in a histedit plan. Future patches will integrate it into the histedit flow, then convert each existing action in to the class form one by one. This is part of a larger refactor aimed at increasing histedit robustness, maintainability, and extensibility.

histedit: allow histedit --continue when not on a descendant (BC) Previously we would not allow --continue if the current working copy parent was not a descendant of the commit produced by the previous histedit step. There's nothing really blocking us from continuing the histedit in this situation, so let's stop aborting in this case. This is technically a BC change, but it is making things more forgiving so I think it's ok. In the future we will likely add an 'exec' action to histedit, which means the user can do whatever they want during the middle of a histedit (like perhaps calling 'hg update'), which means we'll need to support this case eventually anyway.

tags: explicitly log which tags cache file is being written We now have multiple tags cache files. Record exactly which file is being written. This should help aid debugging into performance issues with any single filter.

tags: write a separate tags cache file for unfiltered repos Since we changed the format of the tags cache, we should bump the filename. Before this patch, "tags" was being used for unfiltered repositories. Change the naming scheme to be consistent and ensure that a new cache file is used. While I was here, I updated the docs to describe the existence of multiple caches. I also added explicit test coverage for the creation of the unfiltered tags cache.

tags: return empty list of heads for no .hgtags case The caller only uses the heads to resolve tags from content of .hgtags. Returning a non-empty array is pointless if there is no .hgtags file.

tags: change format of tags cache files .hgtags fnodes are now written to a shared cache file. They don't need to exist in the per-filter tags cache files. Stop writing them. The format of the tags cache file has changed in a backwards incompatible way. This should be acceptable, as we just established per-filter tags cache files and no client should have per-filter tags cache files that will need to be read. So no backwards compatbility concern is present. The new format has a single header line followed by resolved tags entries. The header line is similar to the old first line with a major difference: we now compute and store a hash of the filtered revisions. Before, if the set of filtered revs changed, we may return incorrect results. We now detect that. A test for verifying filtered rev change is handled properly has been added.

tags: don't read .hgtags fnodes from tags cache files Now that we have a standalone and shared cache for .hgtags fnodes, the .hgtags fnodes stored in the tags cache files are redundant. Upcoming patches will change the format of the tags cache files to remove this data and to improve the validation. To prepare for this, we change the tags reading code to ignore all but the tip .hgtags fnodes entries in existing tags cache files. All fnodes lookups now go through our new shared cache, which is why test output changed. Format of tags cache files has not yet changed.

rebase: restore bookmark state on abort The bookmark state was already being preserved, but it wasn't being properly restored.

histedit: store backup file before histedit It's possible for the user to delete some of the commits they started with during a histedit, and aborting the histedit doesn't bring them back. Let's store a backup bundle so we can always recover the stack of commits from before they began.

histedit: replace pickle with custom serialization Pickle is undesirable, so let's serialize it ourselves. We keep the ability to parse existing pickle blobs for now.

devel-warn: add a prefix to all messages ("devel-warn: ") We want to make the origin and importance of the message clear to developers.

push: acquire local 'wlock' if "pushback" is expected (BC) (issue4596) If the client allows "pushback", the bundle2 served back by the server may contains parts that will write to the repository. Such parts may require the 'wlock' (eg: bookmark) so we acquire it in advance to make sure it got acquired before the 'lock'.

annotate: always prepare ancestry context of base fctx (issue4600) This patch extends the workaround introduced by dd01834a696f. Even if the base fctx is the same as intorrev, _ancestrycontext must be built for faster _changeid lookup. repo: https://hg.mozilla.org/releases/mozilla-beta command: hg annotate -r 4954faa47dd0 gfx/thebes/gfxWindowsPlatform.cpp before: 52.450 sec after: 1.820 sec

unbundle: acquire 'wlock' when processing bundle2 (BC) (issue4596) A bundle2 may contain bookmark updates (or other extension content) that requires the 'wlock' to be written. As 'wlock' must be acquired before 'lock', we must stay on the side of caution and use both in all case to ensure their ordering.

run-test: enable the devel warning during tests This should help us to catch new locking order issues as soon as possible. There are two harmless test updates (from the config change). Moreover, some bundle2 tests are displaying warning for a legitimate reason. The use of pushkey during the unbundle process may requires the 'wlock' after 'lock' (around the whole unbundle process was taken). This is non-trivial to fix, so I better have the check on, with the warning in the test than the check off. See issue4596 for details.

wlock: do not warn for non-wait locking We are warning about lock acquired in the wrong order because this can create dead-lock situation. But non-wait acquisition will not block and therefore not create a dead-lock. So we do not need to wait in such case.

develwarn: include call site in the simple message version Just displaying the warning makes it quite hard to recognise the guilty code quickly and using --traceback for all calls is not very convenient. So we include the call site with all simple message to help developer to recognise errors sources.

develwarn: handle the end of line inside the function itself The traceback version should not have a end of line at all. The non-traceback version will requires the same things soon.

develwarn: refactor the developer warning logic The logic is currently duplicated and we plan to make it a bit smarter. So we move it into a function first to make the update more robust and simple.

lock: update the docstring with order information Lock must be acquired in a specific order to avoid dead-lock. This was documented on the wiki, but having this information in the docstring is also useful.

wlock: reword the devel warning We change the wording of the developer warning: - "lock" taken before "wlock" + "wlock" acquired after "lock" The goals here are to: - Put the "subject" as the first word, - use "acquired" instead of "taken" since it seems more accurate.

wlock: only issue devel warning when actually acquiring the lock Before this change, any call to 'wlock' after we acquired a 'lock' was issuing a warning. This is wrong as the 'wlock' have been properly acquired before the 'lock' in a previous call. We move the warning code to only issue such warnings when we are acquiring the 'wlock' -after- acquiring 'lock'. This is the expected behavior of this warning from the start.

bundle2: flush output in a part in all cases We want to preserve output even when the unbundling fails (eg: hook output). So we must make sure that everything we have is flushed into the reply bundle. (This is related to issue4594)

run-tests: also follow symlink when update PATH with 'run-tests.py' dir I'm using 'run-tests.py' from my '$PATH' and use a symlink to get 'run-tests.py' to in that '$PATH'. There is a handful of test helpers (like f) that needs to be in the '$PATH' for the test to run, and they are expected to live next to the 'run-tests.py' binary. Using a symlink confuses this logic, so we add to the '$PATH' both the 'run-tests.py' executable directory, and the actual file location direction.

bundle2: fix names for error part handler The name is not very important but copy paste banshee got there. We make distinctive name.

transaction: introduce a transaction ID, to be available to all hooks The transaction ID is built from the object ID and creation time stamp to make sure it is unique.

transaction: actually use tr.hookargs in pretxnclose The 'tr.hookargs' is a dictionary containing all the arguments to be passed to hooks. It is actually used for hooks now...

bundle2: use unbundle source as transaction name This is what a bundle 1 push is doing. This got caught by trying to run the whole test-suite using bundle2 for exchanges.

tags: have a different cache file per filter level Currently whichever filter level asks for tags last will write the data on disk. This create massive issues when tags are read for "visible" and "unfiltered" on large and multi headed repository (like Mozilla central). See issue4550 for details Each filter level recomputes its own cache without direct collaboration but they all share the same 'hgtagsfnodes' cache. And that is where most of the time is spent.

parsers: when available, use a presized dictionary for the file foldmap On a repo with over 300,000 files, this speeds up perffilefoldmap: before: wall 0.178421 comb 0.180000 user 0.160000 sys 0.020000 (best of 55) after: wall 0.164462 comb 0.160000 user 0.140000 sys 0.020000 (best of 59)

tags: extract .hgtags filenodes cache to a standalone file Resolution of .hgtags filenodes values has historically been a performance pain point for large repositories, where reading individual manifests can take over 100ms. Multiplied by hundreds or even thousands of heads and resolving .hgtags filenodes becomes a performance issue. This patch establishes a standalone cache file holding the .hgtags filenodes for each changeset. After this patch, the .hgtags filenode for any particular changeset should only have to be computed once during the lifetime of the repository. The introduced hgtagsfnodes1 cache file is modeled after the rev branch cache: the cache is effectively an array of entries consisting of a changeset fragment and the filenode for a revision. The file grows in proportion to the length of the repository (24 bytes per changeset) and is truncated when the repository is stripped. The file is not written unless tag info is requested and tags have changed since last time. This patch partially addresses issue4550. Future patches will split the "tags" cache file into per-filter files and will refactor the cache format to not capture the .hgtags fnodes, as these are now stored in the hgtagsfnodes1 cache. This patch is capable of standing alone. We should not have to wait on the tags cache filter split and format refactor for this patch to land.

extensions: extract partial application into a bind() function We use partial function application for wrapping existing Mercurial functions, and it's implemented separately each time. This patch extracts the partial application into a new bind() function that can be used on its own by extensions when appropriate. In particular, the evolve extension needs to wrap functions in the various bundle2 processing dictionaries, which the pre-existing methods don't support.

obsolete: experimental flag to get debug about obsmarkers exchange The obsolescence markers exchange is still experimental. We (developer) need more information about what is going on. I'm adding an experimental flag to add display the amount of data exchanged during bundle2 exchanges.

bundle2: add an 'idx' argument to the 'getbundle2partsgenerator' This argument let extensions control in what order bundle2 part are generated server side during a pull. This is useful to ensure the transaction is in a proper state before some actions or hooks happens.

bundle2: add an 'idx' argument to the 'b2partsgenerator' This argument let extensions control in what order bundle2 part are generated client side during a push. This is useful to ensure the transaction is in a proper state before some actions or hooks happens.

rpms: create missing builds dir if it doesn't exist

rpms: for packages with their own python, put it in /opt/python-hg This is more compliant with the FHS and Fedora packaging guidelines.

branchcache: stay silent if failing to read cache files The warning has in some cases incorrectly attributed unrelated problems to rbc. Instead, just do like the branch head cache does and stay quiet when reading fails. The cache will be missing the first time a repo is used. It is a normal situation and there is no reason to make a note of that.

merge: better debug messages before/after invoking external merge tool

subrepo: use vfs.walk instead of os.walk "dirpath" in the tuple yielded by "vfs.walk()" is relative one from the root of specified vfs, and absolute path in the warning message is composed by "vfs.join()". On the other hand, target file "f" exists in "dirpath", and "reljoin()" is needed to unlink "f" by "vfs.unlink()".

vfs: add walk To eliminate "path prefix" (= "the root of vfs") part from "dirpath" yielded by "os.walk()" correctly, "path prefix" should have "os.sep" at the end of own string, but it isn't easy to ensure it, because: - examination by "path.endswith(os.sep)" isn't portable Some problematic encodings use 0x5c (= "os.sep" on Windows) as the tail byte of some multi-byte characters. - "os.path.join(path, '')" isn't portable With Python 2.7.9, this invocation doesn't add "os.sep" at the end of UNC path (see issue4557 for detail). Python 2.7.9 changed also behavior of "os.path.normpath()" (see *) and "os.path.splitdrive()" for UNC path. vfs root normpath splitdrive os.sep required =============== ============== =================== ============ z:\ z:\ z: + \ no z:\foo z:\foo z: + \foo yes z:\foo\ z:\foo z: + \foo yes [before Python 2.7.9] \\foo\bar \\foo\bar '' + \\foo\bar yes \\foo\bar\ \\foo\bar (*) '' + \\foo\bar yes \\foo\bar\baz \\foo\bar\baz '' + \\foo\bar\baz yes \\foo\bar\baz\ \\foo\bar\baz '' + \\foo\bar\baz yes [Python 2.7.9] \\foo\bar \\foo\bar \\foo\bar + '' yes \\foo\bar\ \\foo\bar\ (*) \\foo\bar + \ no \\foo\bar\baz \\foo\bar\baz \\foo\bar + \baz yes \\foo\bar\baz\ \\foo\bar\baz \\foo\bar + \baz yes If it is ensured that "normpath()"-ed vfs root is passed to "splitdrive()", adding "os.sep" is required only when "path" part of "splitdrive()" result isn't "os.sep" itself. This is just what "pathutil.nameasprefix()" examines. This patch applies "os.path.normpath()" on "self.join(None)" explicitly, because it isn't ensured that vfs root is already normalized: vfs itself is constructed with "realpath=False" (= avoid normalizing in "vfs.__init__()") in many code paths. This normalization should be much cheaper than subsequent file I/O for directory traversal.

subrepo: pass wvfs to _sanitize instead of absolute path to a subrepository As a preparation for vfs migration of "_sanitize()", this patch passes "wvfs" to "_sanitize()" and use "wvfs.base" instead of absolute path to a subrepository.