tests: fix up whitespace complaint

tests: fix test-tags on vfat The unix conditional section wasn't cleaning up sufficiently.

tests: ignore sha512 flag in bundle2 for py2.4

bundle2-localpeer: properly propagate the server output on error (issue4594) In case of errors, output parts salvaged from the reply bundle need to be processed for outputting their content. This concludes our quest for fixing issue4594.

bundle2: add on more layer of exception catching in localrepo.unbundle We are going to add output related logic in this function. We do the indentation first to help next changeset readability. We need a new try except because we want to handle output on any exception, including PushRaced ones.

bundle2-wireproto: properly propagate the server output on error (issue4594) In case of errors, output parts salvaged from the reply bundle are re-injected into the bundle carrying the exception. We still need to fix the situation for non-wireprotocol push.

bundle2: refactor error bundle creation for the wireprotocol We want to add output information to the error bundle. Before doing this, we rework the code to have a single bundler creation and return statement. This will make the update with the output simpler as only one place will have to be touched.

bundle2: store the salvaged output on the exception object The re-handling of output is happening in some 'unbundle' callers. We have to transmit the output information to this place so we stick it on the exception. This is the third step in our quest for preserving the server output on error (issue4594). We want to be able to copy the output part from the aborted reply into the exception bundle.

bundle2: add a 'salvageoutput' method on bundle20 This method returns a copy of all 'output' parts added to the bundler. This is the second step in our quest for preserving the server output on error (issue4594). We want to be able to copy the output parts from the aborted reply into the exception bundle. The function will be used in a later patch.

bundle2: add a 'copy' method on parts This is the first step in our quest for preserving the server output on error (issue4594). We want to be able to copy the output parts from the aborted reply into the exception bundle. The function will be used in a later patch.

hooks: add a 'txnabort' hook This hook will be called whenever a transaction is aborted. This will make it easy for people to clean up temporary content they may have created during a transaction.

help: document the ''HG_TXNID'' environment variable during hooks We forgot to document the new "transaction ID" mechanism.

match: add a subclass for dirstate normalizing of the matched patterns This class is only needed on case insensitive filesystems, and only for wdir context matches. It allows the user to not match the case of the items in the filesystem- especially for naming directories, which dirstate doesn't handle[1]. Making dirstate handle mismatched directory cases is too expensive[2]. Since dirstate doesn't apply to committed csets, this is only created by overriding basectx.match() in workingctx, and only on icasefs. The default arguments have been dropped, because the ctx must be passed to the matcher in order to function. For operations that can apply to both wdir and some other context, this ends up normalizing the filename to the case as it exists in the filesystem, and using that case for the lookup in the other context. See the diff example in the test. Previously, given a directory with an inexact case: - add worked as expected - diff, forget and status would silently ignore the request - files would exit with 1 - commit, revert and remove would fail (even when the commands leading up to them worked): $ hg ci -m "AbCDef" capsdir1/capsdir abort: CapsDir1/CapsDir: no match under directory! $ hg revert -r '.^' capsdir1/capsdir capsdir1\capsdir: no such file in rev 64dae27060b7 $ hg remove capsdir1/capsdir not removing capsdir1\capsdir: no tracked files [1] Globs are normalized, so that the -I and -X don't need to be specified with a case match. Without that, the second last remove (with -X) removes the files, leaving nothing for the last remove. However, specifying the files as 'glob:**.Txt' does not work. Perhaps this requires 're.IGNORECASE'? There are only a handful of places that create matchers directly, instead of being routed through the context.match() method. Some may benefit from changing over to using ctx.match() as a factory function: revset.checkstatus() revset.contains() revset.filelog() revset._matchfiles() localrepository._loadfilter() ignore.ignore() fileset.subrepo() filemerge._picktool() overrides.addlargefiles() lfcommands.lfconvert() kwtemplate.__init__() eolfile.__init__() eolfile.checkrev() acl.buildmatch() Currently, a toplevel subrepo can be named with an inexact case. However, the path auditor gets in the way of naming _anything_ in the subrepo if the top level case doesn't match. That is trickier to handle, because there's the user provided case, the case in the filesystem, and the case stored in .hgsub. This can be fixed next cycle. --- a/tests/test-subrepo-deep-nested-change.t +++ b/tests/test-subrepo-deep-nested-change.t @@ -170,8 +170,15 @@ R sub1/sub2/test.txt $ hg update -Cq $ touch sub1/sub2/folder/bar +#if icasefs + $ hg addremove Sub1/sub2 + abort: path 'Sub1\sub2' is inside nested repo 'Sub1' + [255] + $ hg -q addremove sub1/sub2 +#else $ hg addremove sub1/sub2 adding sub1/sub2/folder/bar (glob) +#endif $ hg status -S A sub1/sub2/folder/bar ? foo/bar/abc The narrowmatcher class may need to be tweaked when that is fixed. [1] http://www.selenic.com/pipermail/mercurial-devel/2015-April/068183.html [2] http://www.selenic.com/pipermail/mercurial-devel/2015-April/068191.html

match: move _normalize() into the match class This will be overridden in an upcoming patch to also deal with dirstate normalization on case insensitive filesystems.

largefiles: always consider updatelfiles 'checked' parameter set mergeupdate already set the flag to update all. This will thus only change overriderevert and scmutilmarktouched ... where the flag effectually also were true. The test coverage thus shows no change. As the flag always is set, it is removed. This is mainly a change for keeping the code simple and consistent and correct, but it should also make it faster in many cases.

largefiles: for update -C, only update largefiles when necessary Before, a --clean update with largefiles would use the "optimization" that it didn't read hashes from standin files before and after the update. Instead of trusting the content of the standin files, it would rehash all the actual largefiles that lfdirstate reported clean and update the standins that didn't have the expected content. It could thus in some "impossible" situations automatically recover from some "largefile got out sync with its standin" issues (even there apparently still were weird corner cases where it could fail). This extra checking is similar to what core --clean intentionally do not do, and it made update --clean unbearable slow. Usually in core Mercurial, --clean will rely on the dirstate to find the files it should update. (It is thus intentionally possible (when trying to trick the system or if there should be bugs) to end up in situations where --clean not will restore the working directory content correctly.) Checking every file when we "know" it is ok is however not an option - that would be too slow. Instead, trust the content of the standin files. Use the same logic for --clean as for linear updates and trust the dirstate and that our "logic" will keep them in sync. It is much cheaper to just rehash the largefiles reported dirty by a status walk and read all standins than to hash largefiles. Most of the changes are just a change of indentation now when the different kinds of updates no longer are handled that differently. Standins for added files are however only written when doing a normal update, while deleted and removed files only will be updated for --clean updates.

subrepo: calculate _relpath for hgsubrepo based on self instead of parent Prior to 105758d1b37b, the subrelpath() (now _relpath) for hgsubrepo was calculated by removing the root path of the outermost repo from the root path of the subrepo. Since the root paths use platform specific separators, and the relative path is printed by various commands, the output of these commands require a glob (and check-code.py enforces this). In an effort to be generic to all subrepos, 105758d1b37b started calculating this path based on the parent repo, and then joining the subrepo path in .hgsub. One of the tests in test-subrepo.t creates a subrepo inside a directory, so the path being joined contained '/' instead of '\'. This made the test fail with a '~' status, because the glob is unnecessary[1]. Removing them made the test work, but then check-code complains. We can't just drop the check-code rule, because sub-subrepos are still joined with '\'. Presumably the other subrepo types have this issue as well, but there likely isn't a test with git or svn repos inside a subdirectory. This simply restores the exact _relpath value (and output) for hgsubrepos prior to 105758d1b37b. [1] http://www.selenic.com/pipermail/mercurial-devel/2015-April/068720.html

subrepo: backout 93b0e0db7929 to restore reporelpath() The path for hgsubrepo needs to be calculated slightly differently from other subrepo types, but can reuse this. See the next patch for details.

rollback: clear resolve state (issue4593)

diff: pass the diff matcher to the copy logic This passes the existing diff matcher instance down to the copy logic so copy tracing can be more efficient when possible and only trace copies for matching files. This only actually affects forwardcopies (i.e. Given A<-B<-C<-D, it works for 'hg diff -r B -r D foo.txt', but not for 'hg diff -r D -r B foo.txt') since backward copies require walking all histories, and not just the individual file's. This reduces 'hg diff -r A -r B foo.txt' time from 15s to 1s when A and B have 80,000 files different.

copies: add matcher parameter to copy logic This allows passing a matcher down the pathcopies() stack to _forwardcopies(). This will let us add logic in a later patch to avoid tracing copies when not necessary (like when doing hg diff -r 1 -r 2 foo.txt).

treemanifest: extract parse method from constructor When we start to lazily load submanifests, it will be useful to be able to create an treemanifest instance before manifest data gets parsed into it. To prepare for this, extract the parsing code from treemanifest's constructor to a separate method.

manifest: duplicate call to addrevision() When we start writing submanifests to their own revlogs, we will not want to write a new revision for a directory if there were no changes to it. To prepare for this, duplicate the call to addrevision() and move them earlier where they can more easily be avoided.

record: add message when starting record's curses interface We are adding this log message to reduce a confusion when a command prints something just before starting the curses interface. Since the interactive mode is taking over the entire screen, starts with no delay and does wait for a key press, the user believes that messages printed before opening the interactive mode were actually printed after using interactive mode, not before. The fix adds the line "Starting interactive mode" helping the user separate the messages that were printed before and after the start of the interactive mode. One particular example where this was a problem is the revert command where we first print the list of changes to be considered for revert, then opens the curses interface right away without letting the user see the messages. The user then selects the changes, validates and then see the messages from before opening the interactive mode and is confused.

subrepo: add include/exclude support for diffing git subrepos Previously, git subrepos were ignored if any type of path selection was done. This can be solved by using subrepo status and filtering matching files.

revset: undocument wdir() until its command outputs get stable wdir() implementation is still incomplete and shouldn't be advertised to users. This patch will be backed out when - template values such as {rev} and {node} are settled - major commands and revsets work without crashing

linkrev: fix issue with annotate of working copy The introrev was appearing as None in new annotate tests, which the code from the stable branch wasn't expecting.

merge with stable

histedit: delete all non-actionclass related code Now that all actions have been migrated to the class format, we can delete all the unnecessary code that supported the old function format.

histedit: improve roll action integration with fold Previously the fold action would inspect it's class to figure out if it was a rollup or not. This was hacky. Now that finishfold is inside the fold class, let's modify it to check a function (which roll can override) to determine if it should be prompting for a commit message.

histedit: move finishfold into fold class Let's move the finishfold function into the fold class so we can modify it from the roll class more easily.

histedit: convert fold/roll actions into a class This converts the fold/roll actions into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness. The tests changed for two reasons: 1) We get a new 'empty changeset' warning because we now warn more consistently between normal histedit and --continue about commits disappearing. 2) Previously we were not putting the histedit-source extra field on the temporary fold commit during normal runs, but we were on --continue runs. By unifying these code paths we now consistently put histedit-source on the temporary fold commit, which changes some of the hashes in the backup bundles.

histedit: convert edit action into a class This converts the edit action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert message action into a class This converts the message action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert drop action into a class This converts the drop action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert pick action into a class This converts the pick action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness. The test output changed because previously pick would only report the commit disappearing if there were no merge conflicts. Now that we've unified the normal and the --continue flows, they act the same and we get the warning even after --continue.

histedit: integrate action class into flow This takes the newly added histeditaction class and integrates it into the histedit run and bootstrapcontinue logic. No actions implement the class yet, but they will be add in upcoming patches.

histedit: add a new histeditaction class This adds a new class called histeditaction. It represents a single action in a histedit plan. Future patches will integrate it into the histedit flow, then convert each existing action in to the class form one by one. This is part of a larger refactor aimed at increasing histedit robustness, maintainability, and extensibility.

histedit: allow histedit --continue when not on a descendant (BC) Previously we would not allow --continue if the current working copy parent was not a descendant of the commit produced by the previous histedit step. There's nothing really blocking us from continuing the histedit in this situation, so let's stop aborting in this case. This is technically a BC change, but it is making things more forgiving so I think it's ok. In the future we will likely add an 'exec' action to histedit, which means the user can do whatever they want during the middle of a histedit (like perhaps calling 'hg update'), which means we'll need to support this case eventually anyway.

tags: explicitly log which tags cache file is being written We now have multiple tags cache files. Record exactly which file is being written. This should help aid debugging into performance issues with any single filter.

tags: write a separate tags cache file for unfiltered repos Since we changed the format of the tags cache, we should bump the filename. Before this patch, "tags" was being used for unfiltered repositories. Change the naming scheme to be consistent and ensure that a new cache file is used. While I was here, I updated the docs to describe the existence of multiple caches. I also added explicit test coverage for the creation of the unfiltered tags cache.

tags: return empty list of heads for no .hgtags case The caller only uses the heads to resolve tags from content of .hgtags. Returning a non-empty array is pointless if there is no .hgtags file.

tags: change format of tags cache files .hgtags fnodes are now written to a shared cache file. They don't need to exist in the per-filter tags cache files. Stop writing them. The format of the tags cache file has changed in a backwards incompatible way. This should be acceptable, as we just established per-filter tags cache files and no client should have per-filter tags cache files that will need to be read. So no backwards compatbility concern is present. The new format has a single header line followed by resolved tags entries. The header line is similar to the old first line with a major difference: we now compute and store a hash of the filtered revisions. Before, if the set of filtered revs changed, we may return incorrect results. We now detect that. A test for verifying filtered rev change is handled properly has been added.

tags: don't read .hgtags fnodes from tags cache files Now that we have a standalone and shared cache for .hgtags fnodes, the .hgtags fnodes stored in the tags cache files are redundant. Upcoming patches will change the format of the tags cache files to remove this data and to improve the validation. To prepare for this, we change the tags reading code to ignore all but the tip .hgtags fnodes entries in existing tags cache files. All fnodes lookups now go through our new shared cache, which is why test output changed. Format of tags cache files has not yet changed.

rebase: restore bookmark state on abort The bookmark state was already being preserved, but it wasn't being properly restored.

histedit: store backup file before histedit It's possible for the user to delete some of the commits they started with during a histedit, and aborting the histedit doesn't bring them back. Let's store a backup bundle so we can always recover the stack of commits from before they began.

histedit: replace pickle with custom serialization Pickle is undesirable, so let's serialize it ourselves. We keep the ability to parse existing pickle blobs for now.

devel-warn: add a prefix to all messages ("devel-warn: ") We want to make the origin and importance of the message clear to developers.

push: acquire local 'wlock' if "pushback" is expected (BC) (issue4596) If the client allows "pushback", the bundle2 served back by the server may contains parts that will write to the repository. Such parts may require the 'wlock' (eg: bookmark) so we acquire it in advance to make sure it got acquired before the 'lock'.

annotate: always prepare ancestry context of base fctx (issue4600) This patch extends the workaround introduced by dd01834a696f. Even if the base fctx is the same as intorrev, _ancestrycontext must be built for faster _changeid lookup. repo: https://hg.mozilla.org/releases/mozilla-beta command: hg annotate -r 4954faa47dd0 gfx/thebes/gfxWindowsPlatform.cpp before: 52.450 sec after: 1.820 sec

unbundle: acquire 'wlock' when processing bundle2 (BC) (issue4596) A bundle2 may contain bookmark updates (or other extension content) that requires the 'wlock' to be written. As 'wlock' must be acquired before 'lock', we must stay on the side of caution and use both in all case to ensure their ordering.

run-test: enable the devel warning during tests This should help us to catch new locking order issues as soon as possible. There are two harmless test updates (from the config change). Moreover, some bundle2 tests are displaying warning for a legitimate reason. The use of pushkey during the unbundle process may requires the 'wlock' after 'lock' (around the whole unbundle process was taken). This is non-trivial to fix, so I better have the check on, with the warning in the test than the check off. See issue4596 for details.

wlock: do not warn for non-wait locking We are warning about lock acquired in the wrong order because this can create dead-lock situation. But non-wait acquisition will not block and therefore not create a dead-lock. So we do not need to wait in such case.

develwarn: include call site in the simple message version Just displaying the warning makes it quite hard to recognise the guilty code quickly and using --traceback for all calls is not very convenient. So we include the call site with all simple message to help developer to recognise errors sources.

develwarn: handle the end of line inside the function itself The traceback version should not have a end of line at all. The non-traceback version will requires the same things soon.

develwarn: refactor the developer warning logic The logic is currently duplicated and we plan to make it a bit smarter. So we move it into a function first to make the update more robust and simple.

lock: update the docstring with order information Lock must be acquired in a specific order to avoid dead-lock. This was documented on the wiki, but having this information in the docstring is also useful.

wlock: reword the devel warning We change the wording of the developer warning: - "lock" taken before "wlock" + "wlock" acquired after "lock" The goals here are to: - Put the "subject" as the first word, - use "acquired" instead of "taken" since it seems more accurate.

wlock: only issue devel warning when actually acquiring the lock Before this change, any call to 'wlock' after we acquired a 'lock' was issuing a warning. This is wrong as the 'wlock' have been properly acquired before the 'lock' in a previous call. We move the warning code to only issue such warnings when we are acquiring the 'wlock' -after- acquiring 'lock'. This is the expected behavior of this warning from the start.

bundle2: flush output in a part in all cases We want to preserve output even when the unbundling fails (eg: hook output). So we must make sure that everything we have is flushed into the reply bundle. (This is related to issue4594)

run-tests: also follow symlink when update PATH with 'run-tests.py' dir I'm using 'run-tests.py' from my '$PATH' and use a symlink to get 'run-tests.py' to in that '$PATH'. There is a handful of test helpers (like f) that needs to be in the '$PATH' for the test to run, and they are expected to live next to the 'run-tests.py' binary. Using a symlink confuses this logic, so we add to the '$PATH' both the 'run-tests.py' executable directory, and the actual file location direction.

bundle2: fix names for error part handler The name is not very important but copy paste banshee got there. We make distinctive name.

transaction: introduce a transaction ID, to be available to all hooks The transaction ID is built from the object ID and creation time stamp to make sure it is unique.

transaction: actually use tr.hookargs in pretxnclose The 'tr.hookargs' is a dictionary containing all the arguments to be passed to hooks. It is actually used for hooks now...