Added tag 3.5.2 for changeset 9a466b9f9792

largefiles: restore archiving largefiles with hgweb (issue4859) This regressed in 7699d3212994, when trying to conditionally disable archiving of largefiles. I'm not sure if wrapfunction() is the right way to do this, but it seems to work. The mysterious issue with lfstatus getting out of sync in the proxy and the unfiltered view crops up again here. See the referenced cset for more info.

i18n-pt_BR: synchronized with f31ddc9bfa5f

help: hide deprecated filesets, revsets and template items if not verbose This allows us to hide {branches} template keyword in a better way. Currently it is achieved by copying keywords table and deleting 'branches' from it.

help: pass around ui to rewriter hooks (API) makeitemsdoc() will hide DEPRECATED items conditionally.

help: pass around ui to doc loader (API) This is necessary to hide DEPRECATED items conditionally. Flagged as API change because it will break "hg help git|subversion".

gendoc: use real ui in place of stdout ui attributes will be required by a help function, so a file object can't be used as a fake ui.

check-seclevel: use ui to show status and error messages Future patches will require ui module to be passed to a help function, so let's use it where appropriate. Additional parens are necessary to silence a check-code warning.

unbundle: test and fix for clean abort on unknown bundle2 feature This introduce a test for the feature and fix the exception name that is different on default.

merge with stable

localrepo: recreate phasecache if changelog was modified (issue4855) Because _phaserevs and _phasesets cache revision numbers, they must be invalidated if there are new commits or stripped revisions. We could do that by calling _phasecache.invalidate(), but it wasn't simple to be integrated with the filecache mechanism. So for now, phasecache will be recreated after repo.invalidate() if 00changelog.i was modified before.

progress: force a repaint of a printed progress bar after a clear() This avoids some visual flickering of the progress bar in convert and probably some other operations. Previously, a line of output would erase the progress bar, and then it would wait `progress.refresh` seconds (default of 0.1) before redrawing the progress bar. Now if we've ever painted a progress bar, we schedule the progress bar for immediate repainting on the next progress call, which helps lend the illusion that the progress bar is "always" there. In practice, it's merely there more of the time, but it ends up being a lot easier to read during convert.

tests: fix test-bundle2-format output Forgot to amend.

localrepo: recreate phasecache if changelog was modified (issue4855) Because _phaserevs and _phasesets cache revision numbers, they must be invalidated if there are new commits or stripped revisions. We could do that by calling _phasecache.invalidate(), but it wasn't simple to be integrated with the filecache mechanism. So for now, phasecache will be recreated after repo.invalidate() if 00changelog.i was modified before.

bundle2: allow compressed bundle This changeset adds support for a 'compression' parameter in bundle2 streams. When set, it controls the compression algorithm used for the payload part of the bundle2. There is currently no usage of this except in tests.

test-bundle2: dump bundle content using f --hexdump Thanks to Greg Szorc for pointing this out.

treemanifest: rework lazy-copying code (issue4840) The old lazy-copy code formed a chain of copied manifests with each copy. Under typical operation, the stack never got more than a couple of manifests deep and was fine. Under conditions like hgsubversion or convert, the stack could get hundreds of manifests deep, and eventually overflow the recursion limit for Python. I was able to consistently reproduce this by converting an hgsubversion clone of svn's history to treemanifests. This may result in fewer manifests staying in memory during operations like convert when treemanifests are in use, and should make those operations faster since there will be significantly fewer noop function calls going on. A previous attempt (never mailed) of mine to fix this problem tried to simply have all treemanifests only have a loadfunc - that caused somewhat weird problems because the gettext() callable passed into read() wasn't idempotent, so the easy solution is to have a loadfunc and a copyfunc.

manifest: rename treemanifest load functions to ease debugging I'm hunting an infinite recursion bug at the moment, and having both of these methods named just _load is muddying the waters slightly.

manifest: add id(self) to treemanifest __repr__ Also rename __str__ to __repr__ since that's what we really want for pdb.

bundlerepo: let bundle repo look in the _mancache When looking up a base revision, we were ignoring the contents that were already available in the manifest's _mancache. This patch allows us to use that data instead of reading from the revlog. This is useful in our pushrebase extension (which allows rebasing on the server side during a push) because it allows us to prefetch the bundle base manifest before aquiring the repo lock (1 second saving), which means doing less work inside the lock, which means a 20% higher commit rate.

check-seclevel: wrap entry point by function This is intended to narrow scope of local variables. The global _verbose flag will be replaced later by ui.verbose.

check-seclevel: set executable bit as it has shebang

bundle20: extract core payload generation in its own function We are about to allow compressing the core of the bundle2. So we extract the generation of this bits in its own parts to make this compression phases easier in a later changesets.

unbundle20: allow registering handlers for stream level parameters As a comment in the code have been asking for, it is now possible to register piece of code that handle parameters for the stream. I've been wondering is such function should be class methods or not. I eventually went for externally decorated methods to stick with the culture of extensibility from an extensions that apply to bundle2.

bundle2: allow to specify unsupported value on error A client may support an argument but not some of its values (eg: coming "compression" parameters). We allow this case to be carried in the exception.

bundle2: rename error exception class for unsupported feature The original name explicitly mention "Part", however it is also used outside of parts related feature. We rename from 'UnsupportedPartError' to 'BundleUnknownFeatureError' to fix this.

changegroup: use a different compression key for BZ in HG10 For "space saving", bundle1 "strip" the first two bytes of the BZ stream since they always are 'BZ'. So the current code boostrap the uncompressor with 'BZ'. This hack is impractical in more generic case so we move it in a dedicated "decompression".

monoblue: provide links to branches, tags and bookmarks by name This is adapted from cd842821db2c, that was added to paper for 3.5 release. It adds another way to refer to branches, tags and bookmarks in urls: by name. It's still possible to navigate to a specific changeset hash, but now you can get more descriptive urls straight from /summary page, for example. branchentry template (and so the whole branches table on /summary and /branches) lost the column that had a plain changeset hash, because tags and bookmarks don't have this column and also because there is already a way to address branch by its changeset hash (changeset link just next to it). Maybe we can instead bring this column with a plain changeset hash to tags and bookmarks, but this, more terse, new look feels fine.

gitweb: provide links to branches, tags and bookmarks by name This is adapted from cd842821db2c, that was added to paper for 3.5 release. It adds another way to refer to branches, tags and bookmarks in urls: by name. It's still possible to navigate to a specific changeset hash, but now you can get more descriptive urls straight from /summary page, for example. branchentry template (and so the whole branches table on /summary and /branches) lost the column that had a plain changeset hash, because tags and bookmarks don't have this column and also because there is already a way to address branch by its changeset hash (changeset link just next to it). Maybe we can instead bring this column with a plain changeset hash to tags and bookmarks, but this, more terse, new look feels fine.

cmdutil: remove HG: prefix from translation strings

i18n-zh_CN: annotate broken HG: translations

lock: recognize parent locks while acquiring This is part of a series that will allow locks to be inherited by subprocesses in limited circumstances. This patch enables the logic introduced in previous patches.

test-lock.py: fix testing for forks The earlier test worked only because the held count went up to 2, so the first release brought it down to 1. Making a copy of the lock fixes that issue.

test-lock.py: allow PID to be changed in test state This will be used in upcoming patches to create locks that appear as if they're being created by child processes.

test-lock.py: add a lock wrapper that allows faking the PID This will be used in upcoming patches to create locks that appear as if they're being created by child processes.

lock: add a wrapper to os.getpid() to make testing easier This will allow us to fake locks across processes more easily.

test-lock.py: move temp dir generation to testcase In upcoming patches we'll want to create multiple test state objects with a common test directory.

test-lock.py: copy-edit assertions about file existing Before: expected lock to exists but actually did not exists After: expected lock to exist but actually did not exist

revlog: don't flush data file after every added revision The current behavior of revlogs is to flush the data file when writing data to it. Tracing system calls revealed that changegroup processing incurred numerous write(2) calls for values much smaller than the default buffer size (Python defaults to 4096, but it can be adjusted based on detected block size at run time by CPython). The reason we flush revlogs is so readers have all data available. For example, the current code in revlog.py will re-open the revlog file (instead of seeking an existing file handle) to read the text of a revision. This happens when starting a new delta chain when adding several revisions from changegroups, for example. Yes, this is likely sub-optimal (we should probably be sharing file descriptors between readers and writers to avoid the flushing and associated overhead of re-opening files). While flushing revlogs is necessary, it appears all callers are diligent about flushing files before a read is performed (see buildtext() in _addrevision()), making the flush in _writeentry() redundant and unncessary. So, we remove it. In practice, this means we incur a write(2) a) when the buffer is full (typically 4096 bytes) b) when a new delta chain is created rather than after every added revision. This applies to every revlog, but by volume it mostly impacts filelogs. Removing the redundant flush from _writeentry() significantly reduces the number of write(2) calls during changegroup processing on my Linux machine. When applying a changegroup of the hg repo based on my local repo, the total number of write(2) calls during application of the mercurial/localrepo.py revlogs dropped from 1,320 to 217 with this patch applied. Total I/O related system calls dropped from 1,577 to 474. When unbundling a mozilla-central gzipped bundle (264,403 changesets with 1,492,215 changes to 222,507 files), total write(2) calls dropped from 1,252,881 to 827,106 and total system calls dropped from 3,601,259 to 3,178,636 - a reduction of 425,775! While the system call reduction is significant, it appears to have no impact on wall time on my Linux and Windows machines. Still, fewer syscalls is fewer syscalls. Surely this can't hurt. If nothing else, it makes examining remaining system call usage simpler and opens the door to experimenting with the performance impact of different buffer sizes.

revlog: use existing file handle when reading during _addrevision _addrevision() may need to read from revlogs as part of computing deltas. Previously, we would flush existing file handles and open a new, short-lived file handle to perform the reading. If we have an existing file handle, it seems logical to reuse it for reading instead of opening a new file handle. This patch makes that the new behavior. After this patch, revlog files are only reopened when adding revisions if the revlog is switched from inline to non-inline. On Linux when unbundling a bundle of the mozilla-central repo, this patch has the following impact on system call counts: Call Before After Delta write 827,639 673,390 -154,249 open 700,103 684,089 -16,014 read 74,489 74,489 0 fstat 493,924 461,896 -32,028 close 249,131 233,117 -16,014 stat 242,001 242,001 0 lstat 18,676 18,676 0 lseek 20,268 20,268 0 ioctl 14,652 13,173 -1,479 TOTAL 3,180,758 2,930,679 -250,079 It's worth noting that many of the open() calls fail due to missing files. That's why there are many more open() calls than close(). Despite the significant system call reduction, this change does not seem to have a significant performance impact on Linux. On Windows 10 (not a VM, on a SSD), this patch appears to reduce unbundle time for mozilla-central from ~960s to ~920s. This isn't as significant as I was hoping. But a decrease it is nonetheless. Still, Windows unbundle performance is still >2x slower than Linux. Despite the lack of significant gains, fewer system calls is fewer system calls. If nothing else, this will narrow the focus of potential areas to optimize in the future.

revlog: always open revlogs for reading and appending An upcoming patch will teach revlogs to use the existing file handle to read revision data instead of opening a new file handle just for quick reads. For this to work, files must be opened for reading as well. This patch is merely cosmetic: there are no behavior changes.

revlog: support using an existing file handle when reading revlogs Currently, the low-level revlog reading code always opens a new file handle. In some key scenarios, the revlog is already opened and an existing file handle could be used to read. This patch paves the road to that by teaching various revlog reading functions to accept an optional existing file handle to read from.

revlog: add docstring for checkinlinesize() The name is deceptive: it does more than just "check." Add a docstring to clarify what's going on.

windows: insert file positioning call between reads and writes fopen() and fdopen() have a unique-to-Windows requirement that transitions between read and write operations in files opened in modes r+, w+, and a+ perform a file positioning call (fsetpos, fseek, or rewind) in between. While the MSDN docs don't say what will happen if this is not done, observations reveal that Python raises an IOError with errno 0. Furthermore, I /think/ this behavior isn't deterministic. But I can reproduce it reliably with subsequent patches applied that open revlogs in a+ mode and perform both reads and writes. This patch introduces a proxy class for file handles opened in r+, w+, and a+ mode on Windows. The class intercepts calls and audits whether a file positioning function has been called between read and write operations. If not, a dummy, no-op seek to the current file position is performed. This appears to be sufficient to "trick" Windows into allowing transitions between read and writes without raising errors.

tests: suppress verbose output of svn transaction Subversion 1.9 shows more verbose messages than 1.8 and the tests fail because of them. These outputs are not important in our tests, so let's suppress them by -q or grep -v.

formatter: use dict.update() to set arguments passed to write functions This isn't important, but update() is better than loop in general.

formatter: verify number of arguments passed to write functions zip() takes the shortest length, which can be a source of bug.

help: unify handling of DEPRECATED/EXPERIMENTAL keywords This fixes listexts() to exclude translated "(DEPRECATED)" marker correctly. On the other hand, help_() doesn't need translated keywords, but I don't think it's worth to separate untranslated keywords just for it.

help: include parens in DEPRECATED/EXPERIMENTAL keywords In some languages that have no caps, "DEPRECATED" and "deprecated" can be translated to the same byte sequence. So it is too wild to exclude messages by _("DEPRECATED").

help: define list of keywords that should be excluded from non-verbose output This list will be reused by the other deprecated/experimental handling.

merge with stable

unbundle: cleanly abort on unknown bundle2 feature The exception wasn't caught and resulted in a traceback.

phases: return zero for no-op operations (issue4751) (BC) It is rather unhelpful to return 1 if there were no changes because the request matches the current state of phases. So we just undo that.

help: add config.troubleshooting section at the top

dispatch: don't stack trace on commands like `hg .log` This used to stack trace because it raised a util.Abort which wasn't handled in this block. We now handle it. Additionally, we error out earlier instead of plodding on and showing the "log" entry of the plain `hg help` output.

gitweb, monoblue: port highlighting linked lines from paper This is adapted from 41c4bdd1d585, fbfe5748ef3e and f3393d458bf5.

gitweb, monoblue: fix vertical align of spans in .sourcelines Empty lines in file view could produce an inexplicable margin before the next line (most noticeable in browsers on webkit/blink engine). That was making empty lines seem taller than the rest. Instead of using default vertical align, let's set it to 'top'. This issue is actually present in paper, and only recently got into gitweb (2239626369f5) and monoblue (119202d4d7a4). There's a bit more to it in paper, so that will be dealt with in a future patch. Recipe to see live: preferably using a webkit/blink browser, such as chromium, browse a file with empty lines, e.g. https://selenic.com/hg/file/3.5/README#l8 Selecting a block of text that includes empty lines will reveal white "breaks" in the selection. Highlighted line (#l8) also shows such a break below itself.

merge: fix mergestate comment

rebase: avoid losing branch commits with --keepbranch (issue4835)

lock.release: do not unlink inherited locks This is part of a series that will allow locks to be inherited by subprocesses in limited circumstances. A subprocess unlinking a lock will lead to potential corruption from other concurrent processes.

lock: add a method to reacquire the lock after subprocesses exit This is part of a series that will allow locks to be inherited by subprocesses in limited circumstances.

lock: add a method to prepare the lock for inheritance This is part of a series that will allow locks to be inherited by subprocesses in limited circumstances.

lock: introduce state to keep track of inheritance This is part of a series that will allow locks to be inherited by subprocesses in limited circumstances. In upcoming patches we will refer to this state.

error: add an exception to indicate lock inheritance API contract violations

clone: check update rev for being True In 30be3aeb5344, there was an attempt to fallback to looking up the update revision assuming it was always a rev but the documentation states: True means update to default rev, anything else is treated as a revision Therefore, we should only fallback to looking up the update rev if it is not True. This bug was found in hg-git and I couldn't think of a test that does this in pure Mercurial since the source repository is checked for the revision as well (and therefore gracefully falls back).

clone: fix over-indented continuation line

resolve: consistently describe re-merge + unresolved

mercurial: add debugextensions command (issue4676) Add debugextensions command to help users debug their extension problems. If there are no extensions command prints nothing, otherwise it prints names of extension modules. If quiet or verbose option is not specified it prints(after extensions name) last version of mercurial in which given module was tested for non internal modules or not tested with user mercurial version. If verbose is specified it prints following information for every extension: extension name, import source, testedwith and buglink information. Extensions are printed sorted by extension name.

dispatch: stop warning about EPIPE in --debug mode It seems silly for "hg --debug manifest | less" to print a scary message after the user hits "q" in less. hg should just exit silently instead, since EPIPE on stdout is a perfectly reasonable result.

rebase: don't rebase obsolete commit whose successor is already rebased This patch avoids unnecessary conflicts to resolve during rebase for the users of changeset evolution. This patch modifies rebase to skip obsolete commits if they are being rebased on their successors. It introduces a new rebase state 'revprecursor' for these revisions that are being skipped and a new message to inform the user of what is happening. This feature is gated behind the config flag experimental.rebaseskipobsolete When an obsolete commit is skipped, the output is: not rebasing 14:9ad579b4a5de "I", already in destination as 17:fc37a630c901 "K"

check-code: forbid mutable value for default argument default value are common to all call. Using mutable value is a classical source of bug in Python. We forbid it. The regexp (Courtesy of Matt Mackall) is only catching such value on the first line of a definition, but that will be good enough for now.

httpconnection: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

transplant: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

mq: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs. The underlying function already handle "None" as an option value, so we do not need to do anything else.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

largefiles: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

histedit: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

templater: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

templater: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

templater: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

templater: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

templater: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

addremove: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

match: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs. We added support for None in the underlying function in the parent commit. therefore we do not need to do anything but passing None.

match: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

match: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

match: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

addremove: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

tag: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs. We just added support of None for the underlying function, so nothing else the required.

commit: remove a mutable default argument Mutable default arguments are know to the state of California to cause bugs.

lock: move acquirefn call to inside the lock We're going to need to call it again as part of reinitialization after a subprocess inherits the lock.

localrepo: call lock.lock with releasefn as a keyword arg We'll be adding an acquirefn argument soon, and this makes that clearer.

copies: move debug statement to appropriate place

bidmerge: choose shortest list of diverge and rename/delete warnings Slightly less arbitrary than choosing the first set.

copies: rename diverge2 to divergeset for clarity

copies: begin separating mergecopies sides

copies: rename ctx() to getfctx() for clarity

monoblue: port code selection without line numbers from gitweb This is adapted from 2239626369f5. It also fixes issue4790 in monoblue; tab characters now have meaningful width on the modified pages (file view, file diff, changeset).

help: copy-edit description of ui.supportcontact

ui: avoid mutable default arguments I almost introduced a bug around this code by accidentally mutating a default argument. There's no reason for these to exist. It is OK to not assign {} to environ in ui.system because util.system knows how to deal with that.

util: avoid mutable default arguments I almost introduced a bug around this code by accidentally mutating a default argument. There's no reason for these to exist.

obsstore: fast path to check if obsstore is empty

obsstore: delay loading markers from obsstore file This will allow us to use cached revisions without parsing obsstore. Because _version isn't determined at __init__, the debugobsconvert command no longer works correctly. I'll fix it by a separate patch for the evolve extension.

obsstore: initialize _all markers without using _addmarkers() The next patch will make _all variable propertycached to avoid costly parsing of obsstore. This means we can't call _addmarkers() to initialize _all. Because all cached markers depend on _all, it isn't necessary to update these caches when _all is initially loaded.

revset: avoid implicit None testing in revset Implicit None testing is a very good way to get in trouble. We explicitly test for None.

revset: speed up existence checks for ordered filtered sets Previously, calling 'if foo:' on a ordered filtered set would start iterating in whatever the current direction was and return if a value was available. If the current direction was ascending, but the set had a fastdesc available, this meant we did a lot more work than necessary. If this was applied without my previous max/min fixes, it would improve max() performance (this was my first attempt at fixing the issue). Since those previous fixes went in though, this doesn't have a visible benefit in the benchmarks, but it does seem clearly better than it was before so I think it should still go in.

revset: remove existence check from min() and max() min() and max() would first do an existence check. Unfortunately existence checks can be slow in certain situations (like if the smartset is a list, and quickly iterable in both ascending and descending directions, then doing an existence check will start from the bottom, even if you want to check the max()). The fix is to not do the check, and just handle the error if it happens. In a large repo, this speeds up: hg log -r 'max(parents(. + .^) - (. + .^) & ::master)' from 3.5s to 0.85s. That revset is contrived and just for testing. In our real case we used 'bundle()' in place of '. + .^' Interesting perf numbers for the revset benchmarks: max(draft() and ::tip) => 0.027s to 0.0005s max(author(lmoscovicz)) => 2.48s to 0.57s min doesn't show any perf changes, but changing it as well will prevent a perf regression in my next patch. Result from revset benchmark revset #0: draft() and ::tip min max 0) 0.001971 0.001991 1) 0.001965 0.000428 21% revset #1: ::tip and draft() min max 0) 0.002017 0.001912 1) 0.001896 94% 0.000421 22% revset #2: author(lmoscovicz) min max 0) 1.049033 1.358913 1) 1.042508 0.319824 23% revset #3: author(lmoscovicz) or author(mpm) min max 0) 1.042512 1.367432 1) 1.019750 0.327750 23% revset #4: author(mpm) or author(lmoscovicz) min max 0) 1.050135 0.324924 1) 1.070698 0.319913 revset #5: roots((tip~100::) - (tip~100::tip)) min max 0) 0.000671 0.001018 1) 0.000605 90% 0.000946 92% revset #6: roots((0::) - (0::tip)) min max 0) 0.149714 0.152369 1) 0.098677 65% 0.100374 65% revset #7: (20000::) - (20000) min max 0) 0.051019 0.042747 1) 0.035586 69% 0.016267 38%

update: move default destination into a revset This is another step toward having "default" destination more clear and unified. Not all the logic is there because some bookmark related computation happened elsewhere. It will be moved later. The function is private because as for the other ones, cleanup is needed before we can proceed.

merge: move default destination computation in a revset This is another step toward having "default" destination more clear and unified.

mergecmd: simplify conditional The previous if tested the same thing with 'if repo._activebookmark'. We make the if/else logic clearer before a bigger refactoring.

rebase: move destination computation in a revset This is the first step toward making the "default destination" logic more clear and unified. The revset is private because I'm happy to delay the bikeshedding until after the clean up happened.

debian: include bash completions in mercurial-common

debian: install hgk as part of mercurial-common (issue4829) Leaving the hgk binary in /usr/bin causes some lintian warnings, and downstream packages poke it in /usr/share/mercurial, so we'll just stash it in there. Rather than patch hgk.py as part of the Mercurial install, just drop a config file in /etc/mercurial/hgrc.d that points to the installed hgk.

debian: install config files as part of mercurial-common Future patches will start putting config files in /etc/mercurial/, this just installs them.

debian: configuration so that cacerts is properly loaded This will be included in the mercurial-common package in a followup patch.

debian: install hg-ssh to /usr/bin just like downstream

highlight: remove temporary hack to overwrite repo.getcwd Now hgweb does it globally.

hgweb: overwrite cwd to resolve file patterns relative to repo (issue4568) It's useless to handle file patterns as relative to the cwd of the server process. The only sensible way in hgweb is to resolve paths relative to the repository root. It seems dirstate.getcwd() isn't used to get a real file path, so this patch won't cause problem.

dirstate: state that getcwd() shouldn't be used to get real file path hgweb will force it to be '' so that file patterns can be resolved relative to the repository root. I want to clarify that is correct.

merge: move merge step to the end Resolving other conflicts before merge ones is better because the state before the merge is as consistent as possible. It will also help with future work involving automatic resolution of merge conflicts with an external merge driver. There are no ordering issues here because it is easy to verify that the same file is never in both the dg/dm and the m sets.

lock: factor out lock testing into a separate function This is going to be needed for upcoming work with lock inheritance.

lock: factor code to read lock into a separate function This is going to be needed for upcoming work with lock inheritance.

tests: add unit tests for locking code We're going to make significant changes to lock behavior soon.

gitweb: port code selection without line numbers from paper This is adapted from f2e4fdb3dd27 and e92d4b8530cb. It also fixes issue4790 in gitweb; tab characters now have meaningful width on the modified pages (file view, file diff, changeset).

obsolete: remove unused _knownrevs function The call site was removed at cd62532c62a1.

bookmark: do not crash when active bookmark is forward and --date is used The logic that decides where to update according to the active bookmark location (when not on ".") was setting the rev to update to before we process --date. This lead to --date processing aborting because of duplicated specification. We reorder the two pieces of code and add a test for this.

bookmark: actually test update behavior in both cases The test claimed to test pull --update behaved the same way as pull + update, but the behavior of pull + update was never tested. We now test both.

coal: reuse style-paper.css and use a separate css file for visual tweaks To remove another big chunk of duplication from coal hgweb style, we can move all visual tweaks to style-extra-coal.css and include it together with style-paper.css.

coal: reuse all templates from paper/map by %including the file Before, coal/map simply had all the templates copied and pasted from paper/map file. It was easy to make a change to paper that would get into coal only partially and just sit there until something like 399e970e35c8 happened. Let's remove this duplication from coal/map by using %include to reuse templates from paper/map directly.

help/config: reword committemplate external editor text

help/config: reword committemplate multibyte character text

help/config: use :hg:`help config....` notation

help/config: omit True or False

help/config: capitalize items

tests: add more doctests for check-translation deprecated

tests: cleanup check-translation deprecated

gitweb: add author to filelogentry The line is copied from shortlogentry template.

monoblue: add author to filelogentry The line is copied from shortlogentry template.

monoblue: split a long line in filelogentry template The whole td has a nowrap class, which disallows breaking the line, so it's not required to use non-breakable spaces.

readbundle: fix typo of None compression The test simulates pre-d1b16a746db6 hgweb that sends "unbundle" capability with no argument.

readbundle: map 'HG10UN' to None compression In line with the other previous changes

getsubset: use None to request uncompressed changegroup

bundle: use None when unpacking uncompressed changegroup Let's be more modern!

writebundle: use 'None' instead of 'UN' for the bundle2 case Let's be modern!

compression: use 'None' for no-compression This seems more idiomatic and clearer. We still support both None and 'UN' for now because no user are migrated.

changegroup: move all compressions utilities in util We'll reuse the compression for other things (next target bundle2), so let's make it more accessible and organised.

obsolete: clarify that 'successorssets' returns the latest successors We do not return the first successors we found, we returns the latest (non obsolete (mostly)) one following the obsolete link transitively. We update the documentation to make this clean.

obsolete: drop an empty line at the end of a docstring This empty line is useless, let's drop it.

traceback: allow providing a local support contact point The extensions blaming code is fine for casual users but pretty terrible for corporate environments that can deploy a large amount of extensions to unsuspecting users. Reports will likely blame a random "innocent" extension (in our case crecord) and the hint in the message will triggers endless debug attempts from the user. We introduce a "ui.supportcontact" option that allow such big company to redirect their users to their own support desk. This disables all extensions blaming and just point people to the local support in all cases.

dispatch: pre-indent the extensions blaming logic We are about to put some real conditional here, but pre-indenting in it's own changesets makes next patch much cleaner.

tests: check for inconsistently translated DEPRECATED Mercurial expects DEPRECATED to be translated consistently, not doing that breaks Mercurial.

i18n-fr: reject translation which is missing DEPRECATED

i18n-pt_BR: select a single definition of DEPRECATED Mercurial expects DEPRECATED to be translated consistently, not doing that breaks Mercurial.

i18n-sv: select a single definition of DEPRECATED Mercurial expects DEPRECATED to be translated consistently, not doing that breaks Mercurial.

i18n-ro: use ()s instead of [] for DEPRECATED

i18n-ro: include translation of DEPRECATED

i18n-zh_CN: include translation of DEPRECATED

i18n-it: include translation of DEPRECATED

i18n-da: include translation of DEPRECATED

test-i18n: always run non gettext tests

localrepo: refresh filecache stats only if transaction finished successfully If commit is aborted by pretxncommit hook, in-memory changelog and manifest have entries that would be added. So they must be discarded on invalidate(). But the mechanism introduced by a710936c3037 doesn't handle this case well. It tries to mitigate the penalty of invalidate() by marking in-memory cache as "clean" on unlock assuming that they are identical to the stored data. But this assumption is wrong if stored data are rolled back by tr.abort(). This patch moves the hook to post-close action so that it will never be triggered on abort. This bug was originally reported to thg, which is only reproducible in command-server process on unix, evolve disabled. https://bitbucket.org/tortoisehg/thg/issues/4285/

localrepo: move closure of lock release to class It only captures "self", so it isn't necessary to be created dynamically.

highlight: add highlightfiles config option which takes a fileset (issue3005) Highlight extension lacked a way to limit files by size, by extension, and/or by any other part of file path. A good solution would be to use a fileset, since it can check file path, extension and size (and more) in one expression. So this change introduces such an option, highlighfiles, which takes a fileset and on each request decides if the requested file should be highlighted. The default "size('<5M')" is, in a way, suggested in issue3005. checkfctx() limits the amount of work to just one file (subset kwarg in fileset.matchctx()). Monkey-patching works around issue4568, otherwise using filesets here while running hgweb in directory mode would say, for example, "Abort: **.py not under root", but this fix is very local and probably far from ideal. I suspect there to be a way to fix this for the whole hgweb and resolve the issue, but I don't know how to do it.

posix: shellquote do not require quoting for "+" (issue4818)

hgweb: consume generator inside context manager (issue4756) If code inside a context manager returns a generator, the context manager exits before the generator is iterated. hgweb was using a context manager to control thread safe access to a localrepository instance. But it was returning a generator, so there was a race condition between a previous request streaming a response to the client and a new request obtaining the released but in use repository. By iterating the generator inside the context manager, we ensure we don't release the repo instance until after the response has finished. With this change, hgweb finally appears to have full localrepository isolation between threads. I can no longer reproduce the 2 exceptions reported in issue4756. test-hgweb-non-interactive.t has been modified to consume the output of calling into a WSGI application. Without this, execution of the WSGI application stalls because of the added yield statement.

histedit: use one editor when multiple folds happen in a row (issue3524) (BC) This was the first ever feature request for histedit, originally filed back on April 4, 2009. Finally fixed. In the future we'll probably want to make it possible for other preprocessing steps to be added to the list, but for now we're skipping that because it's unclear what the API should look like without a proposed consumer.

hgweb: replace .sourcelast with .bottomline that does the same In paper and Coal, basically, div.sourcelast was only used to make a 1px border on the bottom of file source view (and only there). It's better to use bottomline class, that also exists for the same purpose (visually), but is used more widely and works without needing an empty <div>.

coal: copy newer things from paper Basically, coal style in hgweb is intended to be functionally equivalent (just different in style) to paper, and does this by reusing almost all templates from paper (except header.tmpl, where it specifies a different css file). Looks like everybody forgot this and so many improvements to paper templates, that should've also made it into coal, were often only half-done there (usually thanks to template reuse). Let's fix this by bulk-copying missing things from paper/map and style-paper.css to coal/map and style-coal.css. There were many improvements to paper that didn't touch coal, and that makes it hard to untangle the code and split this patch into many, but here are some of the changes (paper-only), that now get into coal: 41c4bdd1d585 - hgweb: color line which is linked to in file source view f3393d458bf5 - hgweb: highlight line which is linked to at annotate view f2e4fdb3dd27 - hgweb: code selection without line numbers in file source view 5ec5097b4c0f - hgweb: add line wrapping switch to file source view bf661a03fddc - hgweb: use css margin instead of empty <p> before diffstat table It also fixes line anchor in annotateline template (#42 vs #l42).

revlog: optionally cache the full text when adding revisions revlog instances can cache the full text of a single revision. Typically the most recently read revision is cached. When adding a delta group via addgroup() and _addrevision(), the full text isn't always computed: sometimes only the passed in delta is sufficient for adding a new revision to the revlog. When writing the changelog from a delta group, the just-added full text revision is always read immediately after it is written because the changegroup code needs to extract the set of files from the entry. In other words, revision() is *always* being called and caching the full text of the just-added revision is guaranteed to result in a cache hit, making the cache worthwhile. This patch adds support to _addrevision() for always building and caching the full text. This option is currently only active when processing changelog entries from a changegroup. While the total number of revision() calls is the same, the location matters: buildtext() calls into revision() on the base revision when building the full text of the just-added revision. Since the previous revision's _addrevision() built the full text and the the previous revision is likely the base revision, this means that the base revision's full text is likely cached and can be used to compute the current full text from just a delta. No extra I/O required. The end result is the changelog isn't opened and read after adding every revision from a changegroup. On my 2013 MacBook Pro running OS X 10.10.5 from an SSD and Python 2.7, this patch impacted the time taken to apply ~262,000 changesets from a mozilla-central gzip bundle: before: ~43s after: ~32s ~25% reduction in changelog processing times. Not bad.

revlog: drop local assignment of cache variable The purpose of this code was to provide thread safety. With the conversion of hgweb to use separate localrepository instances per request/thread, we should no longer have any consumers that need to access revlog instances from multiple threads. Remove the code.

revlog: rename generic "i" variable to "indexdata" Increase readability.

hg: always create new localrepository instance cachedlocalrepo.copy() didn't actually create new localrepository instances. This meant that the new thread isolation code in hgweb wasn't actually using separate localrepository instances, even though it was properly using separate cachedlocalrepo instances. Because the behavior of the API changed, the single caller in hgweb had to be refactored to always call _webifyrepo() or it may not have used the proper filter. I confirmed via print() debugging that id(repo) is in fact different on each thread. This was not the case before. For reasons I can't yet explain, this does not fix issue4756. I suspect there is shared cache somewhere that isn't thread safe.

test-bad-extension: reduce dependencies on other things test-bad-extension would jitter if the format of the first line of hg help changed, which isn't relevant to its goal.

help: fix help argument parsing and documentation support combining -c and -e previously -k was misdocumented: * the first line didn't mention it * the help half implied you could do help -k keyword topic with these changes, -k just changes the search method support -c and -e for -k searches

minirst: establish leveling for nested definitions

dispatch: use the right context manager to deactivate demandimport In e86d12404d69 I very embarrassingly wrote a patch with the completely wrong function name. This should fix it.

ui: improve docs on ui.log This makes the documentation on ui.log line up with the use of that interface in blackbox.

templater: switch ctx of list expression to rev of revset() (BC) Because revset() function generates a list of revisions, it seems sensible to switch the ctx as well where a list expression will be evaluated. I think "{revset(...) % "..."}" expression wasn't considered well when it was introduced at cda9d2b6beab.

fileset: handle error of string unescaping

revset: handle error of string unescaping

parser: move unescape helper from templater revset and fileset have a similar problem, so let's make it a common helper function.

unionrepo: take delta base in account with building unified revlog When general delta is enabled, the base is actually meaningful and should be used. With general delta is enabled, test-unionrepo.t crash without this fix.

extdiff: enable -I/-X with --patch Not sure how useful this really is, but it's trivial to add and ignoring the existing arguments supported seems like a bad UI.

extdiff: add a --patch argument for diffing changeset deltas One of the things I missed the most when transitioning from versioned MQ to evolve was the loss of being able to check that rebase conflicts were properly resolved by: $ hg ci --mq -m "before" $ hg rebase -s qbase -d tip $ hg bcompare --mq The old csets stay in the tree with evolve, but a straight diff includes all of the other changes that were pulled in, obscuring the code that was rebased. Diffing deltas can be confusing, but unless radical changes were made during the resolve, it is very clear when individual hunks are added, dropped or modified. Unlike the MQ technique, this can only compare a single pair of csets/patches at a time. Like the MQ method, this also highlights changes in the commit comment and other metadata. I originally tried monkey patching from the evolve extension, but that is too complicated given that it depends on the order the two different extensions are loaded. This functionality is also useful when comparing grafts however, so implementing it in the core is more than just convenience. The --change argument doesn't make much sense for this, but it isn't harmful so I didn't bother blocking it. The -I/-X options are ignored because of a limitation of cmdutil.export(). We'll fix that next.

extdiff: prepare sections of dodiff() for conditionalizing This is purely indenting under an unconditional branch, so that the actual changes in the next patch are clear. Feel free to fold them if desired.

hgweb: drop unused import

help/config: back out 5f2a1ebd6e78 This breaks building manpages, and by association breaks building debs. timeless has a fix coming, but it turns out we'll need to back out 5f2a1ebd6e78 anyway, so just back it out now to fix building packages.

filemerge: add non-interactive :merge-local and :merge-other There are two non-interactive internal merge tools, :other and :local, but they don't really merge, they just pick all changes from the local or other version of the file. In some situations, it is known that we want a merge and also know that all merge conflicts should be resolved in one direction. Although external merge tools can do this, sometimes it can be convenient to do so from within hg, without invoking a merge tool. These new :merge-local and :merge-other tools can do just that.

simplemerge: enable option to resolve conflicts one way With this change, the simplemerge algorithm grows an option to only return the local or the other hunk in a conflicting region.

templater: add new docheader/footer components for XML (issue4135) The existing header/footer components were templated per-changeset, and thus couldn't be correctly printed for an empty log

import-checker: use modern .endswith for multiple suffixes Suggested by Anton Shestakov <engored@ya.ru> on the list. Thanks!

hgweb: use separate repo instances per thread Before this change, multiple threads/requests could share a localrepository instance. This meant that all of localrepository needed to be thread safe. Many bugs have been reported telling us that localrepository isn't actually thread safe. While making localrepository thread safe is a noble cause, it is a lot of work. And there is little gain from doing so. Due to Python's GIL, only 1 thread may be processing Python code at a time. The benefits to multi-threaded servers are marginal. Thread safety would be a lot of work for little gain. So, we're not going to even attempt it. This patch establishes a pool of repos in hgweb. When a request arrives, we obtain the most recently used repository from the pool or create a new one if none is available. When the request has finished, we put that repo back in the pool. We start with a pool size of 1. For servers using a single thread, the pool will only ever be of size 1. For multi-threaded servers, the pool size will grow to the max number of simultaneous requests the server processes. No logic for pruning the pool has been implemented. We assume server operators either limit the number of threads to something they can handle or restart the Mercurial process after a certain amount of requests or time has passed.

hg: establish a cache for localrepository instances hgweb contained code for determining whether a cached localrepository instance was up to date. This code was way too low-level to be in hgweb. This functionality has been moved to a new "cachedlocalrepo" class in hg.py. The code has been changed slightly to facilitate use inside a class. hgweb has been refactored to use the new API. As part of this refactor, hgweb.repo no longer exists! We're very close to using a distinct repo instance per thread. The new cache records state when it is created. This intelligence prevents an extra localrepository from being created on the first hgweb request. This is why some redundant output from test-extension.t has gone away.

hgweb: create function to perform actions on new repo We perform some common tasks when a new repo instance is obtained. In preparation for changing how we obtain repo instances, factor this functionality into a standalone function.

hgweb: remove proxy to hgweb instance We were temporarily routing attributes until all request-specific attributes from hgweb were moved to requestcontext. We have finally reached that juncture and we can remove the proxy. At this point, only the repo instance is prone to race conditions between threads. This will be dealt with shortly.

dispatch: disable demandimport when invoking the debugger Turns out that demandimport confuses pudb, which does some pretty complicated imports. I think it's reasonable to disable demandimport here.

templater: create string unescape helper (issue4798) This gives us a unified place to do error-handling of string-escaping syntax errors

parsers: use PyTuple_New and SET_ITEM to construct metadata pair of markers With these 2 patches, fm1readmarkers() gets slightly faster: obsolete._fm1readmarkers() for 78644 entries 58.0 -> 56.2msec

parsers: use PyTuple_SET_ITEM() to fill new marker tuples Because we know these tuples have no member yet, PyTuple_SetItem() isn't necessary.

revset: uncache filteredset.__contains__ Since 96b6b3d78697, condition function returns a cached value, so there's little benefit to cache __contains__. No measurable difference found in contrib/base-revsets.txt.

hgweb: assign ctype to requestcontext The very existence of ctype is a bit hacky. But we roll with it. Before this patch, there was possibly a race condition between 2 threads handling file requests: 1 thread could set the ctype and another serving a different file would read and use that potentially wrong ctype.

hgweb: add reponame to requestcontext We have to use object.__setattr__ until the app proxy is gone.

hgweb: don't access self.repo during request processing We want all repository accesses to go through requestcontext.repo so the request is isolated from the application.

hgweb: extract _getview to own function While we're refactoring code, we might as well remove a method that doesn't need to be a method.

hgweb: regenerate web substitutions when repo is refreshed Previously, changes to the configuration would not be picked up by a running server. That feels like a bug. Regenerate the web substitutions table when the repository changes.

add: pass full=False to dirstate walk Previously cmdutil.add would call wctx.walk(), which under the hood calls dirstate.walk with full=True. This means it returns all of the clean files (which we don't need when computing the add set), as well as the unclean files. This results in 1) a lot more work being done and 2) this code path circumventing the hgwatchman extension, resulting in worse performance in hgwatchman environments ('hg add .' went from 9s to 1.8s).

hgweb: drop unused import

obsolete: improve English of successorssets

histedit: fix grammar in cleanupnode comment

hgweb.server: fix _httprequesthandlerssl help text

util: capitalize Python in MBTextWrapper._wrap_chunks comment

hgweb: remove ErrorResponse.message BaseException.message is deprecated: https://www.python.org/dev/peps/pep-0352/#retracted-ideas

manifest: switch add() to heapq.merge (available in Py2.6+)

archival: drop self.filename - deprecated in py2.6

templater: fix get English

help: fix makeitemsdoc English description

help: filesets show hg resolve command

help: filesets show hg status command

hgmanpage: fix grammar remove 's's from places where they don't belong insert the preposition 'in'

check-seclevel: fix file description grammar

profiling: allow logging profile to the blackbox This allows specifying '--config profiling.output=blackbox' which will log the profile output to the blackbox (if enabled). This is useful for doing profiling on the server since it allows us to record the command, it's result, any exceptions, and it's profile, all in one spot. And we get log rotation for free.

bookmark: improve ambiguous documentation for rename

ui: change default path fallback mechanism (issue4796) The previous paths API code always fell back to the default path. This was wrong because if a requested path doesn't exist, that should error. Only if no path was requested should we fall back to the default. As part of implementing the test case for issue 4796, it was discovered that the "repository does not exist" error message raised by localrepository.__init__ wasn't being seen because the paths API validates paths before localrepository.__init__ was being called. The exception and error message from localrepository.__init__ has been introduced to getpath(). This necessitated rewriting expandpath() both to catch the exception and to have proper default fallback. This code is more complicated than I'd like. But making all tests pass was a big chore. As more code moves to getpath(), there will likely be opportunities to improve things a bit.

templater: catch regexp error at sub() function This patch splits re.sub() into re.compile() and sub() so that it can distinguish which argument causes re.error.

graphmod: compute slow revset query once prior to reachableroots (issue4782) Because revsets query is evaluated lazily, "list(revs)" may take long for complicated query. So we shouldn't iterate revs many times. This patch is the easiest workaround for the issue4782. We could introduce more aggressive caching, but it wouldn't be as fast as the simple baseset operation. Gregory Szorc said "this makes `hg wip` on my Firefox clone ~4x faster than 3.5.1 (~6.5s to ~1.5s). This is after a regression in @ to ~45s."

profiling: add config option for enabling profiling Previously you could only enable profiling via the --profile option. This is awkward when trying to debug a server side operation. Let's add a config option to enable profiling. In the future, this could be extended to allow profiling a certain percentage of operations (and potentially reporting that information to an external service).

blackbox: add pid to output This adds the process id to the line header for the blackbox output. This is useful for distinguishing processes when using the blackbox on a server and many processes are writing to the blackbox at once.

exchange: allow fallbackheads to use lazy set behavior The common ancestor set implementation was made lazy a couple years ago, but this piece of code still required processing the entire repo by putting set() around the lazy set. The code was introduced in 5653f2d166ea, a year before the lazy ancestor set was added. Dropping the set() shaves 3.5 seconds off of 'push -r' in repos with hundreds of thousands of commits.

hgweb: move templater instantiation to requestcontext This code needs to access a lot of config options. All our config lookups have moved to requestcontext. It makes sense to move this function there.

help/config: make defaults consistent

help/config: reword allow_push to fix grammar

help/config: change local_hostname help style

help/config: include check values