hgweb: don't access self.repo during request processing We want all repository accesses to go through requestcontext.repo so the request is isolated from the application.

hgweb: extract _getview to own function While we're refactoring code, we might as well remove a method that doesn't need to be a method.

hgweb: regenerate web substitutions when repo is refreshed Previously, changes to the configuration would not be picked up by a running server. That feels like a bug. Regenerate the web substitutions table when the repository changes.

add: pass full=False to dirstate walk Previously cmdutil.add would call wctx.walk(), which under the hood calls dirstate.walk with full=True. This means it returns all of the clean files (which we don't need when computing the add set), as well as the unclean files. This results in 1) a lot more work being done and 2) this code path circumventing the hgwatchman extension, resulting in worse performance in hgwatchman environments ('hg add .' went from 9s to 1.8s).

hgweb: drop unused import

obsolete: improve English of successorssets

histedit: fix grammar in cleanupnode comment

hgweb.server: fix _httprequesthandlerssl help text

util: capitalize Python in MBTextWrapper._wrap_chunks comment

hgweb: remove ErrorResponse.message BaseException.message is deprecated: https://www.python.org/dev/peps/pep-0352/#retracted-ideas

manifest: switch add() to heapq.merge (available in Py2.6+)

archival: drop self.filename - deprecated in py2.6

templater: fix get English

help: fix makeitemsdoc English description

help: filesets show hg resolve command

help: filesets show hg status command

hgmanpage: fix grammar remove 's's from places where they don't belong insert the preposition 'in'

check-seclevel: fix file description grammar

profiling: allow logging profile to the blackbox This allows specifying '--config profiling.output=blackbox' which will log the profile output to the blackbox (if enabled). This is useful for doing profiling on the server since it allows us to record the command, it's result, any exceptions, and it's profile, all in one spot. And we get log rotation for free.

bookmark: improve ambiguous documentation for rename

ui: change default path fallback mechanism (issue4796) The previous paths API code always fell back to the default path. This was wrong because if a requested path doesn't exist, that should error. Only if no path was requested should we fall back to the default. As part of implementing the test case for issue 4796, it was discovered that the "repository does not exist" error message raised by localrepository.__init__ wasn't being seen because the paths API validates paths before localrepository.__init__ was being called. The exception and error message from localrepository.__init__ has been introduced to getpath(). This necessitated rewriting expandpath() both to catch the exception and to have proper default fallback. This code is more complicated than I'd like. But making all tests pass was a big chore. As more code moves to getpath(), there will likely be opportunities to improve things a bit.

templater: catch regexp error at sub() function This patch splits re.sub() into re.compile() and sub() so that it can distinguish which argument causes re.error.

graphmod: compute slow revset query once prior to reachableroots (issue4782) Because revsets query is evaluated lazily, "list(revs)" may take long for complicated query. So we shouldn't iterate revs many times. This patch is the easiest workaround for the issue4782. We could introduce more aggressive caching, but it wouldn't be as fast as the simple baseset operation. Gregory Szorc said "this makes `hg wip` on my Firefox clone ~4x faster than 3.5.1 (~6.5s to ~1.5s). This is after a regression in @ to ~45s."

profiling: add config option for enabling profiling Previously you could only enable profiling via the --profile option. This is awkward when trying to debug a server side operation. Let's add a config option to enable profiling. In the future, this could be extended to allow profiling a certain percentage of operations (and potentially reporting that information to an external service).

blackbox: add pid to output This adds the process id to the line header for the blackbox output. This is useful for distinguishing processes when using the blackbox on a server and many processes are writing to the blackbox at once.

exchange: allow fallbackheads to use lazy set behavior The common ancestor set implementation was made lazy a couple years ago, but this piece of code still required processing the entire repo by putting set() around the lazy set. The code was introduced in 5653f2d166ea, a year before the lazy ancestor set was added. Dropping the set() shaves 3.5 seconds off of 'push -r' in repos with hundreds of thousands of commits.

hgweb: move templater instantiation to requestcontext This code needs to access a lot of config options. All our config lookups have moved to requestcontext. It makes sense to move this function there.

help/config: make defaults consistent

help/config: reword allow_push to fix grammar

help/config: change local_hostname help style

help/config: include check values

help/config: simplify default text

help/config: add trailing periods

help/config: alias, ui.mergemarkertemplate add period for etc.

tests: fix css-related test-hgweb.t breakage from 93258d53ec35

help/scripting: fix HGENCODING indentation

test-strip: fix spelling of initialization (en-US)

help/config: add blank lines

histedit: fix English (en-US)

minirst: handle edge in hunting for parents test for "hg help -c config" in help config ``section`` ----------- section text ``def1`` definition of def1 $ hg help scope.def1 should list this paragraph, but not section text below... section text ``def2`` definition of def2 $ hg help scope.def2 should list this paragraph, but not section text below... section text $ hg help scope.section should show everything from the first section text to this last line. ``section2`` ------------

help: config indent committemplate.changeset section

test-merge-tools: include /usr/sbin in path for sysctl (issue4813) /bin/sh expects to be able to reach sysctl via path (it lives in /usr/sbin)

merge with stable

import-checker: accept .pyc and .pyo files (issue4812) Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin $ ls '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6'/BaseHTTPServer.py* /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.pyc /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.pyo

rebase: fix warning about ignoring tool option on rebase continue (issue4698) Before this patch rebase --continue with specified --tool option outputs warnings "tool option will be ignored". It is false statement because in case of any merge conflicts it uses specified tool to resolve it. This patch makes this warning appears only when user specified --tool when running rebase --abort , in this case tool doesn't have any sense

hgweb: create websubtable on requestcontext

hgweb: move templatepath to requestcontext This does change behavior in that the templatepath could change during the lifetime of the server. But everything else can change, I don't see why template paths can't.

hgweb: extract web substitutions table generation to own function It doesn't use any state in hgweb except for the repo instance. Move it to a standalone function.

hgweb: remove hgweb.configbool It is redundant with requestcontext.configbool.

hgweb: move additional state setting outside of refresh We want refresh() to only be about refreshing repository instances. This state doesn't belong in requestcontext because it is shared across multiple threads.

commands: support hg help scripting.HGPLAIN keywords are lowercased within help/minirst...

run-tests: use $HGTEST_RUN_TESTS_PURE test-run-tests and test-hghave call run-tests; if you don't have a working build environment, and you are trying to use --pure, then if they don't use --pure or --with-hg, they'll break.

minirst: don't treat top level item as children of last item (issue4803) test for "hg help -c config" in help config

scmutil.readonlyvfs: implement join Required in an upcoming patch.

localrepo: move dirstate validate function to class scope This will allow the function to be referred to by subclasses, as required for upcoming patches.

convert: remove restriction on multiple --rev in hg source Multiple --rev args on convert is a new feature, and was initially disabled for all sources. It has since been enabled on git sources, and this patch enables it on mercurial sources.

monoblue: add a margin for help text container

gitweb: add a margin for help text container

monoblue: fix page subtitle on help pages

hgweb: use latest mtime for caching tag (issue4814) Previously, we were using the last mtime we saw when reporting the HTTP cache etag. When we added bookmarks to the end of the list of files checked, unchanged or missing bookmarks would keep the client cache from being invalidated.

hgweb: initialize repostate to None The initial value is irrelevant since refresh() compares it to a tuple of tuples of file mtime and size. None != tuple and None is a better default value than a tuple containing irrelevant values.

debian: switch to using debhelper and dh_python2 to build debs This is a much larger commit than I'd like, but I honestly don't see a good way to break it up and leave things working. Summary: We now use debian/rules with debhelper to build our debs. This is much more standard, and means we use dh_python2 to do things like handle leaving .pyc files out of the built debs. The resulting package is split into mercurial and mercurial-common, with the former being the hg stub and all the native .sos, and the latter being basically everything else. builddeb and dockerdeb are updated to use the new system. The old way (using dpkg by hand) breaks with the above changes because debian/control no longer contains a version string (that's now guessed from the phony changelog.) Tests are updated to assert that the right files end up in the right debs.

test-debian-packages: new test for testing construction of debian packages

test-docker-packaging: test packages built using docker For now this only tests debian packages, but it could easily be extended to also test rpms.

hghave: correct test for debhelper debhelper exits with a nonzero status when you pass --help, at least on jessie.

bundle2: don't try to recover from a GeneratorExit (issue4785) GeneratorExit means the other end of the conversation has already stopped listening, so don't try and yield out error information. Instead, just let the GeneratorExit propagate normally. This should resolve esoteric issues observed with servers that have aggressive timeouts waiting for data to send to clients logging internal Python errors[0]. This has been observed with both gunicorn's gevent worker model and with scm-manager's built-in webserver (which itself is something sitting inside jetty.) 0: Exception RuntimeError: 'generator ignored GeneratorExit' in <generator object getchunks at 0x7fd2f6c586e0> ignored

revset: fix resolving strings from a list When using multiple revsets that get optimized into a list (like hg log -r r1235 -r r1237 in hgsubversion), the revset list code was assuming the strings were resolvable via repo[X]. hgsubversion and other extensions override def stringset() to allow processing different revision identifiers (such as r1235 or g<githash>), and there for the _list() implementation was circumventing that resolution. The fix is to just call stringset(). The default implementaiton does the same thing that _list was already doing (namely repo[X]). This has always been broken, but it was recently exposed by 4ee4f7415095 which made "--rev X --rev Y" produce a combined revset "X | Y".

dispatch: error out on invalid -R path even if optionalrepo (issue4805) (BC) Before this patch, repo could be set to None for wrong -R. It's okay for commands that can reject repo=None, but the command server have a problem because: - it accepts repo=None for "unbound" mode - and it reenters dispatch() where repo object is created for cwd by default Test outputs are changed because the error is detected earlier. I think new message is better than ".hg not found".

test-ssh: remove superfluous triple slashes from absolute path test Because pwd should start with "/", "/`pwd`" is enough to make absolute path.

vagrant: update to official Debian 8.1 base box Debian now rolls their own official Vagrant base boxes, so use that. At the same time, we're updating from Debian 7.4 (wheezy) to 8.1 (jessie), and switching from 32-bit to 64-bit (Debian does not provide 32-bit base boxes).

buildrpm: mkdir -p two needed directories (issue4779) Without this, building RPMs can fail.

merge with stable

hghave: use subprocess instead of os.popen os.popen was deprecated in Python 2.6 in favor of subprocess, so let's move into the future.

hgweb: move archive related attributes to requestcontext As part of this, "archive_specs" was renamed to "archivespecs" to align with naming conventions. "archive_specs" didn't technically need to be moved from hgweb. But it seemed to make sense to have all the archive code in the same class. As part of this, hgweb.configlist is no longer used, so it was deleted.

hgweb: move some config options to requestcontext Various config options from the repository were stored on the hgweb instance. While unlikely, there could be race conditions between a new request updating these values and an in-flight request seeing both old and new values, leading to weird results. We move some of options/attributes from hgweb to requestcontext. As part of this, we establish config* helpers on requestcontext. As part of the move, we changed int() casts to configint() calls. The int() usage likely predates the existence of configint(). We also removed config option updating from once every refresh to every request. I don't believe obtaining config options is expensive enough to warrant only doing when the repository has changed. The excessive use of object.__setattr__ is unfortunate. But it will eventually disappear once the proxy is no longer necessary.

hgweb: establish class for holding per request context Currently, hgweb applications have many instance variables holding mutated state. This is somewhat problematic because multiple threads may race accessing or changing this state. This patch starts a series that will add more thread safety to hgweb applications. It will do this by moving mutated state out of hgweb and into per-request instances of the newly established "requestcontext" class. Our new class currently behaves like a proxy to hgweb instances. This should change once all state is captured in it instead of hgweb. The effectiveness of this proxy is demonstrated by passing instances of it - not hgweb instances/self - to various functions.

hgweb: don't make request optional The "request" argument has been optional in this code since it was introduced in 38170eeed18c in 2009. There are no consumers that don't pass this argument. So don't make it an optional argument.

hgweb: add some documentation It took longer than I wanted to grok how the various parts of hgweb worked. So I added some class and method documentation to help whoever hacks on this next.

hgweb: limit branches shown on summary page to 10 Tags and bookmarks on summary page are already limited to 10, let's limit branches as well. Each of the blocks (tags, bookmarks, branches) currently has a link to the full list.

hgweb: use webutil.branchentries for branches on summary page This allows showing correct status for each branch, which was missing on /summary. Usually that means that closed branches get the same css class (resulting in e.g. different color/shade) as they do on /branches page. The sorting of the branches on summary page also changes and is now the same as on /branches page: closed branches are now at the end of the list.

hgweb: move branchentries code from webcommands to webutil

templater: add optional timezone argument to localdate() The keyword extension uses "utcdate" for a different function, so we can't add new "utcdate" filter or function. Instead, this patch extends "localdate" to a general timezone converter.

templater: port localdate filter to a function It will be extended to accept a timezone argument.

util: extract function that parses timezone string It will be used to parse a timezone argument passed to a template function.

templater: drop unneeded destructuring of argument tuple at buildfilter Because evalfuncarg() accepts an argument tuple, there is little meaning to pass (func, data, filt) in place of ((func, data), filt).

templater: extract helper that evaluates filter or function argument It will be used to get a date tuple from an argument. Perhaps some of "stringify(args[n][0], ...)" can be replaced by this function.

contrib: add showstack extension This allows getting a Python stack trace at any time on Unix by hitting Ctrl-\ (or Ctrl-T on BSDs). Useful for debugging mysterious hangs on the fly. Sample output: $ hg log -k nosuchmessage ^\ File "/home/mpm/hg/mercurial/revset.py", line 3089, in _iterfilter if cond(x): File "/home/mpm/hg/mercurial/util.py", line 415, in f cache[arg] = func(arg) File "/home/mpm/hg/mercurial/revset.py", line 1215, in matches for t in c.files() + [c.user(), c.description()]) File "/home/mpm/hg/mercurial/context.py", line 525, in files return self._changeset[3] File "/home/mpm/hg/mercurial/util.py", line 531, in __get__ result = self.func(obj) File "/home/mpm/hg/mercurial/context.py", line 498, in _changeset return self._repo.changelog.read(self.rev()) File "/home/mpm/hg/mercurial/changelog.py", line 338, in read text = self.revision(node) File "/home/mpm/hg/mercurial/revlog.py", line 1092, in revision bins = self._chunks(chain) File "/home/mpm/hg/mercurial/revlog.py", line 1013, in _chunks ladd(decompress(buffer(data, chunkstart - offset, chunklength))) File "/home/mpm/hg/mercurial/revlog.py", line 91, in decompress return _decompress(bin) ----

Added signature for changeset 1a45e49a6bed

Added tag 3.5.1 for changeset 1a45e49a6bed

hgweb: fix trust of templates path (BC) Long ago we disabled trust of the templates path with a comment describing the (insecure) behavior before the change. At some later refactor, the code was apparently changed back to match the comment, unaware that the intent of the comment was to describe the behavior to avoid. This change disables the trust and updates the comment to explicitly say not only what the old problem was, but also that it was in fact a problem and the action taken to prevent it. Impact: prior to this change, if you had a UNIX-based hgweb server where users can write hgrc files, those users could potentially read any file readable by the web server. This is marked as a backwards compatibility issue because people may have configured templates without proper trust settings. Issue spotted by Greg Szorc.

wix: avoid an abort with 'hg help -k foo' Previously: $ /c/Program\ Files/Mercurial/hg help -k merge-tools abort: No such file or directory: c:\Program Files\Mercurial\help\scripting.txt The Inno installer seems OK, but the TortoiseHg required the same fix. That got queued with an additional change of 'helpFolder.guid' in guids.wxi (probably by Steve). I'm not sure if that is necessary here too.

revlog: add an aggressivemergedelta option This adds an option for delta'ing against both p1 and p2 when applying merge revisions and picking whichever is smallest. Some before and after stats on manifest.d size: internal large repo: before: 1.2 GB after: 930 MB mozilla-central: before: 261 MB after: 92 MB

revlog: change generaldelta delta parent heuristic The old generaldelta heuristic was "if p1 (or p2) was closer than the last full text, use it, otherwise use prev". This was problematic when a repo contained multiple branches that were very different. If commits to branch A were pushed, and the last full text was branch B, it would generate a fulltext. Then if branch B was pushed, it would generate another fulltext. The problem is that the last fulltext (and delta'ing against `prev` in general) has no correlation with the contents of the incoming revision, and therefore will always have degenerate cases. According to the blame, that algorithm was chosen to minimize the chain length. Since there is already code that protects against that (the delta-vs-fulltext code), and since it has been improved since the original generaldelta algorithm went in (2011), I believe the chain length criteria will still be preserved. The new algorithm always diffs against p1 (or p2 if it's closer), unless the resulting delta will fail the delta-vs-fulltext check, in which case we delta against prev. Some before and after stats on manifest.d size. internal large repo old heuristic - 2.0 GB new heuristic - 1.2 GB mozilla-central old heuristic - 242 MB new heuristic - 261 MB The regression in mozilla central is due to the new heuristic choosing p2r as the delta when it's closer to the tip. Switching the algorithm to always prefer p1r brings the size back down (242 MB). This is result of the way in which mozilla does merges and pushes, and the result could easily swing the other direction in other repos (depending on if they merge X into Y or Y into X), but will never be as degenerate as before. I future patch will address the regression by introducing an optional, even more aggressive delta heuristic which will knock the mozilla manifest size down dramatically.

revlog: move textlen calculation to be above delta chooser This moves the textlen calculation to be above the delta chooser. Since textlen is needed for calling isgooddelta, we need it above the delta chooser so future patches can call isgooddelta.

revlog: move delta check to it's own function This moves the delta vs fulltext comparison to its own function. This will allow us to reuse the function in future patches for more efficient delta choices. As a side effect, this will also allow extensions to modify our delta criteria.

monoblue: add archive links on summary page

help: distinguish sections when multiple match (issue4802)

merge with stable

hghave: add a check for docker support This currently refuses to operate if on a non-Linux host. I suspect that Docker running on FreeBSD 11 or on an Illumos derivative would work fine, but I don't have ready access to such a system. On OS X using boot2docker (I used a hacky xhyve-based one for testing), it won't work because $TESTTEMP doesn't end up inside the set of directories that get forwarded to the boot2docker VM, so you can't actually drop debs in the $TESTTEMP at all. It would be possible (probably even trivial) to hack around this by using a randomly-named temporary directory inside the working directory, but that seems unlikely to be useful enough to justify the ugliness.

hghave: add a check for debian packaging tools

run-tests: add support for marking tests as very slow I want to add tests for our packaging rules, but those necessarily run a whole build, or possibly two if both native packaging and docker are available. This lets us flag such tests with a `#require slow` so that they don't unnecessarily slow down normal test runs.

builddeb: rework how output dir and platform are specified This makes it possible to write tests for both builddeb and dockerdeb that actually build .debs and then sanity check the contents.

parsers: fix two cases of unsigned long instead of Py_ssize_t We had to do this before because Python 2.4 didn't understand the n format specifier in Py_BuildValue and friends. We no longer have that problem.

templatefilters: remove redundant 'date' and 'strip' filters These filters are defined as 'date()' and 'strip()' functions. Help messages are moved to the corresponding functions.

templater: introduce unified filter syntax for unary functions "filter(expr)" is allowed already. This patch adds the opposite, "expr|func".

templater: inline getfilter() to buildfilter() This prepares for the unified filter syntax that will be introduced by the next patch.

clone: fix updaterev to update to latest branch changeset (issue4528) Before this patch if clone --updaterev points to branch which head on src repo wasnt in dest repo, clone updated dest repo to default branch. After applying this patch, if changeset from src repo pointing at given branch is not in dest repo, it searches for changeset pointing for given branch locally in dest repo. Lookup in destination repo: 559: uprev = destrepo.lookup(update) is wrapped by try/except block to preserve current behaviour when given revset to -u is not found - it will not fail,but silently update dest repo to head of default branch.

revsets: makes follow() supports file patterns (issue4757) (BC) Before this patch, follow only supports full, exact filenames. This patch makes follow argument to be treated like file pattern same way like log treats their arguments. It preserves current behaviour of follow() matching paths relative to the repository root by default.

i18n-ja: synchronized with b930d4ef7739

histedit: improve discoverability of edit commit message

revset: cache smartset's min/max As the content of a smartset never changes, min and max will never change either. This will save us time when this function is called multiple times. This is relevant for issue4782 but does not fix it.

spelling: behaviour -> behavior

glossary: fixup use of periods at end of entries * Include trailing period for entries * Move trailing period inside quotations for example sentences

histedit: fix grammar for help for safety aborts

revset: mark reachablerootspure as private

reachableroots: construct and sort baseset in revset module This can remove the dependency from changelog to revset, which seems a bit awkward for me.

reachableroots: use smartset min smartset min are likely to be optimised, cached or other magical property.

graphmod: stop building a list out of branchgroupiter Not only does this defeat the purpose of having `branchgroupiter` an iterable, but this also makes revs something else than a smartset preventing to use revs.min() in the call to reachablerevs.

reachableroots: sort the smartset in the pure version too Changeset be8a4e0800d8 uses smartset lazy sorting for the C version. We need to apply the same to the pure version for consistency. This is fixing the tests with --pure.

builddeb: actually run make when building the deb (issue4778) As of this change, we no longer produce broken debs, but I've already got followups written that will produce much more standard-looking packages and test the resulting packages.