repoview: issue a special message when filtering hidden changesets Hidden changesets are by far the most common error case and is the only one[1] that can reach the user. We move to a friendlier message with a hint about how to access the data anyway. We should probably point to a help topic instead but we do not have such a topic yet. Example of the new output abort: hidden revision '4'! (use --hidden to access hidden revisions) [1] Actually, filtering from "served" can also reach the user during certain exchange operations.

repoview: include the filter name in filtered revision error messages This will help user to debug. A more precise message will be issued for the most common case ("visible" filter) in the next changesets. example output: - abort: filtered revision '4'! + abort: filtered revision '4' (not in 'visible' subset)!

largefiles: inline redundant toname function in status Simpler and an optimization.

largefiles: inline redundant inctx function in status

ssl: only use the dummy cert hack if using an Apple Python (issue4410) The hack for using certificate store in addition to the provided CAs resides in Apple's OpenSSL. Apple's own Pythons will use it, but other custom built Pythons might use a custom built OpenSSL without that hack and will fail when exposed to the dummy cacert introduced in d7f7f1860f00. There do not seem to be a simple way to check from Python if we are using a patched OpenSSL or if it is an Apple OpenSSL. Instead, check if the Python executable resides in /usr/bin/python* or in /System/Library/Frameworks/Python.framework/ and assume that all Pythons found there will be native Pythons using the patched OpenSSL. Custom built Pythons will not get the benefit of using the CAs from the certificate store.

largefiles: move initialization of standins variable to clarify its "scope"

largefiles: the update override only needs lfdirstate and status for --check

largefiles: remove confusing rev parameter for lfdirstatestatus Dirstate only works on the repo wctx.

largefiles: replace repo._isaddremove hack with a simple function parameter

status: add more complete tests for --rev The 'status --rev' code is not very well tested, which has bitten us as recently as in issue4321. Let's add some more tests, some of which uncover bugs. Remove the few existing tests that are now covered in a more thorough and consistent way.

cmdserver: include pid of server handling requests in hello message Because unix-mode server forks child process per connection, client does not know the pid of the server that will handle requests. The pid is necessary to interrupt hung process: 1. client connects to socket server 2. server accepts the connection, forks, and tells pid 3. client requests "runcommand pull" .. hung .. 4. client sends SIGINT to the (forked) server 5. server returns from I/O wait Note that getsockopt(SO_PEERCRED) of Linux cannot be used because the server fork()s after accept().

run-tests: clean up unused globals that were removed by tests-refactor series They were demoted to instance variables by cb88d4a04f58, 1ad7aabba14e, 93511a595766, 2e1aa8c1ee37, 8e7b0f4d6ac7 and 02087bc4f143.

docker: use official centos5 image I guess it didn't exist when centos5 support was introduced.

docker: add centos7 target for CentOS / Red Hat 7 support

lock: while releasing, unlink lockfile even if the release function throws Consider a hypothetical bug in the release function that causes it to raise an exception. Also consider the bisect command, which saves its state in a finally clause. Saving the state requires acquiring the wlock. If we don't unlink the lockfile when the exception is thrown, we'll try to acquire the wlock again. We're going to try and acquire a lock again while our old lockfile is on disk. The PID on disk is our own, and of course we're still running, so we won't take over the lock. Hence we'll be stuck waiting for a lock that we left behind ourselves. To avoid this, always unlink the lockfile. This preserves the invariant that self.held > 0 is equivalent to the lockfile existing on disk.

i18n: cache the result of every gettext call In looking at profiler output for 'hg log' on mozilla-central, I noticed we spent a _huge_ amount of time in gettext relative to what it's doing. Caching provides a roughly 15% performance improvement even on repositories as small as hg. == hg repo on linux == Before: % cumulative self time seconds seconds name 5.05 0.19 0.19 i18n.py:62:gettext 4.84 0.18 0.18 revlog.py:88:decompress 2.95 0.17 0.11 changelog.py:201:node 2.32 0.09 0.09 ui.py:577:write 2.11 0.08 0.08 i18n.py:72:gettext 2.11 0.08 0.08 obsolete.py:196:_fm0readmarkers 1.89 0.07 0.07 obsolete.py:569:_load 1.68 0.63 0.06 localrepo.py:29:__get__ real 0m4.026s user 0m3.993s sys 0m0.034s After: % cumulative self time seconds seconds name 8.05 0.26 0.26 revlog.py:88:decompress 2.68 0.22 0.09 color.py:395:write 2.20 0.07 0.07 obsolete.py:196:_fm0readmarkers 1.95 0.06 0.06 obsolete.py:174:_fm0readmarkers 1.95 0.06 0.06 ui.py:577:write 1.95 0.06 0.06 util.py:1228:datestr 1.71 0.06 0.06 utf_8.py:16:decode 1.71 0.06 0.06 revlog.py:273:__len__ real 0m3.519s user 0m3.447s sys 0m0.073s == mozilla-central repo on linux == Before: % cumulative self time seconds seconds name 7.72 2.35 2.35 revlog.py:88:decompress 4.46 1.36 1.36 i18n.py:62:gettext 2.22 0.67 0.67 i18n.py:72:gettext 2.19 1.14 0.67 changelog.py:201:node 2.16 0.66 0.66 ui.py:577:write 1.96 0.60 0.60 utf_8.py:16:decode 1.93 1.97 0.59 color.py:395:write 1.85 0.81 0.56 changelog.py:136:tip real 0m30.822s user 0m30.660s sys 0m0.149s After: % cumulative self time seconds seconds name 9.82 2.49 2.49 revlog.py:88:decompress 2.67 1.31 0.68 localrepo.py:29:__get__ 2.57 0.65 0.65 utf_8.py:16:decode 2.48 1.01 0.63 changelog.py:201:node 2.10 0.82 0.53 changelog.py:136:tip 2.01 0.51 0.51 ui.py:577:write 1.91 0.49 0.49 util.py:1232:datestr 1.85 1.65 0.47 color.py:395:write real 0m25.619s user 0m25.446s sys 0m0.166s == cpython repo on os x = Before: % cumulative self time seconds seconds name 5.05 1.35 1.35 cmdutil.py:982:_show 4.59 1.22 1.22 revlog.py:274:__len__ 3.98 1.06 1.06 i18n.py:62:gettext 3.91 1.04 1.04 revlog.py:1016:revision 3.68 0.98 0.98 revlog.py:337:parents 3.45 0.92 0.92 revlog.py:88:decompress 2.91 0.78 0.78 revlog.py:309:rev 2.62 0.70 0.70 revlog.py:1033:revision real 0m30.414s user 0m28.145s sys 0m0.541s After: % cumulative self time seconds seconds name 7.98 1.66 1.66 cmdutil.py:982:_show 6.83 1.42 1.42 changelog.py:46:decodeextra 5.18 1.08 1.08 revlog.py:274:__len__ 3.94 0.82 0.82 revlog.py:1016:revision 3.41 0.71 0.71 revlog.py:309:rev 3.32 0.69 0.69 revlog.py:88:decompress 2.99 0.63 0.62 revlog.py:1033:revision 2.69 0.56 0.56 revlog.py:341:start real 0m22.811s user 0m21.883s sys 0m0.397s

util.system: avoid buffering of subprocess output when it is piped util.system() copies subprocess' output through pipe if output file is not stdout. Because a file iterator has internal buffering, output won't be flushed until enough data is available. Therefore, it could easily miss important messages such as "waiting for lock".

bundle2: client side support for a part to import external bundles Bundle2 opens doors to advanced features allowing to reduce load on mercurial servers, and improve clone experience for users on unstable or slow networks. For instance, it could be possible to pre-generate a bundle of a repository, and give a pointer to it to clients cloning the repository, followed by another changegroup with the remainder. For significantly big repositories, this could come as several base bundles with e.g. 10k changesets, which, combined with checkpoints (not part of this change), would prevent users with flaky networks from starting over any time their connection fails. While the server-side support for those features doesn't exist yet, it is preferable to have client-side support for this early-on, allowing experiments on servers only requiring a vanilla client with bundle2 enabled.

bundle2: remove _getbundleextrapart Extensions can now add new part generators with the getbundle2partsgenerator decorator instead.

glossary: add deprecated and experimental

test-merge5: remove obsolete '-y' and its motivating comment The '-y' in 'hg update -y' was once needed to answer questions about modify/delete conflicts. That is no longer needed, so remove the '-y' and the comment justifying its use.

test-resolve.t: use redirection to /dev/null instead of grep -q

resolve: run happily after conflict-free merge

test-resolve: add tests for re-merge

test-resolve: add more tests for in conflict-free states We already have a test for 'hg resolve -m' when there is no merge in progress. Add one for 'hg resolve --all' as well. Also add tests for both --all and -m when there is a merge without conflicts in progress. They should both be successful, just as if there had been conflicts that had been marked resolved. However, that is currently broken, so mark the tests broken for now. The behavior will be fixed in a later patch.

test-resolve: add test resolving one of two files The tests for resolve are missing a lot of cases. Let's start by adding another file to the test repo, so we can test resolving one of two files.

test-resolve: clarify test descriptions and consistently use "should"

revset-phases: prefetch attributes in phasesrelated revsets Pre-fetching attributes gives a significant performance boost. Such is Python. draft() 0) wall 0.011661 comb 0.010000 user 0.010000 sys 0.000000 (best of 205) 1) wall 0.009804 comb 0.000000 user 0.000000 sys 0.000000 (best of 231) draft() - ::bookmark() 0) wall 0.014173 comb 0.010000 user 0.010000 sys 0.000000 (best of 177) 1) wall 0.012966 comb 0.010000 user 0.010000 sys 0.000000 (best of 182)

revset-phases: do not cache phase-related filters The phase retrieval is fast enough to not require caching the result of the functions. draft() 0) wall 0.017209 comb 0.020000 user 0.020000 sys 0.000000 (best of 149) 1) wall 0.011654 comb 0.010000 user 0.010000 sys 0.000000 (best of 186) public() 0) wall 0.018687 comb 0.010000 user 0.010000 sys 0.000000 (best of 128) 1) wall 0.013290 comb 0.010000 user 0.010000 sys 0.000000 (best of 181) secret() 0) wall 0.017464 comb 0.020000 user 0.020000 sys 0.000000 (best of 127) 1) wall 0.011499 comb 0.000000 user 0.000000 sys 0.000000 (best of 196) draft() - ::bookmark() 0) wall 0.020099 comb 0.020000 user 0.020000 sys 0.000000 (best of 127) 1) wall 0.014399 comb 0.020000 user 0.020000 sys 0.000000 (best of 169)

changectx: issue a FilteredRepoLookupError when applicable We capture FilteredxxxError and issue a FilteredRepoLookupError instead with a sightly different messsge. The message will likely get more improvement in the future. error: filtered revision '4'

repoview: add a FilteredRepoLookupError This exception is a more precise RepoLookupError that will allow us to issue a special message when we end up accessing a filtered revision.

repoview: add a FilteredLookupError class This exception is a more precise LookupError that will allow us to issue a special message when we end up accessing a filtered revision.

repoview: add a FilteredIndexError class This exception is a more precise IndexError that will allow us to issue a special message when we end up accessing a filtered revision.

changectx: move `IndexError` handling in the top level try except This one can be handled by the top level catching.

changectx: wrap the `changeid` processing in a try/except We are going to introduce more precise exception classes for filtered nodes. So we will have to upgrade them to the `RepoLookupError` level here. We wrap the whole thing into a try/except to ease this future catching. Some of the current exception catching will be moved in this one. But the current changeset focuses on code movement only.

bundle2: detect and disallow a negative chunk size We have no usage planned for 2/3 of them and the support for the planned usecase is not here yet. So we raise a BundleValueError when encountered

bundle2: add an UnsupportedPartError We need the BundleValueError for format errors not related to part support. So we add a specific class for part-support errors.

bundle2: change header size and make them signed (new format) We are changing all integers that denote the size of a chunk to read to int32. There are two main motivations for that. First, we change everything to the same width (32 bits) to make it possible for a reasonably agnostic actor to forward a bundle2 without any extra processing. With this change, this could be achieved by just reading int32s and forwarding chunks of the size read. A bit a smartness would be logic to detect the end of stream but nothing too complicated. Second, we need some capacity to transmit special information during the bundle processing. For example we would like to be able to raise an exception while a part is being read if this exception happend while this part was generated. Having signed integer let us use negative numbers to trigger special events during the parsing of the bundle. The format is renamed for B2X to B2Y because this breaks binary compatibility. The B2X format support is dropped. It was experimental to allow this kind of things. All elements not directly related to the binary format remain flagged "b2x" because they are still compatible.

bundle2: extract processing of part into its own function This is code movement only. This will be useful to have it separated for reuse purposes. We plan to introduce a new feature to the bundle format that allow inserting a part in the middle of another part payload. This will be useful to transmit a exception raised during a part generation.

bundle2: add a test for exceptions raised during the generation process We would like exceptions raised during the generation process to be gracefully handled on the receiver side. We add a test for it. It shows that we are not doing it yet.

dagwalker: drop a useless intermediate variable The variable used to contain revs.set() but as the `.set()` has been removed it is now useless.

revset-node: speedup by a few hundred fold Instead of checking all elements of the subset against a single rev, just check if this rev is in the subset. The old way was inherited from when the subset was a list. Non surprise, this provide massive speedup. id("d82e2223f132") before) wall 0.008205 comb 0.000000 user 0.000000 sys 0.000000 (best of 302) after) wall 0.000069 comb 0.000000 user 0.000000 sys 0.000000 (best of 34518) revset #1: public() and id("d82e2223f132") before) wall 0.019763 comb 0.020000 user 0.020000 sys 0.000000 (best of 124) after) wall 0.000101 comb 0.000000 user 0.000000 sys 0.000000 (best of 20130)

revset-only: use `subset &` instead of filtering The & version is more likely to be optimised. only(.) before) wall 0.003216 comb 0.000000 user 0.000000 sys 0.000000 (best of 768) after) wall 0.001086 comb 0.000000 user 0.000000 sys 0.000000 (best of 2231) only(default, stable) before) wall 0.018469 comb 0.020000 user 0.020000 sys 0.000000 (best of 138) after) wall 0.015888 comb 0.010000 user 0.010000 sys 0.000000 (best of 156)

revset-_ancestor: use & instead of filter The & operation is more likely optimised. ::10 before) wall 0.028189 comb 0.030000 user 0.030000 sys 0.000000 (best of 100) after) wall 0.001050 comb 0.000000 user 0.000000 sys 0.000000 (best of 2326) ::tip before) wall 0.081132 comb 0.080000 user 0.080000 sys 0.000000 (best of 100) after) wall 0.055418 comb 0.050000 user 0.050000 sys 0.000000 (best of 100)

obsstore: record data as floating point in fm0 format For python struct module, "d" is double. But for python string formating, "d" is integer. We want to preserve the floating point nature of the data, so we store it in the metadata as floating point. We use "%r" to make sure we get as many significant digitis as necessary to restore the float to the exact same value on the other side. The fm1 is transmitting the information as float. The lack of this made fm1-stored markers not survive a round-trip to fm0 leading to duplicated markers (or two markers very alike).

bundle2: add a comment about addchangegroup source and url

pull: call the `b2x-transactionclose` hook when closing the transaction We need a wider set of hooks to process all the changes that happened during the pull transaction. We reuse the experimental `b2x-transactionclose` hook set from server's unbundle for consistency. This hook is experimental and will not remains as-is forever, but this will open the door for experimentation in 3.2.

revset-only: use __nonzero__ to check if a revset is empty For some smartsets, computing length is more expensive than checking if the set is empty.

_spanset: drop __getitem__ implementation It is expensive and not part of the official smartset API.

filteredset: drop __getitem__ implementation It is expensive and not part of the official smartset API.

generatorset: implement __len__ It was the only smartset class without a `__len__` implementation.

revset: make __len__ part of the offical API It is common for code to ask for the length of a revset. In fact, all but generatorset already implement it.

cmdserver: add service that listens on unix domain socket and forks process Typical use case of 'unix' mode is a background hg daemon. $ hg serve --cmdserver unix --cwd / -a /tmp/hg-`id -u`.sock Unlike 'pipe' mode in which parent process keeps stdio channel, 'unix' server can be detached. So clients can freely connect and disconnect from server, saving Python start-up time. It might be better to write "--cmdserver socket -a unix:/sockpath" instead of "--cmdserver unix -a /sockpath" in case hgweb gets the ability to listen on unix domain socket.

test-commandserver: add connector for unix domain socket server The next patch will introduce --cmdserver unix.

test-commandserver: allow check() to make connection in different way The next patch will add connector for 'unix' mode server.

test-commandserver: remove unused repopath argument from check() Instead of repopath, check() will receive connect() function as argument. It will allow to connect to server of different mode.