test: demonstrate crash with in-memory rebase and copies In the added test case, there is a merge commit that has one obsolete parent with a rename. Since the rename is not in the other parent, pathcopies() from that other parent will include the copy. Then when we try to rebase this merge commit onto another commit that has the same content changes, but no tracking of the rename (because it was done with "hg remove; hg add" instead of "hg mv"), we try to propagate the copy information. That fails because overlayworkingctx expects a file to be modified if it's going to have copy information. Differential Revision: https://phab.mercurial-scm.org/D6132

manifestcache: actually honor --clear Before this change, the --clear flag was not clearing the on disk cache. (We also remove the extra verbosity when using --clear. Same as what we did for --add)

manifestcache: make sure the entry are ordered by access time This is an LRU cache, let us make sure of that.

manifestcache: adding a second distinct entry Let makes sure the cache can hold multiple value.

manifestcache: test that adding the same entry twice do not duplicates it Simple sanity check.

manifestcache: do not display data when using --add If the command invocation is about adding a new entry, we should remain terse (the same as we do for many commands).

manifestcache: only lock the repository if the debug command touch the cache Not doing so had two consequences: 1) the command cannot be run on read only repositories, 2) when using --add on an empty cache, the command crash prematurely trying to read the cache file on disk.

manifestcache: further fix to debug command output Removing more capital letters. The output will get a test once other issues get fixed.

manifestcache: test and fix some output of the debug command The message was lacking an end of line. In addition we do not capitalize output in Mercurial.

chunkselector: fix typos in instructions when user reviews patch Differential Revision: https://phab.mercurial-scm.org/D6121

scmutil: document matcher argument of movedirstate() Differential Revision: https://phab.mercurial-scm.org/D6120

uncommit: move _movedirstate() to scmutil for reuse The function should be applicable generically when moving from one commit to another. I'll try to add more callers when I find time. I'm not convinced it's handling all the cases correctly, but we should have a generic function for this kind of operation, so I think it belongs somewhere in core (not in the uncommit extension). Differential Revision: https://phab.mercurial-scm.org/D6119

copies: remove dependency on scmutil by directly using match.exact() I want to add a dependency from scmutil.copies(), so I need to remove this dependency first. Differential Revision: https://phab.mercurial-scm.org/D6118

uncommit: convert _fixdirstate() into _movedirstate() _fixdirstate() already also updates to the given commit, so let's rename it to _movedirstate(). Also update the documentation and drop the unnecessary "curctx" argument, since that should always be repo['.']. Differential Revision: https://phab.mercurial-scm.org/D6117

updatecaches: also warm the tags caches Resolving any name requires the tags cache to be warm. We make sure that `hg debugupdatecache` warm the tag cache entry too.

updatecaches: also warm revbranchcache for filtered revisions We are in the "full" case, so we better warm everything we can.

copies: handle a case when both merging csets are not descendant of merge base This patch fix the behaviour of fullcopytracing algorithm in the case when both the merging csets are not the descendant of merge base. Although it seems to be the rare case when both the csets are not descendant of merge base. But it can be seen in most of cases of content-divergence in evolve extension, where merge base is the common predecessor. Previous patch added a test where this algorithm can fail to continue because of an assumption that only one of the two csets can be dirty. This patch fix that error. For refrence I suggest you to look into the previous discussion held on a patch sent by Pulkit: https://phab.mercurial-scm.org/D3896 Differential Revision: https://phab.mercurial-scm.org/D5963

copies: add test that makes both the merging csets dirty and fails This patch is a part of series which is about the case when both the merging csets are not descendant of merge base. The existing code assumes if c1 is dirty there shouldn't be any partial copies from c2 i.e both2['incomplete'] and same for c2, if c2 is dirty both1['incomplete'] should be empty, but this is not the right assumption. Now as we know we can have both c1 and c2 dirty at the same time, it is possible that c1 is dirty and both2['incomplete'] has some value. Or if c2 is dirty and both1['incomplete'] has some value. Added test shows that because of this assumption it could fail. Differential Revision: https://phab.mercurial-scm.org/D5962

copies: add test that makes both the merging csets dirty and run w/o error This series of patches is to cover a case in fullcopytracing algorithms where both the merging csets are not descendant of merge base. In this algorithm we call a merging cset "dirty" if that cset is not the descendant of merge base. That said, added test in this patch cover case when both the merging csets are "dirty". Actually this case of "both dirty" was encountered by Pulkit when he was working on content-divergence where it is possible that both the csets are not descendant of merging base. For reference you can look into: https://phab.mercurial-scm.org/D3896 As this test run fine without any error and correctly traced the copies, I added this test to make sure that it doesn't break even after I will modify some code in next patches to fix an error. Next patch adds the tests where this algorithm throws an error for the same case of "both dirty". Differential Revision: https://phab.mercurial-scm.org/D5961

tests: stabilize test-bundle.t on Windows Similar to 92055d539e49.

discovery-helper: use reflink copy if available A reflink copy will copy the files "as usual" but keep using the same data block underneath. This is only supported by "copy on write" file system like btrfs or zfs. This will achieve similar performance that the existing hardlink clone that Mercurial performs with the same initial space saving. However, it will behave better on revlogs start being touch by strip. Instead of duplicating all data in the touched revlogs, only the block actually affected by the strip will be duplicated. This save a lot of space when building many variants of large repositories. The --reflink=always flag make sure the `cp` call fails if reflink copies are not supported. Falling back to local clone.

discovery-helper: bail out if destination already exists

discovery-helper: move repository creation in a function This makes it easier to update this duplicated code. (we do a small output fix as we go)

discovery-helper: add an extra argument to generate only one repo This is useful to generate left and right in parallel when dealing with very large repositories.

wix: remove enum and future packages These were cargo culted from the THG installer code. I'm not sure what needs them in THG land. But the official MSIs certainly do not - at least not as direct dependencies. .. bc:: The Windows MSI installers no longer include the enum and future Python packages. Differential Revision: https://phab.mercurial-scm.org/D6101

wix: remove pywin32 This dependency was for ancient Mercurial versions. We recently removed it from the Inno Setup installers. So let's remove it from the WiX installers as well. .. bc:: The Windows MSI installers no longer include the pywin32 Python package. Differential Revision: https://phab.mercurial-scm.org/D6100

wix: remove sphinx and dependencies Sphinx was cargo culted into our install environment as part of emulating TortoiseHG's behavior. THG seems to install Sphinx in order to generate THG specific documentation. We don't appear to need Sphinx or any of its dependencies in the official WiX installers. So remove it. This shaves ~1MB off the size of the MSI installers. .. bc:: The Windows MSI installers no longer include the Python sphinx package and its various dependencies. Differential Revision: https://phab.mercurial-scm.org/D6099

wix: functionality to automate building WiX installers Like we did for Inno Setup, we want to make it easier to produce WiX installers. This commit does that. We introduce a new hgpackaging.wix module for performing all the high-level tasks required to produce WiX installers. This required miscellaneous enhancements to existing code in hgpackaging, including support for signing binaries. A new build.py script for calling into the module APIs has been created. It behaves very similarly to the Inno Setup build.py script. Unlike Inno Setup, we didn't have code in the repo previously to generate WiX installers. It appears that all existing automation for building WiX installers lives in the https://bitbucket.org/tortoisehg/thg-winbuild repository - most notably in its setup.py file. My strategy for inventing the code in this commit was to step through the code in that repo's setup.py and observe what it was doing. Despite the length of setup.py in that repository, the actual amount of steps required to produce a WiX installer is actually quite low. It consists of a basic py2exe build plus invocations of candle.exe and light.exe to produce the MSI. One rabbit hole that gave me fits was locating the Visual Studio 9 C Runtime merge modules. These merge modules are only present on your system if you have a full Visual Studio 2008 installation. Fortunately, I have a copy of Visual Studio 2008 and was able to install all the required updates. I then uploaded these merge modules to a personal repository on GitHub. That is where the added code references them from. We probably don't need to ship the merge modules. But that is for another day. The installs from the MSIs produced with the new automation differ from the last official MSI in the following ways: * Our HTML manual pages have UNIX line endings instead of Windows. * We ship modules in the mercurial.pure package. It appears the upstream packaging code is not including this package due to omission (they supply an explicit list of packages that has drifted out of sync with our setup.py). * We do not ship various distutils.* modules. This is because virtualenvs have a custom distutils/__init__.py that automagically imports distutils from its original location and py2exe gets confused by this. We don't use distutils in core Mercurial and don't provide a usable python.exe, so this omission should be acceptable. * The version of the enum package is different and we ship an enum.pyc instead of an enum/__init__.py. * The version of the docutils package is different and we ship a different set of files. * The version of Sphinx is drastically newer and we ship a number of files the old version did not. (I'm not sure why we ship Sphinx - I think it is a side-effect of the way the THG code was installing dependencies.) * We ship the idna package (dependent of requests which is a dependency of newer versions of Sphinx). * The version of imagesize is different and we ship an imagesize.pyc instead of an imagesize/__init__.pyc. * The version of the jinja2 package is different and the sets of files differs. * We ship the packaging package, which is a dependency for Sphinx. * The version of the pygments package is different and the sets of files differs. * We ship the requests package, which is a dependency for Sphinx. * We ship the snowballstemmer package, which is a dependency for Sphinx. * We ship the urllib3 package, which is a dependency for requests, which is a dependency for Sphinx. * We ship a newer version of the futures package, which includes a handful of extra modules that match Python 3 module names. # no-check-commit because foo_bar naming Differential Revision: https://phab.mercurial-scm.org/D6097

wix: move contrib/wix to contrib/packaging/wix We're trying to consolidate all our packaging code into contrib/packaging. Let's move the WiX files there. Differential Revision: https://phab.mercurial-scm.org/D6096

wix: remove hg.cmd This file is not referenced anywhere AFAICT. Differential Revision: https://phab.mercurial-scm.org/D6095