strip: invalidate phase cache after stripping changeset (issue5235) When we remove a changeset from the changelog, the phase cache must be invalidated, otherwise it could refer to changesets that are no longer in the repo. To reproduce the failure, I created an extension querying the phase cache after the strip transaction is over. To do that, I stripped two commits with a bookmark on one of them to force another transaction (we open a transaction for moving bookmarks) after the strip transaction. Without the fix in this patch, the test leads to a stacktrace showing the issue: repair.strip(ui, repo, revs, backup) File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/repair.py", line 205, in strip tr.close() File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/transaction.py", line 44, in _active return func(self, *args, **kwds) File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/transaction.py", line 490, in close self._postclosecallback[cat](self) File "$TESTTMP/crashstrip2.py", line 4, in test [repo.changelog.node(r) for r in repo.revs("not public()")] File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/changelog.py", line 337, in node return super(changelog, self).node(rev) File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/revlog.py", line 377, in node return self.index[rev][7] IndexError: revlog index out of range The situation was encountered in inhibit (evolve's repo) where we would crash following the volatile set invalidation submitted by Augie in e6f490e328635312ee214a12bc7fd3c7d46bf9ce. Before his patch the issue was masked as we were not accessing the phasecache after stripping a revision. This bug uncovered another but in histedit (see explanation in issue5235). I changed the histedit test accordingly to avoid fixing two things at once.

py3: make tests/svn-safe-append.py use absolute_import

py3: make tests/test-atomictempfile.py use absolute_import

py3: tests/test-check-py3-compat.t output updated The lower part of the tests runs with Python 3.5 so its remains unchanged with new commits.

py3: use setattr() to assign new class attribute The old method produces error 'object does not supports item assignment'. So setattr() is used to assign a new class attribute via __dict__ .

localrepo: use dirstate savebackup instead of handling dirstate file manually This is one step towards having dirstate manage its own storage. It will be useful for the implementation of sql dirstate [1]. This introduced a small test change: now we always write the dirstate before saving backup so in some cases where dirstate file didn't exist yet savebackup can create it. [1] https://www.mercurial-scm.org/wiki/SQLDirstatePlan

localrepo: use dirstate restorebackup instead of copying dirstate manually This is one step towards having dirstate manage its own storage. It will be useful for the implementation of sqldirstate [1]. I'm deleting two of the dirstate.invalidate() calls in localrepo because restorebackup method does that for us. [1] https://www.mercurial-scm.org/wiki/SQLDirstatePlan

dirstate: add prefix and suffix arguments to backup This would allow the code explicitly copying dirstate to use this method instead. Use of this method will increase encapsulation (the dirstate class will be sole owner of its on-disk storage).

tests: mark test-atomictempfile.py write as binary

tests: mark test-context.py write as binary

transaction: turn lack of locking into a hard failure (API) We have been warning about transactions without locks for about a year (and three releases), third party extensions had a fair grace period to fix their code, we are moving lack of locking to a hard failure in order to protect users against repository corruption.

test: extract develwarn transaction testing in its own command The lack of locking for a transation is about to change from a warning to an error. We first extract the test decidated to this warning to make the next changeset clearer.

graphmod: update edgemap in-place The edgemap update was not actually propagated to future asciiedge calls; update the edge state dictionary in-place instead.

tests: test-archive.t use mercurial.util for urllib compat

tests: test-archive.t use sys.stdout.buffer for binary output in py3

localrepo: prevent executable-bit only changes from being lost on amend If you have just executable-bit change and amend it twice it will vanish: * After the first amend the commit will have the proper executable bit set in manifest but it won't have the the file on the list of files in changelog. * The second amend will read the wrong list of files from changelog and it will copy the manifest entry from parent for this file. * Voila! The change is lost. This change repairs the bug in localrepo causing this and adds a test for it.

tests: escape bytes setting MSB in input of grep for portability GNU grep (2.21-2 or later) assumes that input is encoded in LC_CTYPE, and input is binary if it contains byte sequence not valid for that encoding. For example, if locale is configured as C, a byte setting most significant bit (MSB) makes such GNU grep show "Binary file <FILENAME> matches" message instead of matched lines unintentionally. This behavior is recognized as a bug, and fixed in GNU grep 2.25-1 or later. But some distributions are shipped with such buggy version (e.g. Ubuntu xenial, which is used by launchpad buildbot). http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19230 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800670 http://packages.ubuntu.com/xenial/grep This causes failure of test-commit-interactive.t, which applies grep on CP932 byte sequence since 1111e84de635. But, explicit setting LC_CTYPE for CP932 might cause another problem, because it can't be assumed that all environment running Mercurial tests allows arbitrary locale setting. To resolve this issue, this patch escapes bytes setting MSB in input of grep. For this purpose: - str.encode('string-escape') isn't useful, because it escapes also control code (less than 0x20), and makes EOL handling complicated - "f --hexdump" isn't useful, because it isn't line-oriented - "sed -n" seems reasonable, but "sed" itself sometimes causes portability issue, too (e.g. 900767dfa80d or afb86ee925bf) This patch is posted with "stable" flag, because 1111e84de635 is on stable branch.

tests: test-archive.t use absolute_import This is a step to adding a mercurial dependency to simplify py3 compat

tests: test-archive.t use open() instead of file() for py3 compat

tests: test-archive.t use print_function

merge with stable

readlink: use print_function

tests: test-addremove-similar.t use print() for py3

tests: add coverage for run-tests.py --whitelist

hg: limit HGUNICODEPEDANTRY to py2 reload is not available in py3, and py3 is fatal anyway

py3: make i18n/hggettext use print_function

py3: make i18n/hggettext use absolute_import

py3: make doc/docchecker use print_function

py3: make doc/docchecker use absolute_import

py3: make contrib/undumprevlog use absolute_import

py3: make contrib/dumprevlog use print_function

py3: make contrib/dumprevlog use absolute_import

py3: make contrib/check-commit use print_function

py3: make contrib/check-commit use absolute_import

hgcia: remove hgcia (BC) As discussed at: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-March/081018.html, cia service is down for years. It also uses socket.setdefaulttimeout() which will break chg. This patch removes the extension.

py3: make tests/hghave use absolute_import

py3: make tests/f use absolute_import

py3: make tests/dummyssh use absolute_import

py3: make raise statement python3 compatible In python3 raise error, message has been changed to raise error(message) In additional to that nodes.SkipNode is changed to nodes.SkipNode() so that it creates an instance directly.

Added signature for changeset aaabed77791a

Added tag 3.8.2 for changeset aaabed77791a

help: search section of help topic by translated section name correctly Before this patch, "hg help topic.section" might show unexpected section of help topic in some encoding. It applies str.lower() instead of encoding.lower(str) on translated message to search section case-insensitively, but some encoding uses 0x41(A) - 0x5a(Z) as the second or later byte of multi-byte character (for example, ja_JP.cp932), and str.lower() causes unexpected result. To search section of help topic by translated section name correctly, this patch replaces str.lower() by encoding.lower(str) for both query string (in commands.help()) and translated help text (in minirst.getsections()).

patch: show lower-ed translated message correctly Before this patch, patch.filterpatch() shows meaningless translation of help message for chunk selection in some encoding. It applies str.lower() instead of encoding.lower(str) on translated message, but some encoding uses 0x41(A) - 0x5a(Z) as the second or later byte of multi-byte character (for example, ja_JP.cp932), and str.lower() causes unexpected result. To show lower-ed translated message correctly, this patch replaces str.lower() by encoding.lower(str).

py3: make i18n/posplit use print_function

py3: make i18n/posplit use absolute_import

wireproto: optimize handling of large batch responses Now that batch can be used by remotefilelog, the quadratic string copying this was doing was actually disastrous. In my local testing, fetching a 56 meg file used to take 3 minutes, and now takes only a few seconds.

cleanup: replace False identity testing with an explicit token object The recommended way to check default value (when None is not as option) is a token object. Identity testing to integer is less explicit and not guaranteed to work in all implementations.

devel: officially deprecate dirstate.write without transaction argument When we introduce the develwarning, we did not had an official deprecation API and infrastructure. We can now officially deprecate the old way with a version deadline.

devel: officially deprecate update without destination When we introduce the develwarning, we did not had an official deprecation API and infrastructure. We can now officially deprecate the old way with a version deadline.

devel: fix a typo in a deprecation warning Credit goes to Sean Farley for spotting it.

devel: officially deprecate old style revset When we introduce the develwarning, we did not had an official deprecation API and infrastructure. We can now officially deprecate the old way with a version deadline.

check-code: handle py3 open divergence open() really wants an encoding attribute

check-code: switch to opener

check-code: handle range/xrange divergence

check-code: fix py3 complaint about \NNN being invalid unicode

hghave: switch from iteritems to items With this, test-hghave.t passes on python 3. Four features fail because mercurial still is not py3 safe: absimport cacheable hardlink defaultcacerts But that will be resolved automatically eventually.

hghave: matchoutput needs to use bytes for regexp file output is bytes in py3, so we need each regexp to be bytes

revset: make dagrange preserve order of input set Unlike range, dagrange has no inverted range (such as '10:0'). So there should be no practical reason to keep dagrange as a function that forces its own ordering. No performance regression is spotted in contrib/base-revsets.txt.

tests: mark test-check-pyflakes.t as requiring hg1.0+ hg does not yet run with py3, so if you try: ./run-tests.py --local test-check-pyflakes.t ... it will try to run the local hg, which does not work and thus, hg locate will return no output to stdout (and stderr is sent to /dev/null). If you do: ./run-tests.py --with-hg=~/bin/hg test-check-pyflakes.t Then it should work, if your hg is new enough to have a locate command (hg0.6 does not have locate).

dirstate: make backup methods public They are called from outside of dirstate anyway and I want the localrepo to use them too.

check-code: add a rule banning `env -u`

subrepo: use unset instead of env -u to fix test on BSDs (issue5229)

graphmod: partial edge styling Allow for a style to only apply to the last N lines (for positive N) or everything but the first N lines (for negative N) of the section along the current node. This allows for more subtle grandparent styling. So from the default: $ hg log -G ... o Lorem ipsum dolor sit :\ amet, consectetur : : adipiscing elit, sed : : do eiusmod tempor : : o : incididunt ut labore | : et dolore magna | : aliqua. Ut enim ad | : minim veniam, quis |/ o nostrud exercitation : ullamco laboris nisi : ut aliquip ex ea : commodo consequat. : o Duis aute irure dolor | in reprehenderit in ~ voluptate velit esse cillum dolore eu to $ hg log -G --config "experimental.graphstyle.grandparent=2." ... o Lorem ipsum dolor sit |\ amet, consectetur | | adipiscing elit, sed . . do eiusmod tempor . . o | incididunt ut labore | | et dolore magna | | aliqua. Ut enim ad | | minim veniam, quis |/ o nostrud exercitation | ullamco laboris nisi | ut aliquip ex ea . commodo consequat. . o Duis aute irure dolor | in reprehenderit in ~ voluptate velit esse cillum dolore eu or $ hg log -G --config "experimental.graphstyle.grandparent=1:" ... o Lorem ipsum dolor sit |\ amet, consectetur | | adipiscing elit, sed | | do eiusmod tempor : : o | incididunt ut labore | | et dolore magna | | aliqua. Ut enim ad | | minim veniam, quis |/ o nostrud exercitation | ullamco laboris nisi | ut aliquip ex ea | commodo consequat. : o Duis aute irure dolor | in reprehenderit in ~ voluptate velit esse cillum dolore eu or $ hg log -G --config "experimental.graphstyle.grandparent=-2!" ... o Lorem ipsum dolor sit |\ amet, consectetur ! ! adipiscing elit, sed ! ! do eiusmod tempor ! ! o | incididunt ut labore | | et dolore magna | | aliqua. Ut enim ad | | minim veniam, quis |/ o nostrud exercitation | ullamco laboris nisi ! ut aliquip ex ea ! commodo consequat. ! o Duis aute irure dolor | in reprehenderit in ~ voluptate velit esse cillum dolore eu

pure: write a really lazy version of pure indexObject On PyPy this version performs reasonably well compared to C version. Example command is "hg id" which gets faster, depending on details of your operating system and hard drive (it's bottlenecked on stat mostly) There is potential for improvements by storing extra as a condensed struct too.