Augie Fackler <raf@durin42.com> [Sun, 01 Oct 2017 05:28:54 -0400] rev 33688
Added signature for changeset 2f427b57bf90
Augie Fackler <raf@durin42.com> [Sun, 01 Oct 2017 05:28:49 -0400] rev 33687
Added tag 4.3.3 for changeset 2f427b57bf90
Jun Wu <quark@fb.com> [Mon, 18 Sep 2017 10:54:00 -0700] rev 33686
rebase: move bookmarks with --keep (issue5682)
This is a regression caused by 3b7cb3d17137. We have documented the behavior
in rebase help:
Rebase will destroy original commits unless you use "--keep". It will
also move your bookmarks (even if you do).
So let's restore the old behavior.
It is done by changing `scmutil.cleanupnodes` to accept more information so
a node could have different "movement destination" from "successors". It
also helps simplifying the callsite as a side effect - the special bookmark
movement logic in rebase is removed.
Differential Revision: https://phab.mercurial-scm.org/D727
Martin von Zweigbergk <martinvonz@google.com> [Wed, 20 Sep 2017 09:32:26 -0700] rev 33685
cleanupnodes: rename "mapping" to "replacements"
The next patch will pass in overrides for bookmark moves, which is a
mapping itself, so let's rename "mapping" to "replacements" to
distinguish them better.
Differential Revision: https://phab.mercurial-scm.org/D749
Martin von Zweigbergk <martinvonz@google.com> [Wed, 20 Sep 2017 09:10:43 -0700] rev 33684
cleanupnodes: separate out bookmark destination calculation from actual update
We will soon want to pass in overrides for bookmark movements and this
will make that patch simpler. I also think this makes the code easier
to follow regardless of the later patch.
Differential Revision: https://phab.mercurial-scm.org/D748
Mark Thomas <mbthomas@fb.com> [Wed, 20 Sep 2017 09:55:52 -0700] rev 33683
ui: check for progress singleton when clearing progress bar (issue5684)
A combination of wrapping `ui` and progress bars interrupted by exceptions can
lead to the progress bar not being cleared when the exception error is printed.
This results in corrupted-looking output like this:
```
updating [===============================> ] 1/2u
nresolved conflicts (see hg resolve, then hg rebase --continue)
```
This is because in `ui._progclear`, we only check the local reference to the
progress bar, not whether or not there is an instance of the singleton. When a
progress bar is interrupted by an exception, the exception printing in
`scmutil.callcatch` uses the original instance of the `ui` object, not the
wrapped copy that has `_progbar` set.
When consider whether or not to clear the progress bar, check for the existence
of the singleton, rather than just whether or not we have a local reference to
it.
Differential Revision: https://phab.mercurial-scm.org/D743
Augie Fackler <raf@durin42.com> [Mon, 18 Sep 2017 11:53:54 -0400] rev 33682
Added signature for changeset 920977f72c7b
Augie Fackler <raf@durin42.com> [Mon, 18 Sep 2017 11:53:53 -0400] rev 33681
Added tag 4.3.2 for changeset 920977f72c7b
Augie Fackler <augie@google.com> [Mon, 18 Sep 2017 11:51:41 -0400] rev 33680
merge with i18n
Wagner Bruna <wbruna@softwareexpress.com.br> [Mon, 31 Jul 2017 12:18:42 -0300] rev 33679
i18n-pt_BR: synchronized with 850d2ec2cf6a
Boris Feld <boris.feld@octobus.net> [Fri, 15 Sep 2017 18:57:50 +0200] rev 33678
hgwebdir: read 'web.template' untrusted
The 'hgweb_mod.py' version of this read it untrusted. For consistency we align
the two versions of this code.
Durham Goode <durham@fb.com> [Mon, 11 Sep 2017 15:59:18 -0700] rev 33677
ssh: fix flakey ssh errors on BSD systems
This is a trivial backport of c037fd655b47 performed by
augie@google.com, but the change is still really Durham's not mine, so
I [augie] am leaving him as the author.
Martin von Zweigbergk <martinvonz@google.com> [Thu, 14 Sep 2017 11:16:57 -0700] rev 33676
repair: preserve phase also when not using generaldelta (issue5678)
It seems like we used to pick the oldest possible version of the
changegroup to use for bundles created by the repair module (used
e.g. by "hg strip" and for temporary bundles by "hg rebase"). I tried
to preserve that behavior when I created the changegroup.safeversion()
method in 3b2ac2115464 (changegroup: introduce safeversion(),
2016-01-19).
However, we have recently chagned our minds and decided that these
commands are only used locally and downgrades are unlikely. That
decicion allowed us to start adding obsmarker and phase information to
these bundles. However, as the bug report shows, it means we get
different behavior e.g. when generaldelta is not enabled (because when
it was enabled, it forced us to use bundle2). The commit that actually
caused the reported bug was 8e3021fd1a44 (strip: include phases in
bundle (BC), 2017-06-15).
So, since we now depend on having more information in the bundles,
let's make sure we instead pick the newest possible changegroup
version.
Differential Revision: https://phab.mercurial-scm.org/D715
Martin von Zweigbergk <martinvonz@google.com> [Thu, 14 Sep 2017 11:16:47 -0700] rev 33675
tests: add test for issue5678
In addition to a test case for the direct problem described in the bug
report, this also adds a test case showing how obsmarkers can also get
lost when not using generaldelta.
Differential Revision: https://phab.mercurial-scm.org/D714
Mads Kiilerich <mads@kiilerich.com> [Mon, 11 Sep 2017 00:42:24 +0200] rev 33674
mq: create non-lossy patches, also with custom global diff configuration
Users with custom [diff] configuration most certainly didn't intend it to make
mq lose changes. It could:
* git is handled perfectly fine.
* nobinary could make mq leave some files out from the patches.
* noprefix could make mq itself (and probably also other tools) fail to apply
patches without the usual a/b prefix.
* ignorews, ignorewsamount, or ignoreblanklines could create patches with
missing whitespace that could fail to apply correctly.
Thus, when refreshing patches, use patch.difffeatureopts, optionally with git
as before, but without the config options for whitespace and format changing
that most likely will cause loss or problems.
(patch.diffopts is just patch.difffeatureopts with all options enabled and can
be replaced with that.)
Mads Kiilerich <mads@kiilerich.com> [Mon, 11 Sep 2017 00:42:22 +0200] rev 33673
mq: test coverage of how [diff] configuration influence can break mq patches
Matt Harbison <matt_harbison@yahoo.com> [Tue, 29 Aug 2017 16:38:10 -0400] rev 33672
debugssl: allow a URL to be specified without a local repository
This was the original intent, but I bungled the logic. Otherwise if there is a
certificate chain issue, the repository can't be cloned in order for there to be
a repo object. I think I missed this case because I was inside of a Mercurial
clone as I was originally developing and testing this.
Jun Wu <quark@fb.com> [Sun, 27 Aug 2017 13:39:17 -0700] rev 33671
record: fix revert -i for lines without newline (issue5651)
This is a regression caused by 66117dae87f9. Code prior to 66117dae87f9
seems to miss the "\ No newline at end of file" line.
Differential Revision: https://phab.mercurial-scm.org/D528
Martin von Zweigbergk <martinvonz@google.com> [Tue, 15 Aug 2017 23:23:55 -0700] rev 33670
templatekw: choose {latesttag} by len(changes), not date (issue5659)
As Augie reported in the bug, the current heuristic of choosing the
best tag of a merge commit by taking the one with newest tag (in terms
of tagging date) currently fails in the Mercurial repo itself. Copying
the example from Yuya:
$ hg glog -T '{node|short} {latesttag}+{latesttagdistance}\n' \
-r '4.2.3: & (merge() + parents(merge()) + tag())'
o 02a745c20121 4.2.3+5
|\
| o 86aca74a063b 4.2.3+4
| |\
| | o e6d8ee3c9ec3 4.3-rc+109
| | |
| | ~
o | a3ce07e2dde5 4.3.1+2
: |
o | 3fee7f7d2da0 4.3.1+0
|/
o 98e990bb7330 4.2.3+3
|\
| ~
o 506d7e48fbe6 4.2.3+2
:
o 943c91326b23 4.2.3+0
|
~
It seems to me like the best choice is the tag with the smallest
number of changes since it (across all paths, not the longest single
path). So that's what this patch does, even though it's
costly. Best-of-5 timings for Yuya's command above shows a slowdown
from 1.293s to 1.610s. We can optimize it later.
Differential Revision: https://phab.mercurial-scm.org/D447
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Aug 2017 12:50:26 -0700] rev 33669
tests: use graph log in {latesttag} tests
The tests are much easier to read if one does not have to re-read the
setup part all the time to understand the graph shape.
Differential Revision: https://phab.mercurial-scm.org/D446
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 15 Aug 2017 13:04:31 -0700] rev 33668
ui: restore behavior to ignore some I/O errors (issue5658)
e9646ff34d55 and 1bfb9a63b98e refactored ui methods to no longer
silently swallow some IOError instances. This is arguably the
correct thing to do. However, it had the unfortunate side-effect
of causing StdioError to bubble up to sensitive code like
transaction aborts, leading to an uncaught exceptions and failures
to e.g. roll back a transaction. This could occur when a remote
HTTP or SSH client connection dropped. The new behavior is
resulting in semi-frequent "abandonded transaction" errors on
multiple high-volume repositories at Mozilla.
This commit effectively reverts e9646ff34d55 and 1bfb9a63b98e to
restore the old behavior.
I agree with the principle that I/O errors shouldn't be ignored.
That makes this change... unfortunate. However, our hands are tied
for what to do on stable. I think the proper solution is for the
ui's behavior to be configurable (possibly via a context manager).
During critical sections like transaction rollback and abort, it
should be possible to suppress errors. But this feature would not
be appropriate on stable.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 14 Aug 2017 13:12:40 -0700] rev 33667
tests: test behavior of IOError during transactions (issue5658)
ui._write(), ui._write_err(), and ui.flush() all trap IOError and
re-raise as error.StdioError. If a caller doesn't catch StdioError
when writing to stdio, it could bubble all the way to dispatch.
This commit adds tests for I/O failures around various transaction
operations.
The most notable badness is during abort. Here, an uncaught StdioError
will result in incomplete transaction rollback, requiring an
`hg rollback` to recover. This can result in a client "corrupting"
a remote repo via terminated HTTP and SSH socket.
Nathan Goldbaum <ngoldbau@illinois.edu> [Wed, 16 Aug 2017 10:24:49 -0500] rev 33666
log: mention ui.logtemplate in the help text
Tristan Seligmann <mithrandi@mithrandi.net> [Sat, 12 Aug 2017 14:29:22 +0200] rev 33665
hg: avoid relying on errno numbers / descriptions
A few tests hardcode errno numbers and/or descriptions in the output, causing
test failures on platforms where these values are different.
Differential Revision: https://phab.mercurial-scm.org/D362
Tristan Seligmann <mithrandi@mithrandi.net> [Sat, 12 Aug 2017 14:24:25 +0200] rev 33664
hg: tolerate long vs. int in test-context.py
The file times here can be longs instead of ints on some platforms, which will
cause a test failure due to these printing with an L suffix; instead always
format with %d which will produce the same output in either case.
Differential Revision: https://phab.mercurial-scm.org/D361
Mike Hommey <mh@glandium.org> [Fri, 11 Aug 2017 10:16:00 +0900] rev 33663
setup: fix installing in a mingw environment
The addition, in 9a4adc76c88a, of a hack for the MSVC compiler class was
overwriting the original class for the Mingw32CCompiler class, leading to an
error when the HackedMingw32CCompiler is instantiated.
Differential Revision: https://phab.mercurial-scm.org/D329
Kevin Bullock <kbullock+mercurial@ringworld.org> [Thu, 10 Aug 2017 21:25:02 -0400] rev 33662
osx: delay version computation on macOS builds
Before this patch, HGVER would be evaluated at the beginning of the make
execution, and would be unset because build/mercurial/ doesn't exist yet
at that point. Now we compute the version after the `make install` run
has completed.
This is backported to stable from 8626b44516c1, but that revision had an
error in the shell invocation syntax.
Mike Hommey <mh@glandium.org> [Thu, 10 Aug 2017 20:47:19 -0700] rev 33661
branchmap: revert c34532365b38 for Python 2.7 compatibility
Old versions of python 2.7 don't like that the second argument to
struct.unpack_from is a bytearray, so the change removing the util.buffer
around that argument in branchmap broke running on older versions of python
2.7.
Differential Revision: https://phab.mercurial-scm.org/D330
Augie Fackler <raf@durin42.com> [Thu, 10 Aug 2017 18:46:55 -0400] rev 33660
Added signature for changeset 3fee7f7d2da0
Augie Fackler <raf@durin42.com> [Thu, 10 Aug 2017 18:46:54 -0400] rev 33659
Added tag 4.3.1 for changeset 3fee7f7d2da0
Yuya Nishihara <yuya@tcha.org> [Mon, 07 Aug 2017 22:22:28 +0900] rev 33658
ssh: unban the use of pipe character in user@host:port string
This vulnerability was fixed by the previous patch and there were more ways
to exploit than using '|shellcmd'. So it doesn't make sense to reject only
pipe character.
Test cases are updated to actually try to exploit the bug. As the SSH bridge
of git/svn subrepos are not managed by our code, the tests for non-hg subrepos
are just removed.
This may be folded into the original patches.
Jun Wu <quark@fb.com> [Fri, 04 Aug 2017 23:54:12 -0700] rev 33657
ssh: quote parameters using shellquote (SEC)
This patch uses shellquote to quote ssh parameters more strictly to avoid
shell injection.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 14:55:11 -0700] rev 33656
subrepo: add tests for git rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.
When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 16:44:17 -0700] rev 33655
subrepo: add tests for svn rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.
When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 16:04:44 -0700] rev 33654
subrepo: add tests for hg rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.
When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 14:40:28 -0700] rev 33653
push: add tests for unsafe ssh url (SEC)
Sean Farley <sean@farley.io> [Fri, 28 Jul 2017 16:47:32 -0700] rev 33652
pull: add tests for unsafe ssh url (SEC)
Sean Farley <sean@farley.io> [Fri, 28 Jul 2017 16:36:36 -0700] rev 33651
clone: add tests for unsafe ssh url (SEC)
Sean Farley <sean@farley.io> [Tue, 01 Aug 2017 14:40:19 -0700] rev 33650
sshpeer: check for safe ssh url (SEC)
Checking in the sshpeer for a rogue ssh:// urls seems like the right
place to do it (instead of whack-a-mole with pull, clone, push, etc).
Augie Fackler <augie@google.com> [Fri, 04 Aug 2017 14:00:03 -0400] rev 33649
ssh: ban any username@host or host that starts with - (SEC)
This paranoia probably isn't required, but it can't hurt either.
Sean Farley <sean@farley.io> [Fri, 28 Jul 2017 16:32:25 -0700] rev 33648
util: add utility method to check for bad ssh urls (SEC)
Our use of SSH has an exploit that will parse the first part of an url
blindly as a hostname. Prior to this set of security patches, a url
with '-oProxyCommand' could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' can be abused to execute
arbitrary commands in a similar fashion.
We defend against this by checking ssh:// URLs and looking for a
hostname that starts with a - or contains a |.
When this happens, let's throw a big abort into the user's face so
that they can inspect what's going on.
Yuya Nishihara <yuya@tcha.org> [Wed, 26 Jul 2017 22:10:15 +0900] rev 33647
pathauditor: disable cache of audited paths by default (issue5628)
The initial attempt was to discard cache when appropriate, but it appears
to be error prone. We had to carefully inspect all places where audit() is
called e.g. without actually updating filesystem, before removing files and
directories, etc.
So, this patch disables the cache of audited paths by default, and enables
it only for the following cases:
- short-lived auditor objects
- repo.vfs, repo.svfs, and repo.cachevfs, which are managed directories
and considered sort of append-only (a file/directory would never be
replaced with a symlink)
There would be more cacheable vfs objects (e.g. mq.queue.opener), but I
decided not to inspect all of them in this patch. We can make them cached
later.
Benchmark result:
- using old clone of http://selenic.com/repo/linux-2.6/ (38319 files)
- on tmpfs
- run HGRCPATH=/dev/null hg up -q --time tip && hg up -q null
- try 4 times and take the last three results
original:
real 7.480 secs (user 1.140+22.760 sys 0.150+1.690)
real 8.010 secs (user 1.070+22.280 sys 0.170+2.120)
real 7.470 secs (user 1.120+22.390 sys 0.120+1.910)
clearcache (the other series):
real 7.680 secs (user 1.120+23.420 sys 0.140+1.970)
real 7.670 secs (user 1.110+23.620 sys 0.130+1.810)
real 7.740 secs (user 1.090+23.510 sys 0.160+1.940)
enable cache only for vfs and svfs (this series):
real 8.730 secs (user 1.500+25.190 sys 0.260+2.260)
real 8.750 secs (user 1.490+25.170 sys 0.250+2.340)
real 9.010 secs (user 1.680+25.340 sys 0.280+2.540)
remove cache function at all (for reference):
real 9.620 secs (user 1.440+27.120 sys 0.250+2.980)
real 9.420 secs (user 1.400+26.940 sys 0.320+3.130)
real 9.760 secs (user 1.530+27.270 sys 0.250+2.970)
Yuya Nishihara <yuya@tcha.org> [Tue, 01 Aug 2017 21:03:25 +0900] rev 33646
tests: show cache of audited paths is never invalidated
Augie Fackler <augie@google.com> [Thu, 10 Aug 2017 14:23:25 -0400] rev 33645
stable: merge heads
Augie Fackler <raf@durin42.com> [Thu, 10 Aug 2017 14:14:49 -0400] rev 33644
Added signature for changeset 943c91326b23
Augie Fackler <raf@durin42.com> [Thu, 10 Aug 2017 14:14:48 -0400] rev 33643
Added tag 4.2.3 for changeset 943c91326b23
Yuya Nishihara <yuya@tcha.org> [Mon, 07 Aug 2017 22:22:28 +0900] rev 33642
ssh: unban the use of pipe character in user@host:port string
This vulnerability was fixed by the previous patch and there were more ways
to exploit than using '|shellcmd'. So it doesn't make sense to reject only
pipe character.
Test cases are updated to actually try to exploit the bug. As the SSH bridge
of git/svn subrepos are not managed by our code, the tests for non-hg subrepos
are just removed.
This may be folded into the original patches.
Jun Wu <quark@fb.com> [Fri, 04 Aug 2017 23:54:12 -0700] rev 33641
ssh: quote parameters using shellquote (SEC)
This patch uses shellquote to quote ssh parameters more strictly to avoid
shell injection.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 14:55:11 -0700] rev 33640
subrepo: add tests for git rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.
When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 16:44:17 -0700] rev 33639
subrepo: add tests for svn rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.
When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 16:04:44 -0700] rev 33638
subrepo: add tests for hg rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.
When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
Sean Farley <sean@farley.io> [Mon, 31 Jul 2017 14:40:28 -0700] rev 33637
push: add tests for unsafe ssh url (SEC)
Sean Farley <sean@farley.io> [Fri, 28 Jul 2017 16:47:32 -0700] rev 33636
pull: add tests for unsafe ssh url (SEC)
Sean Farley <sean@farley.io> [Fri, 28 Jul 2017 16:36:36 -0700] rev 33635
clone: add tests for unsafe ssh url (SEC)
Sean Farley <sean@farley.io> [Tue, 01 Aug 2017 14:40:19 -0700] rev 33634
sshpeer: check for safe ssh url (SEC)
Checking in the sshpeer for a rogue ssh:// urls seems like the right
place to do it (instead of whack-a-mole with pull, clone, push, etc).
Augie Fackler <augie@google.com> [Fri, 04 Aug 2017 14:00:03 -0400] rev 33633
ssh: ban any username@host or host that starts with - (SEC)
This paranoia probably isn't required, but it can't hurt either.