windows: degrade to py2 behavior when reading a non-symlink as a symlink
While waiting for the push to hg-committed in WSL to complete, I ran a
`phabimport` from Windows and got this traceback:
$ hg phabimport 11313
** Unknown exception encountered with possibly-broken third-party extension "mercurial_keyring" (version N/A)
** which supports versions unknown of Mercurial.
** Please disable "mercurial_keyring" and try your action again.
** If that fixes the bug please report it to https://foss.heptapod.net/mercurial/mercurial_keyring/issues
** Python 3.9.5 (default, May 6 2021, 17:29:31) [MSC v.1928 64 bit (AMD64)]
** Mercurial Distributed SCM (version 5.9rc1+hg32.
0e2f5733563d)
** Extensions loaded: absorb, blackbox, evolve 10.3.3, extdiff, fastannotate, fix, mercurial_keyring, mq, phabblocker
20210126, phabricator, rebase, show, strip, topic 0.22.3
Traceback (most recent call last):
File "mercurial.lock", line 279, in _trylock
File "mercurial.vfs", line 202, in makelock
File "mercurial.util", line 2147, in makelock
FileExistsError: [WinError 183] Cannot create a file when that file already exists: b'hp-omen:78348' -> b'C:\\Users\\Matt\\hg/.hg/store/lock'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 24, in <module>
File "mercurial.dispatch", line 144, in run
File "mercurial.dispatch", line 250, in dispatch
File "mercurial.dispatch", line 294, in _rundispatch
File "mercurial.dispatch", line 470, in _runcatch
File "mercurial.dispatch", line 480, in _callcatch
File "mercurial.scmutil", line 153, in callcatch
File "mercurial.dispatch", line 460, in _runcatchfunc
File "mercurial.dispatch", line 1273, in _dispatch
File "mercurial.dispatch", line 918, in runcommand
File "mercurial.dispatch", line 1285, in _runcommand
File "mercurial.dispatch", line 1271, in <lambda>
File "mercurial.util", line 1886, in check
File "mercurial.util", line 1886, in check
File "hgext.mq", line 4239, in mqcommand
File "mercurial.util", line 1886, in check
File "mercurial.util", line 1886, in check
File "hgext.phabricator", line 314, in inner
File "hgext.phabricator", line 2222, in phabimport
File "hgext.phabricator", line 2123, in readpatch
File "hgext.phabricator", line 2199, in _write
File "mercurial.localrepo", line 2956, in lock
File "mercurial.localrepo", line 2918, in _lock
File "mercurial.lock", line 152, in trylock
File "mercurial.lock", line 283, in _trylock
File "mercurial.lock", line 314, in _readlock
File "mercurial.vfs", line 221, in readlock
File "mercurial.util", line 2163, in readlock
File "mercurial.windows", line 619, in readlink
ValueError: not a symbolic link
Both exceptions look accurate (the file exists, and the Windows side can't read
WSL side symlinks). I didn't try to reproduce this entirely within the Windows
side, but we can do better than a cryptic stacktrace. With this change, the
same scenario results in this abort:
abort: C:\Users\Matt\hg/.hg/store/lock: The file cannot be accessed by the system
When both the `push` and `phabimport` are done on the Windows side, it prints a
message about waiting for the lock, and successfully applies the patch after the
push completes.
I'm not sure if there's enough info to be able to convert the abort into the
wait scenario. As it stands now, we don't support symlinks on Windows, which
requires either a UAC Administrator level process or an opt-in in developer
mode, and there are several places where the new symlink on Windows support in
py3 was explicitly disabled in order to get tests to pass quicker.
Differential Revision: https://phab.mercurial-scm.org/D11333
setup
$ cat >> $HGRCPATH <<EOF
> [extensions]
> share =
> [format]
> use-share-safe = True
> [storage]
> revlog.persistent-nodemap.slow-path=allow
> # enforce zlib to ensure we can upgrade to zstd later
> [format]
> revlog-compression=zlib
> # we want to be able to enable it later
> use-persistent-nodemap=no
> EOF
prepare source repo
$ hg init source
$ cd source
$ cat .hg/requires
exp-dirstate-v2 (dirstate-v2 !)
share-safe
$ cat .hg/store/requires
dotencode
fncache
generaldelta
revlogv1
sparserevlog
store
$ hg debugrequirements
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
share-safe
sparserevlog
store
$ echo a > a
$ hg ci -Aqm "added a"
$ echo b > b
$ hg ci -Aqm "added b"
$ HGEDITOR=cat hg config --shared
abort: repository is not shared; can't use --shared
[10]
$ cd ..
Create a shared repo and check the requirements are shared and read correctly
$ hg share source shared1
updating working directory
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd shared1
$ cat .hg/requires
exp-dirstate-v2 (dirstate-v2 !)
share-safe
shared
$ hg debugrequirements -R ../source
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
share-safe
sparserevlog
store
$ hg debugrequirements
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
share-safe
shared
sparserevlog
store
$ echo c > c
$ hg ci -Aqm "added c"
Check that config of the source repository is also loaded
$ hg showconfig ui.curses
[1]
$ echo "[ui]" >> ../source/.hg/hgrc
$ echo "curses=true" >> ../source/.hg/hgrc
$ hg showconfig ui.curses
true
Test that extensions of source repository are also loaded
$ hg debugextensions
share
$ hg extdiff -p echo
hg: unknown command 'extdiff'
'extdiff' is provided by the following extension:
extdiff command to allow external programs to compare revisions
(use 'hg help extensions' for information on enabling extensions)
[10]
$ echo "[extensions]" >> ../source/.hg/hgrc
$ echo "extdiff=" >> ../source/.hg/hgrc
$ hg debugextensions -R ../source
extdiff
share
$ hg extdiff -R ../source -p echo
BROKEN: the command below will not work if config of shared source is not loaded
on dispatch but debugextensions says that extension
is loaded
$ hg debugextensions
extdiff
share
$ hg extdiff -p echo
However, local .hg/hgrc should override the config set by share source
$ echo "[ui]" >> .hg/hgrc
$ echo "curses=false" >> .hg/hgrc
$ hg showconfig ui.curses
false
$ HGEDITOR=cat hg config --shared
[ui]
curses=true
[extensions]
extdiff=
$ HGEDITOR=cat hg config --local
[ui]
curses=false
Testing that hooks set in source repository also runs in shared repo
$ cd ../source
$ cat <<EOF >> .hg/hgrc
> [extensions]
> hooklib=
> [hooks]
> pretxnchangegroup.reject_merge_commits = \
> python:hgext.hooklib.reject_merge_commits.hook
> EOF
$ cd ..
$ hg clone source cloned
updating to branch default
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd cloned
$ hg up 0
0 files updated, 0 files merged, 2 files removed, 0 files unresolved
$ echo bar > bar
$ hg ci -Aqm "added bar"
$ hg merge
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg ci -m "merge commit"
$ hg push ../source
pushing to ../source
searching for changes
adding changesets
adding manifests
adding file changes
error: pretxnchangegroup.reject_merge_commits hook failed: bcde3522682d rejected as merge on the same branch. Please consider rebase.
transaction abort!
rollback completed
abort: bcde3522682d rejected as merge on the same branch. Please consider rebase.
[255]
$ hg push ../shared1
pushing to ../shared1
searching for changes
adding changesets
adding manifests
adding file changes
error: pretxnchangegroup.reject_merge_commits hook failed: bcde3522682d rejected as merge on the same branch. Please consider rebase.
transaction abort!
rollback completed
abort: bcde3522682d rejected as merge on the same branch. Please consider rebase.
[255]
Test that if share source config is untrusted, we dont read it
$ cd ../shared1
$ cat << EOF > $TESTTMP/untrusted.py
> from mercurial import scmutil, util
> def uisetup(ui):
> class untrustedui(ui.__class__):
> def _trusted(self, fp, f):
> if util.normpath(fp.name).endswith(b'source/.hg/hgrc'):
> return False
> return super(untrustedui, self)._trusted(fp, f)
> ui.__class__ = untrustedui
> EOF
$ hg showconfig hooks
hooks.pretxnchangegroup.reject_merge_commits=python:hgext.hooklib.reject_merge_commits.hook
$ hg showconfig hooks --config extensions.untrusted=$TESTTMP/untrusted.py
[1]
Update the source repository format and check that shared repo works
$ cd ../source
Disable zstd related tests because its not present on pure version
#if zstd
$ echo "[format]" >> .hg/hgrc
$ echo "revlog-compression=zstd" >> .hg/hgrc
$ hg debugupgraderepo --run -q
upgrade will perform the following actions:
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (dirstate-v2 !)
added: revlog-compression-zstd
processed revlogs:
- all-filelogs
- changelog
- manifest
$ hg log -r .
changeset: 1:5f6d8a4bf34a
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: added b
#endif
$ echo "[format]" >> .hg/hgrc
$ echo "use-persistent-nodemap=True" >> .hg/hgrc
$ hg debugupgraderepo --run -q -R ../shared1
abort: cannot upgrade repository; unsupported source requirement: shared
[255]
$ hg debugupgraderepo --run -q
upgrade will perform the following actions:
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-zstd no-dirstate-v2 !)
preserved: dotencode, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-zstd dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd dirstate-v2 !)
added: persistent-nodemap
processed revlogs:
- all-filelogs
- changelog
- manifest
$ hg log -r .
changeset: 1:5f6d8a4bf34a
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: added b
Shared one should work
$ cd ../shared1
$ hg log -r .
changeset: 2:155349b645be
tag: tip
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: added c
Testing that nonsharedrc is loaded for source and not shared
$ cd ../source
$ touch .hg/hgrc-not-shared
$ echo "[ui]" >> .hg/hgrc-not-shared
$ echo "traceback=true" >> .hg/hgrc-not-shared
$ hg showconfig ui.traceback
true
$ HGEDITOR=cat hg config --non-shared
[ui]
traceback=true
$ cd ../shared1
$ hg showconfig ui.traceback
[1]
Unsharing works
$ hg unshare
Test that source config is added to the shared one after unshare, and the config
of current repo is still respected over the config which came from source config
$ cd ../cloned
$ hg push ../shared1
pushing to ../shared1
searching for changes
adding changesets
adding manifests
adding file changes
error: pretxnchangegroup.reject_merge_commits hook failed: bcde3522682d rejected as merge on the same branch. Please consider rebase.
transaction abort!
rollback completed
abort: bcde3522682d rejected as merge on the same branch. Please consider rebase.
[255]
$ hg showconfig ui.curses -R ../shared1
false
$ cd ../
Test that upgrading using debugupgraderepo works
=================================================
$ hg init non-share-safe --config format.use-share-safe=false
$ cd non-share-safe
$ hg debugrequirements
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
sparserevlog
store
$ echo foo > foo
$ hg ci -Aqm 'added foo'
$ echo bar > bar
$ hg ci -Aqm 'added bar'
Create a share before upgrading
$ cd ..
$ hg share non-share-safe nss-share
updating working directory
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg debugrequirements -R nss-share
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
shared
sparserevlog
store
$ cd non-share-safe
Upgrade
$ hg debugupgraderepo -q
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
added: share-safe
processed revlogs:
- all-filelogs
- changelog
- manifest
$ hg debugupgraderepo --run
upgrade will perform the following actions:
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
added: share-safe
share-safe
Upgrades a repository to share-safe format so that future shares of this repository share its requirements and configs.
processed revlogs:
- all-filelogs
- changelog
- manifest
beginning upgrade...
repository locked and read-only
creating temporary repository to stage upgraded data: $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes)
upgrading repository requirements
removing temporary repository $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
repository upgraded to share safe mode, existing shares will still work in old non-safe mode. Re-share existing shares to use them in safe mode New shares will be created in safe mode.
$ hg debugrequirements
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
share-safe
sparserevlog
store
$ cat .hg/requires
exp-dirstate-v2 (dirstate-v2 !)
share-safe
$ cat .hg/store/requires
dotencode
fncache
generaldelta
revlogv1
sparserevlog
store
$ hg log -GT "{node}: {desc}\n"
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
Make sure existing shares dont work with default config
$ hg log -GT "{node}: {desc}\n" -R ../nss-share
abort: version mismatch: source uses share-safe functionality while the current share does not
(see `hg help config.format.use-share-safe` for more information)
[255]
Create a safe share from upgrade one
$ cd ..
$ hg share non-share-safe ss-share
updating working directory
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd ss-share
$ hg log -GT "{node}: {desc}\n"
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
$ cd ../non-share-safe
Test that downgrading works too
$ cat >> $HGRCPATH <<EOF
> [extensions]
> share =
> [format]
> use-share-safe = False
> EOF
$ hg debugupgraderepo -q
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
removed: share-safe
processed revlogs:
- all-filelogs
- changelog
- manifest
$ hg debugupgraderepo --run
upgrade will perform the following actions:
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
removed: share-safe
processed revlogs:
- all-filelogs
- changelog
- manifest
beginning upgrade...
repository locked and read-only
creating temporary repository to stage upgraded data: $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes)
upgrading repository requirements
removing temporary repository $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
repository downgraded to not use share safe mode, existing shares will not work and needs to be reshared.
$ hg debugrequirements
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
sparserevlog
store
$ cat .hg/requires
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
sparserevlog
store
$ test -f .hg/store/requires
[1]
$ hg log -GT "{node}: {desc}\n"
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
Make sure existing shares still works
$ hg log -GT "{node}: {desc}\n" -R ../nss-share
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
$ hg log -GT "{node}: {desc}\n" -R ../ss-share
abort: share source does not support share-safe requirement
(see `hg help config.format.use-share-safe` for more information)
[255]
Testing automatic downgrade of shares when config is set
$ touch ../ss-share/.hg/wlock
$ hg log -GT "{node}: {desc}\n" -R ../ss-share --config share.safe-mismatch.source-not-safe=downgrade-abort
abort: failed to downgrade share, got error: Lock held
(see `hg help config.format.use-share-safe` for more information)
[255]
$ rm ../ss-share/.hg/wlock
$ hg log -GT "{node}: {desc}\n" -R ../ss-share --config share.safe-mismatch.source-not-safe=downgrade-abort
repository downgraded to not use share-safe mode
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
$ hg log -GT "{node}: {desc}\n" -R ../ss-share
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
Testing automatic upgrade of shares when config is set
$ hg debugupgraderepo -q --run --config format.use-share-safe=True
upgrade will perform the following actions:
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
preserved: dotencode, exp-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
added: share-safe
processed revlogs:
- all-filelogs
- changelog
- manifest
repository upgraded to share safe mode, existing shares will still work in old non-safe mode. Re-share existing shares to use them in safe mode New shares will be created in safe mode.
$ hg debugrequirements
dotencode
exp-dirstate-v2 (dirstate-v2 !)
fncache
generaldelta
revlogv1
share-safe
sparserevlog
store
$ hg log -GT "{node}: {desc}\n" -R ../nss-share
abort: version mismatch: source uses share-safe functionality while the current share does not
(see `hg help config.format.use-share-safe` for more information)
[255]
Check that if lock is taken, upgrade fails but read operation are successful
$ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgra
abort: share-safe mismatch with source.
Unrecognized value 'upgra' of `share.safe-mismatch.source-safe` set.
(see `hg help config.format.use-share-safe` for more information)
[255]
$ touch ../nss-share/.hg/wlock
$ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-allow
failed to upgrade share, got error: Lock held
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
$ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-allow --config share.safe-mismatch.source-safe.warn=False
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
$ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-abort
abort: failed to upgrade share, got error: Lock held
(see `hg help config.format.use-share-safe` for more information)
[255]
$ rm ../nss-share/.hg/wlock
$ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-abort
repository upgraded to use share-safe mode
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
Test that unshare works
$ hg unshare -R ../nss-share
$ hg log -GT "{node}: {desc}\n" -R ../nss-share
@ f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
|
o f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo