tests/test-share-safe.t

rust-index: using the `hg::index::Index` in ancestors iterator and lazy set

Since there is no Rust implementation for REVLOGV2/CHANGELOGv2, we declare
them to be incompatible with Rust, hence indexes in these formats will use
the implementations from Python `mercurial.ancestor`. If this is an unacceptable
performance hit for current users of these formats, we can later on add Rust
implementations based on the C index for them or implement these formats for
the Rust indexes.

Among the challenges that we had to meet, we wanted to avoid taking the GIL each
time the inner (vcsgraph) iterator has to call the parents function. This would probably
still be acceptable in terms of performance with `AncestorsIterator`, but not with
`LazyAncestors` nor for the upcoming change in `MissingAncestors`.

Hence we enclose the reference to the index in a `PySharedRef`, leading to more
rigourous checking of mutations, which does pass now that there no logically immutable
methods of `hg::index::Index` that take a mutable reference as input.

setup

  $ cat >> $HGRCPATH <<EOF
  > [extensions]
  > share =
  > [format]
  > use-share-safe = True
  > [storage]
  > revlog.persistent-nodemap.slow-path=allow
  > # enforce zlib to ensure we can upgrade to zstd later
  > [format]
  > revlog-compression=zlib
  > # we want to be able to enable it later
  > use-persistent-nodemap=no
  > EOF

prepare source repo

  $ hg init source
  $ cd source
  $ cat .hg/requires
  dirstate-v2 (dirstate-v2 !)
  share-safe
  $ cat .hg/store/requires
  dotencode
  fncache
  generaldelta
  revlogv1
  sparserevlog
  store
  $ hg debugrequirements
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  share-safe
  sparserevlog
  store

  $ echo a > a
  $ hg ci -Aqm "added a"
  $ echo b > b
  $ hg ci -Aqm "added b"

  $ HGEDITOR=cat hg config --shared
  abort: repository is not shared; can't use --shared
  [10]
  $ cd ..

Create a shared repo and check the requirements are shared and read correctly
  $ hg share source shared1
  updating working directory
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ cd shared1
  $ cat .hg/requires
  dirstate-v2 (dirstate-v2 !)
  share-safe
  shared

  $ hg debugrequirements -R ../source
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  share-safe
  sparserevlog
  store

  $ hg debugrequirements
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  share-safe
  shared
  sparserevlog
  store

  $ echo c > c
  $ hg ci -Aqm "added c"

Check that config of the source repository is also loaded

  $ hg showconfig ui.curses
  [1]

  $ echo "[ui]" >> ../source/.hg/hgrc
  $ echo "curses=true" >> ../source/.hg/hgrc

  $ hg showconfig ui.curses
  true

Test that extensions of source repository are also loaded

  $ hg debugextensions
  share
  $ hg extdiff -p echo
  hg: unknown command 'extdiff'
  'extdiff' is provided by the following extension:
  
      extdiff       command to allow external programs to compare revisions
  
  (use 'hg help extensions' for information on enabling extensions)
  [10]

  $ echo "[extensions]" >> ../source/.hg/hgrc
  $ echo "extdiff=" >> ../source/.hg/hgrc

  $ hg debugextensions -R ../source
  extdiff
  share
  $ hg extdiff -R ../source -p echo

BROKEN: the command below will not work if config of shared source is not loaded
on dispatch but debugextensions says that extension
is loaded
  $ hg debugextensions
  extdiff
  share

  $ hg extdiff -p echo

However, local .hg/hgrc should override the config set by share source

  $ echo "[ui]" >> .hg/hgrc
  $ echo "curses=false" >> .hg/hgrc

  $ hg showconfig ui.curses
  false

  $ HGEDITOR=cat hg config --shared
  [ui]
  curses=true
  [extensions]
  extdiff=

  $ HGEDITOR=cat hg config --local
  [ui]
  curses=false

Testing that hooks set in source repository also runs in shared repo

  $ cd ../source
  $ cat <<EOF >> .hg/hgrc
  > [extensions]
  > hooklib=
  > [hooks]
  > pretxnchangegroup.reject_merge_commits = \
  >   python:hgext.hooklib.reject_merge_commits.hook
  > EOF

  $ cd ..
  $ hg clone source cloned
  updating to branch default
  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ cd cloned
  $ hg up 0
  0 files updated, 0 files merged, 2 files removed, 0 files unresolved
  $ echo bar > bar
  $ hg ci -Aqm "added bar"
  $ hg merge
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  (branch merge, don't forget to commit)
  $ hg ci -m "merge commit"

  $ hg push ../source
  pushing to ../source
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  error: pretxnchangegroup.reject_merge_commits hook failed: bcde3522682d rejected as merge on the same branch. Please consider rebase.
  transaction abort!
  rollback completed
  abort: bcde3522682d rejected as merge on the same branch. Please consider rebase.
  [255]

  $ hg push ../shared1
  pushing to ../shared1
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  error: pretxnchangegroup.reject_merge_commits hook failed: bcde3522682d rejected as merge on the same branch. Please consider rebase.
  transaction abort!
  rollback completed
  abort: bcde3522682d rejected as merge on the same branch. Please consider rebase.
  [255]

Test that if share source config is untrusted, we dont read it

  $ cd ../shared1

  $ cat << EOF > $TESTTMP/untrusted.py
  > from mercurial import scmutil, util
  > def uisetup(ui):
  >     class untrustedui(ui.__class__):
  >         def _trusted(self, fp, f):
  >             if util.normpath(fp.name).endswith(b'source/.hg/hgrc'):
  >                 return False
  >             return super(untrustedui, self)._trusted(fp, f)
  >     ui.__class__ = untrustedui
  > EOF

  $ hg showconfig hooks
  hooks.pretxnchangegroup.reject_merge_commits=python:hgext.hooklib.reject_merge_commits.hook

  $ hg showconfig hooks --config extensions.untrusted=$TESTTMP/untrusted.py
  [1]

Update the source repository format and check that shared repo works

  $ cd ../source

Disable zstd related tests because its not present on pure version
#if zstd
  $ echo "[format]" >> .hg/hgrc
  $ echo "revlog-compression=zstd" >> .hg/hgrc

  $ hg debugupgraderepo --run -q
  upgrade will perform the following actions:
  
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (dirstate-v2 !)
     added: revlog-compression-zstd
  
  processed revlogs:
    - all-filelogs
    - changelog
    - manifest
  
  $ hg log -r .
  changeset:   1:5f6d8a4bf34a
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     added b
  
#endif
  $ echo "[format]" >> .hg/hgrc
  $ echo "use-persistent-nodemap=True" >> .hg/hgrc

  $ hg debugupgraderepo --run -q -R ../shared1
  abort: cannot use these actions on a share repository: persistent-nodemap
  (upgrade the main repository directly)
  [255]

  $ hg debugupgraderepo --run -q
  upgrade will perform the following actions:
  
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-zstd no-dirstate-v2 !)
     preserved: dotencode, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, share-safe, sparserevlog, store (no-zstd dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlog-compression-zstd, revlogv1, share-safe, sparserevlog, store (zstd dirstate-v2 !)
     added: persistent-nodemap
  
  processed revlogs:
    - all-filelogs
    - changelog
    - manifest
  
  $ hg log -r .
  changeset:   1:5f6d8a4bf34a
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     added b
  

Shared one should work
  $ cd ../shared1
  $ hg log -r .
  changeset:   2:155349b645be
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     added c
  

Testing that nonsharedrc is loaded for source and not shared

  $ cd ../source
  $ touch .hg/hgrc-not-shared
  $ echo "[ui]" >> .hg/hgrc-not-shared
  $ echo "traceback=true" >> .hg/hgrc-not-shared

  $ hg showconfig ui.traceback
  true

  $ HGEDITOR=cat hg config --non-shared
  [ui]
  traceback=true

  $ cd ../shared1
  $ hg showconfig ui.traceback
  [1]

Unsharing works

  $ hg unshare

Test that source config is added to the shared one after unshare, and the config
of current repo is still respected over the config which came from source config
  $ cd ../cloned
  $ hg push ../shared1
  pushing to ../shared1
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  error: pretxnchangegroup.reject_merge_commits hook failed: bcde3522682d rejected as merge on the same branch. Please consider rebase.
  transaction abort!
  rollback completed
  abort: bcde3522682d rejected as merge on the same branch. Please consider rebase.
  [255]
  $ hg showconfig ui.curses -R ../shared1
  false

  $ cd ../

Test that upgrading using debugupgraderepo works
=================================================

  $ hg init non-share-safe --config format.use-share-safe=false
  $ cd non-share-safe
  $ hg debugrequirements
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  sparserevlog
  store
  $ echo foo > foo
  $ hg ci -Aqm 'added foo'
  $ echo bar > bar
  $ hg ci -Aqm 'added bar'

Create a share before upgrading

  $ cd ..
  $ hg share non-share-safe nss-share
  updating working directory
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg debugrequirements -R nss-share
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  shared
  sparserevlog
  store
  $ cd non-share-safe

Upgrade

  $ hg debugupgraderepo -q
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
     added: share-safe
  
  no revlogs to process
  
  $ hg debugupgraderepo --run
  upgrade will perform the following actions:
  
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
     added: share-safe
  
  share-safe
     Upgrades a repository to share-safe format so that future shares of this repository share its requirements and configs.
  
  no revlogs to process
  
  beginning upgrade...
  repository locked and read-only
  creating temporary repository to stage upgraded data: $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
  (it is safe to interrupt this process any time before data migration completes)
  upgrading repository requirements
  removing temporary repository $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
  repository upgraded to share safe mode, existing shares will still work in old non-safe mode. Re-share existing shares to use them in safe mode New shares will be created in safe mode.

  $ hg debugrequirements
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  share-safe
  sparserevlog
  store

  $ cat .hg/requires
  dirstate-v2 (dirstate-v2 !)
  share-safe

  $ cat .hg/store/requires
  dotencode
  fncache
  generaldelta
  revlogv1
  sparserevlog
  store

  $ hg log -GT "{node}: {desc}\n"
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

Make sure existing shares dont work with default config

  $ hg log -GT "{node}: {desc}\n" -R ../nss-share
  abort: version mismatch: source uses share-safe functionality while the current share does not
  (see `hg help config.format.use-share-safe` for more information)
  [255]


Create a safe share from upgrade one

  $ cd ..
  $ hg share non-share-safe ss-share
  updating working directory
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ cd ss-share
  $ hg log -GT "{node}: {desc}\n"
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  
  $ cd ../non-share-safe

Test that downgrading works too

  $ cat >> $HGRCPATH <<EOF
  > [extensions]
  > share =
  > [format]
  > use-share-safe = False
  > EOF

  $ hg debugupgraderepo -q
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
     removed: share-safe
  
  no revlogs to process
  
  $ hg debugupgraderepo --run
  upgrade will perform the following actions:
  
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
     removed: share-safe
  
  no revlogs to process
  
  beginning upgrade...
  repository locked and read-only
  creating temporary repository to stage upgraded data: $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
  (it is safe to interrupt this process any time before data migration completes)
  upgrading repository requirements
  removing temporary repository $TESTTMP/non-share-safe/.hg/upgrade.* (glob)
  repository downgraded to not use share safe mode, existing shares will not work and need to be reshared.

  $ hg debugrequirements
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  sparserevlog
  store

  $ cat .hg/requires
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  sparserevlog
  store

  $ test -f .hg/store/requires
  [1]

  $ hg log -GT "{node}: {desc}\n"
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

Make sure existing shares still works

  $ hg log -GT "{node}: {desc}\n" -R ../nss-share
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

  $ hg log -GT "{node}: {desc}\n" -R ../ss-share
  abort: share source does not support share-safe requirement
  (see `hg help config.format.use-share-safe` for more information)
  [255]

Testing automatic downgrade of shares when config is set

  $ touch ../ss-share/.hg/wlock
  $ hg log -GT "{node}: {desc}\n" -R ../ss-share --config share.safe-mismatch.source-not-safe=downgrade-abort
  abort: failed to downgrade share, got error: Lock held
  (see `hg help config.format.use-share-safe` for more information)
  [255]
  $ rm ../ss-share/.hg/wlock

  $ cp -R ../ss-share ../ss-share-bck
  $ hg log -GT "{node}: {desc}\n" -R ../ss-share --config share.safe-mismatch.source-not-safe=downgrade-abort
  repository downgraded to not use share-safe mode
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  
  $ rm -rf ../ss-share
  $ mv ../ss-share-bck ../ss-share

  $ hg log -GT "{node}: {desc}\n" -R ../ss-share --config share.safe-mismatch.source-not-safe=downgrade-abort --config share.safe-mismatch.source-not-safe:verbose-upgrade=no
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

  $ hg log -GT "{node}: {desc}\n" -R ../ss-share
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  


Testing automatic upgrade of shares when config is set

  $ hg debugupgraderepo -q --run --config format.use-share-safe=True
  upgrade will perform the following actions:
  
  requirements
     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store (no-dirstate-v2 !)
     preserved: dotencode, use-dirstate-v2, fncache, generaldelta, revlogv1, sparserevlog, store (dirstate-v2 !)
     added: share-safe
  
  no revlogs to process
  
  repository upgraded to share safe mode, existing shares will still work in old non-safe mode. Re-share existing shares to use them in safe mode New shares will be created in safe mode.
  $ hg debugrequirements
  dotencode
  dirstate-v2 (dirstate-v2 !)
  fncache
  generaldelta
  revlogv1
  share-safe
  sparserevlog
  store
  $ hg log -GT "{node}: {desc}\n" -R ../nss-share
  abort: version mismatch: source uses share-safe functionality while the current share does not
  (see `hg help config.format.use-share-safe` for more information)
  [255]

Check that if lock is taken, upgrade fails but read operation are successful
  $ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgra
  abort: share-safe mismatch with source.
  Unrecognized value 'upgra' of `share.safe-mismatch.source-safe` set.
  (see `hg help config.format.use-share-safe` for more information)
  [255]
  $ touch ../nss-share/.hg/wlock
  $ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-allow
  failed to upgrade share, got error: Lock held
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

  $ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-allow --config share.safe-mismatch.source-safe.warn=False
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

  $ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-abort
  abort: failed to upgrade share, got error: Lock held
  (see `hg help config.format.use-share-safe` for more information)
  [255]

  $ rm ../nss-share/.hg/wlock
  $ cp -R ../nss-share ../nss-share-bck
  $ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-abort
  repository upgraded to use share-safe mode
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  
  $ rm -rf ../nss-share
  $ mv ../nss-share-bck ../nss-share
  $ hg log -GT "{node}: {desc}\n" -R ../nss-share --config share.safe-mismatch.source-safe=upgrade-abort --config share.safe-mismatch.source-safe:verbose-upgrade=no
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

Test that unshare works

  $ hg unshare -R ../nss-share
  $ hg log -GT "{node}: {desc}\n" -R ../nss-share
  @  f63db81e6dde1d9c78814167f77fb1fb49283f4f: added bar
  |
  o  f3ba8b99bb6f897c87bbc1c07b75c6ddf43a4f77: added foo
  

Test automatique upgrade/downgrade of main-repository
------------------------------------------------------

create an initial repository

  $ hg init auto-upgrade \
  > --config format.use-share-safe=no
  $ hg debugbuilddag -R auto-upgrade --new-file .+5
  $ hg -R auto-upgrade update
  6 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg debugformat -R auto-upgrade | grep share-safe
  share-safe:          no

upgrade it to share-safe automatically

  $ hg status -R auto-upgrade \
  >     --config format.use-share-safe.automatic-upgrade-of-mismatching-repositories=yes \
  >     --config format.use-share-safe=yes
  automatically upgrading repository to the `share-safe` feature
  (see `hg help config.format.use-share-safe` for details)
  $ hg debugformat -R auto-upgrade | grep share-safe
  share-safe:         yes

downgrade it from share-safe automatically

  $ hg status -R auto-upgrade \
  >     --config format.use-share-safe.automatic-upgrade-of-mismatching-repositories=yes \
  >     --config format.use-share-safe=no
  automatically downgrading repository from the `share-safe` feature
  (see `hg help config.format.use-share-safe` for details)
  $ hg debugformat -R auto-upgrade | grep share-safe
  share-safe:          no