view tests/test-globalopts.t @ 42044:bb271ec2fbfb

compression: introduce a `storage.revlog.zstd.level` configuration

This option control the zstd compression level used when compressing revlog
chunk. The usage of zstd for revlog compression has not graduated from
experimental yet, but we intend to fix that soon.

The option name for the compression level is more straight forward to pick, so
this changesets comes first.  Having a dedicated option for each compression
engine is useful because they don't support the same range of values.

I ran the same measurement as for the zlib compression level (in the parent
changesets). The variation in repository size is stay mostly in the same (small)
range. The "read/write" performance see smallish variation, but are overall much
better than zlib. Write performance show the same tend of having better write
performance for when reaching high-end compression.

Again, we don't intend to change the default zstd compression level (currently:
3) in this series. However this is worth investigating in the future.

The Performance comparison of zlib vs zstd is quite impressive. The repository
size stay in the same range, but the performance are much better in all
situations.

Comparison summary
==================

We are looking at:
- performance range for zlib
- performance range for zstd
- comparison of default zstd (level-3) to default zlib (level 6)
- comparison of the slowest zstd time to the fastest zlib time

Read performance:
-----------------
          |           zlib          |           zstd          | cmp | f2s
mercurial |   0.170159 -   0.189219 |   0.144127 -   0.149624 | 80% | 88%
pypy      |   2.679217 -   2.768691 |   1.532317 -   1.705044 | 60% | 63%
netbeans  | 122.477027 - 141.620281 |  72.996346 -  89.731560 | 58% | 73%
mozilla   | 147.867662 - 170.572118 |  91.700995 - 105.853099 | 56% | 71%

Write performance:
------------------
          |           zlib          |           zstd          | cmp | f2s
mercurial |  53.250304 - 56.2936129 |  40.877025 -  45.677286 | 75% | 86%
pypy      | 460.721984 - 476.589918 | 270.545409 - 301.002219 | 63% | 65%
netbeans  | 520.560316 - 715.930400 | 370.356311 - 428.329652 | 55% | 82%
mozilla   | 739.803002 - 987.056093 | 505.152906 - 591.930683 | 57% | 80%

Raw data
--------

repo      alg lvl  .hg/store size  00manifest.d read       write

mercurial zlib  1      49,402,813     5,963,475   0.170159  53.250304
mercurial zlib  6      47,197,397     5,875,730   0.182820  56.264320
mercurial zlib  9      47,121,596     5,849,781   0.189219  56.293612

mercurial zstd  1      49,737,084     5,966,355   0.144127  40.877025
mercurial zstd  3      48,961,867     5,895,208   0.146376  42.268142
mercurial zstd  5      48,200,592     5,938,676   0.149624  43.162875
mercurial zstd 10      47,833,520     5,913,353   0.145185  44.012489
mercurial zstd 15      47,314,604     5,728,679   0.147686  45.677286
mercurial zstd 20      47,330,502     5,830,539   0.145789  45.025407
mercurial zstd 22      47,330,076     5,830,539   0.143996  44.690460


pypy      zlib  1     370,830,572    28,462,425   2.679217 460.721984
pypy      zlib  6     340,112,317    27,648,747   2.768691 467.537158
pypy      zlib  9     338,360,736    27,639,003   2.763495 476.589918

pypy      zstd  1     362,377,479    27,916,214   1.532317 270.545409
pypy      zstd  3     354,137,693    27,905,988   1.686718 294.951509
pypy      zstd  5     342,640,043    27,655,774   1.705044 301.002219
pypy      zstd 10     334,224,327    27,164,493   1.567287 285.186239
pypy      zstd 15     329,000,363    26,645,965   1.637729 299.561332
pypy      zstd 20     324,534,039    26,199,547   1.526813 302.149827
pypy      zstd 22     324,530,595    26,198,932   1.525718 307.821218


netbeans  zlib  1   1,281,847,810   165,495,457 122.477027 520.560316
netbeans  zlib  6   1,205,284,353   159,161,207 139.876147 715.930400
netbeans  zlib  9   1,197,135,671   155,034,586 141.620281 678.297064

netbeans  zstd  1   1,259,581,737   160,840,613  72.996346 370.356311
netbeans  zstd  3   1,232,978,122   157,691,551  81.622317 396.733087
netbeans  zstd  5   1,208,034,075   160,246,880  83.080549 364.342626
netbeans  zstd 10   1,188,624,176   156,083,417  79.323935 403.594602
netbeans  zstd 15   1,176,973,589   153,859,477  89.731560 428.329652
netbeans  zstd 20   1,162,958,258   151,147,535  82.842667 392.335349
netbeans  zstd 22   1,162,707,029   151,150,220  82.565695 402.840655


mozilla   zlib  1   2,775,497,186   298,527,987 147.867662 751.263721
mozilla   zlib  6   2,596,856,420   286,597,671 170.572118 987.056093
mozilla   zlib  9   2,587,542,494   287,018,264 163.622338 739.803002

mozilla   zstd  1   2,723,159,348   286,617,532  91.700995 570.042751
mozilla   zstd  3   2,665,055,001   286,152,013  95.240155 561.412805
mozilla   zstd  5   2,607,819,817   288,060,030 101.978048 505.152906
mozilla   zstd 10   2,558,761,085   283,967,648 104.113481 497.771202
mozilla   zstd 15   2,526,216,060   275,581,300 105.853099 591.930683
mozilla   zstd 20   2,485,114,806   266,478,859  95.268795 576.515389
mozilla   zstd 22   2,484,869,080   266,456,505  94.429282 572.785537

  $ hg init a
  $ cd a
  $ echo a > a
  $ hg ci -A -d'1 0' -m a
  adding a

  $ cd ..

  $ hg init b
  $ cd b
  $ echo b > b
  $ hg ci -A -d'1 0' -m b
  adding b

  $ cd ..

  $ hg clone a c
  updating to branch default
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ cd c
  $ cat >> .hg/hgrc <<EOF
  > [paths]
  > relative = ../a
  > EOF
  $ hg pull -f ../b
  pulling from ../b
  searching for changes
  warning: repository is unrelated
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 1 changes to 1 files (+1 heads)
  new changesets b6c483daf290
  (run 'hg heads' to see heads, 'hg merge' to merge)
  $ hg merge
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  (branch merge, don't forget to commit)

  $ cd ..

Testing -R/--repository:

  $ hg -R a tip
  changeset:   0:8580ff50825a
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  summary:     a
  
  $ hg --repository b tip
  changeset:   0:b6c483daf290
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  summary:     b
  

-R with a URL:

  $ hg -R file:a identify
  8580ff50825a tip
  $ hg -R file://localhost/`pwd`/a/ identify
  8580ff50825a tip

-R with path aliases:

  $ cd c
  $ hg -R default identify
  8580ff50825a tip
  $ hg -R relative identify
  8580ff50825a tip
  $ echo '[paths]' >> $HGRCPATH
  $ echo 'relativetohome = a' >> $HGRCPATH
  $ HOME=`pwd`/../ hg -R relativetohome identify
  8580ff50825a tip
  $ cd ..

#if no-outer-repo

Implicit -R:

  $ hg ann a/a
  0: a
  $ hg ann a/a a/a
  0: a
  $ hg ann a/a b/b
  abort: no repository found in '$TESTTMP' (.hg not found)!
  [255]
  $ hg -R b ann a/a
  abort: a/a not under root '$TESTTMP/b'
  (consider using '--cwd b')
  [255]
  $ hg log
  abort: no repository found in '$TESTTMP' (.hg not found)!
  [255]

#endif

Abbreviation of long option:

  $ hg --repo c tip
  changeset:   1:b6c483daf290
  tag:         tip
  parent:      -1:000000000000
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  summary:     b
  

earlygetopt with duplicate options (36d23de02da1):

  $ hg --cwd a --cwd b --cwd c tip
  changeset:   1:b6c483daf290
  tag:         tip
  parent:      -1:000000000000
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  summary:     b
  
  $ hg --repo c --repository b -R a tip
  changeset:   0:8580ff50825a
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  summary:     a
  

earlygetopt short option without following space:

  $ hg -q -Rb tip
  0:b6c483daf290

earlygetopt with illegal abbreviations:

  $ hg --confi "foo.bar=baz"
  abort: option --config may not be abbreviated!
  [255]
  $ hg --cw a tip
  abort: option --cwd may not be abbreviated!
  [255]
  $ hg --rep a tip
  abort: option -R has to be separated from other options (e.g. not -qR) and --repository may only be abbreviated as --repo!
  [255]
  $ hg --repositor a tip
  abort: option -R has to be separated from other options (e.g. not -qR) and --repository may only be abbreviated as --repo!
  [255]
  $ hg -qR a tip
  abort: option -R has to be separated from other options (e.g. not -qR) and --repository may only be abbreviated as --repo!
  [255]
  $ hg -qRa tip
  abort: option -R has to be separated from other options (e.g. not -qR) and --repository may only be abbreviated as --repo!
  [255]

Testing --cwd:

  $ hg --cwd a parents
  changeset:   0:8580ff50825a
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  summary:     a
  

Testing -y/--noninteractive - just be sure it is parsed:

  $ hg --cwd a tip -q --noninteractive
  0:8580ff50825a
  $ hg --cwd a tip -q -y
  0:8580ff50825a

Testing -q/--quiet:

  $ hg -R a -q tip
  0:8580ff50825a
  $ hg -R b -q tip
  0:b6c483daf290
  $ hg -R c --quiet parents
  0:8580ff50825a
  1:b6c483daf290

Testing -v/--verbose:

  $ hg --cwd c head -v
  changeset:   1:b6c483daf290
  tag:         tip
  parent:      -1:000000000000
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  files:       b
  description:
  b
  
  
  changeset:   0:8580ff50825a
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  files:       a
  description:
  a
  
  
  $ hg --cwd b tip --verbose
  changeset:   0:b6c483daf290
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  files:       b
  description:
  b
  
  

Testing --config:

  $ hg --cwd c --config paths.quuxfoo=bar paths | grep quuxfoo > /dev/null && echo quuxfoo
  quuxfoo
  $ hg --cwd c --config '' tip -q
  abort: malformed --config option: '' (use --config section.name=value)
  [255]
  $ hg --cwd c --config a.b tip -q
  abort: malformed --config option: 'a.b' (use --config section.name=value)
  [255]
  $ hg --cwd c --config a tip -q
  abort: malformed --config option: 'a' (use --config section.name=value)
  [255]
  $ hg --cwd c --config a.= tip -q
  abort: malformed --config option: 'a.=' (use --config section.name=value)
  [255]
  $ hg --cwd c --config .b= tip -q
  abort: malformed --config option: '.b=' (use --config section.name=value)
  [255]

Testing --debug:

  $ hg --cwd c log --debug
  changeset:   1:b6c483daf2907ce5825c0bb50f5716226281cc1a
  tag:         tip
  phase:       public
  parent:      -1:0000000000000000000000000000000000000000
  parent:      -1:0000000000000000000000000000000000000000
  manifest:    1:23226e7a252cacdc2d99e4fbdc3653441056de49
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  files+:      b
  extra:       branch=default
  description:
  b
  
  
  changeset:   0:8580ff50825a50c8f716709acdf8de0deddcd6ab
  phase:       public
  parent:      -1:0000000000000000000000000000000000000000
  parent:      -1:0000000000000000000000000000000000000000
  manifest:    0:a0c8bcbbb45c63b90b70ad007bf38961f64f2af0
  user:        test
  date:        Thu Jan 01 00:00:01 1970 +0000
  files+:      a
  extra:       branch=default
  description:
  a
  
  

Testing --traceback:

#if no-chg
  $ hg --cwd c --config x --traceback id 2>&1 | grep -i 'traceback'
  Traceback (most recent call last):
  Traceback (most recent call last): (py3 !)
#else
Traceback for '--config' errors not supported with chg.
  $ hg --cwd c --config x --traceback id 2>&1 | grep -i 'traceback'
  [1]
#endif

Testing --time:

  $ hg --cwd a --time id
  8580ff50825a tip
  time: real * (glob)

Testing --version:

  $ hg --version -q
  Mercurial Distributed SCM * (glob)

hide outer repo
  $ hg init

Testing -h/--help:

#if no-extraextensions

  $ hg -h
  Mercurial Distributed SCM
  
  list of commands:
  
  Repository creation:
  
   clone         make a copy of an existing repository
   init          create a new repository in the given directory
  
  Remote repository management:
  
   incoming      show new changesets found in source
   outgoing      show changesets not found in the destination
   paths         show aliases for remote repositories
   pull          pull changes from the specified source
   push          push changes to the specified destination
   serve         start stand-alone webserver
  
  Change creation:
  
   commit        commit the specified files or all outstanding changes
  
  Change manipulation:
  
   backout       reverse effect of earlier changeset
   graft         copy changes from other branches onto the current branch
   merge         merge another revision into working directory
  
  Change organization:
  
   bookmarks     create a new bookmark or list existing bookmarks
   branch        set or show the current branch name
   branches      list repository named branches
   phase         set or show the current phase name
   tag           add one or more tags for the current or given revision
   tags          list repository tags
  
  File content management:
  
   annotate      show changeset information by line for each file
   cat           output the current or given revision of files
   copy          mark files as copied for the next commit
   diff          diff repository (or selected files)
   grep          search revision history for a pattern in specified files
  
  Change navigation:
  
   bisect        subdivision search of changesets
   heads         show branch heads
   identify      identify the working directory or specified revision
   log           show revision history of entire repository or files
  
  Working directory management:
  
   add           add the specified files on the next commit
   addremove     add all new files, delete all missing files
   files         list tracked files
   forget        forget the specified files on the next commit
   remove        remove the specified files on the next commit
   rename        rename files; equivalent of copy + remove
   resolve       redo merges or set/view the merge status of files
   revert        restore files to their checkout state
   root          print the root (top) of the current working directory
   status        show changed files in the working directory
   summary       summarize working directory state
   update        update working directory (or switch revisions)
  
  Change import/export:
  
   archive       create an unversioned archive of a repository revision
   bundle        create a bundle file
   export        dump the header and diffs for one or more changesets
   import        import an ordered set of patches
   unbundle      apply one or more bundle files
  
  Repository maintenance:
  
   manifest      output the current or given revision of the project manifest
   recover       roll back an interrupted transaction
   verify        verify the integrity of the repository
  
  Help:
  
   config        show combined config settings from all hgrc files
   help          show help for a given topic or a help overview
   version       output version and copyright information
  
  additional help topics:
  
  Mercurial identifiers:
  
   filesets      Specifying File Sets
   hgignore      Syntax for Mercurial Ignore Files
   patterns      File Name Patterns
   revisions     Specifying Revisions
   urls          URL Paths
  
  Mercurial output:
  
   color         Colorizing Outputs
   dates         Date Formats
   diffs         Diff Formats
   templating    Template Usage
  
  Mercurial configuration:
  
   config        Configuration Files
   environment   Environment Variables
   extensions    Using Additional Features
   flags         Command-line flags
   hgweb         Configuring hgweb
   merge-tools   Merge Tools
   pager         Pager Support
  
  Concepts:
  
   bundlespec    Bundle File Formats
   glossary      Glossary
   phases        Working with Phases
   subrepos      Subrepositories
  
  Miscellaneous:
  
   deprecated    Deprecated Features
   internals     Technical implementation topics
   scripting     Using Mercurial from scripts and automation
  
  (use 'hg help -v' to show built-in aliases and global options)

  $ hg --help
  Mercurial Distributed SCM
  
  list of commands:
  
  Repository creation:
  
   clone         make a copy of an existing repository
   init          create a new repository in the given directory
  
  Remote repository management:
  
   incoming      show new changesets found in source
   outgoing      show changesets not found in the destination
   paths         show aliases for remote repositories
   pull          pull changes from the specified source
   push          push changes to the specified destination
   serve         start stand-alone webserver
  
  Change creation:
  
   commit        commit the specified files or all outstanding changes
  
  Change manipulation:
  
   backout       reverse effect of earlier changeset
   graft         copy changes from other branches onto the current branch
   merge         merge another revision into working directory
  
  Change organization:
  
   bookmarks     create a new bookmark or list existing bookmarks
   branch        set or show the current branch name
   branches      list repository named branches
   phase         set or show the current phase name
   tag           add one or more tags for the current or given revision
   tags          list repository tags
  
  File content management:
  
   annotate      show changeset information by line for each file
   cat           output the current or given revision of files
   copy          mark files as copied for the next commit
   diff          diff repository (or selected files)
   grep          search revision history for a pattern in specified files
  
  Change navigation:
  
   bisect        subdivision search of changesets
   heads         show branch heads
   identify      identify the working directory or specified revision
   log           show revision history of entire repository or files
  
  Working directory management:
  
   add           add the specified files on the next commit
   addremove     add all new files, delete all missing files
   files         list tracked files
   forget        forget the specified files on the next commit
   remove        remove the specified files on the next commit
   rename        rename files; equivalent of copy + remove
   resolve       redo merges or set/view the merge status of files
   revert        restore files to their checkout state
   root          print the root (top) of the current working directory
   status        show changed files in the working directory
   summary       summarize working directory state
   update        update working directory (or switch revisions)
  
  Change import/export:
  
   archive       create an unversioned archive of a repository revision
   bundle        create a bundle file
   export        dump the header and diffs for one or more changesets
   import        import an ordered set of patches
   unbundle      apply one or more bundle files
  
  Repository maintenance:
  
   manifest      output the current or given revision of the project manifest
   recover       roll back an interrupted transaction
   verify        verify the integrity of the repository
  
  Help:
  
   config        show combined config settings from all hgrc files
   help          show help for a given topic or a help overview
   version       output version and copyright information
  
  additional help topics:
  
  Mercurial identifiers:
  
   filesets      Specifying File Sets
   hgignore      Syntax for Mercurial Ignore Files
   patterns      File Name Patterns
   revisions     Specifying Revisions
   urls          URL Paths
  
  Mercurial output:
  
   color         Colorizing Outputs
   dates         Date Formats
   diffs         Diff Formats
   templating    Template Usage
  
  Mercurial configuration:
  
   config        Configuration Files
   environment   Environment Variables
   extensions    Using Additional Features
   flags         Command-line flags
   hgweb         Configuring hgweb
   merge-tools   Merge Tools
   pager         Pager Support
  
  Concepts:
  
   bundlespec    Bundle File Formats
   glossary      Glossary
   phases        Working with Phases
   subrepos      Subrepositories
  
  Miscellaneous:
  
   deprecated    Deprecated Features
   internals     Technical implementation topics
   scripting     Using Mercurial from scripts and automation
  
  (use 'hg help -v' to show built-in aliases and global options)

#endif

Not tested: --debugger