tests/test-sparse-verbose-json.t

perf: add threading capability to perfbdiff

Since we are releasing the GIL during diffing, it is interesting to see how a
thread pool would perform on diffing. We add a new `--threads` argument to
commands. Synchronizing the thread pool is a bit complex because we want to be
able to reuse it from one run to another.

On my computer (i7 with 4 cores + hyperthreading), I get the following data for
about 12000 revisions:
threads wall        comb    wall gain   comb overhead
none    31.596715   31.59    0.00%       0.00%
1       31.621228   31.62   -0.08%       0.09%
2       16.406202   32.8    48.08%       3.83%
3       11.598334   34.76   63.29%      10.03%
4        9.205421   36.77   70.87%      16.40%
5        8.517604   42.51   73.04%      34.57%
6        7.94645    47.58   74.85%      50.62%
7        7.434972   51.92   76.47%      64.36%
8        7.070638   55.34   77.62%      75.18%

Compared to the feature disabled (threads=0), the overhead is negligible with
the threading code (threads=1), and the gain is already 48% with two threads.

test sparse with --verbose and -T json

  $ hg init myrepo
  $ cd myrepo
  $ cat > .hg/hgrc <<EOF
  > [extensions]
  > sparse=
  > strip=
  > EOF

  $ echo a > show
  $ echo x > hide
  $ hg ci -Aqm 'initial'

  $ echo b > show
  $ echo y > hide
  $ echo aa > show2
  $ echo xx > hide2
  $ hg ci -Aqm 'two'

Verify basic --include and --reset

  $ hg up -q 0
  $ hg debugsparse --include 'hide' -Tjson
  [
   {
    "exclude_rules_added": 0,
    "files_added": 0,
    "files_conflicting": 0,
    "files_dropped": 1,
    "include_rules_added": 1,
    "profiles_added": 0
   }
  ]
  $ hg debugsparse --clear-rules
  $ hg debugsparse --include 'hide' --verbose
  removing show
  Profiles changed: 0
  Include rules changed: 1
  Exclude rules changed: 0

  $ hg debugsparse --reset -Tjson
  [
   {
    "exclude_rules_added": 0,
    "files_added": 1,
    "files_conflicting": 0,
    "files_dropped": 0,
    "include_rules_added": -1,
    "profiles_added": 0
   }
  ]
  $ hg debugsparse --include 'hide'
  $ hg debugsparse --reset --verbose
  getting show
  Profiles changed: 0
  Include rules changed: -1
  Exclude rules changed: 0

Verifying that problematic files still allow us to see the deltas when forcing:

  $ hg debugsparse --include 'show*'
  $ touch hide
  $ hg debugsparse --delete 'show*' --force -Tjson
  pending changes to 'hide'
  [
   {
    "exclude_rules_added": 0,
    "files_added": 0,
    "files_conflicting": 1,
    "files_dropped": 0,
    "include_rules_added": -1,
    "profiles_added": 0
   }
  ]
  $ hg debugsparse --include 'show*' --force
  pending changes to 'hide'
  $ hg debugsparse --delete 'show*' --force --verbose
  pending changes to 'hide'
  Profiles changed: 0
  Include rules changed: -1
  Exclude rules changed: 0