view tests/get-with-headers.py @ 28492:837f1c437d58

changelog: lazily parse date/extra field

This is probably the most complicated patch in the parsing
refactor.

Because the date and extras are encoded in the same field, we
stuff the entire field into a dedicated variable and add a
property for accessing the sub-components of each. There is
some duplicated code here. But the code is relatively simple,
so it shouldn't be a big deal.

We see revset performance wins across the board:

author(mpm)
0.896565
0.876713
0.822961

desc(bug)
0.887169
0.895514
0.847054

date(2015)
0.878797
0.820987
0.811613

extra(rebase_source)
0.865446
0.823811
0.797756

author(mpm) or author(greg)
1.801832
1.784160
1.668172

author(mpm) or desc(bug)
1.812438
1.822756
1.677608

date(2015) or branch(default)
0.968276
0.910981
0.896032

author(mpm) or desc(bug) or date(2015) or extra(rebase_source)
3.656193
3.516788
3.265024

We see a speed-up on revsets accessing date and extras because the new
parsing code only parses what you access. Even though they are stored
the same text field, we avoid parsing dates when accessing extras and
vice-versa.

But strangely revsets accessing both date and extras appeared to speed
up as well! I'm not sure if this is due to refactoring the parsing
code or due to an optimization in revsets. You can't argue with the
results!

#!/usr/bin/env python

"""This does HTTP GET requests given a host:port and path and returns
a subset of the headers plus the body of the result."""

from __future__ import absolute_import

import httplib
import json
import os
import sys

try:
    import msvcrt
    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
    msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)
except ImportError:
    pass

twice = False
if '--twice' in sys.argv:
    sys.argv.remove('--twice')
    twice = True
headeronly = False
if '--headeronly' in sys.argv:
    sys.argv.remove('--headeronly')
    headeronly = True
formatjson = False
if '--json' in sys.argv:
    sys.argv.remove('--json')
    formatjson = True

tag = None
def request(host, path, show):
    assert not path.startswith('/'), path
    global tag
    headers = {}
    if tag:
        headers['If-None-Match'] = tag

    conn = httplib.HTTPConnection(host)
    conn.request("GET", '/' + path, None, headers)
    response = conn.getresponse()
    print response.status, response.reason
    if show[:1] == ['-']:
        show = sorted(h for h, v in response.getheaders()
                      if h.lower() not in show)
    for h in [h.lower() for h in show]:
        if response.getheader(h, None) is not None:
            print "%s: %s" % (h, response.getheader(h))
    if not headeronly:
        print
        data = response.read()

        # Pretty print JSON. This also has the beneficial side-effect
        # of verifying emitted JSON is well-formed.
        if formatjson:
            # json.dumps() will print trailing newlines. Eliminate them
            # to make tests easier to write.
            data = json.loads(data)
            lines = json.dumps(data, sort_keys=True, indent=2).splitlines()
            for line in lines:
                print line.rstrip()
        else:
            sys.stdout.write(data)

        if twice and response.getheader('ETag', None):
            tag = response.getheader('ETag')

    return response.status

status = request(sys.argv[1], sys.argv[2], sys.argv[3:])
if twice:
    status = request(sys.argv[1], sys.argv[2], sys.argv[3:])

if 200 <= status <= 305:
    sys.exit(0)
sys.exit(1)