Mercurial > hg-stable
changeset 33269:ead6749354e1
phabricator: try to fetch differential revisions in batch
Previously, we read Differential Revisions one by one by calling
`differential.query`.
Fetching them one by one is suboptimal. Unfortunately, there is no Conduit
API that allows us to get a stack of diffids using a single API call.
This patch tries to be smarter using a simple heuristic: when fetching D59
as a stack, previous IDs like D51, D52, D53, ..., D58 are likely belonging
to a same stack so just fetch them as well. Since `differential.query` only
returns cheap metadata without expensive diff content, it shouldn't be a big
problem for the server.
Using a test Phabricator instance, this patch reduces `phabread` reading a
10 patch stack from about 13 to 30 seconds to 8 seconds.
author | Jun Wu <quark@fb.com> |
---|---|
date | Tue, 04 Jul 2017 16:36:48 -0700 |
parents | 85391b95961d |
children | f7b635716ef2 |
files | contrib/phabricator.py |
diffstat | 1 files changed, 24 insertions(+), 4 deletions(-) [+] |
line wrap: on
line diff
--- a/contrib/phabricator.py Tue Jul 04 16:36:48 2017 -0700 +++ b/contrib/phabricator.py Tue Jul 04 16:36:48 2017 -0700 @@ -358,14 +358,34 @@ order that the latter ones depend on the former ones. Otherwise, return a list of a unique "Differential Revision dict". """ + prefetched = {} # {id or phid: drev} + def fetch(params): + """params -> single drev or None""" + key = (params.get(r'ids') or params.get(r'phids') or [None])[0] + if key in prefetched: + return prefetched[key] + # Otherwise, send the request. If we're fetching a stack, be smarter + # and fetch more ids in one batch, even if it could be unnecessary. + batchparams = params + if stack and len(params.get(r'ids', [])) == 1: + i = int(params[r'ids'][0]) + # developer config: phabricator.batchsize + batchsize = repo.ui.configint('phabricator', 'batchsize', 12) + batchparams = {'ids': range(max(1, i - batchsize), i + 1)} + drevs = callconduit(repo, 'differential.query', batchparams) + # Fill prefetched with the result + for drev in drevs: + prefetched[drev[r'phid']] = drev + prefetched[int(drev[r'id'])] = drev + if key not in prefetched: + raise error.Abort(_('cannot get Differential Revision %r') % params) + return prefetched[key] + result = [] queue = [params] while queue: params = queue.pop() - drevs = callconduit(repo, 'differential.query', params) - if len(drevs) != 1: - raise error.Abort(_('cannot get Differential Revision %r') % params) - drev = drevs[0] + drev = fetch(params) result.append(drev) if stack: auxiliary = drev.get(r'auxiliary', {})