fastannotate: rip out specialized support for remotefilelog
remotefilelog can choose to collaborate with fastannotate for now if
it needs to, and in the future when we make good on our longstanding
desire to move remotefilelog-like features in-house we'll make sure
things are well-supported via a reasonable interface.
Differential Revision: https://phab.mercurial-scm.org/D4201