[PYTHON] Download the raw link of GitLab private repository with CLI

Download gitlab private repository's raw link using CLI

Starting with GitLab 11.5.1, you can no longer download raw links using the PRIVATE-TOKEN header (cf https://about.gitlab.com/releases/2018/11/28/security-release-gitlab-11 -dot-5-dot-1-released / Improper Enforcement of Token Scope). When the raw link was passed, I parsed and converted the URL so that I could download the file itself as well.

#!/usr/bin/python
#gitlab_download

import os
import sys
import json
import shutil
from contextlib import closing

if sys.version_info[0]>=3:
    import http.client as httplib
    from urllib.request import urlparse
    binstdout = sys.stdout.buffer
else:
    import httplib
    from urlparse import urlparse
    binstdout = sys.stdout

url = urlparse(sys.argv[1])
a=url.path[1:].split('/')
projname='/'.join([a[0],a[1]])
revision=a[3]
filepath='%2F'.join(a[4:])

token = os.environ.get('gitlab_token_'+url.hostname.replace('.','_'),'')

with closing(httplib.HTTPSConnection(url.hostname,port=url.port)) as https:
    https.request('GET','/api/v4/projects?search_namespaces=true&search='+projname,None,{'PRIVATE-TOKEN':token})
    resp = https.getresponse()
    jso = json.loads(resp.read())
    proj = next(e for e in jso if e['path_with_namespace']==projname)
    resolvedPath = '/api/v4/projects/'+str(proj['id'])+'/repository/files/'+filepath.replace('/','%2F')+'/raw?ref='+revision
    sys.stderr.write('resolved: https://%s%s\n'%(url.netloc,resolvedPath))
    https.request('GET',resolvedPath,None,{'PRIVATE-TOKEN':token})
    resp = https.getresponse()
    #print(resp.getheaders())
    shutil.copyfileobj(resp,binstdout)

200602

Changed to give search_namespaces = true when converting from project name to project ID.

Recommended Posts

Download the raw link of GitLab private repository with CLI
Download the file deployed with appcfg.py
Run the IDCF cloud CLI with Docker
Align the size of the colorbar with matplotlib
Check the existence of the file with python
Download files on the web with Python
The third night of the loop with for
The second night of the loop with for
Create a private repository with AWS CodeArtifact
Download the file with PHP [Under construction]
Count the number of characters with echo