Crash when an release asset doesn't exist

Currently, the script crashes whenever a release asset is unable to download (for example a 404 response). This change instead logs the failure and allows the script to continue. No retry logic is enabled, but at least it prevents the crash and allows the backup to complete. Retry logic can be implemented later if wanted.

closes https://github.com/josegonzalez/python-github-backup/issues/129
This commit is contained in:
Ben Baron
2020-01-06 11:13:25 -05:00
parent fac8e4274f
commit 27441b71b6

View File

@@ -569,15 +569,27 @@ def download_file(url, path, auth):
request.add_header('Accept', 'application/octet-stream')
request.add_header('Authorization', 'Basic '.encode('ascii') + auth)
opener = build_opener(S3HTTPRedirectHandler)
response = opener.open(request)
chunk_size = 16 * 1024
with open(path, 'wb') as f:
while True:
chunk = response.read(chunk_size)
if not chunk:
break
f.write(chunk)
try:
response = opener.open(request)
chunk_size = 16 * 1024
with open(path, 'wb') as f:
while True:
chunk = response.read(chunk_size)
if not chunk:
break
f.write(chunk)
except HTTPError as exc:
# Gracefully handle 404 responses (and others) when downloading from S3
log_info('Skipping download of asset {0} due to HTTPError: {1}'.format(url, exc.reason))
except URLError as e:
# Gracefully hadnle other URL errors
log_info('Skipping download of asset {0} due to URLError: {1}'.format(url, e.reason))
except socket.error as e:
# Gracefully handle socket errors
# TODO: Implement retry logic
log_info('Skipping download of asset {0} due to socker error: {1}'.format(url, e.strerror))
def get_authenticated_user(args):