Commit Graph

99 Commits

Author SHA1 Message Date
Ben Baron
869f761c90 Separate release assets and skip re-downloading
Currently the script puts all release assets into the same folder called `releases`. So any time 2 release files have the same name, only the last one downloaded is actually saved. A particularly bad example of this is MacDownApp/macdown where all of their releases are named `MacDown.app.zip`. So even though they have 36 releases and all 36 are downloaded, only the last one is actually saved.

With this change, each releases' assets are now stored in a fubfolder inside `releases` named after the release name. There could still be edge cases if two releases have the same name, but this is still much safer tha the previous behavior.

This change also now checks if the asset file already exists on disk and skips downloading it. This drastically speeds up addiotnal syncs as it no longer downloads every single release every single time. It will now only download new releases which I believe is the expected behavior.

closes https://github.com/josegonzalez/python-github-backup/issues/126
2020-01-06 12:40:47 -05:00
Vladislav Yarmak
a56d27dd8b workaround gist clone in --prefer-ssh mode 2019-09-21 19:22:27 +03:00
2a
e74765ba7f Issue 119: Change retrieve_data to be a generator
See issue #119.
2019-07-03 23:01:00 +02:00
Jose Diaz-Gonzalez
7305871c20 Merge pull request #117 from QuicketSolutions/master
Add option for Releases
2019-06-27 11:15:02 -04:00
Harrison Wright
89f59cc7a2 Make assets it's own flag 2019-06-24 15:49:19 -05:00
Harrison Wright
921aab3729 Fix pull details 2019-06-22 13:19:45 -05:00
Harrison Wright
ea4c3d0f6f Fix super call for python2 2019-06-22 13:05:54 -05:00
Harrison Wright
9b6400932d Fix redirect to s3 2019-06-22 13:00:42 -05:00
Harrison Wright
de0c3f46c6 WIP: download assets 2019-06-21 20:03:14 -05:00
ethan
4b40ae94d7 QKT-42 update: shorter command flag 2019-06-21 16:48:25 -05:00
ethan
41130fc8b0 QKT-42: support saving release information 2019-06-21 11:20:32 -05:00
Gael de Chalendar
3193d120e5 Avoid to crash in case of HTTP 502 error
Survive also on socket.error connections like on HTTPError or URLError.

This should solve issue #110.
2019-06-04 18:53:58 +02:00
Jose Diaz-Gonzalez
c86163bfe6 fix: warn instead of error
Refs #106
2019-02-21 15:40:39 -05:00
JOHN STETIC
57ab5ce1a2 Log URL error https://github.com/josegonzalez/python-github-backup/issues/105 2019-02-20 20:43:00 -05:00
W. Harrison Wright
9e472b74e6 Remove unnecessary sys.exit call 2018-12-27 13:07:13 -06:00
W. Harrison Wright
4b459f9af8 Add org check to avoid incorrect log output 2018-12-27 12:58:57 -06:00
W. Harrison Wright
b70ea87db7 Fix accidental system exit with better logging strategy 2018-12-27 12:53:21 -06:00
Bernd
628f2cbf73 Mark options which are not included in --all
As discussed in Issue #100
2018-12-24 04:19:29 +01:00
Jose Diaz-Gonzalez
899cf42b57 Merge pull request #97 from whwright/94-fix-user-repos
Correctly download repos when user arg != authenticated user
2018-11-28 01:58:37 -05:00
W. Harrison Wright
b5972aaaf0 Correctly download repos when user arg != authenticated user 2018-11-11 19:40:46 -06:00
W. Harrison Wright
4a4a317331 Clone the specified user's gists, not the authenticated user 2018-09-28 21:59:50 -05:00
W. Harrison Wright
5a8e1ac275 Clone the specified user's starred repos, not the authenticated user 2018-09-28 21:46:28 -05:00
Jose Diaz-Gonzalez
413d4381cc fix: cleanup pep8 violations 2018-03-24 01:16:28 -04:00
Jose Diaz-Gonzalez
1a79f755a5 Merge pull request #77 from mayflower/pull-details
Pull Details
2018-03-23 23:40:22 -04:00
Johannes Bornhold
fd33037b1c Mark string as binary in comparison for skip_existing
Found out that the flag "--skip-existing" did not work out as expected on Python
3.6. Tracked it down to the comparison which has to be against a string of bytes
in Python3.
2018-02-26 11:21:25 +01:00
Stephen Greene
eca9f0f7df Add option to fetch followers/following JSON data 2018-02-21 19:29:59 -08:00
W. Harrison Wright
f378254188 Short circuit gists backup process 2018-02-07 21:46:59 -06:00
W. Harrison Wright
83128e986a Formatting 2018-02-07 21:30:55 -06:00
W. Harrison Wright
17e4f9a125 Add ability to backup gists 2018-02-07 21:29:49 -06:00
W. Harrison Wright
7cccd42ec9 Change option to --all-starred 2018-01-14 10:22:10 -06:00
W. Harrison Wright
bd346de898 Put starred clone repoistories under a new option 2018-01-13 17:43:00 -06:00
W. Harrison Wright
6e3cbe841a Add comment 2018-01-13 14:12:26 -06:00
W. Harrison Wright
8b95f187ad Add ability to clone starred repos 2018-01-13 14:08:36 -06:00
Robin Gloster
ef88248c41 Add additional output for the current request
This is useful to have some progress indication for huge repositories.
2017-12-29 23:33:53 +01:00
Robin Gloster
0a4decfb3b Add option to backup additional PR details
Some payload is only included when requesting a single pull request
2017-12-29 21:39:59 +01:00
Sam Libby
4f4785085d update logging_subprocess function
1. added newline for return
2. added one-time warning (once per subprocess)
2017-12-11 09:25:49 -07:00
Sam Libby
76895dcf69 update check_io() to allow scripts to run on Windows 2017-12-10 21:44:26 -07:00
Edward Pfremmer
27a1ba2d04 Fix arg not defined error
Ref: https://github.com/josegonzalez/python-github-backup/issues/69
2017-10-11 15:12:34 -05:00
pieterclaerhout
bb551a83f4 Updated the readme 2017-10-11 15:14:13 +02:00
pieterclaerhout
9b1b4a9ebc Added a check to see if git-lfs is installed when doing an LFS clone 2017-10-11 15:11:14 +02:00
pieterclaerhout
e6b6eb8bef Added support for LFS clones 2017-10-10 19:52:07 +02:00
Dima Gerasimov
922a3c5a6e Fix remotes while updating repository 2017-05-06 12:58:42 +01:00
Nicolai Ehemann
de93824498 Fix error when repository has no updated_at value 2017-04-11 11:10:03 +02:00
Martin O'Reilly
0dfe5c342a Add OS check for OSX specific keychain args
Keychain arguments are only supported on Mac OSX.
Added check for operating system so we give a
"Keychain arguments are only supported on Mac OSX"
error message rather than a "No password item matching the
provided name and account could be found in the osx keychain"
error message
2017-04-05 16:36:52 +01:00
Martin O'Reilly
1d6e1abab1 Add support for storing PAT in OSX keychain
Added additional optional arguments and README guidance for storing
and accessing a Github personal access token (PAT) in the OSX
keychain
2017-04-05 15:17:52 +01:00
Chris Adams
92c619cd01 Avoid remote branch name churn
This avoids the backup output having lots of "[new branch]" messages
because removing the old remote name removed all of the existing branch
references.
2017-03-27 16:26:19 -04:00
Andrzej Maczuga
6592bd8196 Fix detection of bare git directories 2016-11-22 20:11:26 +00:00
Jose Diaz-Gonzalez
88148b4c95 pep8: E501 line too long (83 > 79 characters) 2016-11-22 10:55:37 -07:00
Jose Diaz-Gonzalez
8448add464 pep8: E128 continuation line under-indented for visual indent 2016-11-22 10:51:04 -07:00
Jose Diaz-Gonzalez
5b30b7ebdd fix: properly import version from github_backup package 2016-11-22 10:49:18 -07:00