Compare commits

...

152 Commits

Author SHA1 Message Date
Jose Diaz-Gonzalez
321414d352 Release version 0.19.2 2018-03-24 01:16:34 -04:00
Jose Diaz-Gonzalez
413d4381cc fix: cleanup pep8 violations 2018-03-24 01:16:28 -04:00
Jose Diaz-Gonzalez
0110ea40ed Release version 0.19.1 2018-03-24 01:04:35 -04:00
Jose Diaz-Gonzalez
8d2ef2f528 Release version 0.19.0 2018-03-24 00:54:34 -04:00
Jose Diaz-Gonzalez
1a79f755a5 Merge pull request #77 from mayflower/pull-details
Pull Details
2018-03-23 23:40:22 -04:00
Jose Diaz-Gonzalez
abf45d5b54 Merge pull request #84 from johbo/fix-python36-skip-existing
Mark string as binary in comparison for skip_existing
2018-02-26 10:44:12 -05:00
Johannes Bornhold
fd33037b1c Mark string as binary in comparison for skip_existing
Found out that the flag "--skip-existing" did not work out as expected on Python
3.6. Tracked it down to the comparison which has to be against a string of bytes
in Python3.
2018-02-26 11:21:25 +01:00
Jose Diaz-Gonzalez
87dab293ed Release version 0.18.0 2018-02-22 12:13:06 -05:00
Jose Diaz-Gonzalez
0244af4e05 Merge pull request #82 from sgreene570/add-followers
Add option to fetch followers/following JSON data
2018-02-22 12:11:48 -05:00
Stephen Greene
eca9f0f7df Add option to fetch followers/following JSON data 2018-02-21 19:29:59 -08:00
Jose Diaz-Gonzalez
afa2a6d587 Release version 0.17.0 2018-02-20 13:06:48 -05:00
Jose Diaz-Gonzalez
b77ea48d74 Merge pull request #81 from whwright/gists
Add ability to back up gists
2018-02-19 15:29:20 -05:00
W. Harrison Wright
f378254188 Short circuit gists backup process 2018-02-07 21:46:59 -06:00
W. Harrison Wright
83128e986a Formatting 2018-02-07 21:30:55 -06:00
W. Harrison Wright
17e4f9a125 Add ability to backup gists 2018-02-07 21:29:49 -06:00
Jose Diaz-Gonzalez
e59d1e3a68 Release version 0.16.0 2018-01-22 12:49:31 -05:00
Jose Diaz-Gonzalez
de860ee5a9 Merge pull request #78 from whwright/clone-starred-repos
Clone starred repos
2018-01-22 12:36:42 -05:00
Jose Diaz-Gonzalez
cb054c2631 Update README.rst 2018-01-22 12:36:32 -05:00
W. Harrison Wright
c142707a90 Update documentation 2018-01-22 11:34:27 -06:00
W. Harrison Wright
7cccd42ec9 Change option to --all-starred 2018-01-14 10:22:10 -06:00
W. Harrison Wright
9a539b1d6b JK don't update documentation 2018-01-14 10:18:51 -06:00
W. Harrison Wright
cd2372183e Update documentation 2018-01-13 17:44:09 -06:00
W. Harrison Wright
bd346de898 Put starred clone repoistories under a new option 2018-01-13 17:43:00 -06:00
W. Harrison Wright
6e3cbe841a Add comment 2018-01-13 14:12:26 -06:00
W. Harrison Wright
8b95f187ad Add ability to clone starred repos 2018-01-13 14:08:36 -06:00
Robin Gloster
ef88248c41 Add additional output for the current request
This is useful to have some progress indication for huge repositories.
2017-12-29 23:33:53 +01:00
Robin Gloster
0a4decfb3b Add option to backup additional PR details
Some payload is only included when requesting a single pull request
2017-12-29 21:39:59 +01:00
Jose Diaz-Gonzalez
2b9549ffde Release version 0.15.0 2017-12-11 11:46:16 -05:00
Jose Diaz-Gonzalez
fb2c3ca921 Merge pull request #75 from slibby/slibby-patch-windows
update check_io() to allow scripts to run on Windows
2017-12-11 11:45:58 -05:00
Sam Libby
4f4785085d update logging_subprocess function
1. added newline for return
2. added one-time warning (once per subprocess)
2017-12-11 09:25:49 -07:00
Sam Libby
76895dcf69 update check_io() to allow scripts to run on Windows 2017-12-10 21:44:26 -07:00
Jose Diaz-Gonzalez
1d50a4038b Release version 0.14.1 2017-10-11 16:18:21 -04:00
Jose Diaz-Gonzalez
9d31ccfba9 Merge pull request #70 from epfremmer/patch-1
Fix arg not defined error
2017-10-11 16:17:58 -04:00
Edward Pfremmer
27a1ba2d04 Fix arg not defined error
Ref: https://github.com/josegonzalez/python-github-backup/issues/69
2017-10-11 15:12:34 -05:00
Jose Diaz-Gonzalez
f157ea107f Release version 0.14.0 2017-10-11 11:52:16 -04:00
Jose Diaz-Gonzalez
a129cc759a Merge pull request #68 from pieterclaerhout/master
Added support for LFS clones
2017-10-11 11:51:57 -04:00
pieterclaerhout
bb551a83f4 Updated the readme 2017-10-11 15:14:13 +02:00
pieterclaerhout
9b1b4a9ebc Added a check to see if git-lfs is installed when doing an LFS clone 2017-10-11 15:11:14 +02:00
pieterclaerhout
e6b6eb8bef Added support for LFS clones 2017-10-10 19:52:07 +02:00
Jose Diaz-Gonzalez
0b3f120e2b Merge pull request #66 from albertyw/python3
Explicitly support python 3
2017-09-30 21:01:27 -04:00
Albert Wang
990249b80b Add pypi info to readme 2017-09-30 17:16:38 -07:00
Albert Wang
cefb226545 Explicitly support python 3 in package description 2017-09-30 17:13:47 -07:00
Jose Diaz-Gonzalez
ea22ffdf26 Merge pull request #65 from mumblez/master
add couple examples to help new users
2017-05-30 11:58:42 -06:00
Yusuf Tran
0f21d7b8a4 add couple examples to help new users 2017-05-30 18:52:11 +01:00
Jose Diaz-Gonzalez
cb33b9bab7 Release version 0.13.2 2017-05-06 14:14:08 -06:00
Jose Diaz-Gonzalez
68c48cb0b3 Merge pull request #64 from karlicoss/fix-remotes
Fix remotes while updating repository
2017-05-06 14:13:46 -06:00
Dima Gerasimov
922a3c5a6e Fix remotes while updating repository 2017-05-06 12:58:42 +01:00
Jose Diaz-Gonzalez
d4055eb99c Release version 0.13.1 2017-04-11 09:40:13 -06:00
Jose Diaz-Gonzalez
d8a330559c Merge pull request #61 from McNetic/fix_empty_updated_at
Fix error when repository has no updated_at value
2017-04-11 09:37:15 -06:00
Nicolai Ehemann
de93824498 Fix error when repository has no updated_at value 2017-04-11 11:10:03 +02:00
Jose Diaz-Gonzalez
2efeaa7580 Release version 0.13.0 2017-04-05 11:49:49 -04:00
Jose Diaz-Gonzalez
647810a2f0 Merge pull request #59 from martintoreilly/master
Add support for storing PAT in OSX keychain
2017-04-05 11:49:24 -04:00
Martin O'Reilly
0dfe5c342a Add OS check for OSX specific keychain args
Keychain arguments are only supported on Mac OSX.
Added check for operating system so we give a
"Keychain arguments are only supported on Mac OSX"
error message rather than a "No password item matching the
provided name and account could be found in the osx keychain"
error message
2017-04-05 16:36:52 +01:00
Martin O'Reilly
1d6e1abab1 Add support for storing PAT in OSX keychain
Added additional optional arguments and README guidance for storing
and accessing a Github personal access token (PAT) in the OSX
keychain
2017-04-05 15:17:52 +01:00
Jose Diaz-Gonzalez
dd2b96b172 Release version 0.12.1 2017-03-27 14:55:11 -06:00
Jose Diaz-Gonzalez
7a589f1e63 Merge pull request #57 from acdha/reuse-existing-remotes
Avoid remote branch name churn
2017-03-27 14:54:02 -06:00
Chris Adams
92c619cd01 Avoid remote branch name churn
This avoids the backup output having lots of "[new branch]" messages
because removing the old remote name removed all of the existing branch
references.
2017-03-27 16:26:19 -04:00
Jose Diaz-Gonzalez
9a91dd7733 Merge pull request #55 from amaczuga/master
Fix detection of bare git directories
2016-11-22 13:36:52 -07:00
Andrzej Maczuga
6592bd8196 Fix detection of bare git directories 2016-11-22 20:11:26 +00:00
Jose Diaz-Gonzalez
e9e3b18512 Release version 0.12.0 2016-11-22 10:56:56 -07:00
Jose Diaz-Gonzalez
88148b4c95 pep8: E501 line too long (83 > 79 characters) 2016-11-22 10:55:37 -07:00
Jose Diaz-Gonzalez
8448add464 pep8: E128 continuation line under-indented for visual indent 2016-11-22 10:51:04 -07:00
Jose Diaz-Gonzalez
5b30b7ebdd fix: properly import version from github_backup package 2016-11-22 10:49:18 -07:00
Jose Diaz-Gonzalez
c3a17710d3 fix: support alternate git status output 2016-11-22 10:48:07 -07:00
Jose Diaz-Gonzalez
4462412ec7 Merge pull request #54 from amaczuga/master
Support archivization using bare git clones
2016-11-22 09:44:54 -07:00
Andrzej Maczuga
8d61538e5e Support archivization using bare git clones 2016-11-22 13:07:52 +00:00
Jose Diaz-Gonzalez
4d37ad206f Merge pull request #53 from trel/master
fix typo, 3x
2016-11-18 15:35:53 -05:00
Terrell Russell
1f983863fc fix typo, 3x 2016-11-18 15:17:42 -05:00
Jose Diaz-Gonzalez
f0b28567b9 Release version 0.11.0 2016-10-26 14:14:00 -06:00
Jose Diaz-Gonzalez
77ede50b19 Merge pull request #52 from bjodah/fix-gh-51
Support --token file:///home/user/token.txt (fixes gh-51)
2016-10-26 14:13:35 -06:00
Björn Dahlgren
97e4fbbacb Support --token file:///home/user/token.txt (fixes gh-51) 2016-10-26 01:57:33 +02:00
Jose Diaz-Gonzalez
03604cc654 Merge pull request #48 from albertyw/python3
Support Python 3
2016-10-25 17:38:05 -06:00
Albert Wang
73a62fdee1 Fix some linting 2016-09-11 01:14:36 -07:00
Albert Wang
94e1d62ad5 Fix byte/string conversion for python 3 2016-09-11 01:14:31 -07:00
Albert Wang
54cef11ce7 Support python 3 2016-09-11 01:14:19 -07:00
Jose Diaz-Gonzalez
56397eba1c Merge pull request #46 from remram44/encode-password
Encode special characters in password
2016-09-06 14:31:35 -04:00
Remi Rampin
9f861efccf Encode special characters in password 2016-09-06 14:27:47 -04:00
Jose Diaz-Gonzalez
c1c9ce6dca Merge pull request #45 from remram44/cli-programname
Fix program name
2016-09-06 12:42:45 -04:00
Jose Diaz-Gonzalez
ab18d8aee0 Merge pull request #44 from remram44/readme-git-https
Don't install over insecure connection
2016-09-06 12:42:30 -04:00
Remi Rampin
9d7d98b19e Update README.rst 2016-09-06 12:28:44 -04:00
Remi Rampin
0233bff696 Don't pretend program name is "Github Backup" 2016-09-06 12:24:51 -04:00
Remi Rampin
6154ceda15 Don't install over insecure connection
The git:// protocol is unauthenticated and unencrypted, and no longer advertised by GitHub. Using HTTPS shouldn't impact performance.
2016-09-06 12:11:29 -04:00
Jose Diaz-Gonzalez
9023052e9c Release version 0.10.3 2016-08-20 20:50:29 -04:00
Jose Diaz-Gonzalez
874c235ba5 Merge pull request #30 from jonasrmichel/master
Fixes #29
2016-08-20 20:50:25 -04:00
Jose Diaz-Gonzalez
b7b234d8a5 Release version 0.10.2 2016-08-20 20:49:46 -04:00
Jose Diaz-Gonzalez
ed160eb0ca Add a note regarding git version requirement
Closes #37
2016-08-20 20:49:42 -04:00
Jose Diaz-Gonzalez
1d11d62b73 Release version 0.10.1 2016-08-20 20:45:27 -04:00
Jose Diaz-Gonzalez
9e1cba9817 Release version 0.10.0 2016-08-18 14:20:46 -04:00
Jose Diaz-Gonzalez
3859a80b7a Merge pull request #42 from robertwb/master
Implement incremental updates
2016-08-18 14:20:05 -04:00
Robert Bradshaw
8c12d54898 Implement incremental updates
Guarded with an --incremental flag.

Stores the time of the last update and only downloads issue and
pull request data since this time.  All other data is relatively
small (likely fetched with a single request) and so is simply
re-populated from scratch as before.
2016-08-17 21:31:59 -07:00
Jose Diaz-Gonzalez
b6b6605acd Release version 0.9.0 2016-03-29 13:23:45 -04:00
Jose Diaz-Gonzalez
ff5e0aa89c Merge pull request #36 from zlabjp/fix-cloning-private-repos
Fix cloning private repos with basic auth or token
2016-03-29 13:21:57 -04:00
Kazuki Suda
79726c360d Fix cloning private repos with basic auth or token 2016-03-29 15:23:54 +09:00
Jose Diaz-Gonzalez
a511bb2b49 Release version 0.8.0 2016-02-14 16:04:54 -05:00
Jose Diaz-Gonzalez
aedf9b2c66 Merge pull request #35 from eht16/issue23_store_pullrequests_once
Don't store issues which are actually pull requests
2016-02-14 16:02:18 -05:00
Enrico Tröger
b9e35a50f5 Don't store issues which are actually pull requests
This prevents storing pull requests twice since the Github API returns
pull requests also as issues. Those issues will be skipped but only if
retrieving pull requests is requested as well.
Closes #23.
2016-02-14 16:36:40 +01:00
Jose Diaz-Gonzalez
d0e239b3ef Release version 0.7.0 2016-02-02 14:52:07 -05:00
Jose Diaz-Gonzalez
29c9373d9d Merge pull request #32 from albertyw/soft-fail-hooks
Softly fail if not able to read hooks
2016-01-29 03:36:48 -05:00
Jose Diaz-Gonzalez
eb8b22c81c Merge pull request #33 from albertyw/update-readme
Add note about 2-factor auth in readme
2016-01-29 03:35:28 -05:00
Jose Diaz-Gonzalez
03739ce1be Merge pull request #31 from albertyw/fix-private-repos
Fix reading user's private repositories
2016-01-29 03:34:21 -05:00
Albert Wang
d2bb205b4b Add note about 2-factor auth 2016-01-29 00:33:53 -08:00
Albert Wang
17141c1bb6 Softly fail if not able to read hooks 2016-01-29 00:20:53 -08:00
Albert Wang
d362adbbca Make user repository search go through endpoint capable of reading private repositories 2016-01-28 22:52:58 -08:00
Jonas Michel
1e5a90486c Fixes #29
Reporting an error when the user's rate limit is exceeded causes
the script to terminate after resuming execution from a rate limit
sleep. Instead of generating an explicit error we just want to
inform the user that the script is going to sleep until their rate
limit count resets.
2016-01-20 14:48:02 -06:00
Jonas Michel
9b74aff20b Fixes #29
The errors list was not being cleared out after resuming a backup
from a rate limit sleep. When the backup was resumed, the non-empty
errors list caused the backup to quit after the next `retrieve_data`
request.
2016-01-17 11:10:28 -06:00
Jose Diaz-Gonzalez
89df625e04 Merge pull request #28 from alexmojaki/getpass
Prompt for password if only username given
2016-01-15 10:34:29 -05:00
Alex Hall
675484a215 Update README with new CLI usage 2016-01-12 14:40:29 +02:00
Alex Hall
325f77dcd9 Prompt for password if only username given 2016-01-12 11:18:26 +02:00
Jose Diaz-Gonzalez
f12e9167aa Release version 0.6.0 2015-11-10 15:36:20 -05:00
Jose Diaz-Gonzalez
816447af19 Force proper remote url 2015-11-10 15:36:12 -05:00
Jose Diaz-Gonzalez
d9e15e2be2 Merge pull request #24 from eht16/add_backup_hooks
Add backup hooks
2015-10-21 16:47:26 -04:00
Enrico Tröger
534145d178 Improve error handling in case of HTTP errors
In case of a HTTP status code 404, the returned 'r' was never assigned.
In case of URL errors which are not timeouts, we probably should bail
out.
2015-10-21 22:40:34 +02:00
Enrico Tröger
fe162eedd5 Add --hooks to also include web hooks into the backup 2015-10-21 22:39:45 +02:00
Jose Diaz-Gonzalez
53a9a22afb Merge pull request #22 from eht16/issue_17_create_output_directory
Create the user specified output directory if it does not exist
2015-10-16 15:05:29 -04:00
Jose Diaz-Gonzalez
2aa7d4cf1e Merge pull request #21 from eht16/fix_get_response_missing_auth
Add missing auth argument to _get_response()
2015-10-16 15:05:14 -04:00
Jose Diaz-Gonzalez
804843c128 Merge pull request #20 from eht16/improve_error_msg_on_non_existing_repo
Add repository URL to error message for non-existing repositories
2015-10-16 15:05:05 -04:00
Enrico Tröger
5fc27a4d42 Create the user specified output directory if it does not exist
Fixes #17.
2015-10-16 14:16:47 +02:00
Enrico Tröger
c8b3f048f5 Add repository URL to error message for non-existing repositories
This makes it easier for the user to identify which repository does not
exist or is not initialised, i.e. whether it is the main repository or
the wiki repository and which clone URL was used to check.
2015-10-16 14:09:13 +02:00
Enrico Tröger
2d98251992 Add missing auth argument to _get_response()
When running unauthenticated and Github starts rate-limiting the client,
github-backup crashes because the used auth variable in _get_response()
was not available. This change should fix it.
2015-10-16 14:00:56 +02:00
Jose Diaz-Gonzalez
050f5f1c17 Release version 0.5.0 2015-10-10 00:19:45 -04:00
Jose Diaz-Gonzalez
348a238770 Add release script 2015-10-10 00:19:31 -04:00
Jose Diaz-Gonzalez
708b377918 Refactor to both simplify codepath as well as follow PEP8 standards 2015-10-10 00:16:30 -04:00
Jose Diaz-Gonzalez
6193efb798 Merge pull request #19 from Embed-Engineering/retry-timeout
Retry 3 times when the connection times out
2015-09-04 10:36:57 -04:00
Mathijs Jonker
4b30aaeef3 Retry 3 times when the connection times out 2015-09-04 14:07:45 +02:00
Jose Diaz-Gonzalez
762059d1a6 Merge pull request #15 from kromkrom/master
Preserve Unicode characters in the output file
2015-05-04 14:13:11 -04:00
Kirill Grushetsky
a440bc1522 Update github-backup 2015-05-04 19:16:23 +03:00
Kirill Grushetsky
43793c1e5e Update github-backup 2015-05-04 19:15:55 +03:00
Kirill Grushetsky
24fac46459 Made unicode output defalut 2015-05-04 19:12:47 +03:00
Kirill Grushetsky
c9916e28a4 Import alphabetised 2015-05-04 13:45:39 +03:00
Kirill Grushetsky
ab4b28cdd4 Preserve Unicode characters in the output file
Added option to preserve Unicode characters in the output file
2015-05-04 13:38:28 +03:00
Jose Diaz-Gonzalez
6feb409fc2 Merge pull request #14 from aensley/master
Added backup of labels and milestones.
2015-04-23 15:37:14 -04:00
aensley
8bdbc2cee2 josegonzales/python-github-backup#12 Added backup of labels and milestones. 2015-04-23 14:05:48 -05:00
Jose Diaz-Gonzalez
a4d6272b50 Merge pull request #11 from Embed-Engineering/master
Added test for uninitialized repo's (or wiki's)
2015-04-15 11:03:25 -04:00
Mathijs Jonker
7ce61202e5 Fixed indent 2015-04-15 12:21:58 +02:00
mjonker-embed
3e82d829e4 Update github-backup 2015-04-15 12:14:55 +02:00
mjonker-embed
339ad96876 Skip unitialized repo's
These gave me errors which caused mails from crontab.
2015-04-15 12:10:53 +02:00
Jose Diaz-Gonzalez
b2a942eb43 Merge pull request #10 from Embed-Engineering/master
Added prefer-ssh
2015-03-20 10:54:42 -04:00
mjonker-embed
e8aa38f395 Added prefer-ssh
Was needed for my back-up setup, code includes this but readme wasn't updated
2015-03-20 14:22:53 +01:00
Jose Diaz-Gonzalez
86bdb1420c Merge pull request #9 from acdha/ratelimit-retries
Retry API requests which failed due to rate-limiting
2015-03-13 17:55:25 -04:00
Chris Adams
2e7f325475 Retry API requests which failed due to rate-limiting
This allows operation to continue, albeit at a slower pace,
if you have enough data to trigger the API rate limits
2015-03-13 17:37:01 -04:00
Jose Diaz-Gonzalez
8bf62cd932 Release 0.4.0 2015-03-13 16:33:28 -04:00
Jose Diaz-Gonzalez
63bf7267a6 Merge pull request #7 from acdha/repo-backup-overhaul
Repo backup overhaul
2015-03-13 16:32:49 -04:00
Chris Adams
5612e51153 Update repository back up handling for wikis
* Now wikis will follow the same logic as the main repo
  checkout for --prefer-ssh.
* The regular repository and wiki paths both use the same
  function to handle either cloning or updating a local copy
  of the remote repo
* All git updates will now use “git fetch --all --tags”
  to ensure that tags and branches other than master will
  also be backed up
2015-03-13 15:50:30 -04:00
Chris Adams
c81bf98627 logging_subprocess: always log when a command fails
Previously git clones could fail without any indication 
unless you edited the source to change `logger=None` to use
a configured logger.

Now a non-zero return code will always output a message to
stderr and will display the executed command so it can be
rerun for troubleshooting.
2015-03-13 15:50:04 -04:00
Chris Adams
040516325a Switch to using ssh_url
The previous commit used the wrong URL for a private repo. This was
masked by the lack of error loging in logging_subprocess (which will be
in a separate branch)
2015-03-13 15:39:35 -04:00
Jose Diaz-Gonzalez
dca9f8051b Merge pull request #6 from acdha/allow-clone-over-ssh
Add an option to prefer checkouts over SSH
2015-03-12 17:15:19 -04:00
Chris Adams
3bc23473b8 Add an option to prefer checkouts over SSH
This is really useful with private repos to avoid being nagged
for credentials for every repository
2015-03-12 16:10:46 -04:00
Jose Diaz-Gonzalez
2c9eb80cf2 Release 0.3.0 2015-02-20 12:41:25 -05:00
Jose Diaz-Gonzalez
bb86f0582e Merge pull request #4 from klaude/pull_request_support
Add pull request support
2015-01-16 11:06:01 -05:00
Kevin Laude
e8387f9a7f Add pull request support
Back up reporitory pull requests by passing the --include-pulls
argument. Pull requests are saved to
repositories/<repository name>/pulls/<pull request number>.json. Include
the --pull-request-comments argument to add review comments to the pull
request backup and pass the --pull-request-commits argument to add
commits to the pull request backup.

Pull requests are automatically backed up when the --all argument is
uesd.
2015-01-16 09:57:05 -06:00
Jose Diaz-Gonzalez
39b173f173 Merge pull request #5 from klaude/github-enterprise-support
Add GitHub Enterprise Support
2015-01-15 22:05:33 -05:00
Kevin Laude
883c92753d Add GitHub Enterprise support
Pass the -H or --github-host argument with a GitHub Enterprise hostname
to backup from that GitHub enterprise host. If no argument is passed
then back up from github.com.
2015-01-15 20:20:33 -06:00
6 changed files with 1426 additions and 155 deletions

383
CHANGES.rst Normal file
View File

@@ -0,0 +1,383 @@
Changelog
=========
0.19.2 (2018-03-24)
-------------------
Fix
~~~
- Cleanup pep8 violations. [Jose Diaz-Gonzalez]
0.19.0 (2018-03-24)
-------------------
- Add additional output for the current request. [Robin Gloster]
This is useful to have some progress indication for huge repositories.
- Add option to backup additional PR details. [Robin Gloster]
Some payload is only included when requesting a single pull request
- Mark string as binary in comparison for skip_existing. [Johannes
Bornhold]
Found out that the flag "--skip-existing" did not work out as expected on Python
3.6. Tracked it down to the comparison which has to be against a string of bytes
in Python3.
0.18.0 (2018-02-22)
-------------------
- Add option to fetch followers/following JSON data. [Stephen Greene]
0.17.0 (2018-02-20)
-------------------
- Short circuit gists backup process. [W. Harrison Wright]
- Formatting. [W. Harrison Wright]
- Add ability to backup gists. [W. Harrison Wright]
0.16.0 (2018-01-22)
-------------------
- Change option to --all-starred. [W. Harrison Wright]
- JK don't update documentation. [W. Harrison Wright]
- Put starred clone repoistories under a new option. [W. Harrison
Wright]
- Add comment. [W. Harrison Wright]
- Add ability to clone starred repos. [W. Harrison Wright]
0.14.1 (2017-10-11)
-------------------
- Fix arg not defined error. [Edward Pfremmer]
Ref: https://github.com/josegonzalez/python-github-backup/issues/69
0.14.0 (2017-10-11)
-------------------
- Added a check to see if git-lfs is installed when doing an LFS clone.
[pieterclaerhout]
- Added support for LFS clones. [pieterclaerhout]
- Add pypi info to readme. [Albert Wang]
- Explicitly support python 3 in package description. [Albert Wang]
- Add couple examples to help new users. [Yusuf Tran]
0.13.2 (2017-05-06)
-------------------
- Fix remotes while updating repository. [Dima Gerasimov]
0.13.1 (2017-04-11)
-------------------
- Fix error when repository has no updated_at value. [Nicolai Ehemann]
0.13.0 (2017-04-05)
-------------------
- Add OS check for OSX specific keychain args. [Martin O'Reilly]
Keychain arguments are only supported on Mac OSX.
Added check for operating system so we give a
"Keychain arguments are only supported on Mac OSX"
error message rather than a "No password item matching the
provided name and account could be found in the osx keychain"
error message
- Add support for storing PAT in OSX keychain. [Martin O'Reilly]
Added additional optional arguments and README guidance for storing
and accessing a Github personal access token (PAT) in the OSX
keychain
0.12.1 (2017-03-27)
-------------------
- Avoid remote branch name churn. [Chris Adams]
This avoids the backup output having lots of "[new branch]" messages
because removing the old remote name removed all of the existing branch
references.
- Fix detection of bare git directories. [Andrzej Maczuga]
0.12.0 (2016-11-22)
-------------------
Fix
~~~
- Properly import version from github_backup package. [Jose Diaz-
Gonzalez]
- Support alternate git status output. [Jose Diaz-Gonzalez]
Other
~~~~~
- Pep8: E501 line too long (83 > 79 characters) [Jose Diaz-Gonzalez]
- Pep8: E128 continuation line under-indented for visual indent. [Jose
Diaz-Gonzalez]
- Support archivization using bare git clones. [Andrzej Maczuga]
- Fix typo, 3x. [Terrell Russell]
0.11.0 (2016-10-26)
-------------------
- Support --token file:///home/user/token.txt (fixes gh-51) [Björn
Dahlgren]
- Fix some linting. [Albert Wang]
- Fix byte/string conversion for python 3. [Albert Wang]
- Support python 3. [Albert Wang]
- Encode special characters in password. [Remi Rampin]
- Don't pretend program name is "Github Backup" [Remi Rampin]
- Don't install over insecure connection. [Remi Rampin]
The git:// protocol is unauthenticated and unencrypted, and no longer advertised by GitHub. Using HTTPS shouldn't impact performance.
0.10.3 (2016-08-21)
-------------------
- Fixes #29. [Jonas Michel]
Reporting an error when the user's rate limit is exceeded causes
the script to terminate after resuming execution from a rate limit
sleep. Instead of generating an explicit error we just want to
inform the user that the script is going to sleep until their rate
limit count resets.
- Fixes #29. [Jonas Michel]
The errors list was not being cleared out after resuming a backup
from a rate limit sleep. When the backup was resumed, the non-empty
errors list caused the backup to quit after the next `retrieve_data`
request.
0.10.2 (2016-08-21)
-------------------
- Add a note regarding git version requirement. [Jose Diaz-Gonzalez]
Closes #37
0.10.0 (2016-08-18)
-------------------
- Implement incremental updates. [Robert Bradshaw]
Guarded with an --incremental flag.
Stores the time of the last update and only downloads issue and
pull request data since this time. All other data is relatively
small (likely fetched with a single request) and so is simply
re-populated from scratch as before.
0.9.0 (2016-03-29)
------------------
- Fix cloning private repos with basic auth or token. [Kazuki Suda]
0.8.0 (2016-02-14)
------------------
- Don't store issues which are actually pull requests. [Enrico Tröger]
This prevents storing pull requests twice since the Github API returns
pull requests also as issues. Those issues will be skipped but only if
retrieving pull requests is requested as well.
Closes #23.
0.7.0 (2016-02-02)
------------------
- Softly fail if not able to read hooks. [Albert Wang]
- Add note about 2-factor auth. [Albert Wang]
- Make user repository search go through endpoint capable of reading
private repositories. [Albert Wang]
- Prompt for password if only username given. [Alex Hall]
0.6.0 (2015-11-10)
------------------
- Force proper remote url. [Jose Diaz-Gonzalez]
- Improve error handling in case of HTTP errors. [Enrico Tröger]
In case of a HTTP status code 404, the returned 'r' was never assigned.
In case of URL errors which are not timeouts, we probably should bail
out.
- Add --hooks to also include web hooks into the backup. [Enrico Tröger]
- Create the user specified output directory if it does not exist.
[Enrico Tröger]
Fixes #17.
- Add missing auth argument to _get_response() [Enrico Tröger]
When running unauthenticated and Github starts rate-limiting the client,
github-backup crashes because the used auth variable in _get_response()
was not available. This change should fix it.
- Add repository URL to error message for non-existing repositories.
[Enrico Tröger]
This makes it easier for the user to identify which repository does not
exist or is not initialised, i.e. whether it is the main repository or
the wiki repository and which clone URL was used to check.
0.5.0 (2015-10-10)
------------------
- Add release script. [Jose Diaz-Gonzalez]
- Refactor to both simplify codepath as well as follow PEP8 standards.
[Jose Diaz-Gonzalez]
- Retry 3 times when the connection times out. [Mathijs Jonker]
- Made unicode output defalut. [Kirill Grushetsky]
- Import alphabetised. [Kirill Grushetsky]
- Preserve Unicode characters in the output file. [Kirill Grushetsky]
Added option to preserve Unicode characters in the output file
- Josegonzales/python-github-backup#12 Added backup of labels and
milestones. [aensley]
- Fixed indent. [Mathijs Jonker]
- Skip unitialized repo's. [mjonker-embed]
These gave me errors which caused mails from crontab.
- Added prefer-ssh. [mjonker-embed]
Was needed for my back-up setup, code includes this but readme wasn't updated
- Retry API requests which failed due to rate-limiting. [Chris Adams]
This allows operation to continue, albeit at a slower pace,
if you have enough data to trigger the API rate limits
- Logging_subprocess: always log when a command fails. [Chris Adams]
Previously git clones could fail without any indication
unless you edited the source to change `logger=None` to use
a configured logger.
Now a non-zero return code will always output a message to
stderr and will display the executed command so it can be
rerun for troubleshooting.
- Switch to using ssh_url. [Chris Adams]
The previous commit used the wrong URL for a private repo. This was
masked by the lack of error loging in logging_subprocess (which will be
in a separate branch)
- Add an option to prefer checkouts over SSH. [Chris Adams]
This is really useful with private repos to avoid being nagged
for credentials for every repository
- Add pull request support. [Kevin Laude]
Back up reporitory pull requests by passing the --include-pulls
argument. Pull requests are saved to
repositories/<repository name>/pulls/<pull request number>.json. Include
the --pull-request-comments argument to add review comments to the pull
request backup and pass the --pull-request-commits argument to add
commits to the pull request backup.
Pull requests are automatically backed up when the --all argument is
uesd.
- Add GitHub Enterprise support. [Kevin Laude]
Pass the -H or --github-host argument with a GitHub Enterprise hostname
to backup from that GitHub enterprise host. If no argument is passed
then back up from github.com.
0.2.0 (2014-09-22)
------------------
- Add support for retrieving repositories. Closes #1. [Jose Diaz-
Gonzalez]
- Fix PEP8 violations. [Jose Diaz-Gonzalez]
- Add authorization to header only if specified by user. [Ioannis
Filippidis]
- Fill out readme more. [Jose Diaz-Gonzalez]
- Fix import. [Jose Diaz-Gonzalez]
- Properly name readme. [Jose Diaz-Gonzalez]
- Create MANIFEST.in. [Jose Diaz-Gonzalez]
- Create .gitignore. [Jose Diaz-Gonzalez]
- Create setup.py. [Jose Diaz-Gonzalez]
- Create requirements.txt. [Jose Diaz-Gonzalez]
- Create __init__.py. [Jose Diaz-Gonzalez]
- Create LICENSE.txt. [Jose Diaz-Gonzalez]
- Create README.md. [Jose Diaz-Gonzalez]
- Create github-backup. [Jose Diaz-Gonzalez]

View File

@@ -2,8 +2,15 @@
github-backup
=============
|PyPI| |Python Versions|
backup a github user or organization
Requirements
============
- GIT 1.9+
Installation
============
@@ -13,22 +20,28 @@ Using PIP via PyPI::
Using PIP via Github::
pip install git+git://github.com/josegonzalez/python-github-backup.git#egg=github-backup
pip install git+https://github.com/josegonzalez/python-github-backup.git#egg=github-backup
Usage
=====
CLI Usage is as follows::
Github Backup [-h] [-u USERNAME] [-p PASSWORD] [-t TOKEN]
[-o OUTPUT_DIRECTORY] [--starred] [--watched] [--all]
[--issues] [--issue-comments] [--issue-events]
[--repositories] [--wikis] [--skip-existing]
[-L [LANGUAGES [LANGUAGES ...]]] [-N NAME_REGEX] [-O]
[-R REPOSITORY] [-P] [-F] [-v]
USER
github-backup [-h] [-u USERNAME] [-p PASSWORD] [-t TOKEN]
[-o OUTPUT_DIRECTORY] [-i] [--starred] [--all-starred]
[--watched] [--followers] [--following] [--all]
[--issues] [--issue-comments] [--issue-events] [--pulls]
[--pull-comments] [--pull-commits] [--labels] [--hooks]
[--milestones] [--repositories] [--bare] [--lfs]
[--wikis] [--gists] [--starred-gists] [--skip-existing]
[-L [LANGUAGES [LANGUAGES ...]]] [-N NAME_REGEX]
[-H GITHUB_HOST] [-O] [-R REPOSITORY] [-P] [-F]
[--prefer-ssh] [-v]
[--keychain-name OSX_KEYCHAIN_ITEM_NAME]
[--keychain-account OSX_KEYCHAIN_ITEM_ACCOUNT]
USER
Backup a github users account
Backup a github account
positional arguments:
USER github username
@@ -38,29 +51,105 @@ CLI Usage is as follows::
-u USERNAME, --username USERNAME
username for basic auth
-p PASSWORD, --password PASSWORD
password for basic auth
password for basic auth. If a username is given but
not a password, the password will be prompted for.
-t TOKEN, --token TOKEN
personal access or OAuth token
personal access or OAuth token, or path to token
(file://...)
-o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
directory at which to backup the repositories
--starred include starred repositories in backup
-i, --incremental incremental backup
--starred include JSON output of starred repositories in backup
--all-starred include starred repositories in backup
--watched include watched repositories in backup
--followers include JSON output of followers in backup
--following include JSON output of following users in backup
--all include everything in backup
--issues include issues in backup
--issue-comments include issue comments in backup
--issue-events include issue events in backup
--pulls include pull requests in backup
--pull-comments include pull request review comments in backup
--pull-commits include pull request commits in backup
--labels include labels in backup
--hooks include hooks in backup (works only when
authenticated)
--milestones include milestones in backup
--repositories include repository clone in backup
--bare clone bare repositories
--lfs clone LFS repositories (requires Git LFS to be
installed, https://git-lfs.github.com)
--wikis include wiki clone in backup
--gists include gists in backup
--starred-gists include starred gists in backup
--skip-existing skip project if a backup directory exists
-L [LANGUAGES [LANGUAGES ...]], --languages [LANGUAGES [LANGUAGES ...]]
only allow these languages
-N NAME_REGEX, --name-regex NAME_REGEX
python regex to match names against
-O, --organization whether or not this is a query for an organization
-H GITHUB_HOST, --github-host GITHUB_HOST
GitHub Enterprise hostname
-O, --organization whether or not this is an organization user
-R REPOSITORY, --repository REPOSITORY
name of repository to limit backup to
-P, --private include private repositories
-F, --fork include forked repositories
--prefer-ssh Clone repositories using SSH instead of HTTPS
-v, --version show program's version number and exit
--keychain-name OSX_KEYCHAIN_ITEM_NAME
OSX ONLY: name field of password item in OSX keychain
that holds the personal access or OAuth token
--keychain-account OSX_KEYCHAIN_ITEM_ACCOUNT
OSX ONLY: account field of password item in OSX
keychain that holds the personal access or OAuth token
The package can be used to backup an *entire* organization or repository, including issues and wikis in the most appropriate format (clones for wikis, json files for issues).
Authentication
==============
Note: Password-based authentication will fail if you have two-factor authentication enabled.
Using the Keychain on Mac OSX
=============================
Note: On Mac OSX the token can be stored securely in the user's keychain. To do this:
1. Open Keychain from "Applications -> Utilities -> Keychain Access"
2. Add a new password item using "File -> New Password Item"
3. Enter a name in the "Keychain Item Name" box. You must provide this name to github-backup using the --keychain-name argument.
4. Enter an account name in the "Account Name" box, enter your Github username as set above. You must provide this name to github-backup using the --keychain-account argument.
5. Enter your Github personal access token in the "Password" box
Note: When you run github-backup, you will be asked whether you want to allow "security" to use your confidential information stored in your keychain. You have two options:
1. **Allow:** In this case you will need to click "Allow" each time you run `github-backup`
2. **Always Allow:** In this case, you will not be asked for permission when you run `github-backup` in future. This is less secure, but is required if you want to schedule `github-backup` to run automatically
About Git LFS
=============
When you use the "--lfs" option, you will need to make sure you have Git LFS installed.
Instructions on how to do this can be found on https://git-lfs.github.com.
Examples
========
Backup all repositories::
export ACCESS_TOKEN=SOME-GITHUB-TOKEN
github-backup WhiteHouse --token $ACCESS_TOKEN --organization --output-directory /tmp/white-house --repositories
Backup a single organization repository with everything else (wiki, pull requests, comments, issues etc)::
export ACCESS_TOKEN=SOME-GITHUB-TOKEN
ORGANIZATION=docker
REPO=cli
# e.g. git@github.com:docker/cli.git
github-backup $ORGANIZATION -P -t $ACCESS_TOKEN -o . --all -O -R $REPO
.. |PyPI| image:: https://img.shields.io/pypi/v/github-backup.svg
:target: https://pypi.python.org/pypi/github-backup/
.. |Python Versions| image:: https://img.shields.io/pypi/pyversions/github-backup.svg
:target: https://github.com/albertyw/github-backup

952
bin/github-backup Normal file → Executable file

File diff suppressed because it is too large Load Diff

View File

@@ -1 +1 @@
__version__ = '0.2.0'
__version__ = '0.19.2'

127
release Executable file
View File

@@ -0,0 +1,127 @@
#!/usr/bin/env bash
set -eo pipefail; [[ $RELEASE_TRACE ]] && set -x
PACKAGE_NAME='github-backup'
INIT_PACKAGE_NAME='github_backup'
PUBLIC="true"
# Colors
COLOR_OFF="\033[0m" # unsets color to term fg color
RED="\033[0;31m" # red
GREEN="\033[0;32m" # green
YELLOW="\033[0;33m" # yellow
MAGENTA="\033[0;35m" # magenta
CYAN="\033[0;36m" # cyan
# ensure wheel is available
pip install wheel > /dev/null
command -v gitchangelog >/dev/null 2>&1 || {
echo -e "${RED}WARNING: Missing gitchangelog binary, please run: pip install gitchangelog==2.2.0${COLOR_OFF}\n"
exit 1
}
command -v rst-lint > /dev/null || {
echo -e "${RED}WARNING: Missing rst-lint binary, please run: pip install restructuredtext_lint${COLOR_OFF}\n"
exit 1
}
if [[ "$@" != "major" ]] && [[ "$@" != "minor" ]] && [[ "$@" != "patch" ]]; then
echo -e "${RED}WARNING: Invalid release type, must specify 'major', 'minor', or 'patch'${COLOR_OFF}\n"
exit 1
fi
echo -e "\n${GREEN}STARTING RELEASE PROCESS${COLOR_OFF}\n"
set +e;
git status | grep -Eo "working (directory|tree) clean" &> /dev/null
if [ ! $? -eq 0 ]; then # working directory is NOT clean
echo -e "${RED}WARNING: You have uncomitted changes, you may have forgotten something${COLOR_OFF}\n"
exit 1
fi
set -e;
echo -e "${YELLOW}--->${COLOR_OFF} Updating local copy"
git pull -q origin master
echo -e "${YELLOW}--->${COLOR_OFF} Retrieving release versions"
current_version=$(cat ${INIT_PACKAGE_NAME}/__init__.py |grep '__version__ ='|sed 's/[^0-9.]//g')
major=$(echo $current_version | awk '{split($0,a,"."); print a[1]}')
minor=$(echo $current_version | awk '{split($0,a,"."); print a[2]}')
patch=$(echo $current_version | awk '{split($0,a,"."); print a[3]}')
if [[ "$@" == "major" ]]; then
major=$(($major + 1));
minor="0"
patch="0"
elif [[ "$@" == "minor" ]]; then
minor=$(($minor + 1));
patch="0"
elif [[ "$@" == "patch" ]]; then
patch=$(($patch + 1));
fi
next_version="${major}.${minor}.${patch}"
echo -e "${YELLOW} >${COLOR_OFF} ${MAGENTA}${current_version}${COLOR_OFF} -> ${MAGENTA}${next_version}${COLOR_OFF}"
echo -e "${YELLOW}--->${COLOR_OFF} Ensuring readme passes lint checks (if this fails, run rst-lint)"
rst-lint README.rst > /dev/null
echo -e "${YELLOW}--->${COLOR_OFF} Creating necessary temp file"
tempfoo=$(basename $0)
TMPFILE=$(mktemp /tmp/${tempfoo}.XXXXXX) || {
echo -e "${RED}WARNING: Cannot create temp file using mktemp in /tmp dir ${COLOR_OFF}\n"
exit 1
}
find_this="__version__ = '$current_version'"
replace_with="__version__ = '$next_version'"
echo -e "${YELLOW}--->${COLOR_OFF} Updating ${INIT_PACKAGE_NAME}/__init__.py"
sed "s/$find_this/$replace_with/" ${INIT_PACKAGE_NAME}/__init__.py > $TMPFILE && mv $TMPFILE ${INIT_PACKAGE_NAME}/__init__.py
find_this="${PACKAGE_NAME}.git@$current_version"
replace_with="${PACKAGE_NAME}.git@$next_version"
echo -e "${YELLOW}--->${COLOR_OFF} Updating README.rst"
sed "s/$find_this/$replace_with/" README.rst > $TMPFILE && mv $TMPFILE README.rst
if [ -f docs/conf.py ]; then
echo -e "${YELLOW}--->${COLOR_OFF} Updating docs"
find_this="version = '${current_version}'"
replace_with="version = '${next_version}'"
sed "s/$find_this/$replace_with/" docs/conf.py > $TMPFILE && mv $TMPFILE docs/conf.py
find_this="version = '${current_version}'"
replace_with="release = '${next_version}'"
sed "s/$find_this/$replace_with/" docs/conf.py > $TMPFILE && mv $TMPFILE docs/conf.py
fi
echo -e "${YELLOW}--->${COLOR_OFF} Updating CHANGES.rst for new release"
version_header="$next_version ($(date +%F))"
set +e; dashes=$(yes '-'|head -n ${#version_header}|tr -d '\n') ; set -e
gitchangelog |sed "4s/.*/$version_header/"|sed "5s/.*/$dashes/" > $TMPFILE && mv $TMPFILE CHANGES.rst
echo -e "${YELLOW}--->${COLOR_OFF} Adding changed files to git"
git add CHANGES.rst README.rst ${INIT_PACKAGE_NAME}/__init__.py
if [ -f docs/conf.py ]; then git add docs/conf.py; fi
echo -e "${YELLOW}--->${COLOR_OFF} Creating release"
git commit -q -m "Release version $next_version"
echo -e "${YELLOW}--->${COLOR_OFF} Tagging release"
git tag -a $next_version -m "Release version $next_version"
echo -e "${YELLOW}--->${COLOR_OFF} Pushing release and tags to github"
git push -q origin master && git push -q --tags
if [[ "$PUBLIC" == "true" ]]; then
echo -e "${YELLOW}--->${COLOR_OFF} Creating python release"
cp README.rst README
python setup.py sdist bdist_wheel upload > /dev/null
rm README
fi
echo -e "\n${CYAN}RELEASED VERSION ${next_version}!${COLOR_OFF}\n"

View File

@@ -39,6 +39,8 @@ setup(
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
],
description='backup a github user or organization',
long_description=open_file('README.rst').read(),