Compare commits

..

11 Commits

Author SHA1 Message Date
GitHub Action
2bb83d6d8b Release version 0.56.0 2025-12-11 16:50:28 +00:00
Jose Diaz-Gonzalez
8fcc142621 Merge pull request #465 from Iamrodos/fix/379-lfs-clone-deprecated
fix: replace deprecated git lfs clone with git clone + git lfs fetch --all
2025-12-11 11:49:53 -05:00
Jose Diaz-Gonzalez
7615ce6102 Merge pull request #464 from Iamrodos/fix/246-restore-docs
docs: clarify no inbuilt restore and GitHub API limitations
2025-12-11 11:49:39 -05:00
Jose Diaz-Gonzalez
3f1ef821c3 Merge pull request #466 from Iamrodos/fix/112-windows-support
fix: add Windows support with entry_points and os.replace
2025-12-11 11:48:59 -05:00
Rodos
3684756eaa fix: add Windows support with entry_points and os.replace
- Replace os.rename() with os.replace() for atomic file operations
  on Windows (os.rename fails if destination exists on Windows)
- Add entry_points console_scripts for proper .exe generation on Windows
- Create github_backup/cli.py with main() entry point
- Add github_backup/__main__.py for python -m github_backup support
- Keep bin/github-backup as thin wrapper for backwards compatibility

Closes #112
2025-12-11 22:03:45 +11:00
Rodos
e745b55755 fix: replace deprecated git lfs clone with git clone + git lfs fetch --all
git lfs clone is deprecated - modern git clone handles LFS automatically.
Using git lfs fetch --all ensures all LFS objects across all refs are
backed up, matching the existing bare clone behavior and providing
complete LFS backups.

Closes #379
2025-12-11 20:55:38 +11:00
Rodos
75e6f56773 docs: add "Restoring from Backup" section to README
Clarifies that this tool is backup-only with no inbuilt restore.
Documents that git repos can be pushed back, but issues/PRs have
GitHub API limitations affecting all backup tools.

Closes #246
2025-12-11 20:35:08 +11:00
Jose Diaz-Gonzalez
b991c363a0 Merge pull request #463 from josegonzalez/dependabot/pip/python-packages-9e0978b55f
chore(deps): bump urllib3 from 2.6.0 to 2.6.1 in the python-packages group
2025-12-10 09:39:07 -05:00
dependabot[bot]
6d74af9126 chore(deps): bump urllib3 in the python-packages group
Bumps the python-packages group with 1 update: [urllib3](https://github.com/urllib3/urllib3).


Updates `urllib3` from 2.6.0 to 2.6.1
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.6.0...2.6.1)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-09 13:10:12 +00:00
Jose Diaz-Gonzalez
381d67af96 Merge pull request #462 from josegonzalez/dependabot/pip/python-packages-3a01b12ef5
chore(deps): bump the python-packages group with 3 updates
2025-12-08 16:00:24 -05:00
dependabot[bot]
2fbe8d272c chore(deps): bump the python-packages group with 3 updates
Bumps the python-packages group with 3 updates: [black](https://github.com/psf/black), [pytest](https://github.com/pytest-dev/pytest) and [platformdirs](https://github.com/tox-dev/platformdirs).


Updates `black` from 25.11.0 to 25.12.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.11.0...25.12.0)

Updates `pytest` from 9.0.1 to 9.0.2
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.1...9.0.2)

Updates `platformdirs` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/tox-dev/platformdirs/releases)
- [Changelog](https://github.com/tox-dev/platformdirs/blob/main/CHANGES.rst)
- [Commits](https://github.com/tox-dev/platformdirs/compare/4.5.0...4.5.1)

---
updated-dependencies:
- dependency-name: black
  dependency-version: 25.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-packages
- dependency-name: pytest
  dependency-version: 9.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-packages
- dependency-name: platformdirs
  dependency-version: 4.5.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-08 13:09:32 +00:00
9 changed files with 244 additions and 86 deletions

View File

@@ -1,10 +1,104 @@
Changelog Changelog
========= =========
0.55.0 (2025-12-07) 0.56.0 (2025-12-11)
------------------- -------------------
------------------------ ------------------------
Fix
~~~
- Replace deprecated git lfs clone with git clone + git lfs fetch --all.
[Rodos]
git lfs clone is deprecated - modern git clone handles LFS automatically.
Using git lfs fetch --all ensures all LFS objects across all refs are
backed up, matching the existing bare clone behavior and providing
complete LFS backups.
Closes #379
- Add Windows support with entry_points and os.replace. [Rodos]
- Replace os.rename() with os.replace() for atomic file operations
on Windows (os.rename fails if destination exists on Windows)
- Add entry_points console_scripts for proper .exe generation on Windows
- Create github_backup/cli.py with main() entry point
- Add github_backup/__main__.py for python -m github_backup support
- Keep bin/github-backup as thin wrapper for backwards compatibility
Closes #112
Other
~~~~~
- Docs: add "Restoring from Backup" section to README. [Rodos]
Clarifies that this tool is backup-only with no inbuilt restore.
Documents that git repos can be pushed back, but issues/PRs have
GitHub API limitations affecting all backup tools.
Closes #246
- Chore(deps): bump urllib3 in the python-packages group.
[dependabot[bot]]
Bumps the python-packages group with 1 update: [urllib3](https://github.com/urllib3/urllib3).
Updates `urllib3` from 2.6.0 to 2.6.1
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.6.0...2.6.1)
---
updated-dependencies:
- dependency-name: urllib3
dependency-version: 2.6.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: python-packages
...
- Chore(deps): bump the python-packages group with 3 updates.
[dependabot[bot]]
Bumps the python-packages group with 3 updates: [black](https://github.com/psf/black), [pytest](https://github.com/pytest-dev/pytest) and [platformdirs](https://github.com/tox-dev/platformdirs).
Updates `black` from 25.11.0 to 25.12.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.11.0...25.12.0)
Updates `pytest` from 9.0.1 to 9.0.2
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.1...9.0.2)
Updates `platformdirs` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/tox-dev/platformdirs/releases)
- [Changelog](https://github.com/tox-dev/platformdirs/blob/main/CHANGES.rst)
- [Commits](https://github.com/tox-dev/platformdirs/compare/4.5.0...4.5.1)
---
updated-dependencies:
- dependency-name: black
dependency-version: 25.12.0
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: python-packages
- dependency-name: pytest
dependency-version: 9.0.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: python-packages
- dependency-name: platformdirs
dependency-version: 4.5.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: python-packages
...
0.55.0 (2025-12-07)
-------------------
Fix Fix
~~~ ~~~
- Improve error messages for inaccessible repos and empty wikis. [Rodos] - Improve error messages for inaccessible repos and empty wikis. [Rodos]

View File

@@ -215,6 +215,8 @@ When you use the ``--lfs`` option, you will need to make sure you have Git LFS i
Instructions on how to do this can be found on https://git-lfs.github.com. Instructions on how to do this can be found on https://git-lfs.github.com.
LFS objects are fetched for all refs, not just the current checkout, ensuring a complete backup of all LFS content across all branches and history.
About Attachments About Attachments
----------------- -----------------
@@ -360,6 +362,25 @@ Debug an error/block or incomplete backup into a temporary directory. Omit "incr
github-backup -f $FINE_ACCESS_TOKEN -o /tmp/github-backup/ -l debug -P --all-starred --starred --watched --followers --following --issues --issue-comments --issue-events --pulls --pull-comments --pull-commits --labels --milestones --repositories --wikis --releases --assets --pull-details --gists --starred-gists $GH_USER github-backup -f $FINE_ACCESS_TOKEN -o /tmp/github-backup/ -l debug -P --all-starred --starred --watched --followers --following --issues --issue-comments --issue-events --pulls --pull-comments --pull-commits --labels --milestones --repositories --wikis --releases --assets --pull-details --gists --starred-gists $GH_USER
Restoring from Backup
=====================
This tool creates backups only, there is no inbuilt restore command.
**Git repositories, wikis, and gists** can be restored by pushing them back to GitHub as you would any git repository. For example, to restore a bare repository backup::
cd /tmp/white-house/repositories/petitions/repository
git push --mirror git@github.com:WhiteHouse/petitions.git
**Issues, pull requests, comments, and other metadata** are saved as JSON files for archival purposes. The GitHub API does not support recreating this data faithfully, creating issues via the API has limitations:
- New issue/PR numbers are assigned (original numbers cannot be set)
- Timestamps reflect creation time (original dates cannot be set)
- The API caller becomes the author (original authors cannot be set)
- Cross-references between issues and PRs will break
These are GitHub API limitations that affect all backup and migration tools, not just this one. Recreating issues with these limitations via the GitHub API is an exercise for the reader. The JSON backups remain useful for searching, auditing, or manual reference.
Development Development
=========== ===========

View File

@@ -1,76 +1,18 @@
#!/usr/bin/env python #!/usr/bin/env python
"""
Backwards-compatible wrapper script.
The recommended way to run github-backup is via the installed command
(pip install github-backup) or python -m github_backup.
This script is kept for backwards compatibility with existing installations
that may reference this path directly.
"""
import logging
import os
import sys import sys
from github_backup.github_backup import ( from github_backup.cli import main
backup_account, from github_backup.github_backup import logger
backup_repositories,
check_git_lfs_install,
filter_repositories,
get_auth,
get_authenticated_user,
logger,
mkdir_p,
parse_args,
retrieve_repositories,
)
# INFO and DEBUG go to stdout, WARNING and above go to stderr
log_format = logging.Formatter(
fmt="%(asctime)s.%(msecs)03d: %(message)s",
datefmt="%Y-%m-%dT%H:%M:%S",
)
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(logging.DEBUG)
stdout_handler.addFilter(lambda r: r.levelno < logging.WARNING)
stdout_handler.setFormatter(log_format)
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setLevel(logging.WARNING)
stderr_handler.setFormatter(log_format)
logging.basicConfig(level=logging.INFO, handlers=[stdout_handler, stderr_handler])
def main():
args = parse_args()
if args.private and not get_auth(args):
logger.warning(
"The --private flag has no effect without authentication. "
"Use -t/--token, -f/--token-fine, or -u/--username to authenticate."
)
if args.quiet:
logger.setLevel(logging.WARNING)
output_directory = os.path.realpath(args.output_directory)
if not os.path.isdir(output_directory):
logger.info("Create output directory {0}".format(output_directory))
mkdir_p(output_directory)
if args.lfs_clone:
check_git_lfs_install()
if args.log_level:
log_level = logging.getLevelName(args.log_level.upper())
if isinstance(log_level, int):
logger.root.setLevel(log_level)
if not args.as_app:
logger.info("Backing up user {0} to {1}".format(args.user, output_directory))
authenticated_user = get_authenticated_user(args)
else:
authenticated_user = {"login": None}
repositories = retrieve_repositories(args, authenticated_user)
repositories = filter_repositories(args, repositories)
backup_repositories(args, output_directory, repositories)
backup_account(args, output_directory)
if __name__ == "__main__": if __name__ == "__main__":
try: try:

View File

@@ -1 +1 @@
__version__ = "0.55.0" __version__ = "0.56.0"

13
github_backup/__main__.py Normal file
View File

@@ -0,0 +1,13 @@
"""Allow running as: python -m github_backup"""
import sys
from github_backup.cli import main
from github_backup.github_backup import logger
if __name__ == "__main__":
try:
main()
except Exception as e:
logger.error(str(e))
sys.exit(1)

82
github_backup/cli.py Normal file
View File

@@ -0,0 +1,82 @@
#!/usr/bin/env python
"""Command-line interface for github-backup."""
import logging
import os
import sys
from github_backup.github_backup import (
backup_account,
backup_repositories,
check_git_lfs_install,
filter_repositories,
get_auth,
get_authenticated_user,
logger,
mkdir_p,
parse_args,
retrieve_repositories,
)
# INFO and DEBUG go to stdout, WARNING and above go to stderr
log_format = logging.Formatter(
fmt="%(asctime)s.%(msecs)03d: %(message)s",
datefmt="%Y-%m-%dT%H:%M:%S",
)
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(logging.DEBUG)
stdout_handler.addFilter(lambda r: r.levelno < logging.WARNING)
stdout_handler.setFormatter(log_format)
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setLevel(logging.WARNING)
stderr_handler.setFormatter(log_format)
logging.basicConfig(level=logging.INFO, handlers=[stdout_handler, stderr_handler])
def main():
"""Main entry point for github-backup CLI."""
args = parse_args()
if args.private and not get_auth(args):
logger.warning(
"The --private flag has no effect without authentication. "
"Use -t/--token, -f/--token-fine, or -u/--username to authenticate."
)
if args.quiet:
logger.setLevel(logging.WARNING)
output_directory = os.path.realpath(args.output_directory)
if not os.path.isdir(output_directory):
logger.info("Create output directory {0}".format(output_directory))
mkdir_p(output_directory)
if args.lfs_clone:
check_git_lfs_install()
if args.log_level:
log_level = logging.getLevelName(args.log_level.upper())
if isinstance(log_level, int):
logger.root.setLevel(log_level)
if not args.as_app:
logger.info("Backing up user {0} to {1}".format(args.user, output_directory))
authenticated_user = get_authenticated_user(args)
else:
authenticated_user = {"login": None}
repositories = retrieve_repositories(args, authenticated_user)
repositories = filter_repositories(args, repositories)
backup_repositories(args, output_directory, repositories)
backup_account(args, output_directory)
if __name__ == "__main__":
try:
main()
except Exception as e:
logger.error(str(e))
sys.exit(1)

View File

@@ -1038,7 +1038,7 @@ def download_attachment_file(url, path, auth, as_app=False, fine=False):
bytes_downloaded += len(chunk) bytes_downloaded += len(chunk)
# Atomic rename to final location # Atomic rename to final location
os.rename(temp_path, path) os.replace(temp_path, path)
metadata["size_bytes"] = bytes_downloaded metadata["size_bytes"] = bytes_downloaded
metadata["success"] = True metadata["success"] = True
@@ -1459,7 +1459,7 @@ def download_attachments(
# Rename to add extension (already atomic from download) # Rename to add extension (already atomic from download)
try: try:
os.rename(filepath, final_filepath) os.replace(filepath, final_filepath)
metadata["saved_as"] = os.path.basename(final_filepath) metadata["saved_as"] = os.path.basename(final_filepath)
except Exception as e: except Exception as e:
logger.warning( logger.warning(
@@ -1490,7 +1490,7 @@ def download_attachments(
manifest_path = os.path.join(attachments_dir, "manifest.json") manifest_path = os.path.join(attachments_dir, "manifest.json")
with open(manifest_path + ".temp", "w") as f: with open(manifest_path + ".temp", "w") as f:
json.dump(manifest, f, indent=2) json.dump(manifest, f, indent=2)
os.rename(manifest_path + ".temp", manifest_path) # Atomic write os.replace(manifest_path + ".temp", manifest_path) # Atomic write
logger.debug( logger.debug(
"Wrote manifest for {0} #{1}: {2} attachments".format( "Wrote manifest for {0} #{1}: {2} attachments".format(
item_type_display, number, len(attachment_metadata_list) item_type_display, number, len(attachment_metadata_list)
@@ -1811,7 +1811,7 @@ def backup_issues(args, repo_cwd, repository, repos_template):
with codecs.open(issue_file + ".temp", "w", encoding="utf-8") as f: with codecs.open(issue_file + ".temp", "w", encoding="utf-8") as f:
json_dump(issue, f) json_dump(issue, f)
os.rename(issue_file + ".temp", issue_file) # Unlike json_dump, this is atomic os.replace(issue_file + ".temp", issue_file) # Atomic write
def backup_pulls(args, repo_cwd, repository, repos_template): def backup_pulls(args, repo_cwd, repository, repos_template):
@@ -1886,7 +1886,7 @@ def backup_pulls(args, repo_cwd, repository, repos_template):
with codecs.open(pull_file + ".temp", "w", encoding="utf-8") as f: with codecs.open(pull_file + ".temp", "w", encoding="utf-8") as f:
json_dump(pull, f) json_dump(pull, f)
os.rename(pull_file + ".temp", pull_file) # Unlike json_dump, this is atomic os.replace(pull_file + ".temp", pull_file) # Atomic write
def backup_milestones(args, repo_cwd, repository, repos_template): def backup_milestones(args, repo_cwd, repository, repos_template):
@@ -2090,11 +2090,13 @@ def fetch_repository(
git_command.pop() git_command.pop()
logging_subprocess(git_command, cwd=local_dir) logging_subprocess(git_command, cwd=local_dir)
else: else:
if lfs_clone: git_command = ["git", "clone", remote_url, local_dir]
git_command = ["git", "lfs", "clone", remote_url, local_dir]
else:
git_command = ["git", "clone", remote_url, local_dir]
logging_subprocess(git_command) logging_subprocess(git_command)
if lfs_clone:
git_command = ["git", "lfs", "fetch", "--all", "--prune"]
if no_prune:
git_command.pop()
logging_subprocess(git_command, cwd=local_dir)
def backup_account(args, output_directory): def backup_account(args, output_directory):
@@ -2203,5 +2205,5 @@ def json_dump_if_changed(data, output_file_path):
temp_file = output_file_path + ".temp" temp_file = output_file_path + ".temp"
with codecs.open(temp_file, "w", encoding="utf-8") as f: with codecs.open(temp_file, "w", encoding="utf-8") as f:
f.write(new_content) f.write(new_content)
os.rename(temp_file, output_file_path) # Atomic on POSIX systems os.replace(temp_file, output_file_path) # Atomic write
return True return True

View File

@@ -1,5 +1,5 @@
autopep8==2.3.2 autopep8==2.3.2
black==25.11.0 black==25.12.0
bleach==6.3.0 bleach==6.3.0
certifi==2025.11.12 certifi==2025.11.12
charset-normalizer==3.4.4 charset-normalizer==3.4.4
@@ -8,7 +8,7 @@ colorama==0.4.6
docutils==0.22.3 docutils==0.22.3
flake8==7.3.0 flake8==7.3.0
gitchangelog==3.0.4 gitchangelog==3.0.4
pytest==9.0.1 pytest==9.0.2
idna==3.11 idna==3.11
importlib-metadata==8.7.0 importlib-metadata==8.7.0
jaraco.classes==3.4.0 jaraco.classes==3.4.0
@@ -21,7 +21,7 @@ mypy-extensions==1.1.0
packaging==25.0 packaging==25.0
pathspec==0.12.1 pathspec==0.12.1
pkginfo==1.12.1.2 pkginfo==1.12.1.2
platformdirs==4.5.0 platformdirs==4.5.1
pycodestyle==2.14.0 pycodestyle==2.14.0
pyflakes==3.4.0 pyflakes==3.4.0
Pygments==2.19.2 Pygments==2.19.2
@@ -35,6 +35,6 @@ setuptools==80.9.0
six==1.17.0 six==1.17.0
tqdm==4.67.1 tqdm==4.67.1
twine==6.2.0 twine==6.2.0
urllib3==2.6.0 urllib3==2.6.1
webencodings==0.5.1 webencodings==0.5.1
zipp==3.23.0 zipp==3.23.0

View File

@@ -33,7 +33,11 @@ setup(
author="Jose Diaz-Gonzalez", author="Jose Diaz-Gonzalez",
author_email="github-backup@josediazgonzalez.com", author_email="github-backup@josediazgonzalez.com",
packages=["github_backup"], packages=["github_backup"],
scripts=["bin/github-backup"], entry_points={
"console_scripts": [
"github-backup=github_backup.cli:main",
],
},
url="http://github.com/josegonzalez/python-github-backup", url="http://github.com/josegonzalez/python-github-backup",
license="MIT", license="MIT",
classifiers=[ classifiers=[