Compare commits

..

22 Commits

Author SHA1 Message Date
GitHub Action
2bb83d6d8b Release version 0.56.0 2025-12-11 16:50:28 +00:00
Jose Diaz-Gonzalez
8fcc142621 Merge pull request #465 from Iamrodos/fix/379-lfs-clone-deprecated
fix: replace deprecated git lfs clone with git clone + git lfs fetch --all
2025-12-11 11:49:53 -05:00
Jose Diaz-Gonzalez
7615ce6102 Merge pull request #464 from Iamrodos/fix/246-restore-docs
docs: clarify no inbuilt restore and GitHub API limitations
2025-12-11 11:49:39 -05:00
Jose Diaz-Gonzalez
3f1ef821c3 Merge pull request #466 from Iamrodos/fix/112-windows-support
fix: add Windows support with entry_points and os.replace
2025-12-11 11:48:59 -05:00
Rodos
3684756eaa fix: add Windows support with entry_points and os.replace
- Replace os.rename() with os.replace() for atomic file operations
  on Windows (os.rename fails if destination exists on Windows)
- Add entry_points console_scripts for proper .exe generation on Windows
- Create github_backup/cli.py with main() entry point
- Add github_backup/__main__.py for python -m github_backup support
- Keep bin/github-backup as thin wrapper for backwards compatibility

Closes #112
2025-12-11 22:03:45 +11:00
Rodos
e745b55755 fix: replace deprecated git lfs clone with git clone + git lfs fetch --all
git lfs clone is deprecated - modern git clone handles LFS automatically.
Using git lfs fetch --all ensures all LFS objects across all refs are
backed up, matching the existing bare clone behavior and providing
complete LFS backups.

Closes #379
2025-12-11 20:55:38 +11:00
Rodos
75e6f56773 docs: add "Restoring from Backup" section to README
Clarifies that this tool is backup-only with no inbuilt restore.
Documents that git repos can be pushed back, but issues/PRs have
GitHub API limitations affecting all backup tools.

Closes #246
2025-12-11 20:35:08 +11:00
Jose Diaz-Gonzalez
b991c363a0 Merge pull request #463 from josegonzalez/dependabot/pip/python-packages-9e0978b55f
chore(deps): bump urllib3 from 2.6.0 to 2.6.1 in the python-packages group
2025-12-10 09:39:07 -05:00
dependabot[bot]
6d74af9126 chore(deps): bump urllib3 in the python-packages group
Bumps the python-packages group with 1 update: [urllib3](https://github.com/urllib3/urllib3).


Updates `urllib3` from 2.6.0 to 2.6.1
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.6.0...2.6.1)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-09 13:10:12 +00:00
Jose Diaz-Gonzalez
381d67af96 Merge pull request #462 from josegonzalez/dependabot/pip/python-packages-3a01b12ef5
chore(deps): bump the python-packages group with 3 updates
2025-12-08 16:00:24 -05:00
dependabot[bot]
2fbe8d272c chore(deps): bump the python-packages group with 3 updates
Bumps the python-packages group with 3 updates: [black](https://github.com/psf/black), [pytest](https://github.com/pytest-dev/pytest) and [platformdirs](https://github.com/tox-dev/platformdirs).


Updates `black` from 25.11.0 to 25.12.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.11.0...25.12.0)

Updates `pytest` from 9.0.1 to 9.0.2
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.1...9.0.2)

Updates `platformdirs` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/tox-dev/platformdirs/releases)
- [Changelog](https://github.com/tox-dev/platformdirs/blob/main/CHANGES.rst)
- [Commits](https://github.com/tox-dev/platformdirs/compare/4.5.0...4.5.1)

---
updated-dependencies:
- dependency-name: black
  dependency-version: 25.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-packages
- dependency-name: pytest
  dependency-version: 9.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-packages
- dependency-name: platformdirs
  dependency-version: 4.5.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-08 13:09:32 +00:00
GitHub Action
eb5779ac23 Release version 0.55.0 2025-12-07 13:59:35 +00:00
Jose Diaz-Gonzalez
5b52931ebf Merge pull request #461 from Iamrodos/fix-cli-ux-and-cleanup
fix: CLI UX improvements and cleanup
2025-12-07 08:58:59 -05:00
Rodos
1d6d474408 fix: improve error messages for inaccessible repos and empty wikis 2025-12-07 21:50:49 +11:00
Rodos
b80049e96e test: add missing test coverage for case sensitivity fix 2025-12-07 21:21:37 +11:00
Rodos
58ad1c2378 docs: fix RST formatting in Known blocking errors section 2025-12-07 21:21:26 +11:00
Rodos
6e2a7e521c fix: --all-starred now clones repos without --repositories 2025-12-07 21:21:14 +11:00
Rodos
aba048a3e9 fix: warn when --private used without authentication 2025-12-07 21:20:54 +11:00
Jose Diaz-Gonzalez
9f7c08166f Merge pull request #460 from josegonzalez/dependabot/pip/urllib3-2.6.0
chore(deps): bump urllib3 from 2.5.0 to 2.6.0
2025-12-06 22:23:09 -05:00
dependabot[bot]
fdfaaec1ba chore(deps): bump urllib3 from 2.5.0 to 2.6.0
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.5.0 to 2.6.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.5.0...2.6.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.6.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-06 04:51:42 +00:00
Jose Diaz-Gonzalez
8f9cf7ff89 Merge pull request #459 from Iamrodos/issue-93-starred-gists-warning
fix: warn and skip when --starred-gists used for different user
2025-12-03 23:07:29 -05:00
Rodos
899ab5fdc2 fix: warn and skip when --starred-gists used for different user
GitHub's API only allows retrieving starred gists for the authenticated
user. Previously, using --starred-gists when backing up a different user
would silently return no relevant data.

Now warns and skips the retrieval entirely when the target user differs
from the authenticated user. Uses case-insensitive comparison to match
GitHub's username handling.

Fixes #93
2025-12-04 10:07:43 +11:00
11 changed files with 590 additions and 104 deletions

View File

@@ -1,10 +1,143 @@
Changelog
=========
0.54.0 (2025-12-03)
0.56.0 (2025-12-11)
-------------------
------------------------
Fix
~~~
- Replace deprecated git lfs clone with git clone + git lfs fetch --all.
[Rodos]
git lfs clone is deprecated - modern git clone handles LFS automatically.
Using git lfs fetch --all ensures all LFS objects across all refs are
backed up, matching the existing bare clone behavior and providing
complete LFS backups.
Closes #379
- Add Windows support with entry_points and os.replace. [Rodos]
- Replace os.rename() with os.replace() for atomic file operations
on Windows (os.rename fails if destination exists on Windows)
- Add entry_points console_scripts for proper .exe generation on Windows
- Create github_backup/cli.py with main() entry point
- Add github_backup/__main__.py for python -m github_backup support
- Keep bin/github-backup as thin wrapper for backwards compatibility
Closes #112
Other
~~~~~
- Docs: add "Restoring from Backup" section to README. [Rodos]
Clarifies that this tool is backup-only with no inbuilt restore.
Documents that git repos can be pushed back, but issues/PRs have
GitHub API limitations affecting all backup tools.
Closes #246
- Chore(deps): bump urllib3 in the python-packages group.
[dependabot[bot]]
Bumps the python-packages group with 1 update: [urllib3](https://github.com/urllib3/urllib3).
Updates `urllib3` from 2.6.0 to 2.6.1
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.6.0...2.6.1)
---
updated-dependencies:
- dependency-name: urllib3
dependency-version: 2.6.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: python-packages
...
- Chore(deps): bump the python-packages group with 3 updates.
[dependabot[bot]]
Bumps the python-packages group with 3 updates: [black](https://github.com/psf/black), [pytest](https://github.com/pytest-dev/pytest) and [platformdirs](https://github.com/tox-dev/platformdirs).
Updates `black` from 25.11.0 to 25.12.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.11.0...25.12.0)
Updates `pytest` from 9.0.1 to 9.0.2
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.1...9.0.2)
Updates `platformdirs` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/tox-dev/platformdirs/releases)
- [Changelog](https://github.com/tox-dev/platformdirs/blob/main/CHANGES.rst)
- [Commits](https://github.com/tox-dev/platformdirs/compare/4.5.0...4.5.1)
---
updated-dependencies:
- dependency-name: black
dependency-version: 25.12.0
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: python-packages
- dependency-name: pytest
dependency-version: 9.0.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: python-packages
- dependency-name: platformdirs
dependency-version: 4.5.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: python-packages
...
0.55.0 (2025-12-07)
-------------------
Fix
~~~
- Improve error messages for inaccessible repos and empty wikis. [Rodos]
- --all-starred now clones repos without --repositories. [Rodos]
- Warn when --private used without authentication. [Rodos]
- Warn and skip when --starred-gists used for different user. [Rodos]
GitHub's API only allows retrieving starred gists for the authenticated
user. Previously, using --starred-gists when backing up a different user
would silently return no relevant data.
Now warns and skips the retrieval entirely when the target user differs
from the authenticated user. Uses case-insensitive comparison to match
GitHub's username handling.
Fixes #93
Other
~~~~~
- Test: add missing test coverage for case sensitivity fix. [Rodos]
- Docs: fix RST formatting in Known blocking errors section. [Rodos]
- Chore(deps): bump urllib3 from 2.5.0 to 2.6.0. [dependabot[bot]]
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.5.0 to 2.6.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.5.0...2.6.0)
---
updated-dependencies:
- dependency-name: urllib3
dependency-version: 2.6.0
dependency-type: direct:production
...
0.54.0 (2025-12-03)
-------------------
Fix
~~~
- Send INFO/DEBUG to stdout, WARNING/ERROR to stderr. [Rodos]

View File

@@ -215,6 +215,8 @@ When you use the ``--lfs`` option, you will need to make sure you have Git LFS i
Instructions on how to do this can be found on https://git-lfs.github.com.
LFS objects are fetched for all refs, not just the current checkout, ensuring a complete backup of all LFS content across all branches and history.
About Attachments
-----------------
@@ -281,7 +283,7 @@ If the incremental argument is used, this will result in the next backup only re
It's therefore recommended to only use the incremental argument if the output/result is being actively monitored, or complimented with periodic full non-incremental runs, to avoid unexpected missing data in a regular backup runs.
1. **Starred public repo hooks blocking**
**Starred public repo hooks blocking**
Since the ``--all`` argument includes ``--hooks``, if you use ``--all`` and ``--all-starred`` together to clone a users starred public repositories, the backup will likely error and block the backup continuing.
@@ -301,6 +303,8 @@ Starred gists vs starred repo behaviour
The starred normal repo cloning (``--all-starred``) argument stores starred repos separately to the users own repositories. However, using ``--starred-gists`` will store starred gists within the same directory as the users own gists ``--gists``. Also, all gist repo directory names are IDs not the gist's name.
Note: ``--starred-gists`` only retrieves starred gists for the authenticated user, not the target user, due to a GitHub API limitation.
Skip existing on incomplete backups
-----------------------------------
@@ -358,6 +362,25 @@ Debug an error/block or incomplete backup into a temporary directory. Omit "incr
github-backup -f $FINE_ACCESS_TOKEN -o /tmp/github-backup/ -l debug -P --all-starred --starred --watched --followers --following --issues --issue-comments --issue-events --pulls --pull-comments --pull-commits --labels --milestones --repositories --wikis --releases --assets --pull-details --gists --starred-gists $GH_USER
Restoring from Backup
=====================
This tool creates backups only, there is no inbuilt restore command.
**Git repositories, wikis, and gists** can be restored by pushing them back to GitHub as you would any git repository. For example, to restore a bare repository backup::
cd /tmp/white-house/repositories/petitions/repository
git push --mirror git@github.com:WhiteHouse/petitions.git
**Issues, pull requests, comments, and other metadata** are saved as JSON files for archival purposes. The GitHub API does not support recreating this data faithfully, creating issues via the API has limitations:
- New issue/PR numbers are assigned (original numbers cannot be set)
- Timestamps reflect creation time (original dates cannot be set)
- The API caller becomes the author (original authors cannot be set)
- Cross-references between issues and PRs will break
These are GitHub API limitations that affect all backup and migration tools, not just this one. Recreating issues with these limitations via the GitHub API is an exercise for the reader. The JSON backups remain useful for searching, auditing, or manual reference.
Development
===========

View File

@@ -1,69 +1,18 @@
#!/usr/bin/env python
"""
Backwards-compatible wrapper script.
The recommended way to run github-backup is via the installed command
(pip install github-backup) or python -m github_backup.
This script is kept for backwards compatibility with existing installations
that may reference this path directly.
"""
import logging
import os
import sys
from github_backup.github_backup import (
backup_account,
backup_repositories,
check_git_lfs_install,
filter_repositories,
get_authenticated_user,
logger,
mkdir_p,
parse_args,
retrieve_repositories,
)
# INFO and DEBUG go to stdout, WARNING and above go to stderr
log_format = logging.Formatter(
fmt="%(asctime)s.%(msecs)03d: %(message)s",
datefmt="%Y-%m-%dT%H:%M:%S",
)
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(logging.DEBUG)
stdout_handler.addFilter(lambda r: r.levelno < logging.WARNING)
stdout_handler.setFormatter(log_format)
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setLevel(logging.WARNING)
stderr_handler.setFormatter(log_format)
logging.basicConfig(level=logging.INFO, handlers=[stdout_handler, stderr_handler])
def main():
args = parse_args()
if args.quiet:
logger.setLevel(logging.WARNING)
output_directory = os.path.realpath(args.output_directory)
if not os.path.isdir(output_directory):
logger.info("Create output directory {0}".format(output_directory))
mkdir_p(output_directory)
if args.lfs_clone:
check_git_lfs_install()
if args.log_level:
log_level = logging.getLevelName(args.log_level.upper())
if isinstance(log_level, int):
logger.root.setLevel(log_level)
if not args.as_app:
logger.info("Backing up user {0} to {1}".format(args.user, output_directory))
authenticated_user = get_authenticated_user(args)
else:
authenticated_user = {"login": None}
repositories = retrieve_repositories(args, authenticated_user)
repositories = filter_repositories(args, repositories)
backup_repositories(args, output_directory, repositories)
backup_account(args, output_directory)
from github_backup.cli import main
from github_backup.github_backup import logger
if __name__ == "__main__":
try:

View File

@@ -1 +1 @@
__version__ = "0.54.0"
__version__ = "0.56.0"

13
github_backup/__main__.py Normal file
View File

@@ -0,0 +1,13 @@
"""Allow running as: python -m github_backup"""
import sys
from github_backup.cli import main
from github_backup.github_backup import logger
if __name__ == "__main__":
try:
main()
except Exception as e:
logger.error(str(e))
sys.exit(1)

82
github_backup/cli.py Normal file
View File

@@ -0,0 +1,82 @@
#!/usr/bin/env python
"""Command-line interface for github-backup."""
import logging
import os
import sys
from github_backup.github_backup import (
backup_account,
backup_repositories,
check_git_lfs_install,
filter_repositories,
get_auth,
get_authenticated_user,
logger,
mkdir_p,
parse_args,
retrieve_repositories,
)
# INFO and DEBUG go to stdout, WARNING and above go to stderr
log_format = logging.Formatter(
fmt="%(asctime)s.%(msecs)03d: %(message)s",
datefmt="%Y-%m-%dT%H:%M:%S",
)
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(logging.DEBUG)
stdout_handler.addFilter(lambda r: r.levelno < logging.WARNING)
stdout_handler.setFormatter(log_format)
stderr_handler = logging.StreamHandler(sys.stderr)
stderr_handler.setLevel(logging.WARNING)
stderr_handler.setFormatter(log_format)
logging.basicConfig(level=logging.INFO, handlers=[stdout_handler, stderr_handler])
def main():
"""Main entry point for github-backup CLI."""
args = parse_args()
if args.private and not get_auth(args):
logger.warning(
"The --private flag has no effect without authentication. "
"Use -t/--token, -f/--token-fine, or -u/--username to authenticate."
)
if args.quiet:
logger.setLevel(logging.WARNING)
output_directory = os.path.realpath(args.output_directory)
if not os.path.isdir(output_directory):
logger.info("Create output directory {0}".format(output_directory))
mkdir_p(output_directory)
if args.lfs_clone:
check_git_lfs_install()
if args.log_level:
log_level = logging.getLevelName(args.log_level.upper())
if isinstance(log_level, int):
logger.root.setLevel(log_level)
if not args.as_app:
logger.info("Backing up user {0} to {1}".format(args.user, output_directory))
authenticated_user = get_authenticated_user(args)
else:
authenticated_user = {"login": None}
repositories = retrieve_repositories(args, authenticated_user)
repositories = filter_repositories(args, repositories)
backup_repositories(args, output_directory, repositories)
backup_account(args, output_directory)
if __name__ == "__main__":
try:
main()
except Exception as e:
logger.error(str(e))
sys.exit(1)

View File

@@ -1038,7 +1038,7 @@ def download_attachment_file(url, path, auth, as_app=False, fine=False):
bytes_downloaded += len(chunk)
# Atomic rename to final location
os.rename(temp_path, path)
os.replace(temp_path, path)
metadata["size_bytes"] = bytes_downloaded
metadata["success"] = True
@@ -1459,7 +1459,7 @@ def download_attachments(
# Rename to add extension (already atomic from download)
try:
os.rename(filepath, final_filepath)
os.replace(filepath, final_filepath)
metadata["saved_as"] = os.path.basename(final_filepath)
except Exception as e:
logger.warning(
@@ -1490,7 +1490,7 @@ def download_attachments(
manifest_path = os.path.join(attachments_dir, "manifest.json")
with open(manifest_path + ".temp", "w") as f:
json.dump(manifest, f, indent=2)
os.rename(manifest_path + ".temp", manifest_path) # Atomic write
os.replace(manifest_path + ".temp", manifest_path) # Atomic write
logger.debug(
"Wrote manifest for {0} #{1}: {2} attachments".format(
item_type_display, number, len(attachment_metadata_list)
@@ -1565,6 +1565,12 @@ def retrieve_repositories(args, authenticated_user):
repos.extend(gists)
if args.include_starred_gists:
if not authenticated_user.get("login") or args.user.lower() != authenticated_user["login"].lower():
logger.warning(
"Cannot retrieve starred gists for '%s'. GitHub only allows access to the authenticated user's starred gists.",
args.user,
)
else:
starred_gists_template = "https://{0}/gists/starred".format(
get_github_api_host(args)
)
@@ -1666,9 +1672,10 @@ def backup_repositories(args, output_directory, repositories):
repo_url = get_github_repo_url(args, repository)
include_gists = args.include_gists or args.include_starred_gists
include_starred = args.all_starred and repository.get("is_starred")
if (args.include_repository or args.include_everything) or (
include_gists and repository.get("is_gist")
):
) or include_starred:
repo_name = (
repository.get("name")
if not repository.get("is_gist")
@@ -1804,7 +1811,7 @@ def backup_issues(args, repo_cwd, repository, repos_template):
with codecs.open(issue_file + ".temp", "w", encoding="utf-8") as f:
json_dump(issue, f)
os.rename(issue_file + ".temp", issue_file) # Unlike json_dump, this is atomic
os.replace(issue_file + ".temp", issue_file) # Atomic write
def backup_pulls(args, repo_cwd, repository, repos_template):
@@ -1879,7 +1886,7 @@ def backup_pulls(args, repo_cwd, repository, repos_template):
with codecs.open(pull_file + ".temp", "w", encoding="utf-8") as f:
json_dump(pull, f)
os.rename(pull_file + ".temp", pull_file) # Unlike json_dump, this is atomic
os.replace(pull_file + ".temp", pull_file) # Atomic write
def backup_milestones(args, repo_cwd, repository, repos_template):
@@ -2017,12 +2024,9 @@ def fetch_repository(
):
if bare_clone:
if os.path.exists(local_dir):
clone_exists = (
subprocess.check_output(
clone_exists = subprocess.check_output(
["git", "rev-parse", "--is-bare-repository"], cwd=local_dir
)
== b"true\n"
)
) == b"true\n"
else:
clone_exists = False
else:
@@ -2037,10 +2041,13 @@ def fetch_repository(
"git ls-remote " + remote_url, stdout=FNULL, stderr=FNULL, shell=True
)
if initialized == 128:
if ".wiki.git" in remote_url:
logger.info(
"Skipping {0} ({1}) since it's not initialized".format(
name, masked_remote_url
"Skipping {0} wiki (wiki is enabled but has no content)".format(name)
)
else:
logger.info(
"Skipping {0} (repository not accessible - may be empty, private, or credentials invalid)".format(name)
)
return
@@ -2082,12 +2089,14 @@ def fetch_repository(
if no_prune:
git_command.pop()
logging_subprocess(git_command, cwd=local_dir)
else:
if lfs_clone:
git_command = ["git", "lfs", "clone", remote_url, local_dir]
else:
git_command = ["git", "clone", remote_url, local_dir]
logging_subprocess(git_command)
if lfs_clone:
git_command = ["git", "lfs", "fetch", "--all", "--prune"]
if no_prune:
git_command.pop()
logging_subprocess(git_command, cwd=local_dir)
def backup_account(args, output_directory):
@@ -2196,5 +2205,5 @@ def json_dump_if_changed(data, output_file_path):
temp_file = output_file_path + ".temp"
with codecs.open(temp_file, "w", encoding="utf-8") as f:
f.write(new_content)
os.rename(temp_file, output_file_path) # Atomic on POSIX systems
os.replace(temp_file, output_file_path) # Atomic write
return True

View File

@@ -1,5 +1,5 @@
autopep8==2.3.2
black==25.11.0
black==25.12.0
bleach==6.3.0
certifi==2025.11.12
charset-normalizer==3.4.4
@@ -8,7 +8,7 @@ colorama==0.4.6
docutils==0.22.3
flake8==7.3.0
gitchangelog==3.0.4
pytest==9.0.1
pytest==9.0.2
idna==3.11
importlib-metadata==8.7.0
jaraco.classes==3.4.0
@@ -21,7 +21,7 @@ mypy-extensions==1.1.0
packaging==25.0
pathspec==0.12.1
pkginfo==1.12.1.2
platformdirs==4.5.0
platformdirs==4.5.1
pycodestyle==2.14.0
pyflakes==3.4.0
Pygments==2.19.2
@@ -35,6 +35,6 @@ setuptools==80.9.0
six==1.17.0
tqdm==4.67.1
twine==6.2.0
urllib3==2.5.0
urllib3==2.6.1
webencodings==0.5.1
zipp==3.23.0

View File

@@ -33,7 +33,11 @@ setup(
author="Jose Diaz-Gonzalez",
author_email="github-backup@josediazgonzalez.com",
packages=["github_backup"],
scripts=["bin/github-backup"],
entry_points={
"console_scripts": [
"github-backup=github_backup.cli:main",
],
},
url="http://github.com/josegonzalez/python-github-backup",
license="MIT",
classifiers=[

161
tests/test_all_starred.py Normal file
View File

@@ -0,0 +1,161 @@
"""Tests for --all-starred flag behavior (issue #225)."""
import pytest
from unittest.mock import Mock, patch
from github_backup import github_backup
class TestAllStarredCloning:
"""Test suite for --all-starred repository cloning behavior.
Issue #225: --all-starred should clone starred repos without requiring --repositories.
"""
def _create_mock_args(self, **overrides):
"""Create a mock args object with sensible defaults."""
args = Mock()
args.user = "testuser"
args.output_directory = "/tmp/backup"
args.include_repository = False
args.include_everything = False
args.include_gists = False
args.include_starred_gists = False
args.all_starred = False
args.skip_existing = False
args.bare_clone = False
args.lfs_clone = False
args.no_prune = False
args.include_wiki = False
args.include_issues = False
args.include_issue_comments = False
args.include_issue_events = False
args.include_pulls = False
args.include_pull_comments = False
args.include_pull_commits = False
args.include_pull_details = False
args.include_labels = False
args.include_hooks = False
args.include_milestones = False
args.include_releases = False
args.include_assets = False
args.include_attachments = False
args.incremental = False
args.incremental_by_files = False
args.github_host = None
args.prefer_ssh = False
args.token_classic = None
args.token_fine = None
args.username = None
args.password = None
args.as_app = False
args.osx_keychain_item_name = None
args.osx_keychain_item_account = None
for key, value in overrides.items():
setattr(args, key, value)
return args
@patch('github_backup.github_backup.fetch_repository')
@patch('github_backup.github_backup.get_github_repo_url')
def test_all_starred_clones_without_repositories_flag(self, mock_get_url, mock_fetch):
"""--all-starred should clone starred repos without --repositories flag.
This is the core fix for issue #225.
"""
args = self._create_mock_args(all_starred=True)
mock_get_url.return_value = "https://github.com/otheruser/awesome-project.git"
# A starred repository (is_starred flag set by retrieve_repositories)
starred_repo = {
"name": "awesome-project",
"full_name": "otheruser/awesome-project",
"owner": {"login": "otheruser"},
"private": False,
"fork": False,
"has_wiki": False,
"is_starred": True, # This flag is set for starred repos
}
with patch('github_backup.github_backup.mkdir_p'):
github_backup.backup_repositories(args, "/tmp/backup", [starred_repo])
# fetch_repository should be called for the starred repo
assert mock_fetch.called, "--all-starred should trigger repository cloning"
mock_fetch.assert_called_once()
call_args = mock_fetch.call_args
assert call_args[0][0] == "awesome-project" # repo name
@patch('github_backup.github_backup.fetch_repository')
@patch('github_backup.github_backup.get_github_repo_url')
def test_starred_repo_not_cloned_without_all_starred_flag(self, mock_get_url, mock_fetch):
"""Starred repos should NOT be cloned if --all-starred is not set."""
args = self._create_mock_args(all_starred=False)
mock_get_url.return_value = "https://github.com/otheruser/awesome-project.git"
starred_repo = {
"name": "awesome-project",
"full_name": "otheruser/awesome-project",
"owner": {"login": "otheruser"},
"private": False,
"fork": False,
"has_wiki": False,
"is_starred": True,
}
with patch('github_backup.github_backup.mkdir_p'):
github_backup.backup_repositories(args, "/tmp/backup", [starred_repo])
# fetch_repository should NOT be called
assert not mock_fetch.called, "Starred repos should not be cloned without --all-starred"
@patch('github_backup.github_backup.fetch_repository')
@patch('github_backup.github_backup.get_github_repo_url')
def test_non_starred_repo_not_cloned_with_only_all_starred(self, mock_get_url, mock_fetch):
"""Non-starred repos should NOT be cloned when only --all-starred is set."""
args = self._create_mock_args(all_starred=True)
mock_get_url.return_value = "https://github.com/testuser/my-project.git"
# A regular (non-starred) repository
regular_repo = {
"name": "my-project",
"full_name": "testuser/my-project",
"owner": {"login": "testuser"},
"private": False,
"fork": False,
"has_wiki": False,
# No is_starred flag
}
with patch('github_backup.github_backup.mkdir_p'):
github_backup.backup_repositories(args, "/tmp/backup", [regular_repo])
# fetch_repository should NOT be called for non-starred repos
assert not mock_fetch.called, "Non-starred repos should not be cloned with only --all-starred"
@patch('github_backup.github_backup.fetch_repository')
@patch('github_backup.github_backup.get_github_repo_url')
def test_repositories_flag_still_works(self, mock_get_url, mock_fetch):
"""--repositories flag should still clone repos as before."""
args = self._create_mock_args(include_repository=True)
mock_get_url.return_value = "https://github.com/testuser/my-project.git"
regular_repo = {
"name": "my-project",
"full_name": "testuser/my-project",
"owner": {"login": "testuser"},
"private": False,
"fork": False,
"has_wiki": False,
}
with patch('github_backup.github_backup.mkdir_p'):
github_backup.backup_repositories(args, "/tmp/backup", [regular_repo])
# fetch_repository should be called
assert mock_fetch.called, "--repositories should trigger repository cloning"
if __name__ == "__main__":
pytest.main([__file__, "-v"])

View File

@@ -0,0 +1,112 @@
"""Tests for case-insensitive username/organization filtering."""
import pytest
from unittest.mock import Mock
from github_backup import github_backup
class TestCaseSensitivity:
"""Test suite for case-insensitive username matching in filter_repositories."""
def test_filter_repositories_case_insensitive_user(self):
"""Should filter repositories case-insensitively for usernames.
Reproduces issue #198 where typing 'iamrodos' fails to match
repositories with owner.login='Iamrodos' (the canonical case from GitHub API).
"""
# Simulate user typing lowercase username
args = Mock()
args.user = "iamrodos" # lowercase (what user typed)
args.repository = None
args.name_regex = None
args.languages = None
args.exclude = None
args.fork = False
args.private = False
args.public = False
args.all = True
# Simulate GitHub API returning canonical case
repos = [
{
"name": "repo1",
"owner": {"login": "Iamrodos"}, # Capital I (canonical from API)
"private": False,
"fork": False,
},
{
"name": "repo2",
"owner": {"login": "Iamrodos"},
"private": False,
"fork": False,
},
]
filtered = github_backup.filter_repositories(args, repos)
# Should match despite case difference
assert len(filtered) == 2
assert filtered[0]["name"] == "repo1"
assert filtered[1]["name"] == "repo2"
def test_filter_repositories_case_insensitive_org(self):
"""Should filter repositories case-insensitively for organizations.
Tests the example from issue #198 where 'prai-org' doesn't match 'PRAI-Org'.
"""
args = Mock()
args.user = "prai-org" # lowercase (what user typed)
args.repository = None
args.name_regex = None
args.languages = None
args.exclude = None
args.fork = False
args.private = False
args.public = False
args.all = True
repos = [
{
"name": "repo1",
"owner": {"login": "PRAI-Org"}, # Different case (canonical from API)
"private": False,
"fork": False,
},
]
filtered = github_backup.filter_repositories(args, repos)
# Should match despite case difference
assert len(filtered) == 1
assert filtered[0]["name"] == "repo1"
def test_filter_repositories_case_variations(self):
"""Should handle various case combinations correctly."""
args = Mock()
args.user = "TeSt-UsEr" # Mixed case
args.repository = None
args.name_regex = None
args.languages = None
args.exclude = None
args.fork = False
args.private = False
args.public = False
args.all = True
repos = [
{"name": "repo1", "owner": {"login": "test-user"}, "private": False, "fork": False},
{"name": "repo2", "owner": {"login": "TEST-USER"}, "private": False, "fork": False},
{"name": "repo3", "owner": {"login": "TeSt-UsEr"}, "private": False, "fork": False},
{"name": "repo4", "owner": {"login": "other-user"}, "private": False, "fork": False},
]
filtered = github_backup.filter_repositories(args, repos)
# Should match first 3 (all case variations of same user)
assert len(filtered) == 3
assert set(r["name"] for r in filtered) == {"repo1", "repo2", "repo3"}
if __name__ == "__main__":
pytest.main([__file__, "-v"])