Compare commits

..

7 Commits

Author SHA1 Message Date
Frederik Ring
34d04211eb Update README TOC 2022-02-11 20:06:23 +01:00
Mauro Molin
8dfdd14527 Added custom notification messages using text/template (#60)
* Added custom notification messages using text/template

* Change notification template path and removed automatic newline trim

* Added stats and changed structure of template params

* Stat file hotfix

* Embedded and fixed default notification templates


Fix

* Changed Output to LogOutput

* Changed stats integer to unsigned

* Bytes formatting in template func


fix

* Changed Archives to Storages

* Removed unecessary sleep for pruning leeway

* Set EndTime after pruning is completed

* Added custom notifications documentation

* Added 5s sleep in swarm test

* Fixed documentation

* Dockerfile copies all files in cmd/backup
2022-02-11 20:05:16 +01:00
Frederik Ring
3bb99a7117 Update package targz 2022-02-08 15:12:46 +01:00
Fridgemagnet
ddc34be55d Updated README.md "Restoring..." section example (#56)
Edited the "Restoring a volume from a backup" section example to be able to better differentiate between the names of the temporary restore container and the name of the mounted restore volume.
New example container name: temp_restore_container
The restore volume name remains the same: backup_restore

Also changed "one-off container" to "once-off container".
2022-02-04 11:52:59 +01:00
Joshua Noble
cb9b4bfcff Add support for Filebase (#54) 2022-02-02 17:22:42 +01:00
Frederik Ring
62bd2f4a5a Update base docker image to alpine 3.15 (#53) 2022-01-27 14:40:56 +01:00
Frederik Ring
6fe629ce87 Allow path to be set for bucket storage (#52) 2022-01-25 21:16:16 +01:00
8 changed files with 325 additions and 49 deletions

View File

@@ -6,10 +6,10 @@ FROM golang:1.17-alpine as builder
WORKDIR /app WORKDIR /app
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY cmd/backup/main.go ./cmd/backup/main.go COPY cmd/backup ./cmd/backup/
RUN go build -o backup cmd/backup/main.go RUN go build -o backup cmd/backup/main.go
FROM alpine:3.14 FROM alpine:3.15
WORKDIR /root WORKDIR /root

View File

@@ -19,6 +19,7 @@ It handles __recurring or one-off backups of Docker volumes__ to a __local direc
- [Stopping containers during backup](#stopping-containers-during-backup) - [Stopping containers during backup](#stopping-containers-during-backup)
- [Automatically pruning old backups](#automatically-pruning-old-backups) - [Automatically pruning old backups](#automatically-pruning-old-backups)
- [Send email notifications on failed backup runs](#send-email-notifications-on-failed-backup-runs) - [Send email notifications on failed backup runs](#send-email-notifications-on-failed-backup-runs)
- [Customize notifications](#customize-notifications)
- [Encrypting your backup using GPG](#encrypting-your-backup-using-gpg) - [Encrypting your backup using GPG](#encrypting-your-backup-using-gpg)
- [Restoring a volume from a backup](#restoring-a-volume-from-a-backup) - [Restoring a volume from a backup](#restoring-a-volume-from-a-backup)
- [Set the timezone the container runs in](#set-the-timezone-the-container-runs-in) - [Set the timezone the container runs in](#set-the-timezone-the-container-runs-in)
@@ -27,6 +28,7 @@ It handles __recurring or one-off backups of Docker volumes__ to a __local direc
- [Update deprecated email configuration](#update-deprecated-email-configuration) - [Update deprecated email configuration](#update-deprecated-email-configuration)
- [Recipes](#recipes) - [Recipes](#recipes)
- [Backing up to AWS S3](#backing-up-to-aws-s3) - [Backing up to AWS S3](#backing-up-to-aws-s3)
- [Backing up to Filebase](#backing-up-to-filebase)
- [Backing up to MinIO](#backing-up-to-minio) - [Backing up to MinIO](#backing-up-to-minio)
- [Backing up to WebDAV](#backing-up-to-webdav) - [Backing up to WebDAV](#backing-up-to-webdav)
- [Backing up locally](#backing-up-locally) - [Backing up locally](#backing-up-locally)
@@ -135,7 +137,7 @@ You can populate below template according to your requirements and use it as you
# Please note that you will need to escape the `$` when providing the value # Please note that you will need to escape the `$` when providing the value
# in a docker-compose.yml file, i.e. using $$VAR instead of $VAR. # in a docker-compose.yml file, i.e. using $$VAR instead of $VAR.
# BACKUP_FILENAME_TEMPLATE="true" # BACKUP_FILENAME_EXPAND="true"
# When storing local backups, a symlink to the latest backup can be created # When storing local backups, a symlink to the latest backup can be created
# in case a value is given for this key. This has no effect on remote backups. # in case a value is given for this key. This has no effect on remote backups.
@@ -156,6 +158,11 @@ You can populate below template according to your requirements and use it as you
# AWS_S3_BUCKET_NAME="backup-bucket" # AWS_S3_BUCKET_NAME="backup-bucket"
# If you want to store the backup in a non-root location on your bucket
# you can provide a path. The path must not contain a leading slash.
# AWS_S3_PATH="my/backup/location"
# Define credentials for authenticating against the backup storage and a bucket # Define credentials for authenticating against the backup storage and a bucket
# name. Although all of these keys are `AWS`-prefixed, the setup can be used # name. Although all of these keys are `AWS`-prefixed, the setup can be used
# with any S3 compatible storage. # with any S3 compatible storage.
@@ -402,6 +409,30 @@ Refer to the documentation of [shoutrrr][shoutrrr-docs] to find out about option
[shoutrrr-docs]: https://containrrr.dev/shoutrrr/v0.5/services/overview/ [shoutrrr-docs]: https://containrrr.dev/shoutrrr/v0.5/services/overview/
### Customize notifications
The title and body of the notifications can be easily tailored to your needs using [go templates](https://pkg.go.dev/text/template).
Templates must be mounted inside the container in `/etc/dockervolumebackup/notifications.d/`: any file inside this directory will be parsed.
The files have to define [nested templates](https://pkg.go.dev/text/template#hdr-Nested_template_definitions) in order to override the original values. An example:
```
{{ define "title_success" -}}
✅ Successfully ran backup {{ .Config.BackupStopContainerLabel }}
{{- end }}
{{ define "body_success" -}}
▶️ Start time: {{ .Stats.StartTime | formatTime }}
⏹️ End time: {{ .Stats.EndTime | formatTime }}
⌛ Took time: {{ .Stats.TookTime }}
🛑 Stopped containers: {{ .Stats.Containers.Stopped }}/{{ .Stats.Containers.All }} ({{ .Stats.Containers.StopErrors }} errors)
⚖️ Backup size: {{ .Stats.BackupFile.Size | formatBytesBin }} / {{ .Stats.BackupFile.Size | formatBytesDec }}
🗑️ Pruned backups: {{ .Stats.Storages.Local.Pruned }}/{{ .Stats.Storages.Local.Total }} ({{ .Stats.Storages.Local.PruneErrors }} errors)
{{- end }}
```
Overridable template names are: `title_success`, `body_success`, `title_failure`, `body_failure`.
For a full list of available variables and functions, see [this page](https://github.com/offen/docker-volume-backup/blob/master/docs/NOTIFICATION-TEMPLATES.md).
### Encrypting your backup using GPG ### Encrypting your backup using GPG
@@ -423,12 +454,12 @@ In case you need to restore a volume from a backup, the most straight forward pr
```console ```console
tar -C /tmp -xvf backup.tar.gz tar -C /tmp -xvf backup.tar.gz
``` ```
- Using a temporary one-off container, mount the volume (the example assumes it's named `data`) and copy over the backup. Make sure you copy the correct path level (this depends on how you mount your volume into the backup container), you might need to strip some leading elements - Using a temporary once-off container, mount the volume (the example assumes it's named `data`) and copy over the backup. Make sure you copy the correct path level (this depends on how you mount your volume into the backup container), you might need to strip some leading elements
```console ```console
docker run -d --name backup_restore -v data:/backup_restore alpine docker run -d --name temp_restore_container -v data:/backup_restore alpine
docker cp /tmp/backup/data-backup backup_restore:/backup_restore docker cp /tmp/backup/data-backup temp_restore_container:/backup_restore
docker stop backup_restore docker stop temp_restore_container
docker rm backup_restore docker rm temp_restore_container
``` ```
- Restart the container(s) that are using the volume - Restart the container(s) that are using the volume
@@ -528,6 +559,28 @@ volumes:
data: data:
``` ```
### Backing up to Filebase
```yml
version: '3'
services:
# ... define other services using the `data` volume here
backup:
image: offen/docker-volume-backup:latest
environment:
AWS_ENDPOINT: s3.filebase.com
AWS_BUCKET_NAME: filebase-bucket
AWS_ACCESS_KEY_ID: FILEBASE-ACCESS-KEY
AWS_SECRET_ACCESS_KEY: FILEBASE-SECRET-KEY
volumes:
- data:/backup/my-app-backup:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
volumes:
data:
```
### Backing up to MinIO ### Backing up to MinIO
```yml ```yml

View File

@@ -6,6 +6,7 @@ package main
import ( import (
"bytes" "bytes"
"context" "context"
_ "embed"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -15,6 +16,7 @@ import (
"path/filepath" "path/filepath"
"sort" "sort"
"strings" "strings"
"text/template"
"time" "time"
"github.com/containrrr/shoutrrr" "github.com/containrrr/shoutrrr"
@@ -36,6 +38,9 @@ import (
"golang.org/x/crypto/openpgp" "golang.org/x/crypto/openpgp"
) )
//go:embed notifications.tmpl
var defaultNotifications string
func main() { func main() {
unlock := lock("/var/lock/dockervolumebackup.lock") unlock := lock("/var/lock/dockervolumebackup.lock")
defer unlock() defer unlock()
@@ -83,6 +88,8 @@ func main() {
s.must(s.encryptBackup()) s.must(s.encryptBackup())
s.must(s.copyBackup()) s.must(s.copyBackup())
s.must(s.pruneOldBackups()) s.must(s.pruneOldBackups())
s.stats.EndTime = time.Now()
s.stats.TookTime = s.stats.EndTime.Sub(s.stats.EndTime)
} }
// script holds all the stateful information required to orchestrate a // script holds all the stateful information required to orchestrate a
@@ -93,17 +100,64 @@ type script struct {
webdavClient *gowebdav.Client webdavClient *gowebdav.Client
logger *logrus.Logger logger *logrus.Logger
sender *router.ServiceRouter sender *router.ServiceRouter
template *template.Template
hooks []hook hooks []hook
hookLevel hookLevel hookLevel hookLevel
start time.Time file string
file string stats *Stats
output *bytes.Buffer
c *config c *Config
} }
type config struct { // ContainersStats stats about the docker containers
type ContainersStats struct {
All uint
ToStop uint
Stopped uint
StopErrors uint
}
// BackupFileStats stats about the created backup file
type BackupFileStats struct {
Name string
FullPath string
Size uint64
}
// StorageStats stats about the status of an archival directory
type StorageStats struct {
Total uint
Pruned uint
PruneErrors uint
}
// StoragesStats stats about each possible archival location (Local, WebDAV, S3)
type StoragesStats struct {
Local StorageStats
WebDAV StorageStats
S3 StorageStats
}
// Stats global stats regarding script execution
type Stats struct {
StartTime time.Time
EndTime time.Time
TookTime time.Duration
LogOutput *bytes.Buffer
Containers ContainersStats
BackupFile BackupFileStats
Storages StoragesStats
}
// NotificationData data to be passed to the notification templates
type NotificationData struct {
Error error
Config *Config
Stats *Stats
}
type Config struct {
BackupSources string `split_words:"true" default:"/backup"` BackupSources string `split_words:"true" default:"/backup"`
BackupFilename string `split_words:"true" default:"backup-%Y-%m-%dT%H-%M-%S.tar.gz"` BackupFilename string `split_words:"true" default:"backup-%Y-%m-%dT%H-%M-%S.tar.gz"`
BackupFilenameExpand bool `split_words:"true"` BackupFilenameExpand bool `split_words:"true"`
@@ -115,6 +169,7 @@ type config struct {
BackupStopContainerLabel string `split_words:"true" default:"true"` BackupStopContainerLabel string `split_words:"true" default:"true"`
BackupFromSnapshot bool `split_words:"true"` BackupFromSnapshot bool `split_words:"true"`
AwsS3BucketName string `split_words:"true"` AwsS3BucketName string `split_words:"true"`
AwsS3Path string `split_words:"true"`
AwsEndpoint string `split_words:"true" default:"s3.amazonaws.com"` AwsEndpoint string `split_words:"true" default:"s3.amazonaws.com"`
AwsEndpointProto string `split_words:"true" default:"https"` AwsEndpointProto string `split_words:"true" default:"https"`
AwsEndpointInsecure bool `split_words:"true"` AwsEndpointInsecure bool `split_words:"true"`
@@ -145,15 +200,18 @@ var msgBackupFailed = "backup run failed"
func newScript() (*script, error) { func newScript() (*script, error) {
stdOut, logBuffer := buffer(os.Stdout) stdOut, logBuffer := buffer(os.Stdout)
s := &script{ s := &script{
c: &config{}, c: &Config{},
logger: &logrus.Logger{ logger: &logrus.Logger{
Out: stdOut, Out: stdOut,
Formatter: new(logrus.TextFormatter), Formatter: new(logrus.TextFormatter),
Hooks: make(logrus.LevelHooks), Hooks: make(logrus.LevelHooks),
Level: logrus.InfoLevel, Level: logrus.InfoLevel,
}, },
start: time.Now(), stats: &Stats{
output: logBuffer, StartTime: time.Now(),
LogOutput: logBuffer,
Storages: StoragesStats{},
},
} }
if err := envconfig.Process("", s.c); err != nil { if err := envconfig.Process("", s.c); err != nil {
@@ -166,7 +224,7 @@ func newScript() (*script, error) {
s.c.BackupLatestSymlink = os.ExpandEnv(s.c.BackupLatestSymlink) s.c.BackupLatestSymlink = os.ExpandEnv(s.c.BackupLatestSymlink)
s.c.BackupPruningPrefix = os.ExpandEnv(s.c.BackupPruningPrefix) s.c.BackupPruningPrefix = os.ExpandEnv(s.c.BackupPruningPrefix)
} }
s.file = timeutil.Strftime(&s.start, s.file) s.file = timeutil.Strftime(&s.stats.StartTime, s.file)
_, err := os.Stat("/var/run/docker.sock") _, err := os.Stat("/var/run/docker.sock")
if !os.IsNotExist(err) { if !os.IsNotExist(err) {
@@ -272,6 +330,31 @@ func newScript() (*script, error) {
}) })
} }
tmpl := template.New("")
tmpl.Funcs(template.FuncMap{
"formatTime": func(t time.Time) string {
return t.Format(time.RFC3339)
},
"formatBytesDec": func(bytes uint64) string {
return formatBytes(bytes, true)
},
"formatBytesBin": func(bytes uint64) string {
return formatBytes(bytes, false)
},
})
tmpl, err = tmpl.Parse(defaultNotifications)
if err != nil {
return nil, fmt.Errorf("newScript: unable to parse default notifications templates: %w", err)
}
if _, err := os.Stat("/etc/dockervolumebackup/notifications.d"); err == nil {
tmpl, err = tmpl.ParseGlob("/etc/dockervolumebackup/notifications.d/*.*")
if err != nil {
return nil, fmt.Errorf("newScript: unable to parse user defined notifications templates: %w", err)
}
}
s.template = tmpl
return s, nil return s, nil
} }
@@ -282,28 +365,39 @@ func (s *script) registerHook(level hookLevel, action func(err error) error) {
s.hooks = append(s.hooks, hook{level, action}) s.hooks = append(s.hooks, hook{level, action})
} }
// notifyFailure sends a notification about a failed backup run // notify sends a notification using the given title and body templates.
func (s *script) notifyFailure(err error) error { // Automatically creates notification data, adding the given error
body := fmt.Sprintf( func (s *script) notify(titleTemplate string, bodyTemplate string, err error) error {
"Running docker-volume-backup failed with error: %s\n\nLog output of the failed run was:\n\n%s\n", err, s.output.String(), params := NotificationData{
) Error: err,
title := fmt.Sprintf("Failure running docker-volume-backup at %s", s.start.Format(time.RFC3339)) Stats: s.stats,
if err := s.sendNotification(title, body); err != nil { Config: s.c,
}
titleBuf := &bytes.Buffer{}
if err := s.template.ExecuteTemplate(titleBuf, titleTemplate, params); err != nil {
return fmt.Errorf("notifyFailure: error executing %s template: %w", titleTemplate, err)
}
bodyBuf := &bytes.Buffer{}
if err := s.template.ExecuteTemplate(bodyBuf, bodyTemplate, params); err != nil {
return fmt.Errorf("notifyFailure: error executing %s template: %w", bodyTemplate, err)
}
if err := s.sendNotification(titleBuf.String(), bodyBuf.String()); err != nil {
return fmt.Errorf("notifyFailure: error notifying: %w", err) return fmt.Errorf("notifyFailure: error notifying: %w", err)
} }
return nil return nil
} }
// notifyFailure sends a notification about a failed backup run
func (s *script) notifyFailure(err error) error {
return s.notify("title_failure", "body_failure", err)
}
// notifyFailure sends a notification about a successful backup run // notifyFailure sends a notification about a successful backup run
func (s *script) notifySuccess() error { func (s *script) notifySuccess() error {
title := fmt.Sprintf("Success running docker-volume-backup at %s", s.start.Format(time.RFC3339)) return s.notify("title_success", "body_success", nil)
body := fmt.Sprintf(
"Running docker-volume-backup succeeded.\n\nLog output was:\n\n%s\n", s.output.String(),
)
if err := s.sendNotification(title, body); err != nil {
return fmt.Errorf("notifySuccess: error notifying: %w", err)
}
return nil
} }
// sendNotification sends a notification to all configured third party services // sendNotification sends a notification to all configured third party services
@@ -381,6 +475,12 @@ func (s *script) stopContainers() (func() error, error) {
) )
} }
s.stats.Containers = ContainersStats{
All: uint(len(allContainers)),
ToStop: uint(len(containersToStop)),
Stopped: uint(len(stoppedContainers)),
}
return func() error { return func() error {
servicesRequiringUpdate := map[string]struct{}{} servicesRequiringUpdate := map[string]struct{}{}
@@ -524,8 +624,19 @@ func (s *script) encryptBackup() error {
// as per the given configuration. // as per the given configuration.
func (s *script) copyBackup() error { func (s *script) copyBackup() error {
_, name := path.Split(s.file) _, name := path.Split(s.file)
if stat, err := os.Stat(s.file); err != nil {
return fmt.Errorf("copyBackup: unable to stat backup file: %w", err)
} else {
size := stat.Size()
s.stats.BackupFile = BackupFileStats{
Size: uint64(size),
Name: name,
FullPath: s.file,
}
}
if s.minioClient != nil { if s.minioClient != nil {
if _, err := s.minioClient.FPutObject(context.Background(), s.c.AwsS3BucketName, name, s.file, minio.PutObjectOptions{ if _, err := s.minioClient.FPutObject(context.Background(), s.c.AwsS3BucketName, filepath.Join(s.c.AwsS3Path, name), s.file, minio.PutObjectOptions{
ContentType: "application/tar+gzip", ContentType: "application/tar+gzip",
}); err != nil { }); err != nil {
return fmt.Errorf("copyBackup: error uploading backup to remote storage: %w", err) return fmt.Errorf("copyBackup: error uploading backup to remote storage: %w", err)
@@ -574,12 +685,7 @@ func (s *script) pruneOldBackups() error {
return nil return nil
} }
if s.c.BackupPruningLeeway != 0 { deadline := time.Now().AddDate(0, 0, -int(s.c.BackupRetentionDays)).Add(s.c.BackupPruningLeeway)
s.logger.Infof("Sleeping for %s before pruning backups.", s.c.BackupPruningLeeway)
time.Sleep(s.c.BackupPruningLeeway)
}
deadline := time.Now().AddDate(0, 0, -int(s.c.BackupRetentionDays))
// Prune minio/S3 backups // Prune minio/S3 backups
if s.minioClient != nil { if s.minioClient != nil {
@@ -603,6 +709,10 @@ func (s *script) pruneOldBackups() error {
} }
} }
s.stats.Storages.S3 = StorageStats{
Total: uint(lenCandidates),
Pruned: uint(len(matches)),
}
if len(matches) != 0 && len(matches) != lenCandidates { if len(matches) != 0 && len(matches) != lenCandidates {
objectsCh := make(chan minio.ObjectInfo) objectsCh := make(chan minio.ObjectInfo)
go func() { go func() {
@@ -618,6 +728,7 @@ func (s *script) pruneOldBackups() error {
removeErrors = append(removeErrors, result.Err) removeErrors = append(removeErrors, result.Err)
} }
} }
s.stats.Storages.S3.PruneErrors = uint(len(removeErrors))
if len(removeErrors) != 0 { if len(removeErrors) != 0 {
return fmt.Errorf( return fmt.Errorf(
@@ -626,10 +737,11 @@ func (s *script) pruneOldBackups() error {
join(removeErrors...), join(removeErrors...),
) )
} }
s.logger.Infof( s.logger.Infof(
"Pruned %d out of %d remote backup(s) as their age exceeded the configured retention period of %d days.", "Pruned %d out of %d remote backup(s) as their age exceeded the configured retention period of %d days.",
len(matches), s.stats.Storages.S3.Pruned,
lenCandidates, s.stats.Storages.S3.Total,
s.c.BackupRetentionDays, s.c.BackupRetentionDays,
) )
} else if len(matches) != 0 && len(matches) == lenCandidates { } else if len(matches) != 0 && len(matches) == lenCandidates {
@@ -658,14 +770,33 @@ func (s *script) pruneOldBackups() error {
} }
} }
s.stats.Storages.WebDAV = StorageStats{
Total: uint(lenCandidates),
Pruned: uint(len(matches)),
}
if len(matches) != 0 && len(matches) != lenCandidates { if len(matches) != 0 && len(matches) != lenCandidates {
var removeErrors []error
for _, match := range matches { for _, match := range matches {
if err := s.webdavClient.Remove(filepath.Join(s.c.WebdavPath, match.Name())); err != nil { if err := s.webdavClient.Remove(filepath.Join(s.c.WebdavPath, match.Name())); err != nil {
return fmt.Errorf("pruneOldBackups: error removing a file from remote storage: %w", err) removeErrors = append(removeErrors, err)
} else {
s.logger.Infof("Pruned %s from WebDAV: %s", match.Name(), filepath.Join(s.c.WebdavUrl, s.c.WebdavPath))
} }
s.logger.Infof("Pruned %s from WebDAV: %s", match.Name(), filepath.Join(s.c.WebdavUrl, s.c.WebdavPath))
} }
s.logger.Infof("Pruned %d out of %d remote backup(s) as their age exceeded the configured retention period of %d days.", len(matches), lenCandidates, s.c.BackupRetentionDays) s.stats.Storages.WebDAV.PruneErrors = uint(len(removeErrors))
if len(removeErrors) != 0 {
return fmt.Errorf(
"pruneOldBackups: %d error(s) removing files from remote storage: %w",
len(removeErrors),
join(removeErrors...),
)
}
s.logger.Infof(
"Pruned %d out of %d remote backup(s) as their age exceeded the configured retention period of %d days.",
s.stats.Storages.WebDAV.Pruned,
s.stats.Storages.WebDAV.Total,
s.c.BackupRetentionDays,
)
} else if len(matches) != 0 && len(matches) == lenCandidates { } else if len(matches) != 0 && len(matches) == lenCandidates {
s.logger.Warnf("The current configuration would delete all %d remote backup copies.", len(matches)) s.logger.Warnf("The current configuration would delete all %d remote backup copies.", len(matches))
s.logger.Warn("Refusing to do so, please check your configuration.") s.logger.Warn("Refusing to do so, please check your configuration.")
@@ -720,6 +851,10 @@ func (s *script) pruneOldBackups() error {
} }
} }
s.stats.Storages.Local = StorageStats{
Total: uint(len(candidates)),
Pruned: uint(len(matches)),
}
if len(matches) != 0 && len(matches) != len(candidates) { if len(matches) != 0 && len(matches) != len(candidates) {
var removeErrors []error var removeErrors []error
for _, match := range matches { for _, match := range matches {
@@ -728,6 +863,7 @@ func (s *script) pruneOldBackups() error {
} }
} }
if len(removeErrors) != 0 { if len(removeErrors) != 0 {
s.stats.Storages.Local.PruneErrors = uint(len(removeErrors))
return fmt.Errorf( return fmt.Errorf(
"pruneOldBackups: %d error(s) deleting local files, starting with: %w", "pruneOldBackups: %d error(s) deleting local files, starting with: %w",
len(removeErrors), len(removeErrors),
@@ -736,8 +872,8 @@ func (s *script) pruneOldBackups() error {
} }
s.logger.Infof( s.logger.Infof(
"Pruned %d out of %d local backup(s) as their age exceeded the configured retention period of %d days.", "Pruned %d out of %d local backup(s) as their age exceeded the configured retention period of %d days.",
len(matches), s.stats.Storages.Local.Pruned,
len(candidates), s.stats.Storages.Local.Total,
s.c.BackupRetentionDays, s.c.BackupRetentionDays,
) )
} else if len(matches) != 0 && len(matches) == len(candidates) { } else if len(matches) != 0 && len(matches) == len(candidates) {
@@ -855,6 +991,26 @@ func join(errs ...error) error {
return errors.New("[" + strings.Join(msgs, ", ") + "]") return errors.New("[" + strings.Join(msgs, ", ") + "]")
} }
// formatBytes converts an amount of bytes in a human-readable representation
// the decimal parameter specifies if using powers of 1000 (decimal) or powers of 1024 (binary)
func formatBytes(b uint64, decimal bool) string {
unit := uint64(1024)
format := "%.1f %ciB"
if decimal {
unit = uint64(1000)
format = "%.1f %cB"
}
if b < unit {
return fmt.Sprintf("%d B", b)
}
div, exp := unit, 0
for n := b / unit; n >= unit; n /= unit {
div *= unit
exp++
}
return fmt.Sprintf(format, float64(b)/float64(div), "kMGTPE"[exp])
}
// buffer takes an io.Writer and returns a wrapped version of the // buffer takes an io.Writer and returns a wrapped version of the
// writer that writes to both the original target as well as the returned buffer // writer that writes to both the original target as well as the returned buffer
func buffer(w io.Writer) (io.Writer, *bytes.Buffer) { func buffer(w io.Writer) (io.Writer, *bytes.Buffer) {

View File

@@ -0,0 +1,26 @@
{{ define "title_failure" -}}
Failure running docker-volume-backup at {{ .Stats.StartTime | formatTime }}
{{- end }}
{{ define "body_failure" -}}
Running docker-volume-backup failed with error: {{ .Error }}
Log output of the failed run was:
{{ .Stats.LogOutput }}
{{- end }}
{{ define "title_success" -}}
Success running docker-volume-backup at {{ .Stats.StartTime | formatTime }}
{{- end }}
{{ define "body_success" -}}
Running docker-volume-backup succeeded.
Log output was:
{{ .Stats.LogOutput }}
{{- end }}

View File

@@ -0,0 +1,38 @@
# Notification templates reference
In order to customize title and body of notifications you'll have to write a [go template](https://pkg.go.dev/text/template) and mount it inside the `/etc/dockervolumebackup/notifications.d/` directory.
Configuration, data about the backup run and helper functions will be passed to this template, this page documents them fully.
## Data
Here is a list of all data passed to the template:
* `Config`: this object holds the configuration that has been passed to the script. The field names are the name of the recognized environment variables converted in PascalCase. (e.g. `BACKUP_STOP_CONTAINER_LABEL` becomes `BackupStopContainerLabel`)
* `Error`: the error that made the backup fail. Only available in the `title_failure` and `body_failure` templates
* `Stats`: objects that holds stats regarding script execution. In case of an unsuccessful run, some information may not be available.
* `StartTime`: time when the script started execution
* `EndTime`: time when the backup has completed successfully (after pruning)
* `TookTime`: amount of time it took for the backup to run. (equal to `EndTime - StartTime`)
* `LogOutput`: full log of the application
* `Containers`: object containing stats about the docker containers
* `All`: total number of containers
* `ToStop`: number of containers matched by the stop rule
* `Stopped`: number of containers successfully stopped
* `StopErrors`: number of containers that were unable to be stopped (equal to `ToStop - Stopped`)
* `BackupFile`: object containing information about the backup file
* `Name`: name of the backup file (e.g. `backup-2022-02-11T01-00-00.tar.gz`)
* `FullPath`: full path of the backup file (e.g. `/archive/backup-2022-02-11T01-00-00.tar.gz`)
* `Size`: size in bytes of the backup file
* `Storages`: object that holds stats about each storage
* `Local`, `S3` or `WebDAV`:
* `Total`: total number of backup files
* `Pruned`: number of backup files that were deleted due to pruning rule
* `PruneErrors`: number of backup files that were unable to be pruned
## Functions
Some formatting functions are also available:
* `formatTime`: formats a time object using [RFC3339](https://datatracker.ietf.org/doc/html/rfc3339) format (e.g. `2022-02-11T01:00:00Z`)
* `formatBytesBin`: formats an amount of bytes using powers of 1024 (e.g. `7055258` bytes will be `6.7 MiB`)
* `formatBytesDec`: formats an amount of bytes using powers of 1000 (e.g. `7055258` bytes will be `7.1 MB`)

2
go.mod
View File

@@ -8,7 +8,7 @@ require (
github.com/gofrs/flock v0.8.1 github.com/gofrs/flock v0.8.1
github.com/kelseyhightower/envconfig v1.4.0 github.com/kelseyhightower/envconfig v1.4.0
github.com/leekchan/timeutil v0.0.0-20150802142658-28917288c48d github.com/leekchan/timeutil v0.0.0-20150802142658-28917288c48d
github.com/m90/targz v0.0.0-20211229090208-2f22c2d9278e github.com/m90/targz v0.0.0-20220208141135-d3baeef59a97
github.com/minio/minio-go/v7 v7.0.16 github.com/minio/minio-go/v7 v7.0.16
github.com/otiai10/copy v1.7.0 github.com/otiai10/copy v1.7.0
github.com/sirupsen/logrus v1.8.1 github.com/sirupsen/logrus v1.8.1

2
go.sum
View File

@@ -452,6 +452,8 @@ github.com/leekchan/timeutil v0.0.0-20150802142658-28917288c48d/go.mod h1:hO90vC
github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII= github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
github.com/m90/targz v0.0.0-20211229090208-2f22c2d9278e h1:Kzm2zfxS40RUGD5UVtVtOo9RT5TtGoNJnmWORtCEaxM= github.com/m90/targz v0.0.0-20211229090208-2f22c2d9278e h1:Kzm2zfxS40RUGD5UVtVtOo9RT5TtGoNJnmWORtCEaxM=
github.com/m90/targz v0.0.0-20211229090208-2f22c2d9278e/go.mod h1:YZK3bSO/oVlk9G+v00BxgzxW2Us4p/R4ysHOBjk0fJI= github.com/m90/targz v0.0.0-20211229090208-2f22c2d9278e/go.mod h1:YZK3bSO/oVlk9G+v00BxgzxW2Us4p/R4ysHOBjk0fJI=
github.com/m90/targz v0.0.0-20220208141135-d3baeef59a97 h1:Uc/WzUKI/zvhkqIzk5TyaPE6AY1SD1DWGc7RV7cky4s=
github.com/m90/targz v0.0.0-20220208141135-d3baeef59a97/go.mod h1:YZK3bSO/oVlk9G+v00BxgzxW2Us4p/R4ysHOBjk0fJI=
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/mailru/easyjson v0.0.0-20190403194419-1ea4449da983/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190403194419-1ea4449da983/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=

View File

@@ -23,6 +23,7 @@ docker run --rm -it \
echo "[TEST:PASS] Found relevant files in untared backup." echo "[TEST:PASS] Found relevant files in untared backup."
sleep 5
if [ "$(docker ps -q | wc -l)" != "5" ]; then if [ "$(docker ps -q | wc -l)" != "5" ]; then
echo "[TEST:FAIL] Expected all containers to be running post backup, instead seen:" echo "[TEST:FAIL] Expected all containers to be running post backup, instead seen:"
docker ps -a docker ps -a