Introduction

rustic is a fast and secure backup program. In the following sections, we will present typical workflows, starting with installing, preparing a new repository, and making the first backup.

Note: Parts of this documentation are shamelessly copied from the restic documentation and then adapted to rustic as most workflows work in rustic exactly like restic. See also the restic documentation for more information about restic.

Contact

You can ask questions in the Discussions or have a look at the FAQ.

ContactWhere?
Issue TrackerGitHub Issues
DiscordDiscord
DiscussionsGitHub Discussions
Last change: 2024-12-17, commit: cb15d19

Contributing

Thank you for your interest in contributing to the rustic ecosystem!

We appreciate your help in making this project better.

Table of Contents

Code of Conduct

Please review and abide by the general Rust Community Code of Conduct when contributing to this project. In the future, we might create our own Code of Conduct and supplement it at this location.

How to Contribute

Reporting Bugs

If you find a bug, please open an issue on GitHub and provide as much detail as possible. Include steps to reproduce the bug and the expected behavior.

Issue and Pull Request labels

Our Issues and Pull Request labels follow the official Rust style:

A - Area
C - Category
D - Diagnostic
E - Call for participation
F - Feature
I - Issue e.g. I-crash
M - Meta
O - Operating systems
P - priorities e.g. P-{low, medium, high, critical}
PG - Project Group
perf - Performance
S - Status e.g. S-{blocked, experimental, inactive}
T - Team relevancy
WG - Working group

Suggesting Enhancements

If you have an idea for an enhancement or a new feature, we’d love to hear it! Open an issue on GitHub and describe your suggestion in detail.

Developer’s documentation

For more information about developing around rustic, see the developer’s documentation.

License

By contributing to rustic or any crates contained in this repository, you agree that your contributions will be licensed under:

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Last change: 2024-12-17, commit: cb15d19

Frequently asked questions

Can I use rustic with my existing restic repositories?

Yes, you can. rustic uses the same repository format as restic, so you can use rustic and restic on the same repository. The only thing you have to take care of is that you don’t run prune with restic and rustic at the same time.

What are the differences between rustic and restic?

  • Written in Rust instead of golang
  • Optimized for small resource usage (in particular memory usage, but also overall CPU usage)
  • Philosophy of development (release new features early)
  • New features (e.g. hot/cold repositories, lock-free pruning)
  • Some commands or options act a bit different or have slightly different syntax

A more detailed comparison can be found in the comparison table.

Why is rustic written in Rust

Rust is a powerful language designed to build reliable and efficient software. This is a very good fit for a backup tool.

How does rustic work with cold storages like AWS Glacier?

If you want to use cold storage, make sure you always specify an extra repository --repo-hot which contains the hot data. This repository acts like a cache for all metadata, i.e. config/key/snapshot/index files and tree packs. As all commands except restore only need to access the metadata, they are fully functional but only need the cold storage to list files while everything else is read from the “hot repo”. Note that the “hot repo” on its own is not a valid rustic repository. The “cold repo”, however, contains all files and is nothing but a standard rustic repository.

If you additionally use a cache, you effectively have a first level cache on your local disc and a second level cache with the “hot repo”. Note that the “hot repo” can be also a remote repo, so hot/cold repositories also work for multiple rustic clients backing up to the same repository.

More details

rustic doesn’t support single repositories/buckets which have objects that can be in different “states” (something like this is not contained in the restic REST protocol; rclone therefore isn’t able to provide this kind of information).

So, to use S3 glacier, you should use 2 buckets: A hot one which is always accessible and a cold one where all object should be transitioned/transferred directly to Glacier when they are stored. The “hot” objects are then in fact stored in both buckets, but this shouldn’t be much overhead as it is only metadata which usually is less than 1% of the repo size. (The side effect is that the cold repo contains a full restic-compatible repository; if you warm up it completely, you can use it a standard repository within rustic and even restic).

TL;DR: I think you have to do the following:

  1. To store data (e.g. init or backup command):
  • Create a regular bucket and use it as repo-hot
  • Create a bucket for cold and configure it to store data directly in Glacier (don’t know if you can configure this in rclone or in AWS)
  1. To retrieve data (e.g. restore):
  • configure warm-up-command such that objects from the cold buckets are restored, see e.g https://docs.aws.amazon.com/AmazonS3/latest/userguide/restoring-objects.html. That is, the config file should look have a line like warm-up-command = 'aws s3api restore-object --bucket BUCKET --key path/data/%id --restore-request '{"Days":25,"GlacierJobParameters":{"Tier":"Standard"}}' ' (may need some adaption)
  • configure a suitable warm-up-wait to ensure that the data is warmed up and can be accessed after that
  • Note: All commands which need to read cold data now will warm this data up and wait for the given time before processing the data
  1. removing cold data (prune) This actually should work out of the box, but note that by default prune doesn’t repack any cold pack file; instead it keeps them until the last blob within is no longer used.
  • to do more repacking, have a look at the repack-cacheable option which does allow to repack cold pack files.
  • Use the keep-pack option if you have some minimum holding duration, i.e. if you would get charged if you remove objects too early. This option ensures that only pack files older than the given time will be removed.

Once you get a working configuration, please share it in the main repository so other users can use it as well!

Are all operations lock free?

Yes, all operations are designed lock-free. This means all commands can run parallel. This is especially true for multiple backup runs and backup runs parallel to prune. However, make sure that each individual backup run won’t take longer as the --keep-delete option (default: 23h) if you run prune parallel to backup - and that you don’t use --instant-delete.

Of course, any read-only command also safely runs parallel to any other command.

Multiple parallel forget or prune runs are designed to work, too. But I have to admit that we have not tested this in detail so far. Because of this and other reasons (like ACLs to set up and scheduling) I would recommend to schedule forget and prune for the whole repository using a single host to do so. However feedback on this with multiple parallel runs is highly appreciated.

There is one general caveat: Error handling is not perfect yet in rustic and we may run into cases where parallel runs may throw errors. E.g. the combination forget/check may lead to check trying to load just-removed snapshots. Or forget/forget may want to remove a just-removed snapshots which then fails. These are however all cases which should not affect the general repository consistency. If you encounter such an error handling problem, please report it in our issue tracker!

How does the lock-free prune work?

Like the prune within restic, rustic decides for each pack whether to keep it, remove it or repack it. Instead of removing packs, it however only marks the packs to remove in a separate index structure. Packs which are marked for removal are checked if they are really not needed and have been marked long enough ago. Depending on these checks they are either finally removed, recovered or kept in the state of being marked for removal.

This two-phase deletion is needed for rustic to work lock-free: If a backup runs parallel to a prune run (or forget --prune), it could be that prune decides that some blobs can be removed, but the parallel backup uses these blobs for the newly generated snapshot.

The time to hold marked packs should be long enough to guarantee that a possibly parallel backup run has finished in between. It can be set by the --keep-delete option and defaults to 23 hours. In any case, packs will be kept marked and only deleted by the next prune run.

Note that there is the option --instant-delete which circumvents this two-phase deletion. Only use this option, if you REALLY KNOW that there is no parallel access to your repo, else you risk losing data!

You said “rustic uses less resources than restic” but I’m observing the opposite

In general rustic uses less resources, but there may be some exceptions. For instance the crypto libraries of Rust and golang both have optimizations for some CPUs. But it might be that your CPU benefits from a golang optimization which is not present in the Rust implementation. If you observe some unexpected resource usage, please don’t hesitate to submit an issue.

File exclusion for Windows Defender

If you are on Windows and you are using Windows Defender, you might want to exclude the rustic.exe binary from being scanned to speed up all operations. This can be done by adding an exclusion in the Windows Security settings. Here is how you can do it on the command line (elevated):

Add-MpPreference -ExclusionProcess "rustic.exe"

Pass environment variables to RCLONE

Environment variables are set by rustic before calling a subcommand, e.g. rclone or commands defined in the repository options. For example, to set the RCLONE_FAST_LIST environment variable you add the following to your <profile>.toml:

[global.env]
RCLONE_FAST_LIST = "true"

Make sure to check the latest config documentation for the correct syntax.

Last change: 2024-12-17, commit: cb15d19

Comparison between rustic and restic

Note that we regularly update this document to compare the latest versions of rustic and restic. Currently, we compare restic 0.17.3 with rustic 0.9.5.

General differences

resticrustic
programming languageGoRust
development philosopyconservative with changesmoving fast, add new features early
test coverage❌ (42% in rustic_core)
returns error code(✅) only 0 or 1; not all commands support it
available as libraryrustic_core

Core features introduced by rustic

rustic’s goal is to implement all functionality/features restic offers - and make some of them even better. It also implements new features which are missing in restic.

This section is an advertisement of the most important features uniquely introduced by rustic. Some have been already adopted by restic.

featureresticrustic
cold storage support❌ (may work in special cases)✅ (full support including warm-up of needed data)
config profile support❌ (wrapper tools available)
lock-free❌ (roadmap: 0.19)✅ (lock-free operations, two-phase pruning)
in-place restore
additional snapshot information(✅) (partly added)✅ (see below for details)
in-repo config✅ (see below for details)
<snapshot>:<path> syntax✅ (most commands)
new command: merge
new command: webdav
diff with local files
backup can use .gitignore❌ (roadmap: 0.19)
backup multiple snapshots at once
check uses existing cache❌ (roadmap: 0.18)
show file history✅ (rustic find --path)
more snapshot filter options✅ (see below for details)
allow to log to file
interactive mode (TUI)
log verbosity-v or --quiet--log-level

Supported storage backends

backendresticrustic
local✅ (built-in)✅ (built-in)
sftp✅ (using external ssh command)✅ (built-in using opendal, windows not supported)
rest✅ (built-in)✅ (built-in)
s3✅ (built-in)✅ (built-in using opendal)
swift✅ (built-in)✅ (built-in using opendal)
b2✅ (built-in)✅ (built-in using opendal)
azure✅ (built-in)✅ (built-in using opendal)
gs✅ (built-in)✅ (built-in using opendal)
dropbox✅ (built-in using opendal)
ftp✅ (built-in using opendal)
gdrive✅ (built-in using opendal)
onedrive✅ (built-in using opendal)
webdav✅ (built-in using opendal)
opendal (other services)✅ (built-in using opendal) (see here)
rclone✅ (via stdin, using external rclone command)✅ (via http on localhost, using external rclone command)

Commands

commandresticrustic
backup
cache
cat
config❌ (no in-repo config)
check
copy
diff
dump
find
forget
generatecompletions
init
key list
key add
key remove
key passwd
list
ls
merge
migrate❌ (not needed; repo version migration via config)
mount✅ (linux only)
prune
recover
repair index
repair packs
repair snapshots
repoinfo
restore
rewrite
self-update
show-config
snapshots
stats❌ (but there is repoinfo)
tag
unlocklock-free
webdav

Information saved in snapshots

informationresticrustic
from repo design info
program version used
summary (size,…)
used command
label
description
delete (protection)

General options

optionresticrustic
--cacert
--cache-dir✅ (or in config profile)
--cleanup-cache
--compression✅ (auto,max,off); needed in every call✅ (-7..22) configure once in in-repo config
--dry-run✅ (or in config profile)
--insecure-no-password✅ (empty passwords work without extra option)
--insecure-tls
--json
--key-hint
--limit-download(✅) (for opendal option trottle)
--limit-upload(✅) (for opendal option trottle)
--log-file✅ (or in config profile)
--no-cache✅ (or in config profile)
--no-extra-verify✅ needed in every call✅ configure once using config
--no-lock✅ all operations are lock-free
--no-progress✅ (or in config profile)
--progress-intervall✅ (via env variable)✅ (or in config profile)
--option✅ as cmd arg or env variable✅ via config profile or env variable
--pack-size✅ fix limit; needed in every call✅ fix or dynamic limit, configure once in in-repo config
--password-command✅ (or in config profile)
--password-file✅ (or in config profile)
--quiet
--repo✅ (or in config profile)
--repo-hot❌ (no cold-storage support)✅ (or in config profile)
--repository-file❌ (use repository in config profile instead)
--retry-locknot needed; lock-free
--tls-client-cert
--use-profile❌ (no config profile support)✅ (or in config profile for recursively using profiles)
--verbose (multiple times)--log-level
--warm-up❌ (no cold-storage support)✅ (or in config profile)

rustic in-repo config options

optionresticrustic
append_only
compression✅ (by --compression)
treepack_size❌ (only all packs: --pack-size)
treepack_growfactor
treepack_size_limit
datapack_size❌ (only all packs: --pack-size)
datapack_growfactor
datapack_size_limit
min_packsize_tolerate_percent❌ (hardcoded 80% for prune --repack-small)
max_packsize_tolerate_percent
extra_verify✅ (default, can be unset using --no-extra-verify)✅ (default)

Snapshot filtering

filterresticrustic (options also in config profile)
by host--host--filter-host
by label--filter-label
by paths--paths--filter-paths
by exact pathlists--filter-paths-exact
by tags--tags--filter-tags
by exact tagists--filter-tags-exact
by date/time--filter-before, filter-after
by size--filter-size
by size added to repo--filter-size-added
custom Rhai--filter-fn (using Rhai)
custom jq syntax--filter-jq

Comparison of important commands

init

optionresticrustic
--copy-chunker-params(not needed, use copy --init)
--from-*(not needed, see copy command)
--hostname❌ (always sets hostname)
--repository-version✅ (use --set-version)
--set-*❌ (no in-repo config support)
--username❌ (always sets username)
--with-created❌ (always sets creation time)

backup

generalresticrustic
allow to create multiple snapshot in single run
allow to backup relative paths(✅) full path in snapshots
optionresticrustic (options also in config profile)
--as-path
--command
--custom-ignorefile
--description
--description-from
--delete-never
--delete-after
--exclude--glob
--exclude-file--glob-file
--exclude-caches❌ (use --exclude-if-present)
--exclude-if-present✅ (+ support for header parsing)✅ (but no header parsing)
--exclude-larger-than
--files-from
--files-from-raw
--files-from-verbatim
--force
--git-ignore❌ (roadmap: 0.19)
--group-by✅ (host/paths/tags)✅ (host/label/paths/tags)
--host
--iexclude--iglob
--iexclude-file--iglob-file
--ignore-ctime
--ignore-inode
--ignore-devid
--init
--label
--no-require-git❌ (no --git-ignore)
--no-scan
--one-file-system
--parent
--read-concurrency❌ (hardcoded)
--skip-if-unchanged--skip-identical-parent
--stdin✅ (use - as backup source)
--stdin-filename
--stdin-from-command
--tag
--time
--with-atime

restore

generalresticrustic
scan and use already existing files
resumable restore
restore hard links
<snapshotID>:<subfolder> syntax
<snapshotID>:<subfolder>/file syntax
dry run support
optionresticrustic
filtering options for latest
--delete
--exclude--glob
--iexclude--iglob
--iinclude--iglob
--include--glob
--no-ownership
--numeric-id
--overwrite❌ (missing functionality: if-newer, never)
--sparse
--target✅ (give target as second CLI argument)
--verify❌ (but diff can be used to verify after)
--verify-existing--overwrite always

dump

generalresticrustic
dump files
dump dirs
allow auto-detection of used format
optionresticrustic
snapshot filtering options for latest
--archive✅ (tar,zip)✅ (tar,targz,zip,content,auto)
--target--file

forget

generalresticrustic
allow to keep all XXX
respect “no delete” options in snapshot
optionresticrustic (options also in config profile)
snapshot filtering options
--keep-last
--keep-minutely
--keep-hourly
--keep-daily
--keep-daily
--keep-weekly
--keep-monthly
--keep-quarter-yearly
--keep-half-yearly
--keep-yearly
--keep-within
--keep-within-minutely
--keep-within-hourly
--keep-within-daily
--keep-within-weekly
--keep-within-monthly
--keep-within-quarter-yearly
--keep-within-half-yearly
--keep-within-yearly
--keep-tag
--usafe-allow-remove-all--keep-none
--compact
--group-by✅ (host/paths/tags)✅ (host/label/paths/tags)
--prune

prune

generalresticrustic
prune plan without reading pack files
prune parallel to backup (two-phase prune)❌ (roadmap: 0.19)
different pack sizes for tree/data packs
resumable prune
(option to) resize packs
optionresticrustic
--fast-repack
--instant-delete✅ (default, no two-phase)
--keep-pack
--keep-delete❌ (no two-phase)
--max-repack-size--max-repack (size/%/unlimited)
--max-unused
--repack-all
--repack-cacheable-only
--repack-small✅ (default behavior; to unset use --no-resize)
--repack-uncompressed
--unsafe-recover-no-free-space--early-delete-index

check

generalresticrustic
check index files
check index vs packs
check snapshot files
(optionally) check pack files
only check given snapshots
cache policycreate temporary (use existing: roadmap 0.18)use existing
check cache integrity
check hot/cold integrity❌ (no cold storage support)
optionresticrustic
--read-data
--read-data-subset
--trust-cache❌ (no cache integrity check)
--with-cache✅ (default behavior)

copy

generalresticrustic
source/target given byCLI optionsin config profile
multiple targets
check for matching chunker parameters
optionresticrustic
snapshot filtering options
--from-*✅ (target is --repository)✅ (source is --repository, target in config profile)
--init❌ (extra run of init --copy-chunker-params)

snapshots

generalresticrustic
summarize identical snapshots (like +3)
show summary information (sizes)
optionresticrustic
snapshot filtering options
--all
--compact
--group-by✅ (host/paths/tags)✅ (host/label/paths/tags)
--latest
--long

ls

optionresticrustic
snapshot filtering options for latest
--glob
--glob-file
--human-readable
--iglob
--iglob-file
--long
--numeric-uid-gid
--summary
--recursive

find

generalresticrustic
group and sort snapshots by date before finding
summarize snapshots with identical result (like +3)
fast searching for given full paths
optionresticrustic
--all❌ (no summarizing)
--blob
--glob✅ (give patterns as args)
--group-by
--ignore-case✅ (--iglob)
--long❌ (default: long output)
--newest✅ use --filter-before
--numeric-uid-gid❌ (default: numeric ids)
--oldest✅ use --filter-after
--pack
--path (filter snapshots)--filter-path
--path (full path to search)
--show-pack-id
--show-misses
--snapshot✅ (give ids as args)
--tag--filter-tags
--tree

diff

generalresticrustic
allow latest
diff with local files
<snapshotID>:<subfolder> syntax
<snapshotID>:<subfolder>/file syntax
optionresticrustic
snapshot filtering options for latest❌ (no latest support)
--glob
--glob-file
--iglob
--iglob-file
--metadata
--no-content
exclude options for local files❌ (no diff with local files)

simple comparison of borg, restic and rustic blogpost.

Last change: 2024-12-17, commit: cb15d19

Breaking Changes

This document lists all user facing breaking changes in rustic and provides guidance on how to migrate from one version to another.

0.9.0

Configuration File

Using String in password-command and warm-up-command

Using a String for password-command or warm-up-command have been re-allowed.

Note: The former version to give the command in an array is no longer supported. Instead of

password-command = ["echo", "password"]

please use either of

password-command = "echo password"
password-command = { command = "echo", args = ["password"] }

Copy Command

Targets for the copy command must now be given by using existing rustic config profile(s).

[copy]
targets = ["full", "rustic"]

Key Naming Conventions

Consistently apply singular and plural naming conventions for keys in the configuration file that accept an array of values. This change affects the following keys:

For [global]:

  • use-profile-> use-profiles in config profile

For [snapshot-filter] and [forget]:

  • filter-host -> filter-hosts in config profile
  • filter-label -> filter-labels in config profile

For [backup]:

  • glob -> globs in config profile
  • iglob -> iglobs in config profile
  • glob-file -> glob-files in config profile
  • iglob-file -> iglob-files in config profile
  • custom-ignore-file -> custom-ignore-files in config profile
  • tag-> tags in config profile
  • source -> sources in config profile
  • [[backup.sources]] -> [[backup.snapshots]] in config profile

For [forget]:

  • keep-tags -> now only array
  • keep-ids -> now only array

Update your configuration file accordingly, changing the key names from singular to plural. For cases, where the key name was plural before, the value must be wrapped in an array.

So, for example

[[backup.sources]]
tag = "important"
source = "~/folder1 ~/folder2"

becomes:

[[backup.snapshots]]
tags = ["important"]
sources = ["~/folder1", "~/folder2"]

Filters being affected are:

filter-hosts = ["host2"] # Default: []
filter-labels = ["label1"] # Default: []

The changes can be also derived from the PR that introduced this change. So you may want to take a look at the PR for more details.

Last change: 2024-12-17, commit: cb15d19

Error Codes

This page lists the error codes that rustic and rustic_core can return. The error codes are grouped by the module that returns them.

rustic_core

  • C001 (Cryptography): Data decryption failed, MAC check failed.
  • C002 (Password): The password that has been entered, seems to be incorrect. No suitable key found for the given password. Please check your password and try again.
  • C003 (Verification): Verification failed: After decrypting and decompressing
Last change: 2024-12-17, commit: cb15d19

C001

Data decryption failed, MAC check failed.

Possible Causes

  • The data has been corrupted.
  • The data has been tampered with.

Solutions

  • Restore the data from a backup.
  • If the data is still available, try to decrypt it again.
Last change: 2024-12-17, commit: cb15d19

C002

The password that has been entered, seems to be incorrect. No suitable key found for the given password. Please check your password and try again.

Possible Causes

  • The password has been entered incorrectly.
  • The password has been changed.
  • A Keyfile has been changed or removed.

Solutions

  • Check the password and try again.
  • If the password has been changed, use the new password.
  • If a Keyfile has been changed or removed, restore the Keyfile or use a different one.
Last change: 2024-12-17, commit: cb15d19

C003

Verification failed: After decrypting and decompressing the data changed! The data may be corrupted.\nPlease check the backend for corruption and try again. You can also try to run rustic check --read-data to check for corruption. This may take a long time.

Possible Causes

  • The data has been corrupted.
  • The backend has a bug.

Solutions

  • Check the backend for corruption.
  • Try running rustic check --read-data to check for corruption.
  • If the error persists, please report the issue to the (backend) developers for further investigation.
Last change: 2024-12-17, commit: cb15d19

Installation

Official Binaries

Stable Releases

cargo-binstall

cargo binstall rustic-rs

Windows

Scoop
scoop install rustic

You can download the latest stable release versions of rustic from the rustic release page. These builds are considered stable and releases are made regularly in a controlled manner.

There’s both pre-compiled binaries for different platforms as well as the source code available for download. Just download and run the one matching your system.

Once downloaded, the official binaries can be updated in place using the rustic self-update command (needs rustic 0.3.1 or later):

$ rustic  self-update
Checking target-arch... x86_64-unknown-linux-musl
Checking current version... v0.3.0-dev
Checking latest released version... v0.3.1
New release found! v0.3.0-dev --> v0.3.1
New release is *NOT* compatible

rustic release status:
    * Current exe: "/usr/local/bin/rustic"
    * New exe release: "rustic-v0.3.1-x86_64-unknown-linux-musl.tar.gz"
    * New exe download url: "https://api.github.com/repos/rustic/rustic/releases/assets/75146490"

The new release will be downloaded/extracted and the existing binary will be replaced.
Do you want to continue? [Y/n] Y
Downloading...
[00:00:00] [========================================] 4.29MiB/4.29MiB (0s) Done
Extracting archive... Done
Replacing binary file... Done
Update status: `0.3.1`!

Note: Please be aware that the user executing the rustic self-update command must have the permission to replace the rustic binary.

Unstable Builds

Another option is to use the nightly builds for the main branch, available on the nightly download page. These too are pre-compiled and ready to run, and a new version is built every night from the main branch of various repositories.

From Source

Beware: This installs the latest development version, which might be unstable.

rustic is written in Rust and you need a current Rust version.

In order to build rustic from source, execute the following steps:

Github

cargo install --git https://github.com/rustic-rs/rustic.git rustic-rs

crates.io

You can also directly install the latest crate from crates.io.

cargo install --locked rustic-rs

Cross-compile

You can easily cross-compile rustic for all supported platforms, make sure that the cross-compile toolchain is installed for your target. Then run the build for your chosen target like this

cargo build --target aarch64-unknown-linux-gnu --release
Last change: 2024-12-17, commit: cb15d19

Nightly builds

Nightly builds of rustic’s, rustic_server’s, and rustic_scheduler’s main branch are available here for download.

WARNING: These builds are not guaranteed to be stable, and may contain bugs. Use at your own risk.

Verification

Minisign/Rsign2

Install

  • rsign2 with cargo install rsign2 or
  • minisign with scoop install minisign (on Windows, check other installation instructions here).

Run

rsign verify <filename>.tar.gz \
  -x <filename>.tar.gz.sig \
  -P RWSWSCEJEEacVeCy0va71hlrVtiW8YzMzOyJeso0Bfy/ZXq5OryWi/8T

PGP

Download our public key or copy and paste it from below:

wget https://github.com/rustic-rs/nightly/raw/main/pub/pgp.pub

Check the fingerprint:

12B7166D9FD59124416952E34018C5DE3BF8C081

against the output of: gpg --show-keys <PUBLIC_KEY_FILE>

Import the key with gpg --import <PUBLIC_KEY_FILE>

Verify the signature with gpg --verify <filename>.tar.gz.asc <filename>.tar.gz

The output should say “Good Signature”.

Note: We use the .asc extension for the files because .sig was already taken for supporting minisign used by cargo-binstall.

Status


Download matrix

Platformrusticrustic_schedulerrustic_server
Linux x86_64 / gnu #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
Linux x86_64 / musl (static) #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
Linux i686 / gnu #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
Linux aarch64 / gnu #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
Linux armv7 / raspberry pi #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
MacOS x86_64 #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
MacOS aarch64 #️⃣ 🔑 #️⃣ 🔑 n.a., #6
Windows x86_64 / msvc (exp) #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑
Windows x86_64 / gnu (exp) #️⃣ 🔑 #️⃣ 🔑 #️⃣ 🔑

Key: ⏬ Download | #️⃣ SHA256 checksum | 🔑 Signature

Last change: 2024-12-17, commit: cb15d19

How to install shell completions

All completion files are generated by invoking rustic completions command. So run:

Bash

rustic completions bash > /etc/bash_completion.d/rustic.bash

Fish

rustic completions fish > $HOME/.config/fish/completions/rustic.fish

Powershell

Linux

rustic completions powershell >> ~/.config/powershell/Microsoft.PowerShell_profile.ps1.

Windows

rustic completions powershell >> $HOME\Documents\PowerShell\Microsoft.PowerShell_profile.ps1.

macOs

rustic completions powershell >> ~/.config/powershell/Microsoft.PowerShell_profile.ps1

Zsh

ZSH completions are commonly stored in any directory listed in your $fpath variable. To use these completions, write completions script (_rustic) to one of those directories, or add your own to this list.

This list includes, for example, these directories:

  • /usr/local/share/zsh/site-functions
  • /usr/share/zsh/site-functions

So you can run:

rustic completions zsh > /usr/local/share/zsh/site-functions/_rustic
Last change: 2024-12-17, commit: cb15d19

Getting started

rustic getting started

rustic is a commandline backup tool that provides encrypted, deduplicated backups to various types of repositories.

Below is an example set of steps you can follow. They are similar but not identical to the steps in the screencast above. By the end of these steps, you should have some basic familiarity with rustic, how to set it up, backup, and restore. You will be able to set up and use rustic in a configuration similar to the one presented here.

By default, rustic expects to find a configuration file in the home directory of the user who runs it. For Linux users, that file will almost always be ~/.config/rustic/rustic.toml.

$ cat ~/.config/rustic/rustic.toml
[repository]
repository = "/tmp/repo"
password = "test"

First, we need to initialize the repository, as it doesn’t exist yet:

$ rustic init
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] key e65d9789 successfully added.
[INFO] repository 3f64ae74 successfully created.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601
$ ls /tmp/repo
total 68
drwxr-xr-x 1 jamesvasile jamesvasile    56 Mar 22 12:31 .
drwxrwxrwt 1 root        root  13618 Mar 22 12:31 ..
-rw-r--r-- 1 jamesvasile jamesvasile   155 Mar 22 12:31 config
drwxr-xr-x 1 jamesvasile jamesvasile  1024 Mar 22 12:31 data
drwxr-xr-x 1 jamesvasile jamesvasile     0 Mar 22 12:31 index
drwxr-xr-x 1 jamesvasile jamesvasile   128 Mar 22 12:31 keys
drwxr-xr-x 1 jamesvasile jamesvasile     0 Mar 22 12:31 snapshots

Note that rustic init created the directory for us. Let’s make a test directory to practice backing up and then we can start our first backup.

$ mkdir src
$ echo "test me" > src/test.txt
$ rustic backup src
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601
[00:00:00] reading index...               ████████████████████████████████████████          0/0
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          0/0
[INFO] using no parent
[INFO] starting to backup "src"...
[00:00:00] backing up...                  ████████████████████████████████████████        8 B/8 B   432 B/s  (ETA 0s)
Files:       1 new, 0 changed, 0 unchanged
Dirs:        2 new, 0 changed, 0 unchanged
Added to the repo: 476 B (raw: 529 B)
processed 1 files, 8 B
snapshot 8171d606 successfully saved.
[INFO] backup of "src" done.

Let’s use the rustic snapshots command to see our snapshot:

$ rustic snapshots
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601

snapshots for (host [francium], label [], paths [src])
| ID       | Time                | Host     | Label | Tags | Paths | Files | Dirs | Size |
|----------|---------------------|----------|-------|------|-------|-------|------|------|
| 8171d606 | 2024-03-22 12:39:04 | francium |       |      | src   |     1 |    2 |  8 B |
1 snapshot(s)

total: 1 snapshot(s)

Let’s add a file and backup again.

$ echo "Another test" > src/test2.txt
$ rustic backup src
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601
[00:00:00] reading index...               ████████████████████████████████████████          1/1
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          1/1
[INFO] using parent 8171d606
[INFO] starting to backup "src"...
[00:00:00] backing up...                  ████████████████████████████████████████       21 B/21 B  1.09 KiB/s (ETA 0s)
Files:       1 new, 0 changed, 1 unchanged
Dirs:        0 new, 2 changed, 0 unchanged
Added to the repo: 556 B (raw: 902 B)
processed 2 files, 21 B
snapshot 27fda888 successfully saved.
[INFO] backup of "src" done.

This again was very fast as it only needed to process the added file. But still, it generated a full snapshot:

$ rustic snapshots
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601

snapshots for (host [francium], label [], paths [src])
| ID            | Time                | Host     | Label | Tags | Paths | Files | Dirs | Size |
|---------------|---------------------|----------|-------|------|-------|-------|------|------|
| 8171d606      | 2024-03-22 12:39:04 | francium |       |      | src   |     1 |    2 |  8 B |
| 27fda888      | 2024-03-22 12:55:23 | francium |       |      | src   |     2 |    2 | 21 B |
2 snapshot(s)

total: 2 snapshot(s)

What happens if we do a backup of a directory in which nothing changed?

$ rustic backup src
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601
[00:00:00] reading index...               ████████████████████████████████████████          2/2
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          2/2
[INFO] using parent 27fda888
[INFO] starting to backup "src"...
[00:00:00] backing up...                  ████████████████████████████████████████       21 B/21 B 2.69 KiB/s (ETA 0s)
Files:       0 new, 0 changed, 2 unchanged
Dirs:        0 new, 0 changed, 2 unchanged
Added to the repo: 0 B (raw: 0 B)
processed 2 files, 21 B
snapshot 2627ccdf successfully saved.
[INFO] backup of "src" done.
$ rustic snapshots
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601

snapshots for (host [francium], label [], paths [src])
| ID            | Time                | Host     | Label | Tags | Paths | Files | Dirs | Size |
|---------------|---------------------|----------|-------|------|-------|-------|------|------|
| 8171d606      | 2024-03-22 12:39:04 | francium |       |      | src   |     1 |    2 |  8 B |
| 27fda888 (+1) | 2024-03-22 12:55:23 | francium |       |      | src   |     2 |    2 | 21 B |
3 snapshot(s)

total: 3 snapshot(s)

On this latest backup run, rustic determined that there was no change and created a snapshot with content identical to the previous one. This is indicated by the (+1), which shows how many identical backups exist. Note that the timestamp will match the earliest identical snapshot, not the later ones.

Now we remove the added file:

rm src/test2.txt

But it is still contained in the latest snapshot. Let’s restore from that snapshot.

$ rustic restore latest:src/ src/
[INFO] using config /home/jamesvasile/.config/rustic/rustic.toml
[INFO] repository local:/tmp/repo: password is correct.
[INFO] using cache at /home/jamesvasile/.cache/rustic/3f64ae7425245b259ff98d1dcb6f49035af391f271da7b9ff643fd3f26408601
[00:00:00] reading index...               ████████████████████████████████████████          2/2
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          3/3
[00:00:00] collecting file information...
Files:  1 to restore, 1 unchanged, 0 verified, 0 to modify, 0 additional
Dirs:   0 to restore, 0 to modify, 0 additional
[INFO] total restore size: 13 B
[INFO] using 8 B of existing file contents.
[00:00:00] restoring file contents...     ████████████████████████████████████████       13 B/13 B  6.12 KiB/s (ETA 0s)
[00:00:00] setting metadata...
restore done.
$ cat src/test2.txt
Another test

Note that rustic checks existing contents and only restores what’s needed.

Those are the basics! From here you might want to:

  • edit the configuration file to tune your backups and their location
  • setup a remote backup repository
  • script or schedule your rustic invocations for convenience and reliability
  • test restoration from your snapshots
Last change: 2024-12-17, commit: cb15d19

Init - Preparing a new repository

The place where your backups will be saved is called a “repository”. This chapter explains how to create (“init”) such a repository. The repository can be stored locally, or on some remote server or service. We’ll first cover using a local repository; the remaining sections of this chapter cover all the other options. You can skip to the next chapter once you’ve read the relevant section here.

Note that rustic supports to store configuration in a config profile file in the TOML format which is the preferred way to configure rustic.

For the password, several options exist:

  • Setting the password directly in the config profile file:
[repository]
password = "secret"
  • Setting the environment variable RUSTIC_PASSWORD

  • Specifying the path to a file with the password via the CLI option --password-file, the environment variable RUSTIC_PASSWORD_FILE, or setting in the configuration profile

[repository]
password-file = "/path/to/my/password.txt"
  • Configuring a program to be called when the password is needed via the CLI option --password-command , the environment variable RUSTIC_PASSWORD_COMMAND or setting in the configuration profile
[repository]
password-command = "get_my_password.sh"

If none of these password options is used, rustic will query for a password (followed by a confirmation query if the password is to be set).

If you have configured your repository and password in the config profile, simply run

rustic init

and your repository is created.

The init command has an option called --set-version which can be used to explicitly set the version for the new repository.

The below table shows which rustic version is required to use a certain repository version and shows new features introduced by the repository format.

Repository versionMinimum rustic versionMajor new features
1any version
2>=0.2.0Compression support

Moreover, there are different options which can be set when initializing a repository:

Options to specify the target pack size:

  • --set-treepack-size, --set-datapack-size specify the default target pack size for tree and data pack files. Arguments can given using TODO For example, valid sizes are “4048kiB”, “2MB”, “30MiB”, etc. If not specified, the default is 4 MiB for tree packs and 32 MiB for data packs.

  • --set-treepack-growfactor, --set-datapack-growfactor specify how much the target pack size should be increased per square root of the total pack size in bytes of the given type. This equals to 32kiB per square root of the total pack size in GiB.

Note that larger pack sizes have advantages, especially for large repository or remote repositories. They lead to less packs in the repository and transfer larger datasets to the repository which can increase the throughput. But there are also disadvantages. rustic keeps the whole pack in memory before writing it to the backend. As writes are parallelized, multiple packs are kept. So larger pack sizes increase the memory usage of the backup command. Moreover larger pack sizes lead to increased repack rates during prune or forget --prune.

Last change: 2024-12-17, commit: cb15d19

Configuration file

rustic supports configuration profile files in the TOML format. This allows you to use rustic on the command line without the need to specify required or wanted options. When using config profiles, rustic commands can be simply called as rustic backup, rustic snapshots or rustic forget and all essential options are read from the profile. So, using a config profile is the preferred way to use rustic.

In this documentation, when giving example commands, we assume that a suitable config profile is present and omit options which are usually set there.

Important: For always up-to-date information, please make sure to check the in-repository documentation for the config files available here.

The configuration profile files are searched in the following locations:

By default, rustic uses the file rustic.toml. This can be overwritten by the -P <PROFILE> option which tells rustic to search for a <PROFILE>.toml configuration file. For example, if you have a local.toml configuration for backing up to a local dir and a remote.toml configuration for a remote storage, you can use rustic -P local <COMMAND> and rustic -P remote <COMMAND>, respectively to switch between you two backup configurations.

Note that options in the config profile file can always be overwritten by ENV variables.

In the configuration profile file, you can specify all global and repository-specific options as well as options/sources for the backup command and forget options. Using a config file like

# rustic config file to backup /home and /etc to a local repository

[repository]
repository = "/backup/rustic"
password-file = "/root/key-rustic"
no-cache = true # no cache needed for local repository

[forget]
keep-daily = 14
keep-weekly = 5

[backup]
exclude-if-present = [".nobackup", "CACHEDIR.TAG"]
glob-file = ["/root/rustic-local.glob"]

[[backup.sources]]
source = "/home"
git-ignore = true

[[backup.sources]]
source = "/etc"

allows you to use rustic backup and rustic forget --prune in your regularly backup/cleanup scripts.

For more config file examples check the config here

Configuration profile inheritance

rustic allows to read options from multiple configuration profiles and combines them. This can be seen as inheritance of options from a parent profile. In fact the -P command line option can be given multiple times and all config profiles will be read and used.

For example, using repo.toml:

[repository]
repository = "/backup/rustic"
password-file = "/root/key-rustic"
no-cache = true # no cache needed for local repository

and retention.toml:

[forget]
keep-daily = 14
keep-weekly = 5

we can use rustic -P repo -P retention forget and get repository information from the first and retention information from the second profile. This can be used to stucture your configuration if you need to handle multiple repositories or multiple use-cases.

Morever, there is also the possibiltiy to (recursively) read other profiles from within a profile by using use-profiles, e.g.:

[global]
use-profiles = ["repo", "retention"]
Last change: 2024-12-17, commit: cb15d19

Local backend

In order to create a repository at /srv/rustic-repo, run the following command and enter the same password twice:

$ rustic init -r /srv/rustic-repo
enter password for new repository:
created rustic repository 085b3c76b9 at /srv/rustic-repo

Warning: Remembering your password is important! If you lose it, you won’t be able to access data stored in the repository.

Last change: 2024-12-17, commit: cb15d19

REST Server

In order to backup data to the remote server via HTTP or HTTPS protocol, you must first set up a remote REST server instance. Once the server is configured, accessing it is achieved by changing the URL scheme like this:

[repository]
repository = "rest:http://host:8000/"

Depending on your REST server setup, you can use HTTPS protocol, password protection, multiple repositories or any combination of those features. The TCP/IP port is also configurable. Here are some more examples:

repository = "rest:http://host:8000/"
repository = "rest:http://user:pass@host:8000/"
repository = "rest:http://user:pass@host:8000/my_backup_repo"

If you use TLS, rustic will use the system’s CA certificates to verify the server certificate. When the verification fails, rustic refuses to proceed and exits with an error. If you have your own self-signed certificate, or a custom CA certificate should be used for verification, you can pass rustic the certificate filename via the --cacert option. It will then verify that the server’s certificate is contained in the file passed to this option, or signed by a CA certificate in the file. In this case, the system CA certificates are not considered at all.

REST server uses exactly the same directory structure as local backend, so you should be able to access it both locally and via HTTP, even simultaneously.

Last change: 2024-12-17, commit: cb15d19

Supported Services

rustic integrates opendal, a data access layer for many different services.

The following services are enabled in rustic:

ServiceDescriptionWindowsLinuxMacOS
azblobAzure Blob Storage
azdlsAzure Data Lake Storage
azfileAzure File Storage
b2Backblaze B2
cosTencent Cloud Object Storage
dropboxDropbox
fsLocal Filesystem
gcsGoogle Cloud Storage
gdriveGoogle Drive
ghacGitHub Actions Cache
httpHTTP
ipmfsIPFS based on IPFS MFS API
memoryIn-memory storage
obsHuawei Cloud Object Storage
onedriveOneDrive
ossAliyun Object Storage Service
sftpSFTP
swiftOpenStack Swift
s3Amazon S3
webdavWebDAV
webhdfsWebHDFS
yandex-diskYandex Disk

Note: FTP is temporarily deactivated, due to build issues.

Configuration

The configuration for the services is done via the rustic configuration file.

For example, to configure the Amazon S3 service, you would add the following to the configuration file:

[repository]
repository = "opendal:s3"
password = "password"

# Other options can be given here - note that opendal also support reading config from env
# files or AWS config dirs, see the opendal S3 docs for more information
# https://opendal.apache.org/docs/rust/opendal/services/struct.S3.html
[repository.options]
access_key_id = "xxx" # this can be ommited, when AWS config is used
secret_access_key = "xxx" # this can be ommited, when AWS config is used
bucket = "bucket_name"
root = "/path/to/repo"

opendal is used to access the services. The s3 postfix is used to specify the Amazon S3 service. For other services, the postfix is replaced with the respective service postfix.

You can find more service templates in the rustic repository.

To see the service-dependent options to be set in [repository.options], please refer to the opendal service documentation.

Last change: 2024-12-17, commit: cb15d19

Other Services via rclone

The program rclone can be used to access many other different services and store data there. First, you need to install and configure rclone. The general backend specification format is rclone:<remote>:<path>, the <remote>:<path> component will be directly passed to rclone. When you configure a remote named foo, you can then call rustic as follows to initiate a new repository in the path bar in the repo:

rustic -r rclone:foo:bar init

rustic takes care of starting and stopping rclone.

As a more concrete example, suppose you have configured a remote named b2prod for Backblaze B2 with rclone, with a bucket called yggdrasil. You can then use rclone to list files in the bucket like this:

rclone ls b2prod:yggdrasil

In order to create a new repository in the root directory of the bucket, call rustic like this:

rustic -r rclone:b2prod:yggdrasil init

If you want to use the path foo/bar/baz in the bucket instead, pass this to rustic:

rustic -r rclone:b2prod:yggdrasil/foo/bar/baz init

Listing the files of an empty repository directly with rclone should return a listing similar to the following:

$ rclone ls b2prod:yggdrasil/foo/bar/baz
    155 bar/baz/config
    448 bar/baz/keys/4bf9c78049de689d73a56ed0546f83b8416795295cda12ec7fb9465af3900b44

Rclone can be configured with environment variables prefixed by RCLONE_, so for instance configuring a bandwidth limit for rclone can be achieved by setting the RCLONE_BWLIMIT environment variable:

export RCLONE_BWLIMIT=1M

As rustic allows to set environment variables in the config profile, you can alternatively use:

[global.env]
RCLONE_BWLIMIT = "1M"

For debugging rclone, you can set the environment variable RCLONE_VERBOSE=2.

Last change: 2024-12-17, commit: cb15d19

Cold storage

rustic supports to store the repository in a so-called cold storage. These are storages which are designed for long-term storage and offer usually cheap storage for the price of retarded or expensive access. Examples are Amazon S3 Glacier or OVH Cloud Archive.

To use a cold storage and not access any data in the storage for every-day operations, rustic needs an extra repository to store hot data. This repository can be specified by the hot-repo option or the RUSTIC_REPO_HOT environmental variable, e.g.:

[repository]
repository = "rclone:foo:cold-repo"
repo-hot = "rclone:foo:hot-repo"

or, as CLI-arguments

rustic -r rclone:foo:cold-repo --repo-hot rclone:foo:hot-repo init

In this example in the repository rclone:foo:cold-repo all data is saved. In the repository rclone:foo:hot-repo only hot data is saved, i.e. this is not a complete repository.

Note that you can also specify repository options individually for hot and cold repository, e.g.:

[repository]
repository = "opendal:s3"
repo-hot = "opendal:s3"

# options for both parts
[repository.options]
access_key_id = "xxx"
secret_access_key = "yyy"

# options only for hot part
[repository.options-hot]
bucket = "bucket_name_hot"
root = "/path/to/repo"

# options only cold part
[repository.options-cold]
bucket = "bucket_name_cold"
root = "/path/to/repo"
default_storage_class = "DEEP_ARCHIVE"

Warning: You have to specify both the cold repository and the hot repository in the init command and all other commands which access and work with the repository. Best, you use a config profile to set both within.

Last change: 2024-12-17, commit: cb15d19

Backup - Backing up data

Backing up your data is important. This guide will show you how to backup your data. And what else you can do with rustic.

Last change: 2024-12-17, commit: cb15d19

Creating snapshots

Now we’re ready to backup some data. The contents of a directory at a specific point in time is called a “snapshot” in rustic. Run the following command and enter the repository password you chose above again:

$ rustic --log-level debug backup ~/work
open repository
enter password for repository:
password is correct
lock repository
load index files
start scan
start backup
scan finished in 1.837s
processed 1.720 GiB in 0:12
Files:        5307 new,     0 changed,     0 unmodified
Dirs:         1867 new,     0 changed,     0 unmodified
Added:      1.200 GiB
snapshot 40dc1520 saved

As you can see, rustic created a backup of the directory and was pretty fast! The specific snapshot just created is identified by a sequence of hexadecimal characters, 40dc1520 in this case.

You can see that rustic tells us it processed 1.720 GiB of data, this is the size of the files and directories in ~/work on the local file system. It also tells us that only 1.200 GiB was added to the repository. This means that some of the data was duplicate and rustic was able to efficiently reduce it.

If you don’t pass the --log-level option, rustic will print less data. You’ll still get a nice live status display. Be aware that the live status shows the processed files and not the transferred data. Transferred volume might be lower (due to de-duplication) or higher.

If you run the backup command again, rustic will create another snapshot of your data, but this time it’s even faster and no new data was added to the repository (since all data is already there). This is de-duplication at work!

$ rustic --log-level debug backup ~/work
open repository
enter password for repository:
password is correct
lock repository
load index files
using parent snapshot d875ae93
start scan
start backup
scan finished in 1.881s
processed 1.720 GiB in 0:03
Files:           0 new,     0 changed,  5307 unmodified
Dirs:            0 new,     0 changed,  1867 unmodified
Added:      0 B
snapshot 79766175 saved

You can even backup individual files in the same repository (not passing --log-level means less output):

$ rustic backup ~/work.txt
enter password for repository:
password is correct
snapshot 249d0210 saved

Now is a good time to run rustic check to verify that all data is properly stored in the repository. You should run this command regularly to make sure the internal structure of the repository is free of errors.

Last change: 2024-12-17, commit: cb15d19

File change detection

When rustic encounters a file that has already been backed up, whether in the current backup or a previous one, it makes sure the file’s contents are only stored once in the repository. To do so, it normally has to scan the entire contents of every file. Because this can be very expensive, rustic also uses a change detection rule based on file metadata to determine whether a file is likely unchanged since a previous backup. If it is, the file is not scanned again.

Change detection is only performed for regular files (not special files, symlinks or directories) that have the exact same path as they did in a previous backup of the same location. If a file or one of its containing directories was renamed, it is considered a different file and its entire contents will be scanned again.

Metadata changes (permissions, ownership, etc.) are always included in the backup, even if file contents are considered unchanged.

On Unix (including Linux and Mac), given that a file lives at the same location as a file in a previous backup, the following file metadata attributes have to match for its contents to be presumed unchanged:

  • Modification timestamp (mtime).
  • Metadata change timestamp (ctime).
  • File size.
  • Inode number (internal number used to reference a file in a filesystem).

The reason for requiring both mtime and ctime to match is that Unix programs can freely change mtime (and some do). In such cases, a ctime change may be the only hint that a file did change.

The following rustic backup command line flags modify the change detection rules:

  • --force: turn off change detection and rescan all files.
  • --ignore-ctime: require mtime to match, but allow ctime to differ.
  • --ignore-inode: require mtime to match, but allow inode number and ctime to differ.

The option --ignore-inode exists to support FUSE-based filesystems and pCloud, which do not assign stable inodes to files.

Note that the device id of the containing mount point is never taken into account. Device numbers are not stable for removable devices and ZFS snapshots. If you want to force a re-scan in such a case, you can change the mountpoint.

Last change: 2024-12-17, commit: cb15d19

Dry Run

You can perform a backup in dry run mode to see what would happen without modifying the repo.

  • --dry-run/-n Report what would be done, without writing to the repository
Last change: 2024-12-17, commit: cb15d19

Excluding Files

You can exclude folders and files by specifying exclude patterns, currently the exclude options are:

  • --git-ignore Respect .gitignore files and exclude paths/files not handled by git.
  • --glob include/exclude files and dirs based on given glob patterns
  • --iglob Same as --glob but ignores the case of paths
  • --glob-file Specified one or more times to exclude items listed in a given file
  • --iglob-file Same as --glob-file but ignores cases like in --iglob
  • --exclude-if-present foo Specified one or more times to exclude a folder’s content if it contains a file called foo. For example, to exclude cache dirs, specify --exclude-if-present CACHEDIR.TAG.
  • --exclude-larger-than size Specified once to excludes files larger than the given size

Please see rustic help backup for more specific information about each exclude option.

Let’s say we have a file called glob.txt with the following content:

# exclude go-files
!*.go
# exclude foo/x/y/z/bar foo/x/bar foo/bar
!foo/**/bar

It can be used like this:

rustic backup ~/work --glob="!*.c" --glob-file=glob.txt

This instructs rustic to exclude files matching the following criteria:

  • All files matching *.c (parameter --glob)
  • All files matching *.go (second line in glob.txt)
  • All files and sub-directories named bar which reside somewhere below a directory called foo (fourth line in glob.txt)

By specifying the option --one-file-system you can instruct rustic to only backup files from the file systems the initially specified files or directories reside on. In other words, it will prevent rustic from crossing filesystem boundaries and subvolumes when performing a backup.

For example, if you backup / with this option and you have external media mounted under /media/usb then rustic will not back up /media/usb at all because this is a different filesystem than /. Virtual filesystems such as /proc are also considered different and thereby excluded when using --one-file-system:

rustic -r /srv/rustic-repo backup --one-file-system /

Please note that this does not prevent you from specifying multiple filesystems on the command line, e.g:

rustic backup --one-file-system / /media/usb

will back up both the / and /media/usb filesystems, but will not include other filesystems like /sys and /proc.

Note: --one-file-system is currently unsupported on Windows, and will cause the backup to immediately fail with an error.

Files larger than a given size can be excluded using the --exclude-larger-than option:

rustic backup ~/work --exclude-larger-than 1M

This excludes files in ~/work which are larger than 1 MiB from the backup.

The default unit for the size value is bytes, so e.g. --exclude-larger-than 2048 would exclude files larger than 2048 bytes (2 KiB). To specify other units, suffix the size value with one of k/K for KiB (1024 bytes), m/M for MiB (1024^2 bytes), g/G for GiB (1024^3 bytes) and t/T for TiB (1024^4 bytes), e.g. 1k, 10K, 20m, 20M, 30g, 30G, 2t or 2T).

Last change: 2024-12-17, commit: cb15d19

Comparing Snapshots

rustic has a diff command which shows the difference between two snapshots or a snapshot and a local path/dir

$ rustic diff 5845b002 2ab627a6
password is correct
comparing snapshot ea657ce5 to 2ab627a6:

C   /rustic/cmd_diff.go
+   /rustic/foo
C   /rustic/rustic
Last change: 2024-12-17, commit: cb15d19

Backup special items and metadata

Symlinks are archived as symlinks, rustic does not follow them. When you restore, you get the same symlink again, with the same link target and the same timestamps.

If there is a bind-mount below a directory that is to be saved, rustic descends into it.

Device files are saved and restored as device files. This means that e.g. /dev/sda is archived as a block device file and restored as such. This also means that the content of the corresponding disk is not read, at least not from the device file.

By default, rustic does not save the access time (atime) for any files or other items, since it is not possible to reliably disable updating the access time by rustic itself. This means that for each new backup a lot of metadata is written, and the next backup needs to write new metadata again. If you really want to save the access time for files and directories, you can pass the --with-atime option to the backup command.

Note that rustic does not back up some metadata associated with files. Of particular note are::

  • file creation date on Unix platforms
  • inode flags on Unix platforms
  • xattr information
Last change: 2024-12-17, commit: cb15d19

Reading data from StdIn

Sometimes it can be nice to directly save the output of a program, e.g. mysqldump so that the SQL can later be restored. rustic supports this mode of operation, just supply - as backup source to the backup command like this:

set -o pipefail
mysqldump [...] | rustic backup -

This creates a new snapshot of the output of mysqldump. You can then use e.g. the fuse mounting option (see below) to mount the repository and read the file.

By default, the file name stdin is used, a different name can be specified with --stdin-filename, e.g. like this:

mysqldump [...] | rustic --stdin-filename production.sql -

The option pipefail is highly recommended so that a non-zero exit code from one of the programs in the pipe (e.g. mysqldump here) makes the whole chain return a non-zero exit code. Refer to the Use the Unofficial Bash Strict Mode <http://redsymbol.net/articles/unofficial-bash-strict-mode/>__ for more details on this.

Last change: 2024-12-17, commit: cb15d19

Tags for backup

Snapshots can have one or more tags, short strings which add identifying information. Just specify the tags for a snapshot one by one with --tag:

$ rustic backup --tag projectX --tag foo --tag bar ~/work
[...]

The tags can later be used to keep (or forget) snapshots with the forget command. The command tag can be used to modify tags on an existing snapshot.

Last change: 2024-12-17, commit: cb15d19

Scheduling backups

rustic does not have a built-in way of scheduling backups, as it’s a tool that runs when executed rather than a daemon. There are plenty of different ways to schedule backup runs on various different platforms, e.g. systemd and cron on Linux/BSD and Task Scheduler in Windows, depending on one’s needs and requirements. When scheduling rustic to run recurringly, please make sure to detect already running instances before starting the backup.

Last change: 2024-12-17, commit: cb15d19

Space requirements

rustic currently assumes that your backup repository has sufficient space for the backup operation you are about to perform. This is a realistic assumption for many cloud providers, but may not be true when backing up to local disks.

Should you run out of space during the middle of a backup, there will be some additional data in the repository, but the snapshot will never be created as it would only be written at the very (successful) end of the backup operation. Previous snapshots will still be there and will still work.

Last change: 2024-12-17, commit: cb15d19

Environment Variables

Important: For always up-to-date information, please make sure to check the in-repository documentation for the config files available here.

In addition to command-line options, rustic supports passing various options in environment variables. The following lists these environment variables:

RUSTIC_REPOSITORY                   Location of repository (replaces -r)
RUSTIC_REPO_HOT                     Location of hot repository (replaces -repo-hot)
RUSTIC_PASSWORD                     The actual password for the repository (replaces --password)
RUSTIC_PASSWORD_FILE                Location of password file (replaces --password-file)
RUSTIC_PASSWORD_COMMAND             Command printing the password for the repository to stdout (replaces --password-command)
RUSTIC_CACHE_DIR                    Location of the cache directory (replaces --cache-dir)
RUSTIC_NO_CACHE                     Use no cache (replaces --no-cache)

rustic may execute rclone (for rclone backends) which may respond to further environment variables and configuration files.

Last change: 2024-12-17, commit: cb15d19

Miscellaneous - Working with repositories

A repository is a storage location for all of your snapshots.

The repository is created with the init command:

$ rustic init
enter password for new repository:
enter password again:
created rustic repository 7a8c3b2a0c at /srv/rustic-repo
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

The repository is now ready for use.

Note: You can use latest to select the last snapshot for a command. For example, rustic snapshots latest will show the last snapshot.

Last change: 2024-12-17, commit: cb15d19

Listing snapshots

To list all snapshots in the repository, use the snapshots command:

$ rustic snapshots
enter password for repository:
ID        Date                 Host    Tags   Directory
----------------------------------------------------------------------
40dc1520  2015-05-08 21:38:30  kasimir        /home/user/work
79766175  2015-05-08 21:40:19  kasimir        /home/user/work
bdbd3439  2015-05-08 21:45:17  luigi          /home/art
590c8fc8  2015-05-08 21:47:38  kazik          /srv
9f0bc19e  2015-05-08 21:46:11  luigi          /srv

You can filter the listing by directory path:

$ rustic snapshots --filter-paths="/srv"
enter password for repository:
ID        Date                 Host    Tags   Directory
----------------------------------------------------------------------
590c8fc8  2015-05-08 21:47:38  kazik          /srv
9f0bc19e  2015-05-08 21:46:11  luigi          /srv

Or filter by host:

$ rustic snapshots --filter-host luigi
enter password for repository:
ID        Date                 Host    Tags   Directory
----------------------------------------------------------------------
bdbd3439  2015-05-08 21:45:17  luigi          /home/art
9f0bc19e  2015-05-08 21:46:11  luigi          /srv

Combining filters is also possible.

Furthermore you can group the output by the same filters (host, paths, tags):

$ rustic snapshots --group-by host

enter password for repository:
snapshots for (host [kasimir])
ID        Date                 Host    Tags   Directory
----------------------------------------------------------------------
40dc1520  2015-05-08 21:38:30  kasimir        /home/user/work
79766175  2015-05-08 21:40:19  kasimir        /home/user/work
2 snapshots
snapshots for (host [luigi])
ID        Date                 Host    Tags   Directory
----------------------------------------------------------------------
bdbd3439  2015-05-08 21:45:17  luigi          /home/art
9f0bc19e  2015-05-08 21:46:11  luigi          /srv
2 snapshots
snapshots for (host [kazik])
ID        Date                 Host    Tags   Directory
----------------------------------------------------------------------
590c8fc8  2015-05-08 21:47:38  kazik          /srv
1 snapshots
Last change: 2024-12-17, commit: cb15d19

Copying snapshots between repositories

In case you want to transfer snapshots between two repositories, for example from a local to a remote repository, you can use the copy command:

$ rustic copy --target target-profile
repository d6504c63 opened successfully, password is correct
repository 3dd0878c opened successfully, password is correct

snapshot 410b18a2 of [/home/user/work] at 2020-06-09 23:15:57.305305 +0200 CEST)
    copy started, this may take a while...
snapshot 7a746a07 saved

snapshot 4e5d5487 of [/home/user/work] at 2020-05-01 22:44:07.012113 +0200 CEST)
skipping snapshot 4e5d5487, was already copied to snapshot 50eb62b7

The example command copies all snapshots from the source repository /srv/rustic-repo to the destination repository /srv/rustic-repo-copy. Snapshots which have previously been copied between repositories will be skipped by later copy runs.

Important: This process will have to both download (read) and upload (write) the entire snapshot(s) due to the different encryption keys used in the source and destination repository. This may incur higher bandwidth usage and costs than expected during normal backup runs.

Copying to a repository with different chunker parameters

Instead of pre-initializing the repos you want to copy snapshots into, just use rustic copy --init for the first run - this will set the correct chunker parameters.

Note: rustic currently refuses to copy to a repository with different chunker parameters. If you want to copy to a repository with different chunker parameters, you have to set the chunker parameters of the target repository to the same values as the source repository.

Last change: 2024-12-17, commit: cb15d19

Filtering snapshots

rustic allows to filter snapshots by certain criteria. This can be useful when listing snapshots, copying snapshots between repositories.

Filters can by given in the config profile

filter-host = ["host2"] # Default: []
filter-label = ["label1"] # Default: []
filter-tags = ["tag1,tag2"] # Default: []
filter-paths = ["path1"] # Default: no paths filter
filter-fn = '|sn| {sn.host == "host1" || sn.description.contains("test")}' # Default: no filter function

or alternatively as CLI options --filter-host, --filter-label, --filter-tags, --filter-paths and --filter-fn.

The filter-host option allows to filter snapshots by the host name. The filter-label option allows to filter snapshots by the label. The filter-tags option allows to filter snapshots by tags. The filter-paths option allows to filter snapshots by the paths in the backup. The filter-fn option allows to filter snapshots by a custom filter function. The filter function is a string that is parsed as a Rust closure. The closure takes a Snapshot struct as argument and returns a boolean value.

Multiple filter arguments are connected with an or condition, i.e. snapshots are not filtered out if they match anything of what is the given in the array. For instance, this shows snapshots having either a tag tag1 or a tag tag2:

filter-tags = ["tag1", "tag2"]

Moreover the different filter-* options are combined using an and condition - if a snapshot does not satisfy any of the filters, they are sorted out. For example, to list snapshots that are tagged with tag1 and tag2 and are located in the path /srv, use the following configuration:

filter-tags = ["tag1,tag2"]
filter-paths = ["/srv"]

Note: For paths and tags it is already possible to give multiple values - and then they are anded: filter-tags=["a,b"] matches snapshots which have both “a” and “b” as tags, whereas filter-tags=["a","b"] matches snapshots which have either “a” or “b” as tag.

What is (currently) not possible is to filter exactly for a tag list or path list: A snapshot with tags “a,b,c” is currently shown when you specify filter-tags=["a,b"].

Filtering snapshots to copy

The list of snapshots to copy can be filtered by using the above described filter-* options:

rustic copy --filter-host luigi --filter-tags foo,bar

It is also possible to explicitly specify the list of snapshots to copy, in which case only these instead of all snapshots will be copied:

rustic copy 410b18a2 4e5d5487 latest
Last change: 2024-12-17, commit: cb15d19

File History

Sometimes you know about a file which is present in your repository and you want to know when this given file has been changed. Basically you want to see the history of this file in the snapshots that have been saved in your repository.

To do so, simply use the --path option of rustic find. This will find the given path in all snapshots. However, by default identical results are shown as (+x) (like in the rustic snapshots output). So, this gives you exactly the history of that file.

Typically, you also know that you can filter out some snapshots by using the path(s) stored in the snapshots, so using a suitable --filter-paths will reduce the search by excluding snapshots which cannot contain that file.

An example call:

$ rustic find --path /home/user/.config/mc/ini --filter-paths /home
[00:00:01] reading index...               ████████████████████████████████████████        451/451

searching in snapshots group (host [myhost], label [], paths [/home])...
found in 03521ca4 from 2022-06-11 06:29:55
-rw-r--r--     user     user      4036 28 Mar 07:22 "/home/user/.config/mc/ini" 
found in 772fc402 from 2022-06-30 23:30:21 (+1)
-rw-r--r--     user     user      4036 13 Jun 08:19 "/home/user/.config/mc/ini" 
found in dd4d75e2 from 2022-08-31 23:30:33 (+16)
-rw-r--r--     user     user      4036 12 Aug 12:55 "/home/user/.config/mc/ini" 
found in b31f134e from 2023-10-29 23:30:20 (+230)
-rw-r--r--     user     user      4078 24 Oct 06:41 "/home/user/.config/mc/ini" 
found in bfacf88a from 2024-01-29 09:30:27 (+256)
-rw-r--r--     user     user      4078 28 Jan 23:32 "/home/user/.config/mc/ini" 
found in fef232e8 from 2024-04-23 23:30:52 (+432)
-rw-r--r--     user     user      4078 15 Apr 12:18 "/home/user/.config/mc/ini"

This shows that the given file exists in quite some snapshots - we exactly get the snapshots where the file content changed. So we can easily restore the content of the file from the snapshot we want.

The snapshots are sorted by the snapshot timestamp. Moreover, by default identical nodes are grouped together, this means that

found in dd4d75e2 from 2022-08-31 23:30:33 (+16)
-rw-r--r--     user     user      4036 12 Aug 12:55 "/home/user/.config/mc/ini"

means that this file was found in the shown snapshot dd4d75e2 and in the 16 subsequent snapshots. Moreover, the contents and metadata of it is exactly the same for all 17 snapshots.

Note, this also works for directories - if rustic find groups dirs which were found, the content and metadata of the dir for all contained files and subdirectories are exactly identical.

So, this grouping only shows that there is something different or same. To really see what is different, you have to use rustic diff or maybe even show the contents using rustic dump.

Last change: 2024-12-17, commit: cb15d19

Ensuring deduplication for copied snapshots

Different repositories usually have different parameters for splitting larger files into smaller chunks. When copying snapshots between arbitrary repository, deduplication between snapshots from the source and destination repository doesn’t work unless the repositories share the same parameters, the so-called chunker parameters.

rustic enforces identical chunker parameters - if you try to copy to a repository with different chunker parameter, you get an error like

cannot copy to repository with different chunker parameter (re-chunking not implemented)!

Note: Currently it is not possible to change the chunker parameters of existing repositories (re-chunking is not yet implemented).

To create a repository with identical chunker parameters to a source repository, don’t initialize the target repository, but instead run the first copy command with the --init option. This option initializes non-existing repositories with the correct chunker parameter:

rustic copy --init [SNAPSHOTS]

Note: The target repositories must be defined in the config file.

Last change: 2024-12-17, commit: cb15d19

Checking integrity and consistency

Imagine your repository is saved on a server that has a faulty hard drive, or even worse, attackers get privileged access and modify the files in your repository with the intention to make you restore malicious data:

echo "boom" > /srv/rustic-repo/index/de30f3231ca2e6a59af4aa84216dfe2ef7339c549dc11b09b84000997b139628

Trying to restore a snapshot which has been modified as shown above will yield an error:

$ rustic restore c23e491f /tmp/restore-work
...
Fatal: unable to load index de30f323: load <index/de30f3231c>: invalid data returned

In order to detect these things before they become a problem, it’s a good idea to regularly use the check command to test whether your repository is healthy and consistent, and that your precious backup data is unharmed. There are two types of checks that can be performed:

  • Structural consistency and integrity, e.g. snapshots, trees and pack files (default)
  • Integrity of the actual data that you backed up (enabled with flags, see below)

To verify the structure of the repository, issue the check command. If the repository is damaged like in the example above, check will detect this and yield the same error as when you tried to restore:

$ rustic check
...
load indexes
error: error loading index de30f323: load <index/de30f3231c>: invalid data returned
Fatal: LoadIndex returned errors

If the repository structure is intact, rustic will show that no errors were found:

$ rustic check
...
load indexes
check all packs
check snapshots, trees and blobs
no errors were found

By default, the check command does not verify that the actual pack files on disk in the repository are unmodified, because doing so requires reading a copy of every pack file in the repository. To tell rustic to also verify the integrity of the pack files in the repository, use the --read-data flag:

$ rustic check --read-data
...
load indexes
check all packs
check snapshots, trees and blobs
read all data
[0:00] 100.00%  3 / 3 items
duration: 0:00
no errors were found

Note: Since --read-data has to download all pack files in the repository, beware that it might incur higher bandwidth costs than usual and also that it takes more time than the default check.

Alternatively, use the --read-data-subset parameter to check only a subset of the repository pack files at a time. It supports three ways to select a subset. One selects a specific part of pack files, the second and third selects a random subset of the pack files by the given percentage or size.

Use --read-data-subset=n/t to check a specific part of the repository pack files at a time. The parameter takes two values, n and t. When the check command runs, all pack files in the repository are logically divided in t (roughly equal) groups, and only files that belong to group number n are checked. For example, the following commands check all repository pack files over 5 separate invocations:

rustic check --read-data-subset=1/5
rustic check --read-data-subset=2/5
rustic check --read-data-subset=3/5
rustic check --read-data-subset=4/5
rustic check --read-data-subset=5/5

Use --read-data-subset=x% to check a randomly chosen subset of the repository pack files. It takes one parameter, x, the percentage of pack files to check as an integer or floating point number. This will not guarantee to cover all available pack files after sufficient runs, but it is easy to automate checking a small subset of data after each backup. For a floating point value the following command may be used:

rustic check --read-data-subset=2.5%

When checking bigger subsets you most likely want to specify the percentage as an integer:

rustic check --read-data-subset=10%

Use --read-data-subset=nS to check a randomly chosen subset of the repository pack files. It takes one parameter, nS, where ‘n’ is a whole number representing file size and ‘S’ is the unit of file size (K/M/G/T) of pack files to check. Behind the scenes, the specified size will be converted to percentage of the total repository size. The behaviour of the check command following this conversion will be the same as the percentage option above. For a file size value the following command may be used:

rustic check --read-data-subset=50M
rustic check --read-data-subset=10G
Last change: 2024-12-17, commit: cb15d19

Key - Manage repository keys

The key command allows you to set multiple access keys or passwords per repository. In fact, you can use the list, add, remove, and passwd (changes a password) sub-commands to manage these keys very precisely:

    $ rustic key list
    enter password for repository:
     ID          User        Host        Created
    ----------------------------------------------------------------------
    *eb78040b    username    kasimir   2015-08-12 13:29:57

    $ rustic key add
    enter password for repository:
    enter password for new key:
    enter password again:
    saved new key as <Key of username@kasimir, created on 2015-08-12 13:35:05.316831933 +0200 CEST>

    $ rustic key list
    enter password for repository:
     ID          User        Host        Created
    ----------------------------------------------------------------------
     5c657874    username    kasimir   2015-08-12 13:35:05
    *eb78040b    username    kasimir   2015-08-12 13:29:57

Note: that the currently used key is indicated by an asterisk (*).

Last change: 2024-12-17, commit: cb15d19

Upgrading the repository format version

Repositories created using earlier rustic versions use an older repository format version and have to be upgraded to allow using all new features. Upgrading must be done explicitly as a newer repository version increases the minimum rustic version required to access the repository. For example the repository format version 2 is only readable using rustic 0.2.0 or newer.

Upgrading to repo version 2 is a two step process: first run migrate upgrade_repo_v2 which will check the repository integrity and then upgrade the repository version. Repository problems must be corrected before the migration will be possible. After the migration is complete, run prune to compress the repository metadata. To limit the amount of data rewritten in at once, you can use the prune --max-repack-size size parameter, see :ref:customize-pruning for more details.

File contents stored in the repository will not be rewritten, data from new backups will be compressed. Over time more and more of the repository will be compressed. To speed up this process and compress all not yet compressed data, you can run prune --repack-uncompressed.

Last change: 2024-12-17, commit: cb15d19

Hooks

rustic supports hooks that are executed before and after certain commands. This allows you to run custom scripts or commands before or after a backup, restore, or other commands.

Overview

rustic supports the following hooks (in order):

  • run-before is executed either on a global level, before accessing the repository, or before a specific command (e.g. backup).
  • run-after is executed either on a global level, after accessing the repository, or after a specific command (e.g. backup).
  • run-failed is executed if a command before has failed.
  • run-finally is executed after all other hooks of the same type have been executed, regardless of whether they succeeded or failed.

Configuration

Important: For always up-to-date information, please make sure to check the in-repository documentation for the config files available here.

Hooks are configured in the profile configuration file. Here is an example of a configuration file with all possible hooks:

[global.hooks]
run-before = []
run-after = []
run-failed = []
run-finally = []

[repository.hooks]
run-before = []
run-after = []
run-failed = []
run-finally = []

[backup.hooks]
run-before = []
run-after = []
run-failed = []
run-finally = []

[[backup.snapshots]]
sources = []

[backup.snapshots.hooks]
run-before = []
run-after = []
run-failed = []
run-finally = []

Each hook is a list of commands that are executed in order. If you want to run a script or command, you can add it to the list.

Hooks are also not executed in a shell, so you can’t use shell features like pipes or redirects. If you need to use shell features, you can run a shell command that executes the desired command. Within the shell command, you can use shell features.

For example:

[global.hooks]
run-before = [
  "sh -c  'echo Hello, > test.out'",
  "sh -c  'echo World! >> test.out'",
]
run-after = ["sh -c 'echo Goodbye, world! >> test.out'"]

In this example, the run-before hook is executed globally before any command is executed. Within the hook, two commands are executed. The first command writes Hello, to a file called test.out, and the second command appends World! to the same file on a new line. The commands within the list of a hook are executed in order.

So, after any other command is executed globally, the run-after hook is executed. In this case, the command echo 'Goodbye, world!' is executed, and the output is appended to the file test.out in a new line.

You can use hooks, e.g. to send a notification when a backup has finished:

[backup.hooks]
run-after = ["notify-send 'Backup finished successfully!'"]

Use cases

Here are some use cases which might be interesting to use hooks for:

Global hooks

  • Send messages after successful / failed runs
  • Feed your custom logging/monitoring with information about start/end of rustic calls

Repository hooks

  • Mount/umount the drive where the repository is located
  • Sync your repo to some remote destination after each command
  • Start extra integrity checks like checking the SHA256 against repo files

Backup hooks

  • Mount/umount a drive with data-to-backup
  • Run commands to save some systems state, e.g. dpkg --get-selections on debian-based systems
  • Dump a database into a file and remove the dump after backup (using stdin-command may be an alternative)

If you have a nice use-case yourself, please share it with others by making a pull request to this documentation.

Last change: 2024-12-17, commit: cb15d19

Restore - Restoring from backup

Restoring from a snapshot is as easy as it sounds, just use the following command to restore the contents of the latest snapshot to /tmp/restore-work:

$ rustic restore 79766175 /tmp/restore-work
enter password for repository:
restoring <Snapshot of [/home/user/work] at 2015-05-08 21:40:19.884408621 +0200 CEST> to /tmp/restore-work

Use the word latest to restore the last backup. You can also combine latest with the --filter-host and --filter-path filters to choose the last backup for a specific host, path or both.

$ rustic restore latest /tmp/restore-art --filter-path "/home/art" --filter-host luigi
enter password for repository:
restoring <Snapshot of [/home/art] at 2015-05-08 21:45:17.884408621 +0200 CEST> to /tmp/restore-art

Use --glob (pattern to exclude/include (can be specified multiple times)) to restrict the restore to a subset of files in the snapshot. For example, to restore a single file:

$ rustic restore 79766175 /tmp/restore-work --glob /work/foo
enter password for repository:
restoring <Snapshot of [/home/user/work] at 2015-05-08 21:40:19.884408621 +0200 CEST> to /tmp/restore-work

This will restore the file foo to /tmp/restore-work/work/foo.

You can use the command rustic ls latest

to find

the path to the file within the snapshot. This path you can then pass to --glob in verbatim to only restore the single file or directory.

There is case insensitive variants of --glob called --iglob. This option will behave the same way but ignore the casing of paths.

Important: The restore command will restore the permissions of the files and directories to the state they were in when the snapshot was taken. GUIDs and UIDs will be restored as well, but this requires elevated privileges to work correctly.

Last change: 2024-12-17, commit: cb15d19

Restore using mount

NOTE: rustic currently only supports mount for linux; we are working on support for other operation systemss.

Browsing your backup as a regular file system is also very easy. First, create a mount point such as /mnt/rustic and then use the following command to serve the repository with FUSE:

$ mkdir /mnt/rustic
$ rustic mount /mnt/rustic

Now you can access the backup content like a normal filesystem, i.e. you can simply copy files from the respository snapshots.

NOTE: Copying files using mount may be not as efficient as using restore as restore is highly optimized and knows in the beginning what exactly will be restored.

NOTE Keep in mind that every file access on the mounted repository may involve access to the repository. Especially read access to lots of data may be expensive depending on the backend. It is not advised to run compare tools or something like rsync on the mounted dir if you use a remote backend. Use diff or restore if you want to compare or sync content with your lokal saved files.

NOTE: When using a hot/cold repositories, file access is only possible if the needed data is warmed-up in the cold repository part. By default rustic therefore forbids file-access by default, see --file-access below.

There are various options which can be used with the mount command:

  • You can specify the exact snapshot/path to mount, e.g. rustic mount /mnt/rustic latest:/home
  • If no snapshot/path is given, all snapshots are displayed in a tree, you can use --path-template and --time-template to define the tree structure, e.g. first group by hostname and then by snapshot date/time.
  • You can give additional mount options using --exclusive or --option.
  • --file-access allows to restict access to only listing dirs and.
Last change: 2024-12-17, commit: cb15d19

Restore using webdav

Another possibility is browsing your backup as webdav share. This option can be also used to mount the repository on either remote computers or locally if mount is not supported for your operation system. To start a webdav server, just run

$ rustic webdav

Now you can connect using a standard webdav client.

NOTE: Copying files using webdav may be not as efficient as using restore as restore is highly optimized and knows in the beginning what exactly will be restored.

NOTE Keep in mind that every file access via webdav may involve access to the repository. Especially read access to lots of data may be expensive depending on the backend. It is not advised to run compare tools or something like rsync if you mount the webdav in case you are use a remote backend. Use diff or restore if you want to compare or sync repository content with each other or with your local saved files.

NOTE: When using a hot/cold repositories, file access is only possible if the needed data is warmed-up in the cold repository part. By default rustic therefore forbids file-access by default, see --file-access below.

There are various options which can be used with the webdav command (most similar to mount)

  • You can specify the exact snapshot/path to mount, e.g. rustic webdav latest:/home
  • If no snapshot/path is given, all snapshots are displayed in a tree, you can use --path-template and --time-template to define the tree structure, e.g. first group by hostname and then by snapshot date/time.
  • --address allows you to give the bind address.
  • --symlinks also enables symlinks.
  • --file-access allows to restict access to only listing dirs and.
Last change: 2024-12-17, commit: cb15d19

Printing files to stdout

Sometimes it’s helpful to print files to stdout so that other programs can read the data directly. This can be achieved by using the dump command, like this:

rustic dump latest production.sql | mysql

If you have saved multiple different things into the same repo, the latest snapshot may not be the right one. For example, consider the following snapshots in a repo:

$ rustic snapshots
ID        Date                 Host        Tags        Directory
----------------------------------------------------------------------
562bfc5e  2018-07-14 20:18:01  mopped                  /home/user/file1
bbacb625  2018-07-14 20:18:07  mopped                  /home/other/work
e922c858  2018-07-14 20:18:10  mopped                  /home/other/work
098db9d5  2018-07-14 20:18:13  mopped                  /production.sql
b62f46ec  2018-07-14 20:18:16  mopped                  /home/user/file1
1541acae  2018-07-14 20:18:18  mopped                  /home/other/work
----------------------------------------------------------------------

Here, rustic would resolve latest to the snapshot 1541acae, which does not contain the file we’d like to print at all (production.sql). In this case, you can pass rustic the snapshot ID of the snapshot you like to restore:

rustic dump 098db9d5 production.sql | mysql

Or you can pass rustic a path that should be used for selecting the latest snapshot. The path must match the patch printed in the “Directory” column, e.g.:

rustic dump --path /production.sql latest production.sql | mysql
Last change: 2024-12-17, commit: cb15d19

Forget - Removing backup snapshots

All backup space is finite, so rustic allows removing old snapshots. This can be done either manually (by specifying a snapshot ID to remove) or by using a policy that describes which snapshots to forget. For all remove operations, two commands need to be called in sequence: forget to remove snapshots, and prune to remove the remaining data that was referenced only by the removed snapshots. The latter can be automated with the --prune option of forget, which runs prune automatically if any snapshots were actually removed.

Pruning snapshots can be a time-consuming process, depending on the number of snapshots and data to process. During a prune operation, the repository is locked and backups cannot be completed. Please plan your pruning so that there’s time to complete it and it doesn’t interfere with regular backup runs.

It is advisable to run rustic check after pruning, to make sure you are alerted, should the internal data structures of the repository be damaged.

Last change: 2024-12-17, commit: cb15d19

Remove a single snapshot

The command snapshots can be used to list all snapshots in a repository like this:

$ rustic snapshots
enter password for repository:
ID        Date                 Host      Tags  Directory
----------------------------------------------------------------------
40dc1520  2015-05-08 21:38:30  kasimir         /home/user/work
79766175  2015-05-08 21:40:19  kasimir         /home/user/work
bdbd3439  2015-05-08 21:45:17  luigi           /home/art
590c8fc8  2015-05-08 21:47:38  kazik           /srv
9f0bc19e  2015-05-08 21:46:11  luigi           /srv

In order to remove the snapshot of /home/art, use the forget command and specify the snapshot ID on the command line:

$ rustic forget bdbd3439
enter password for repository:
removed snapshot bdbd3439

Afterwards this snapshot is removed:

$ rustic snapshots
enter password for repository:
ID        Date                 Host     Tags  Directory
----------------------------------------------------------------------
40dc1520  2015-05-08 21:38:30  kasimir        /home/user/work
79766175  2015-05-08 21:40:19  kasimir        /home/user/work
590c8fc8  2015-05-08 21:47:38  kazik          /srv
9f0bc19e  2015-05-08 21:46:11  luigi          /srv

But the data that was referenced by files in this snapshot is still stored in the repository. To cleanup unreferenced data, the prune command must be run:

$ rustic prune
enter password for repository:
repository 33002c5e opened successfully, password is correct
loading all snapshots...
loading indexes...
finding data that is still in use for 4 snapshots
[0:00] 100.00%  4 / 4 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:00] 100.00%  5 / 5 packs processed

to repack:            69 blobs / 1.078 MiB
this removes:         67 blobs / 1.047 MiB
to delete:             7 blobs / 25.726 KiB
total prune:          74 blobs / 1.072 MiB
remaining:            16 blobs / 38.003 KiB
unused size after prune: 0 B (0.00% of remaining size)

repacking packs
[0:00] 100.00%  2 / 2 packs repacked
rebuilding index
[0:00] 100.00%  3 / 3 packs processed
deleting obsolete index files
[0:00] 100.00%  3 / 3 files deleted
removing 3 old packs
[0:00] 100.00%  3 / 3 files deleted
done

Afterwards the repository is smaller.

You can automate this two-step process by using the --prune switch to forget:

    $ rustic forget --keep-last 1 --prune
    snapshots for host mopped, directories /home/user/work:

    keep 1 snapshots:
    ID        Date                 Host        Tags        Directory
    ----------------------------------------------------------------------
    4bba301e  2017-02-21 10:49:18  mopped                  /home/user/work

    remove 1 snapshots:
    ID        Date                 Host        Tags        Directory
    ----------------------------------------------------------------------
    8c02b94b  2017-02-21 10:48:33  mopped                  /home/user/work

    1 snapshots have been removed, running prune
    loading all snapshots...
    loading indexes...
    finding data that is still in use for 1 snapshots
    [0:00] 100.00%  1 / 1 snapshots
    searching used packs...
    collecting packs for deletion and repacking
    [0:00] 100.00%  5 / 5 packs processed

    to repack:           69 blobs / 1.078 MiB
    this removes         67 blobs / 1.047 MiB
    to delete:            7 blobs / 25.726 KiB
    total prune:         74 blobs / 1.072 MiB
    remaining:           16 blobs / 38.003 KiB
    unused size after prune: 0 B (0.00% of remaining size)

    repacking packs
    [0:00] 100.00%  2 / 2 packs repacked
    rebuilding index
    [0:00] 100.00%  3 / 3 packs processed
    deleting obsolete index files
    [0:00] 100.00%  3 / 3 files deleted
    removing 3 old packs
    [0:00] 100.00%  3 / 3 files deleted
    done
Last change: 2024-12-17, commit: cb15d19

Removing snapshots according to a policy

Removing snapshots manually is tedious and error-prone, therefore rustic allows specifying a policy (one or more --keep-* options) for which snapshots to keep. You can for example define how many hourly, daily, weekly, monthly and yearly snapshots to keep, and any other snapshots will be removed.

Warning: If you use an append-only repository with policy-based snapshot removal, some security considerations are important. Please refer to the section below for more information.

Note: You can always use the --dry-run option of the forget command, which instructs rustic to not remove anything but instead just print what actions would be performed.

The forget command accepts the following policy options:

  • --keep-last n keep the n last (most recent) snapshots.
  • --keep-hourly n for the last n hours which have one or more snapshots, keep only the most recent one for each hour.
  • --keep-daily n for the last n days which have one or more snapshots, keep only the most recent one for each day.
  • --keep-weekly n for the last n weeks which have one or more snapshots, keep only the most recent one for each week.
  • --keep-monthly n for the last n months which have one or more snapshots, keep only the most recent one for each month.
  • --keep-yearly n for the last n years which have one or more snapshots, keep only the most recent one for each year.
  • --keep-tag keep all snapshots which have all tags specified by this option (can be specified multiple times).
  • --keep-within duration keep all snapshots having a timestamp within the specified duration of the latest snapshot, where duration is a number of years, months, days, and hours. E.g. 2y5m7d3h will keep all snapshots made in the two years, five months, seven days and three hours before the latest (most recent) snapshot.
  • --keep-within-hourly duration keep all hourly snapshots made within the specified duration of the latest snapshot. The duration is specified in the same way as for --keep-within and the method for determining hourly snapshots is the same as for --keep-hourly.
  • --keep-within-daily duration keep all daily snapshots made within the specified duration of the latest snapshot.
  • --keep-within-weekly duration keep all weekly snapshots made within the specified duration of the latest snapshot.
  • --keep-within-monthly duration keep all monthly snapshots made within the specified duration of the latest snapshot.
  • --keep-within-yearly duration keep all yearly snapshots made within the specified duration of the latest snapshot.

Note: All calendar related options (--keep-{hourly,daily,...}) work on natural time boundaries and not relative to when you run forget. Weeks are Monday 00:00 to Sunday 23:59, days 00:00 to 23:59, hours :00 to :59, etc. They also only count hours/days/weeks/etc which have one or more snapshots.

Note: All duration related options (--keep-{within,-*}) ignore snapshots with a timestamp in the future (relative to when the forget command is run) and these snapshots will hence not be removed.

Note: Specifying --keep-tag '' will match untagged snapshots only.

When forget is run with a policy, rustic first loads the list of all snapshots and groups them by their host name and paths. The grouping options can be set with --group-by, e.g. using --group-by paths,tags to instead group snapshots by paths and tags. The policy is then applied to each group of snapshots individually. This is a safety feature to prevent accidental removal of unrelated backup sets. To disable grouping and apply the policy to all snapshots regardless of their host, paths and tags, use --group-by '' (that is, an empty value to --group-by).

Additionally, you can restrict the policy to only process snapshots which have a particular hostname with the --host parameter, or tags with the --tag option. When multiple tags are specified, only the snapshots which have all the tags are considered. For example, the following command removes all but the latest snapshot of all snapshots that have the tag foo:

rustic forget --tag foo --keep-last 1

This command removes all but the last snapshot of all snapshots that have either the foo or bar tag set:

rustic forget --tag foo --tag bar --keep-last 1

To only keep the last snapshot of all snapshots with both the tag foo and bar set use:

rustic forget --tag foo,bar --keep-last 1

To ensure only untagged snapshots are considered, specify the empty string ‘’ as the tag.

rustic forget --tag '' --keep-last 1

Let’s look at a simple example: Suppose you have only made one backup every Sunday for 12 weeks:

rustic snapshots repository f00c6e2a opened successfully, password is correct ID Time Host Tags Paths

## 0a1f9759 2019-09-01 11:00:00 mopped /home/user/work 46cfe4d5 2019-09-08 11:00:00 mopped /home/user/work f6b1f037 2019-09-15 11:00:00 mopped /home/user/work eb430a5d 2019-09-22 11:00:00 mopped /home/user/work 8cf1cb9a 2019-09-29 11:00:00 mopped /home/user/work 5d33b116 2019-10-06 11:00:00 mopped /home/user/work b9553125 2019-10-13 11:00:00 mopped /home/user/work e1a7b58b 2019-10-20 11:00:00 mopped /home/user/work 8f8018c0 2019-10-27 11:00:00 mopped /home/user/work 59403279 2019-11-03 11:00:00 mopped /home/user/work dfee9fb4 2019-11-10 11:00:00 mopped /home/user/work e1ae2f40 2019-11-17 11:00:00 mopped /home/user/work

12 snapshots

Then forget --keep-daily 4 will keep the last four snapshots, for the last four Sundays, and remove the other snapshots:

$ rustic forget --keep-daily 4 --dry-run repository f00c6e2a opened successfully, password is correct Applying Policy: keep the last 4 daily snapshots keep 4 snapshots: ID Time Host Tags Reasons Paths

8f8018c0 2019-10-27 11:00:00 mopped daily snapshot /home/user/work 59403279 2019-11-03 11:00:00 mopped daily snapshot /home/user/work dfee9fb4 2019-11-10 11:00:00 mopped daily snapshot /home/user/work e1ae2f40 2019-11-17 11:00:00 mopped daily snapshot /home/user/work

4 snapshots

remove 8 snapshots: ID Time Host Tags Paths

0a1f9759 2019-09-01 11:00:00 mopped /home/user/work 46cfe4d5 2019-09-08 11:00:00 mopped /home/user/work f6b1f037 2019-09-15 11:00:00 mopped /home/user/work eb430a5d 2019-09-22 11:00:00 mopped /home/user/work 8cf1cb9a 2019-09-29 11:00:00 mopped /home/user/work 5d33b116 2019-10-06 11:00:00 mopped /home/user/work b9553125 2019-10-13 11:00:00 mopped /home/user/work e1a7b58b 2019-10-20 11:00:00 mopped /home/user/work

8 snapshots

The processed snapshots are evaluated against all --keep-* options but a snapshot only need to match a single option to be kept (the results are ORed). This means that the most recent snapshot on a Sunday would match both hourly, daily and weekly --keep-* options, and possibly more depending on calendar.

For example, suppose you make one backup every day for 100 years. Then forget --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 75 would keep the most recent 7 daily snapshots and 4 last-day-of-the-week ones (since the 7 dailies already include 1 weekly). Additionally, 12 or 11 last-day-of-the-month snapshots will be kept (depending on whether one of them ends up being the same as a daily or weekly). And finally 75 or 74 last-day-of-the-year snapshots are kept, depending on whether one of them ends up being the same as an already kept snapshot. All other snapshots are removed.

You might want to maintain the same policy as in the example above, but have irregular backups. For example, the 7 snapshots specified with --keep-daily 7 might be spread over a longer period. If what you want is to keep daily snapshots for the last week, weekly for the last month, monthly for the last year and yearly for the last 75 years, you can instead specify forget --keep-within-daily 7d --keep-within-weekly 1m --keep-within-monthly 1y --keep-within-yearly 75y (note that 1w is not a recognized duration, so you will have to specify 7d instead).

For safety reasons, rustic refuses to act on an “empty” policy. For example, if one were to specify --keep-last 0 to forget all snapshots in the repository, rustic will respond that no snapshots will be removed. To delete all snapshots, use --keep-last 1 and then finally remove the last snapshot manually (by passing the ID to forget).

Last change: 2024-12-17, commit: cb15d19

Security considerations in append-only mode

Note: TL;DR: With append-only repositories, one should specifically use the --keep-within option of the forget command when removing snapshots.

To prevent a compromised backup client from deleting its backups (for example due to a ransomware infection), a repository service/backend can serve the repository in a so-called append-only mode. This means that the repository is served in such a way that it can only be written to and read from, while delete and overwrite operations are denied. rustic’s rest-server features an append-only mode, but few other standard backends do. To support append-only with such backends, one can use rclone as a complement in between the backup client and the backend service.

To remove snapshots and recover the corresponding disk space, the forget and prune commands require full read, write and delete access to the repository. If an attacker has this, the protection offered by append-only mode is naturally void. The usual and recommended setup with append-only repositories is therefore to use a separate and well-secured client whenever full access to the repository is needed, e.g. for administrative tasks such as running forget, prune and other maintenance commands.

However, even with append-only mode active and a separate, well-secured client used for administrative tasks, an attacker who is able to add garbage snapshots to the repository could bring the snapshot list into a state where all the legitimate snapshots risk being deleted by an unsuspecting administrator that runs the forget command with certain --keep-* options, leaving only the attacker’s useless snapshots.

For example, if the forget policy is to keep three weekly snapshots, and the attacker adds an empty snapshot for each of the last three weeks, all with a timestamp (see the backup command’s --time option) slightly more recent than the existing snapshots (but still within the target week), then the next time the repository administrator (or a scheduled job) runs the forget command with this policy, the legitimate snapshots will be removed (since the policy will keep only the most recent snapshot within each week). Even without running prune, recovering data would be messy and some metadata lost.

To avoid this, forget policies applied to append-only repositories should use the --keep-within option, as this will keep not only the attacker’s snapshots but also the legitimate ones. Assuming the system time is correctly set when forget runs, this will allow the administrator to notice problems with the backup or the compromised host (e.g. by seeing more snapshots than usual or snapshots with suspicious timestamps). This is, of course, limited to the specified duration: if forget --keep-within 7d is run 8 days after the last good snapshot, then the attacker can still use that opportunity to remove all legitimate snapshots.

Last change: 2024-12-17, commit: cb15d19

Customize pruning

To understand the custom options, we first explain how the pruning process works:

  1. All snapshots and directories within snapshots are scanned to determine which data is still in use.

  2. For all files in the repository, rustic finds out if the file is fully used, partly used or completely unused.

  3. Completely unused files are marked for deletion. Fully used files are kept. A partially used file is either kept or marked for repacking depending on user options.

    Note that for repacking, rustic must download the file from the repository storage and re-upload the needed data in the repository. This can be very time-consuming for remote repositories.

  4. After deciding what to do, prune will actually perform the repack, modify the index according to the changes and delete the obsolete files.

The prune command accepts the following options:

  • --max-unused limit allow unused data up to the specified limit within the repository. This allows rustic to keep partly used files instead of repacking them.

    The limit can be specified in several ways:

    • As an absolute size (e.g. 200M). If you want to minimize the space used by your repository, pass 0 to this option.
    • As a size relative to the total repo size (e.g. 10%). This means that after prune, at most 10% of the total data stored in the repo may be unused data. If the repo after prune has a size of 500MB, then at most 50MB may be unused.
    • If the string unlimited is passed, there is no limit for partly unused files. This means that as long as some data is still used within a file stored in the repo, rustic will just leave it there. Use this if you want to minimize the time and bandwidth used by the prune operation. Note that metadata will still be repacked.

    rustic tries to repack as little data as possible while still ensuring this limit for unused data. The default value is 5%.

  • --max-repack-size size if set limits the total size of files to repack. As prune first stores all repacked files and deletes the obsolete files at the end, this option might be handy if you expect many files to be repacked and fear to run low on storage.

  • --repack-cacheable-only if set to true only files which contain metadata and would be stored in the cache are repacked. Other pack files are not repacked if this option is set. This allows a very fast repacking using only cached data. It can, however, imply that the unused data in your repository exceeds the value given by --max-unused. The default value is false.

  • --dry-run only show what prune would do.

  • --verbose increased verbosity shows additional statistics for prune.

Last change: 2024-12-17, commit: cb15d19

Recovering from “no free space” errors

In some cases when a repository has grown large enough to fill up all disk space or the allocated quota, then prune might fail to free space. prune works in such a way that a repository remains usable no matter at which point the command is interrupted. However, this also means that prune requires some scratch space to work.

In most cases it is sufficient to instruct prune to remove all packs marked for removal and use as little scratch space as possible. Note that packs marked for removal are automatically removed by a prune run once they are old enough. If you can guarantee that the repository is not used by parallel processes, you can also use rustic prune --instant-delete.

To use as little scratch space as possibe, run rustic prune --max-repack-size 0. This removes all unneeded packs without repacking partly used packs. Obviously, this can only work if several snapshots have been removed using forget before. This then allows the prune command to actually remove data from the repository. If the command succeeds, but there is still little free space, then remove a few more snapshots and run prune again.

Last change: 2024-12-17, commit: cb15d19

Stories

Stories of people using rustic:

Some talks about restic can be found here

Last change: 2024-12-17, commit: cb15d19