README.md 9.5 KB
Newer Older
Pieter Lange's avatar
Pieter Lange committed
1
2
kube-backup
===========
Pieter Lange's avatar
cleanup    
Pieter Lange committed
3
[![Docker Repository on Quay](https://quay.io/repository/plange/kube-backup/status "Docker Repository on Quay")](https://quay.io/repository/plange/kube-backup)
Pieter Lange's avatar
Pieter Lange committed
4
5
[![Docker Repository on Docker Hub](https://img.shields.io/docker/automated/ptlange/kube-backup.svg "Docker Repository on Docker Hub")](https://hub.docker.com/r/ptlange/kube-backup/)

Pieter Lange's avatar
Pieter Lange committed
6
7
8
9
10
11
Quick 'n dirty kubernetes state backup script, designed to be ran as kubernetes Job. Think of it like [RANCID](http://www.shrubbery.net/rancid/) for kubernetes.

Props to @gianrubio for coming up with the idea.

Setup
-----
Elouan Martinet's avatar
Elouan Martinet committed
12
Use the deployment example ([ssh](cronjob-ssh.yaml) authentication) and deploy a kubernetes `CronJob` primitive in your kubernetes (1.5 and up) cluster ensuring backups of kubernetes resource definitions to your private git repo.
Pieter Lange's avatar
Pieter Lange committed
13

Pieter Lange's avatar
Pieter Lange committed
14
15
Define the following environment parameters:
  * `GIT_REPO` - GIT repo url. **Required**
krogon-dp's avatar
krogon-dp committed
16
  * `GIT_PREFIX_PATH` - Path to the subdirectory in your repository. Default: `.`
Pieter Lange's avatar
Pieter Lange committed
17
  * `NAMESPACES` - List of namespaces to export. Default: all
Pieter Lange's avatar
Pieter Lange committed
18
  * `GLOBALRESOURCES` - List of global resource types to export. Default: `namespace`
19
  * `RESOURCETYPES` - List of resource types to export. Default: `ingress deployment configmap svc rc ds networkpolicy statefulset storageclass cronjob`. Notice that `Secret` objects are intentionally not exported by default (see [git-crypt section](#git-crypt) for details).
Pieter Lange's avatar
Pieter Lange committed
20
21
  * `GIT_USERNAME` - Display name of git user. Default: `kube-backup`
  * `GIT_EMAIL` - Email address of git user. Default: `kube-backup@example.com`
Pieter Lange's avatar
Pieter Lange committed
22
  * `GIT_BRANCH` - Use a specific git branch . Default: `master`
krogon-dp's avatar
krogon-dp committed
23
24
25
  * `GITCRYPT_ENABLE` - Use git-crypt for data encryption. See [git-crypt section](#git-crypt) for details. Default: `false`
  * `GITCRYPT_PRIVATE_KEY` - Path to private gpg key for git-crypt. See [git-crypt section](#git-crypt) for details. Default: `/secrets/gpg-private.key`
  * `GITCRYPT_SYMMETRIC_KEY` - Path to shared symmetric key for git-crypt. See [git-crypt section](#git-crypt). Default: `/secrets/symmetric.key`
Pieter Lange's avatar
Pieter Lange committed
26

Elouan Martinet's avatar
Elouan Martinet committed
27
Mount a configured ssh directory in `/backup/.ssh` with the following files:
28
29
30
31
32
33
34
35
36
37
38
39

    * `known_hosts` - Preloaded with SSH host key of `$GIT_REPO` host.
    * `id_rsa` - SSH private key of user allowed to push to `$GIT_REPO`.

  Easiest way of doing this is:
  ```bash
  ssh-keygen -f ./id_rsa
  ssh-keyscan $YOUR_GIT_HOST > known_hosts

  kubectl create secret generic kube-backup-ssh -n kube-system --from-file=id_rsa --from-file=known_hosts
  ```

Pieter Lange's avatar
Pieter Lange committed
40
41
Optional:
  * Modify the snapshot frequency in `spec.schedule` using the [cron format](https://en.wikipedia.org/wiki/Cron).
42
  * Modify the number of successful and failed finished jobs to retain in `spec.successfulJobsHistoryLimit` and `spec.failedJobsHistoryLimit`.
Victor Robertson's avatar
Victor Robertson committed
43
  * If using RBAC (1.6+), use the ClusterRole and ClusterRoleBindings in rbac.yaml.
Pieter Lange's avatar
Pieter Lange committed
44

krogon-dp's avatar
krogon-dp committed
45
46
git-crypt
---------
Pieter Lange's avatar
cleanup    
Pieter Lange committed
47
For security reasons `Secret` objects are not exported by default. However there is a possibility to store them safely using the [git-crypt project](https://github.com/AGWA/git-crypt).
krogon-dp's avatar
krogon-dp committed
48
49

#### Prerequisites
Pieter Lange's avatar
cleanup    
Pieter Lange committed
50
Your repository has to be already initialized with git-crypt. Minimal configuration is listed below. For details and full information see [using git-crypt](https://github.com/AGWA/git-crypt#using-git-crypt).
krogon-dp's avatar
krogon-dp committed
51
52
53
54
55

```
cd repo
git-crypt init
cat <<EOF > .gitattributes
56
*.secret.yaml filter=git-crypt diff=git-crypt
krogon-dp's avatar
krogon-dp committed
57
58
59
60
61
62
63
64
65
.gitattributes !filter !diff
EOF
git-crypt add-gpg-user <USER_ID>
git add -A
git commit -a -m "initialize git-crypt"
```

Optional:
  * You may choose any subdirectory for storing .gitattributes file (useful when using `GIT_PREFIX_PATH`).
Pieter Lange's avatar
cleanup    
Pieter Lange committed
66
  * You may encrypt additional files other than secret.yaml. Add additional lines before the .gitattribute filter. You may also use wildcard `*` to encrypt all files within the directory.
krogon-dp's avatar
krogon-dp committed
67
68
69

#### Enable git-crypt
To enable encryption feature:
Pieter Lange's avatar
cleanup    
Pieter Lange committed
70
  * Set pod environment variable `GITCRYPT_ENABLE` to `true`
krogon-dp's avatar
krogon-dp committed
71
72
73
74
75
76
77
78
79
80
81
82
83
    ```
    spec:
      containers:
      - env:
        - name: GITCRYPT_ENABLE
          value: "true"
    ```
  * Create additional `Secret` object containing **either** gpg-private or symmetric key
    ```
    apiVersion: v1
    kind: Secret
    metadata:
      name: kube-backup-gpg
84
      namespace: kube-system
krogon-dp's avatar
krogon-dp committed
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
    data:
      gpg-private.key: <base64_encoded_key>
      symmetric.key: <base64_encoded_key>
    ```
  * Mount keys from `Secret` as additional volume
    ```
    spec:
      containers:
      - volumeMounts:
        - mountPath: /secrets
          name: gpgkey
      volumes:
      - name: gpgkey
        secret:
          defaultMode: 420
          secretName: kube-backup-gpg
    ```
  * Add secret object name to `RESOURCETYPES` variable
    ```
    spec:
      containers:
      - env:
        - name: RESOURCETYPES
          value: "ingress deployment configmap secret svc rc ds thirdpartyresource networkpolicy statefulset storageclass cronjob"
    ```
110
111
112
113
114
115
116
117
  * If using RBAC (1.6+), add `secrets` to `resources`
    ```
    rules:
    - apiGroups: ["*"]
      resources: [
        "configmaps",
        "secrets",
    ```
krogon-dp's avatar
krogon-dp committed
118
119
120
121

  * (Optional): `$GITCRYPT_PRIVATE_KEY` and `$GITCRYPT_SYMMETRIC_KEY` variables are the combination of path where `Secret` volume is mounted and the name of item key from that object. If you change any value of them from the above example you may need to set this variables accordingly.


Pieter Lange's avatar
Pieter Lange committed
122
123
Result
------
Pieter Lange's avatar
Pieter Lange committed
124
All configured resources will be exported into a directory tree structure in YAML format following a `$namespace/$name.$type.yaml` file structure.
Pieter Lange's avatar
Pieter Lange committed
125
126
127
128

```
.
├── kube-system
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
│   ├── attachdetach-controller.serviceaccounts.yaml
│   ├── canal-config.configmap.yaml
│   ├── canal.daemonset.yaml
│   ├── canal.serviceaccounts.yaml
│   ├── certificate-controller.serviceaccounts.yaml
│   ├── cronjob-controller.serviceaccounts.yaml
│   ├── daemon-set-controller.serviceaccounts.yaml
│   ├── default.serviceaccounts.yaml
│   ├── deployment-controller.serviceaccounts.yaml
│   ├── disruption-controller.serviceaccounts.yaml
│   ├── dns-controller.deployment.yaml
│   ├── dns-controller.serviceaccounts.yaml
│   ├── endpoint-controller.serviceaccounts.yaml
│   ├── generic-garbage-collector.serviceaccounts.yaml
│   ├── horizontal-pod-autoscaler.serviceaccounts.yaml
│   ├── job-controller.serviceaccounts.yaml
│   ├── kube-backup-gpg.secret.yaml
│   ├── kube-backup.serviceaccounts.yaml
│   ├── kube-backup-ssh.secret.yaml
│   ├── kube-dns-autoscaler.configmap.yaml
│   ├── kube-dns-autoscaler.deployment.yaml
│   ├── kube-dns-autoscaler.serviceaccounts.yaml
│   ├── kube-dns.deployment.yaml
│   ├── kube-dns.serviceaccounts.yaml
│   ├── kube-dns.service.yaml
│   ├── kubelet.service.yaml
│   ├── kube-prometheus-exporter-kube-controller-manager.service.yaml
│   ├── kube-prometheus-exporter-kube-dns.service.yaml
│   ├── kube-prometheus-exporter-kube-etcd.service.yaml
│   ├── kube-prometheus-exporter-kube-scheduler.service.yaml
│   ├── kube-proxy.serviceaccounts.yaml
│   ├── kube-state-backup-new.cronjob.yaml
│   ├── kube-sysctl.daemonset.yaml
│   ├── letsencrypt-prod.secret.yaml
│   ├── namespace-controller.serviceaccounts.yaml
│   ├── node-controller.serviceaccounts.yaml
│   ├── openvpn-ccd.configmap.yaml
│   ├── openvpn-crl.configmap.yaml
│   ├── openvpn.deployment.yaml
│   ├── openvpn-ingress.service.yaml
│   ├── openvpn-pki.secret.yaml
│   ├── openvpn-portmapping.configmap.yaml
│   ├── openvpn-settings.configmap.yaml
│   ├── persistent-volume-binder.serviceaccounts.yaml
│   ├── pod-garbage-collector.serviceaccounts.yaml
│   ├── replicaset-controller.serviceaccounts.yaml
│   ├── replication-controller.serviceaccounts.yaml
│   ├── resourcequota-controller.serviceaccounts.yaml
│   ├── route53-config.secret.yaml
│   ├── service-account-controller.serviceaccounts.yaml
│   ├── service-controller.serviceaccounts.yaml
│   ├── statefulset-controller.serviceaccounts.yaml
│   ├── sysctl-options.configmap.yaml
│   ├── tiller-deploy.deployment.yaml
│   ├── tiller-deploy.service.yaml
│   ├── tiller.serviceaccounts.yaml
│   └── ttl-controller.serviceaccounts.yaml
Pieter Lange's avatar
Pieter Lange committed
186
├── prd
187
188
189
190
191
192
193
194
195
196
197
198
│   ├── initdb.configmap.yaml
│   ├── example-app.deployment.yaml
│   ├── example-app.ingress.yaml
│   ├── example-app.secret.yaml
│   ├── example-app.service.yaml
│   ├── postgres-admin.secret.yaml
│   ├── postgresql.deployment.yaml
│   ├── postgresql.service.yaml
│   ├── postgres.secret.yaml
│   ├── prd.example.com.secret.yaml
│   ├── redis.service.yaml
│   └── redis-standalone.rc.yaml
Pieter Lange's avatar
Pieter Lange committed
199
└── staging
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
    ├── initdb.configmap.yaml
    ├── example-app.deployment.yaml
    ├── example-app.ingress.yaml
    ├── example-app.secret.yaml
    ├── example-app.service.yaml
    ├── postgres-admin.secret.yaml
    ├── postgresql.deployment.yaml
    ├── postgresql.service.yaml
    ├── postgres.secret.yaml
    ├── staging.example.com.secret.yaml
    ├── redis.service.yaml
    └── redis-standalone.rc.yaml

3 directories, 80 files
```
Pieter Lange's avatar
cleanup    
Pieter Lange committed
215

Pieter Lange's avatar
Pieter Lange committed
216
217
-------
This project is MIT licensed.