diff options
| author | Silvio Rhatto <rhatto@riseup.net> | 2024-07-12 13:38:51 -0300 | 
|---|---|---|
| committer | Silvio Rhatto <rhatto@riseup.net> | 2024-07-12 13:38:51 -0300 | 
| commit | 10912a5f713fc40473e4870f524dc8934f5559e7 (patch) | |
| tree | 5dcae9e8d7a528829195fa73fcf341c85900e9f5 | |
| parent | c6dde9cfcc68531c58394d9806b2c83dbd4fae8d (diff) | |
| download | hydra-10912a5f713fc40473e4870f524dc8934f5559e7.tar.gz hydra-10912a5f713fc40473e4870f524dc8934f5559e7.tar.bz2  | |
Feat: docs: improvements about backups and TPC volume handling
| -rw-r--r-- | docs/backups.md | 89 | ||||
| -rw-r--r-- | docs/tpc.md | 81 | 
2 files changed, 108 insertions, 62 deletions
diff --git a/docs/backups.md b/docs/backups.md index c612116..d645207 100644 --- a/docs/backups.md +++ b/docs/backups.md @@ -11,33 +11,68 @@ This mostly has to do with `hydractl` commands such as  * But all this does not exempt a team from manually testing backups and to make    their own offline copies in external drives. +## Parameters + +Start by running the following commands in your terminal, adjusting to your +case: + +    export HYDRA="hydra-name" +    export DISK="disk-name" +    export DEVICE="/dev/sdc" +    export USER="`whoami`" +    export DOMAIN="`facter domain`" + +    if [ "$USER" != "root" ]; then +      export SUDO="sudo" +    fi +  ## New external drive -Proceed as follows +To add a new external drive into the pool, proceed as follows. + +First make sure to create a passphrase for the new disk into your hydra's keyring: + +    keyringer $HYDRA pwgen disks/$DISK.$DOMAIN/luks/root + +If you're using a USB drive, run this before plugging it on the computer: + +    hydractl usb-enable + +With the disk plugged in, make a layout with single `${DEVICE}1` partition +(example: a disk with only one partition like `/dev/sdc1`): -    disk=new-disk-name -    device=/dev/sdc -    hydractl usb-enable   # run this for USB drives, then connect the disk -    sudo cfdisk ${device} # layout with single ${device}1 partition -    sudo cryptsetup luksFormat ${device}1 -    sudo cryptsetup luksOpen ${device}1 $disk -    sudo mkfs.ext4 /dev/mapper/$disk -    sudo mkdir /media/$disk -    sudo mount /dev/mapper/$disk /media/$disk -    sudo mkdir /media/$disk/media.`facter domain` -    sudo chown `whoami`: /media/$disk/media.`facter domain` +    $SUDO cfdisk ${DEVICE} -## NAS +Now create the LUKS encryption header in this new disk, using the passphrase +generated above by keyringer: + +    $SUDO cryptsetup luksFormat ${DEVICE}1 + +Open the encrypted volume and create the basic filesystem structure needed +to sync content: + +    $SUDO cryptsetup luksOpen ${DEVICE}1 $DISK +    $SUDO mkfs.ext4 /dev/mapper/$DISK +    $SUDO mkdir /media/$DISK +    $SUDO mount /dev/mapper/$DISK /media/$DISK +    $SUDO mkdir /media/$DISK/media.$DOMAIN +    $SUDO chown ${USER}: /media/$DISK/media.$DOMAIN + +Finally, umount the drive: + +    hydractl umount-media $DISK + +## Regular sync  These commands should be enough to sync all media archives:      hydractl usb-enable  # run this for USB drives, then connect the disk -    hydractl mount-media $MEDIA -    hydractl sync-media  $MEDIA +    hydractl mount-media $DISK +    hydractl sync-media  $DISK  As this should handle syncing all backups: -    hydractl sync-backups $MEDIA +    hydractl sync-backups $DISK  ## TPC @@ -57,6 +92,11 @@ Then do the following:      hydractl mount-media $TPC      hydractl sync-tpc    $TPC +If you don't want to do a full TPC sync, but just want to sync the home folders +(which is faster), use the following instead of the last command above: + +    hydractl sync-home $TPC +  To sync archives and remote backups, proceed with as explained in the NAS  section. @@ -71,10 +111,10 @@ homedir backups in the external archive/backup volume.  You might want to backup the whole SSD, M-SATA or microSD from your appliances.  If so, proceed as follows with the appliance device connected in your TPC: -    export appliance=appliance-name -    export dest="/var/backups/remote/$appliance.`facter domain`/image/`date +%Y%m%d`" -    sudo mkdir -p $dest -    dcfldd if=/dev/sdb | bzip2 > $dest/$appliance.img.bz2 +    export APPLIANCE="appliance-name" +    export DEST="/var/backups/remote/${APPLIANCE}.${DOMAIN}/image/`date +%Y%m%d`" +    $SUDO mkdir -p $dest +    dcfldd if=/dev/sdb | bzip2 > $DEST/${APPLIANCE}.img.bz2  ## Smartphone @@ -100,7 +140,9 @@ A Backup Kit is a box with the following items:  * External encrypted archive/backup disk.  * Case for SSD transportation.  * Laptop power adapter and cables. -* Dockstation SATA/USB/M.2/microSD/etc (with power adapter). +* Dockstation SATA/USB/M.2/microSD/etc (with power adapter -- usually 12V). +* Power adapter for external hard drives (usually 12V, and might be compatible +  with the dockstationadapter, so you might carry just one).  * USB power adapter and cable (including USB 2, USB 3 and USB C).  * USB cables (USB 2, USB 3 and USB C) for the Dockstation and the external drive.  * TPC laptop with battery and TPS (SSD, M.2 etc), optionally with a UltraBase/Dockstation. @@ -128,10 +170,11 @@ For [duplicity][]:  For [Borg][]: +    export PATH_TO_RESTORE="path/to/be/restored"      mkdir ~/temp/misc/restore      cd    ~/temp/misc/restore      borg list    ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg -    borg extract ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg::$USER-2018-06-11T17:07:39 mnt/crypt/home/$USER/$FILE_OR_FOLDER +    borg extract ssh://$USER@$SERVER:$PORT//var/backups/users/$USER/borg::$USER-2018-06-11T17:07:39 mnt/crypt/home/$USER/$PATH_TO_RESTORE  Make sure to cleanup `~/temp/misc/restore` after recovering what you need. @@ -162,6 +205,6 @@ Note on backup keys:  For [eCryptfs][]: -    sudo ecryptfs-recover-private /media/$MEDIA/home/.ecryptfs/$USER/.Private +    $SUDO ecryptfs-recover-private /media/$DISK/home/.ecryptfs/$USER/.Private  [eCryptfs]: https://www.ecryptfs.org/ diff --git a/docs/tpc.md b/docs/tpc.md index d3e7e06..2d8a125 100644 --- a/docs/tpc.md +++ b/docs/tpc.md @@ -6,7 +6,10 @@ The Hydra Suite was made with the [Trusted Personal Computer (TPC)][]  This guide documents the lifecycle of a TPC storage device, or a "TPS  Cartridge": -    Cartridge = TPS = Trusted Personal Storage (like and SSD, microSD or M.2) +    Cartridge = TPS = Trusted Personal Storage + +A TPS can usually be whatever removable block device like an SSD, microSD, M.2 +etc.  The idea is that a number of TPCs can have interchangeable cartridges, as long  as each cartridge has an unique name. @@ -19,12 +22,12 @@ as each cartridge has an unique name.  Start by running the following commands in your terminal, adjusting to your  case: -    export HYDRA="nome-da-hydra" -    export VOLNAME="nome-do-disco" -    export DOMAIN="dominio.example.org" +    export HYDRA="hydra-name" +    export DISK="disk-name"      export DEVICE="/dev/sdb" -    export USER=`whoami` -    export DEST=/media/$VOLNAME +    export DOMAIN="`facter domain`" +    export USER="`whoami`" +    export DEST="/media/$DISK"      export CONFIG_FOLDER="`hydra $HYDRA folder`"      if [ "$USER" != "root" ]; then @@ -46,52 +49,52 @@ procedures from this document can be done from another TPC.  If non-existing, create entries and keys: -    hydra $HYDRA newnode $VOLNAME tpc +    hydra $HYDRA newnode $DISK tpc  Customize: -    vim $CONFIG_FOLDER/config/provision/$VOLNAME.conf -    vim $CONFIG_FOLDER/puppet/config/node/$VOLNAME.$DOMAIN.yaml -    vim $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml +    vim $CONFIG_FOLDER/config/provision/$DISK.conf +    vim $CONFIG_FOLDER/puppet/config/node/$DISK.$DOMAIN.yaml +    vim $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml  ### Secrets  Proceed adding other passwords and secrets at -`$CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN`, make sure to +`$CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN`, make sure to  encode them with [hiera-yaml][]: -    hydra $HYDRA eyaml $VOLNAME encrypt -q -o block -p -l some::password +    hydra $HYDRA eyaml $DISK encrypt -q -o block -p -l some::password  If you already have the secret somewhere, you can use a construction like this  which already adds the eyaml block into the hiera config file:      keyringer $HYDA decrypt /path/to/some/secret | \        tr -d '\n' | \ -      hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::password >> \ -      $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml +      hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::password >> \ +      $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml  You can also ensure a fresh random passphrase is used, using your favourite  generator like this:      head -c ${1:-20} /dev/urandom | base64 | \        tr -d '\n' | \ -      hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \ -      $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml +      hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \ +      $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml  For `passwd(5)` and `shadow(5)` hashed passphrases, use something like this:      mkpasswd -m sha-512 | \        tr -d '\n' | \ -      hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \ -      $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml +      hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \ +      $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml  Or this, for a random passphrase:      head -c ${1:-20} /dev/urandom | base64 | \        mkpasswd -m sha-512 --stdin | \        tr -d '\n' | \ -      hydra $HYDRA eyaml $VOLNAME encrypt -q -o block --stdin -l some::random:password >> \ -      $CONFIG_FOLDER/puppet/config/secrets/node/$VOLNAME.$DOMAIN.yaml +      hydra $HYDRA eyaml $DISK encrypt -q -o block --stdin -l some::random:password >> \ +      $CONFIG_FOLDER/puppet/config/secrets/node/$DISK.$DOMAIN.yaml  [hiera-yaml]: https://github.com/voxpupuli/hiera-eyaml @@ -100,7 +103,7 @@ Or this, for a random passphrase:  The proceed with the system installation, which should create all volumes in  the TPS and setup the base Operating System: -    hydra $HYDRA provision $VOLNAME +    hydra $HYDRA provision $DISK  ### Metadata @@ -112,23 +115,23 @@ can be useful during recovery procedures.  1. Write down UUIDS, models and serials from the new disk. Examples: -        lsblk $DEVICE -n -o +UUID | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/lsblk -        udevadm info --query=all --name=$DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/udevadm +        lsblk $DEVICE -n -o +UUID | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/lsblk +        udevadm info --query=all --name=$DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/udevadm          # For non-USB and non-microSD disks only -        $SUDO hdparm -I $DEVICE   | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/hdparm -        $SUDO smartctl -i $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/smartctl +        $SUDO hdparm -I $DEVICE   | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/hdparm +        $SUDO smartctl -i $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/smartctl  2. Save partition tables, the LUKS header and other relevant information: -        $SUDO LC_ALL=C sfdisk -d $DEVICE | keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/sfdisk +        $SUDO LC_ALL=C sfdisk -d $DEVICE | keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/sfdisk -        $SUDO cryptsetup luksDump /dev/mapper/$VOLNAME-root | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-root -        $SUDO cryptsetup luksDump /dev/mapper/$VOLNAME-swap | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-swap +        $SUDO cryptsetup luksDump /dev/mapper/$DISK-root | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-root +        $SUDO cryptsetup luksDump /dev/mapper/$DISK-swap | keyringer $HYDRA encrypt nodes/houdini.fluxo.info/luksDump-swap -        $SUDO cryptsetup luksHeaderBackup /dev/mapper/$VOLNAME-root --header-backup-file luksHeaderBackup-root +        $SUDO cryptsetup luksHeaderBackup /dev/mapper/$DISK-root --header-backup-file luksHeaderBackup-root          $SUDO chown $(whoami) luksHeaderBackup-root -        keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/luksHeaderBackup-root luksHeaderBackup-root +        keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/luksHeaderBackup-root luksHeaderBackup-root          chmod +w luksHeaderBackup-root          wipe luksHeaderBackup-root @@ -137,15 +140,15 @@ can be useful during recovery procedures.  * Optionally, configure ATA passphrases for the new disk. If so, make sure to    save the passphrase in the keyring: -        keyringer $HYDRA encrypt nodes/$VOLNAME.$DOMAIN/ata +        keyringer $HYDRA encrypt nodes/$DISK.$DOMAIN/ata  ## Mounting  Use the following whenever you need to mount the cartridge in another system: -    hydractl mount-media $VOLNAME +    hydractl mount-media $DISK -This will make the volume available at `/media/$VOLNAME`. +This will make the volume available at `/media/$DISK`.  ## Initial user @@ -161,7 +164,7 @@ home folder encrypted, backup your data and then proceed running  ### Syncing the user and encrypted home in the new cartridge -Then, with the new TPS cartridge mounted (`hydractl mount-media $VOLNAME`), do +Then, with the new TPS cartridge mounted (`hydractl mount-media $DISK`), do  the following to replicate your user and homedir there:      $SUDO chroot $DEST adduser $USER @@ -183,7 +186,7 @@ and is logged in there as `root`.  From another system, and with the cartridge mounted: -    hydractl sync-tpc $VOLNAME +    hydractl sync-tpc $DISK  This might involve a lot of data being transmitted.  If you want `sync-tpc` to  ignore all your QEMU virtual machine images hosted at `/var/cache/quemu`, setup @@ -201,7 +204,7 @@ Now that basic provisioning is completed, you can unmount the cartridge.  From another system, and with the cartridge mounted: -    hydractl umount-media $VOLNAME +    hydractl umount-media $DISK  ## Booting @@ -209,10 +212,10 @@ The current provisioning procedure (as of 2024-02) may yield into  a partially bootable system, and may need some additional manual  steps right after installation: -    hydractl mount-media $VOLNAME +    hydractl mount-media $DISK      sudo chroot $DEST      update-initramfs -v -u -    hydractl umount-media $VOLNAME +    hydractl umount-media $DISK  During boot, at the `(initramfs)` prompt, type this to successfully  unlock the encrypted volumes: @@ -221,7 +224,7 @@ unlock the encrypted volumes:      cryptsetup luksOpen /dev/mapper/volumename-swap swap      ^Ctrl-D -Where `volumename` is `$VOLNAME`. +Where `volumename` is `$DISK`.  Then, in the running system, do:  | 
