Auto Back up folders Linux

Coding

There are some web applications such as Sonarr v2 and Radarr that not have an auto back up wich with some basic bash scripting and Cron can be set up in less than five minutes, that way the next time you format your Pi you can easily restore a back up of your library and save those 15 mins of tedious setup.

Script

1) create your script: create a new file and call it myrsync.sh, copy/paste the lines below:

  #!/bin/bash
sudo rsync -av --progress --delete --log-file=/home/your-username/Desktop/$(date +%Y%m%d)_rsync.log --exclude "/home/your-username/.folder" /home/data /media/dataBackup_$(date +%Y%m%d_%T)

Meaning of the flags

-av bit: 'a' means archive, or copy everything recursively, preserving things like permissions, ownership and time stamps. 
-'v' is verbose, so it tells you what its doing, either in the terminal, in this case, in the log file.
--progress gives you more specific info about progress.
--delete checks for changes between source and destination, and deletes any files at the destination that you've deleted at the source.
--log-file saves a copy of the rsync result to a date-stamped file on my desktop.
--exclude leaves out any files or directories you don't want copied. In the command above, the .folder directory

/home/data is the directory I want copied. /home/data copies the directory and its contents, /home/data would just copy the contents. 

/media/dataBackup_$(date +%Y%m%d_%T) is the separate drive. Change this to whatever your backup location is. Note that `rsync` will name every sync differently based on day/time of sync

Save myrsync.sh in your ~$HOME and make it executable by typing:

sudo chmod +x /home/your-username/Desktop/rsync-shell.sh

You can now double click that .sh file, choose Run in Terminal, it will ask you for your password and run, then leave a log file on your desktop. Or, you can make a cron job to do it for you! and that’s where the auto back up happens.

The cron job

Copy your myrsync.sh file to /root by typing:

sudo cp /home/your-username/Desktop/myrsync.sh /root

Then type:

sudo crontab -e

You’ll see a line which reads: minute hour day month year command

Under that, type: 0 22 * * * /root/myrsync.sh > $HOME/readme.log 2>&1

This means:

The hour in military time (24 hour) format (0 to 23)
The day of the month (1 to 31)
The month (1 to 12)
The day of the week(0 or 7 is Sun, or use name)
The command to run
So at 22:00 (10pm) every day root will run the shell script, without prompting you for sudo password (because its running as root already).

Now press Control-X, then type “Y”, then press Enter

In order to delete older back ups, one way of doing this is to create a file with the timestamp of every sync in it. For example add the following command after the command rsync in myrsync.sh

date +%Y%m%d_%T >> time.txt

Use the command find to delete backups that matches the timestamp e.g: add this command after the date +%Y%m%d_%T >> time.txt in myrsync.sh

find . -type f ! -newer /tmp/timestamp -delete

Or

find . ! -newermt $date ! -type d -delete

This will delete back ups before specific date/time.

More details and sample codes for hourly/daily/monthly backups can be found here

That is how you can set up an easy auto back up of applications that don’t possess one, basic script and Cron job on Linux, this is applicable for most of distros Linux.

Continue Reading Auto Back up folders Linux

Home Assistant and Docker

home_asistant_docker

Nowadays smart homes are becoming pretty common and having multiple devices from different manufacturers are becoming a millenial problem where we dont want to have multiple apps in order to check routines or whats off, of course this helps in saving electricity but is not an easy task but with Home Assistant and Docker that can change.

There are other alternatives to this set up such as Alexa or Google Assistant but Home Assistant is a really good alternative if you want to keep your privacy locally and not shared with those two internet monsters.

Home Assistant is open-source, self-hosted software that will allow you to control various devices in your home.

Docker has become one of the easiest ways to set up virtual enviorements for apps on Linux, Mac and even Windows Machines.

Be sure to run sudo apt-get update and sudo apt-get upgrade.

Install Docker

Run the following command:

curl -sSL https://get.docker.com | sh

Permissions

You are going to want to add the proper permissions to your primary user account to easily run docker commands. To do that, run the following command:

sudo usermod -aG docker YOURUSER

Replace YOURUSER with the name of the user you are logged in as.

Since we made some changes to our user, we will now need to log out and log back in for it to take effect.

You can log out by running the following command in the terminal.

logout

Once you have logged back in, you can verify that the docker group has been successfully added to your user by running the following command.

groups

This command will list out all the groups that the current user is a part of. If everything worked as it should, the group docker should be listed here.

Test

Docker should be running by this point. Running the following command should pull an image from the Docker Hub and display hello-world on your screen.

docker run hello-world

Troubleshooting Docker Installation

If you get the following error:

error_1
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

Run

sudo dockerd

If dockerd shows the following error:

error_2
failed to start daemon: Error initializing network controller: list bridge addresses failed: PredefinedLocalScopeDefaultNetworks List:

Then run the following two commands:

sudo ip link add name docker0 type bridge
sudo ip addr add dev docker0 172.17.0.1/16

Activate the service:

sudo systemctl enable docker
sudo systemctl start docker

Test Docker:

docker run hello-world

If you get the following error:

error_3
Unable to find image 'hello-world:latest' locally

Then type:

docker login

You have to have an account with docker if you dont have it then sign up for one.

and then test again:

docker run hello-world

Install Dependencies (Docker Compose)

To keep everything running smoothly, you’ll want to run the following commands to install the following dependencies:

sudo apt-get install -y libffi-dev libssl-dev
sudo apt-get install -y python3 python3-pip
sudo apt-get remove python-configparser

Install Docker Compose (Optional)

For more advanced scenarios, you can install Docker Compose to allow you multiple containers that are dependent on each other with a single command:1

sudo pip3 -v install docker-compose

Create a Volume for Home Assistant

In order to store persistent data, we have to first create a volume in Docker. This is where Home Assistant will store configuration data. Make sure you store it in a location that you can easily access the files. In order to properly work with Home Assistant, you will be consistently editing various configuration .yaml.

docker volume create home_assistant

Home Assistant Install

Below is the docker run command. After creating the home_assistant volume, you can copy and paste this command into your terminal in order to download and run the Home Assistant image from the Docker hub. Make sure you change the environment variable -e TZ=America/New_York to your time zone. You can locate your TZ database name here.

Also, it is required to use the docker host network (--net=host) for the Home Assistant container image.

home_assistant_install
docker run --init -d \
--name="home-assistant" \
-e TZ=America/New_York \
-v home_assistant:/config \
--net=host \
homeassistant/home-assistant:stable

Variables

Below is the list of variables in the above run code.

VariableDescription
--name="home-assistant"The name of your docker container
-e TZ=America/New_YorkYour time zone from the TZ Database
-v home_assistant:/configPersistent storage location for the /config folder within the container
--net=hostPlaces the container on the host network

Finished

After you run the above Docker run command, you will be able to access your installation at http://YOURIPADDRESS:8123. Now you will be prompted to set up your user account and can begin adding your devices.

Now you have installed Home assistant and Docker.

Remove Docker from System.

To completely uninstall Docker:

Step 1

dpkg -l | grep -i docker

To identify what installed package you have:

Step 2

sudo apt-get purge -y docker-engine docker docker.io docker-ce docker-ce-cli
sudo apt-get autoremove -y --purge docker-engine docker docker.io docker-ce

The above commands will not remove images, containers, volumes, or user created configuration files on your host. If you wish to delete all images, containers, and volumes run the following commands:

sudo rm -rf /var/lib/docker /etc/docker
sudo rm /etc/apparmor.d/docker
sudo groupdel docker
sudo rm -rf /var/run/docker.sock

Continue Reading Home Assistant and Docker

MDADM

Mdadm is the modern tool most Linux distributions use these days to manage software RAID arrays.

You should be able to use any block storage devices in a RAID. Any standard directions for setting up a RAID using SATA HDD’s should be applicable when using USB storage as well. You’ll have to set it up so that the USB devices are assembled as members of the RAID array.

Continue Reading MDADM

NZBGet Pi

NZBGet is the must have software if you are into the usenet databases, basically this will help you searching the files that you requested in Sonarr, Radarr or any other data management tool that you are using for your server, I ended up using NZBGet because it requires less resources than Sabnzbd for my Raspberry PI.

The interface is clean and has plugins that you can set up, such as renaming (if you need help renaming you can also do it with Sonarr or Radarr), detecting files extensions that you dont need for example “.iso”, they dont have the feature to auto comment like sabzbd but usenet providers dont like that feature anyways.

First SSH into Pi

Now we’ll download the and install the NZBGet to /opt.
To do this we’ll need to make a directory

sudo mkdir /opt/nzbget

Download the run script to /tmp using wget with the -P flag to determine the destination directory

Continue Reading NZBGet Pi