Auto Back up folders Linux

Coding

There are some web applications such as Sonarr v2 and Radarr that not have an auto back up wich with some basic bash scripting and Cron can be set up in less than five minutes, that way the next time you format your Pi you can easily restore a back up of your library and save those 15 mins of tedious setup.

Script

1) create your script: create a new file and call it myrsync.sh, copy/paste the lines below:

  #!/bin/bash
sudo rsync -av --progress --delete --log-file=/home/your-username/Desktop/$(date +%Y%m%d)_rsync.log --exclude "/home/your-username/.folder" /home/data /media/dataBackup_$(date +%Y%m%d_%T)

Meaning of the flags

-av bit: 'a' means archive, or copy everything recursively, preserving things like permissions, ownership and time stamps. 
-'v' is verbose, so it tells you what its doing, either in the terminal, in this case, in the log file.
--progress gives you more specific info about progress.
--delete checks for changes between source and destination, and deletes any files at the destination that you've deleted at the source.
--log-file saves a copy of the rsync result to a date-stamped file on my desktop.
--exclude leaves out any files or directories you don't want copied. In the command above, the .folder directory

/home/data is the directory I want copied. /home/data copies the directory and its contents, /home/data would just copy the contents. 

/media/dataBackup_$(date +%Y%m%d_%T) is the separate drive. Change this to whatever your backup location is. Note that `rsync` will name every sync differently based on day/time of sync

Save myrsync.sh in your ~$HOME and make it executable by typing:

sudo chmod +x /home/your-username/Desktop/rsync-shell.sh

You can now double click that .sh file, choose Run in Terminal, it will ask you for your password and run, then leave a log file on your desktop. Or, you can make a cron job to do it for you! and that’s where the auto back up happens.

The cron job

Copy your myrsync.sh file to /root by typing:

sudo cp /home/your-username/Desktop/myrsync.sh /root

Then type:

sudo crontab -e

You’ll see a line which reads: minute hour day month year command

Under that, type: 0 22 * * * /root/myrsync.sh > $HOME/readme.log 2>&1

This means:

The hour in military time (24 hour) format (0 to 23)
The day of the month (1 to 31)
The month (1 to 12)
The day of the week(0 or 7 is Sun, or use name)
The command to run
So at 22:00 (10pm) every day root will run the shell script, without prompting you for sudo password (because its running as root already).

Now press Control-X, then type “Y”, then press Enter

In order to delete older back ups, one way of doing this is to create a file with the timestamp of every sync in it. For example add the following command after the command rsync in myrsync.sh

date +%Y%m%d_%T >> time.txt

Use the command find to delete backups that matches the timestamp e.g: add this command after the date +%Y%m%d_%T >> time.txt in myrsync.sh

find . -type f ! -newer /tmp/timestamp -delete

Or

find . ! -newermt $date ! -type d -delete

This will delete back ups before specific date/time.

More details and sample codes for hourly/daily/monthly backups can be found here

That is how you can set up an easy auto back up of applications that don’t possess one, basic script and Cron job on Linux, this is applicable for most of distros Linux.

Continue Reading Auto Back up folders Linux

Home Assistant and Docker

home_asistant_docker

Nowadays smart homes are becoming pretty common and having multiple devices from different manufacturers are becoming a millenial problem where we dont want to have multiple apps in order to check routines or whats off, of course this helps in saving electricity but is not an easy task but with Home Assistant and Docker that can change.

There are other alternatives to this set up such as Alexa or Google Assistant but Home Assistant is a really good alternative if you want to keep your privacy locally and not shared with those two internet monsters.

Home Assistant is open-source, self-hosted software that will allow you to control various devices in your home.

Docker has become one of the easiest ways to set up virtual enviorements for apps on Linux, Mac and even Windows Machines.

Be sure to run sudo apt-get update and sudo apt-get upgrade.

Install Docker

Run the following command:

curl -sSL https://get.docker.com | sh

Permissions

You are going to want to add the proper permissions to your primary user account to easily run docker commands. To do that, run the following command:

sudo usermod -aG docker YOURUSER

Replace YOURUSER with the name of the user you are logged in as.

Since we made some changes to our user, we will now need to log out and log back in for it to take effect.

You can log out by running the following command in the terminal.

logout

Once you have logged back in, you can verify that the docker group has been successfully added to your user by running the following command.

groups

This command will list out all the groups that the current user is a part of. If everything worked as it should, the group docker should be listed here.

Test

Docker should be running by this point. Running the following command should pull an image from the Docker Hub and display hello-world on your screen.

docker run hello-world

Troubleshooting Docker Installation

If you get the following error:

error_1
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

Run

sudo dockerd

If dockerd shows the following error:

error_2
failed to start daemon: Error initializing network controller: list bridge addresses failed: PredefinedLocalScopeDefaultNetworks List:

Then run the following two commands:

sudo ip link add name docker0 type bridge
sudo ip addr add dev docker0 172.17.0.1/16

Activate the service:

sudo systemctl enable docker
sudo systemctl start docker

Test Docker:

docker run hello-world

If you get the following error:

error_3
Unable to find image 'hello-world:latest' locally

Then type:

docker login

You have to have an account with docker if you dont have it then sign up for one.

and then test again:

docker run hello-world

Install Dependencies (Docker Compose)

To keep everything running smoothly, you’ll want to run the following commands to install the following dependencies:

sudo apt-get install -y libffi-dev libssl-dev
sudo apt-get install -y python3 python3-pip
sudo apt-get remove python-configparser

Install Docker Compose (Optional)

For more advanced scenarios, you can install Docker Compose to allow you multiple containers that are dependent on each other with a single command:1

sudo pip3 -v install docker-compose

Create a Volume for Home Assistant

In order to store persistent data, we have to first create a volume in Docker. This is where Home Assistant will store configuration data. Make sure you store it in a location that you can easily access the files. In order to properly work with Home Assistant, you will be consistently editing various configuration .yaml.

docker volume create home_assistant

Home Assistant Install

Below is the docker run command. After creating the home_assistant volume, you can copy and paste this command into your terminal in order to download and run the Home Assistant image from the Docker hub. Make sure you change the environment variable -e TZ=America/New_York to your time zone. You can locate your TZ database name here.

Also, it is required to use the docker host network (--net=host) for the Home Assistant container image.

home_assistant_install
docker run --init -d \
--name="home-assistant" \
-e TZ=America/New_York \
-v home_assistant:/config \
--net=host \
homeassistant/home-assistant:stable

Variables

Below is the list of variables in the above run code.

VariableDescription
--name="home-assistant"The name of your docker container
-e TZ=America/New_YorkYour time zone from the TZ Database
-v home_assistant:/configPersistent storage location for the /config folder within the container
--net=hostPlaces the container on the host network

Finished

After you run the above Docker run command, you will be able to access your installation at http://YOURIPADDRESS:8123. Now you will be prompted to set up your user account and can begin adding your devices.

Now you have installed Home assistant and Docker.

Remove Docker from System.

To completely uninstall Docker:

Step 1

dpkg -l | grep -i docker

To identify what installed package you have:

Step 2

sudo apt-get purge -y docker-engine docker docker.io docker-ce docker-ce-cli
sudo apt-get autoremove -y --purge docker-engine docker docker.io docker-ce

The above commands will not remove images, containers, volumes, or user created configuration files on your host. If you wish to delete all images, containers, and volumes run the following commands:

sudo rm -rf /var/lib/docker /etc/docker
sudo rm /etc/apparmor.d/docker
sudo groupdel docker
sudo rm -rf /var/run/docker.sock

Continue Reading Home Assistant and Docker

Google Sheets and Python

Nowadays Google Docs has been integrating a whole bunch of features with its amazing API that you can use from triggering a Python script hosted in a local server by a request from IFTTT webhooks for example, which is the final project of this series of posts about my self-education in python and a whole bunch of boring stuff that I love.

Over here I will not explain about basics of Python or the project mentioned before, here I will just remind my self and hopefully others about creating your JSON file that I will use in my python script, for access to Google sheets and update a specific cell, necessary to have a sheet created for this project.

I will assume that we have a Google console already set up if you don’t have one, why are you reading this post?.

Creating a New Project on Google Cloud

  1. Go to the Google APIs Console.
  2. Create a new project.
  3. Click Enable API. Search for and enable the Google Sheets API.
  4. Click Enable API. Search for and enable the Google Drive API.
  5. Create credentials for a Web Server to access Application Data.
  6. Name the service account and grant it a Project Role of Editor.
  7. Download the JSON file.
  8. Copy the JSON file to your code directory and rename it to creds.json

If you need more details check the video below.

Sharing the Sheet

  • find the  client_email inside. Back
  • In your spreadsheet, click the Share button in the top right, and
  • paste the client email into the People field to give it edit rights.
  • Hit Send.

If you skip this step, you’ll get a gspread.exceptions.SpreadsheetNotFound error when you try to access the sheet from Python.

Read Data from a Gsheet with Python

This is the easy part now that we have the code from the Sheet URL, which we will use in our python code.

  1. oauth2client – to authorize with the Google Drive API
  2. gspread – to interact with Google Spreadsheets

Install Modules via pip

pip install gspread oauth2client

Then paste this code into a new file called spreadsheet.py:

import gspread
from oauth2client.service_account import ServiceAccountCredentials

# use creds to create a client to interact with the Google Drive API
scope = ["https://spreadsheets.google.com/feeds",'https://www.googleapis.com/auth/spreadsheets',"https://www.googleapis.com/auth/drive.file","https://www.googleapis.com/auth/drive"]
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
client = gspread.authorize(creds)
#I have the JSON File saved in the same directory as this script.
# Find a workbook by name and open the first sheet
# Make sure you use the right name here.
sheet = client.open("wanatux test").sheet1
#For some reason this only works in sheet1, you wont be able to use another one as far as today 7/29/2020
# Extract and print all of the values
list_of_hashes = sheet.get_all_records()
print(list_of_hashes)

data = sheet.get_all_records() # Get a list of all records

row = sheet.row_values(3) # Get a specific row
col = sheet.col_values(3) # Get a specific column
cell = sheet.cell(1,2).value # Get the value of a specific cell

insertRow = ["hello", 5, "red", "blue"]
sheet.add_rows(insertRow, 4) # Insert the list as a row at index 4

sheet.update_cell(2,2, "CHANGED") # Update one cell

numRows = sheet.row_count # Get the number of rows in the sheet

Run python spreadsheet.py and you will be able to see the data or manipulate the sheet, above you can some commands for testing..

Insert, Update, and Delete from a Spreadsheet with Python

We’ve just scratched the surface of gspreads’ well documented and comprehensive functionality. 

For instance, we extracted the data into a list of hashes, but you can get a list of lists if you’d prefer:

sheet.get_all_values()

Or you could just pull the data from a single row, column, or cell:

sheet.row_values(1)

sheet.col_values(1)

sheet.cell(1, 1).value

You can write to the spreadsheet by changing a specific cell:

sheet.update_cell(1, 1, "I just wrote to a spreadsheet using Python!")

Or you can insert a row in the spreadsheet:

row = ["I'm","inserting","a","row","into","a,","Spreadsheet","with","Python"]
index = 1
sheet.insert_row(row, index)

You can also delete a row from the spreadsheet:

sheet.delete_row(1)

And find out the total number of rows:

sheet.row_count

Check the gspread API reference for the full details on these functions along with a few dozen others. 

Continue Reading Google Sheets and Python

Crypto Mining ETH and Monero

bitcoin

I took some time to post something related to Crypto Mining since is not profitable as a home user anymore but still is a way to learn new things and gamble some money in a possible bitcoin wav, nowadays we dont have only bitcoin, there are some others coins that might have a bright future, but over here I will just cover the basics in mining Etherium and Monero using a specific mining software not nicehash, kudo miner or honeyminer .

Crypto Mining is possible with a single video card but if that is that you want to do just for the hell of it keep in mind that you will not meaking a lot of money, with a single card I have been making between $0.60 to $0.85 with a NVIDIA GTX 1660 ti and I am having between 25MH/s to 27MH/s overclocked with Asus GPU Tweak.

Phoenix Miner

Phoenix Miner is the software I use for Etherium mining and I joined Ethermine pool .

PhoenixMiner is Ethash (ETH, ETC, Muiscoin, EXP, UBQ, etc.) miner that supports both AMD and Nvidia cards (including in mixed mining rigs). It runs under Windows x64 and Linux x64 and has a developer fee of 0.65%. This means that every 90 minutes the miner will mine for us, its developers, for 35 seconds.

The settings are pretty straight you can just copy and paste the following code, just remember to change the ETH wallet address with your own.

REM
REM Example bat file for starting PhoenixMiner.exe to mine ETH
REM

setx GPU_FORCE_64BIT_PTR 0
setx GPU_MAX_HEAP_SIZE 100
setx GPU_USE_SYNC_OBJECTS 1
setx GPU_MAX_ALLOC_PERCENT 100
setx GPU_SINGLE_ALLOC_PERCENT 100

REM IMPORTANT: Replace the ETH address with your own ETH wallet address in the -wal option (Rig001 is the name of the rig)
PhoenixMiner.exe -pool ssl://us1.ethermine.org:5555 -pool2 ssl://us2.ethermine.org:5555 -wal 0xbdbe490381Fd9d477da73c61bDFfBdfa120F5d7c.rig001
pause

In that code you need to change the wallet address and name to whatever you want in that example I have: “0xbdbe490381Fd9d477da73c61bDFfBdfa820F7d7c.rig001”

The address is the 0x… the end of the code rig001 is your PC name and it can be changed to anything you want.

The pool address are the default link and port for Ethermine pool, you can keep that if you want to mine in that pool otherwhise just change it to whatever protocols your desired pool provides to its users.

If you dont have a ETH wallet address get one downloading the multi wallet “Atomic Wallet” its free and is amazing, or use Ledger which is a must have hardware if you are planing in mining for the long term.

XMRIG Monero Miner

Monero and Bitcoin are the most used coins in the tor community, Monero is beign accepted because of its annonymity since you cant see the balance or transfer for a specific wallet not like bitcoin that people can track down you wallet and see your balance since they care about transparency.

In order to get a Monero wallet address follow this link.

The software for mining i use is XMRIG whenever you have it in your PC just edit a few settings in the config.json file, go to lines 88-90 since those are the lines that you have to change.

    "pools": [
        {
            "algo": null,
            "coin": null,
            "url": "gulf.moneroocean.stream:10001",
            "user": "1A9tTWRtjMV3Jh11NTqwa6KFUfKikMJARB5MguJsR7WwCN9C4Bza8dF63ze8M1msdE1A9BPuU98jGCELxCVRJ4np3T2C7yS",
            "pass": "change@please.com",
            "rig-id": null,
            "nicehash": false,
            "keepalive": false,
            "enabled": true,
            "tls": false,
            "tls-fingerprint": null,
            "daemon": false,
            "socks5": null,
            "self-select": null
        }
    ]

The url line is the pool address and port in that example I am using MoneroOcean pool, user is the Monero wallet and pass you can enter whatever you want but for that pool if you use your email you can get notifications for whenever your rig mines or stops mining.

Crypto Mining is so extend that this is just a bare example of what you can do and how to set up the config files for a couple of miners, there is also software that can recognize which Crytocurrency is the more profitable depending on the time such as Nice Hash, Beeminer, Cudo Miner just to mention some, Crypto Mining is fun but keep in mind that you will not make money unless you invest money in a good mining rig.

Continue Reading Crypto Mining ETH and Monero

PGP tool Gpg4win

cyber-security

Most likely if you found this post is because you are already reading about PGP posible in the DN Bible or maybe you landed an onion page that is asking for a PGP auth for login, so here is a little introduction about PGP and how to install a PGP tool named Gpg4win.

PGP stands for Pretty Good Privacy, is a data encryption tool that gives cryptographic privacy that you can use for emails and authentication for some websites (which I wish we could use on surface internet). you create “keys” which are associated with a username and a password, this key is stored in your PC.

Basically the key you create will act as a lock box and that lock box has two keys, a key you can use to share to your contacts or websites and they can use it to encrypt the messages, if they encrypt that message they will not be able to decrypt it, the only way to decrypt it is with your private key.

Gpg4win is a suite of five utilities. The utilities included are:

  • Kleopatra.
  • GPA.
  • GpgO.
  • GpgEX.
  • Claws Mail.
  • Gpg4win Compendium.

Install Gpg4win

  1. Download the software from here GPG4Win download site 
  2. The installation window will have by defult the required software and in them we have Kleopatra which is the must have at this moment, we can read more about the others but that will no be covered here.

Generate Your Key Pair

Open Kleopatra component.

  1. Click File > New Certificate.
  2. Click on Create a personal OpenPGP key pair.
  3. It will as for a username, email and comment, the only field that is a must to enter is the username.
  4. Some websites will require a specific type of encryption, you can find that by clicking on advanced some will ask for 4,096 bits option.
  5. Enter a value in the Passphrase this can be a strong password or multiple words such as “new york houston texas”
  6. Click “Finish”.

Obtain Your Public Key

This is what you will use to share to others, and this is what you import in your kleopatra app for whenever you want to send encrypted messages to specific contacts.

  1. Open Kleopatra. Right-click on your key and select Export Certificates.
  2. Browse where you want to save the certificate, give the file a name, and save.
  3. This will create a file with extension “.asc” which you can open using notepad and there you can see your public PGP Key that you can share to others.

Obtain your Private Key

Do not share this key to anyone, back it up otherwhise you wont be able to decrypt or use any website that has your public PGP for auth.

  1. Kleopatra right-click on your key and select Export Secret Keys.
  2. Select where you will save it with extention “.asc”
  3. Be sure to keep this key in a safe place and never share it.

Import a Public Key

This is how you will import your contacts or markets keys to your Kleopatra.

  1. Copy the entire key block including ---BEGIN PGP PUBLIC KEY BLOCK--- to ---END PGP PUBLIC KEY BLOCK---.
  2. Save it in a new notepad with extention “.asc”
  3. Kleopatra Click file> Import.

Importing Your Private Key

  1. Kleopatra Click File > Import.
  2. Use the File Explorer to browse to where you’ve saved your private key. Select the key file and click open.
  3. The imported private key now displays under the My Certificates tab.

Decrypting a Message

Only your private key can decrypt encrypted messages. To decrypt a message using GPG4Win, do the following:

  1. Copy the entire contents of the message including the ---BEGIN PGP PUBLIC KEY BLOCK--- to ---END PGP PUBLIC KEY BLOCK---.
  2. Kleopatra click on notepad paste the message and click Decrypt/Verify.
  3. It will ask for the passphrase you created when you generated your key pair.
  4. After entering the password it will show a message that the message was decrypted and you will be able to see the contents.

Continue Reading PGP tool Gpg4win