Tutorials

Duplicate Password Remover for Bitwarden

Bitwarden is an awesome, cross platform, password manager and generator. With a free account you are able to sync your passwords between your computers regardless of the operating system they are running. On a mobile device? No worries! Bitwarden will also sync your password to that device as well. Overall it is a great little tool to keep your passwords organized and protected. After using it for about a year, the only I really do not like about it is its inability delete, or even view duplicate passwords. After reading through part of Reddit where users were suggesting that I block out a part of my day to import the passwords and secure notes to excel and manually delete passwords that way I decided that there had to be a better option…..

After 10 minutes of DuckDuckGoing (It will catch on just wait) I was able to create a python script that takes the exported Bitwarden Vault as a Comma Separated Values File (CSV) as an input and exports it to a new CSV with the duplicates removed.

To run the code you will need to install python as well as the pandas plugin.

Steps:
1. Export from Bitwarden and Rename

  • First we are going to export our Bitwarden Vault to a CSV. This can be done by opening Bitwarden and going to —> File —> Export Vault.
  • Select.CSV when it asks for file format.
  • Enter your password and download the file.
  • Make a copy of the the exported .CSV file and name it to bitwarden.csv.

2. Download the Python Script and Configure Dependenices

duplicatefinder.py

The code for the duplicate finder is shown below:

import pandas as pd
df = pd.read_csv('bitwarden.csv', usecols=['folder', 'favorite', 'type', 'name', 'notes', 'fields', 'login_uri', 'login_username', 'login_password', 'login_totp' ]).drop_duplicates(keep='first').reset_index()
file_name = "bitwarden_deduped.csv"
df.to_csv(file_name, index=False) # you don't need to set sep in this because to_csv makes it comma delimited.

3. Run the script!

  • Simply run the script by navigating to the directory where you downloaded it and run python duplicatefinder.py.

4. Re-Upload Bitwarden Vault

  • After the script is run there should be a file called bitwarden_deduped.csv.
  • Open up this CSV file in the editor of your choice and check to make sure your important logins, secure notes, etc are still present and not duplicated.
  • If everything looks good it is time to re-upload.
  • Navigate to your Bitwarden Webvault (typically, Bitwarden Web Vault).
  • Verify one last time that all modfications made by the script are valid and that you are not missing any important passwords, secure notes, etc.
  • Click Settings at the top of the screen and then scroll to the bottom.
    Bitwarden Settings
  • Select Purge Vault, within the “Danger Zone” area at the bottom of the page.
    Danger Zone
  • Select Purge Vault
  • Navigate to the top of the page and select “Tools”
    Tools
  • Select Import Data
    Import Data
  • Select the CSV file type and upload bitwarden_deduped.csv
    Bitwarden CSV
  • Select Import Data

Congratulations!
All of your logins and notes should now be de-duplicated and should sync across your devices shortly! Hopefully this saves you some time so you can get back to doing what you do best!

Posted by admin in Tutorials

Hetzner Proxmox Network Setup

Setting up Proxmox can be somewhat difficult on a Hetzner dedicated server. Once the initial install is completed it gives one the false sense of hope that they are done with the install and ready to use it. In reality, setting up the network interfaces can be a time-consuming and quite annoying process. To alleviate the frustration from this, I have created a python script that is embedded below that will generate the file for /etc/network/interfaces. I recommend testing it while you still have KVM access to the server but that is not required and all “should” work fine. If you have any questions please email me at henry@hwrrobotics.com or use the contact form on this site. 

The Host’s IP, Host’s Gateway IP, and Hosts’s Netmask can be found in the Robot Console

https://robot.your-server.de/server

The Additional IP Information can be found in the Subnets section on the Robot page:

Clicking the second subnet so the checkmark appears and then clicking the plus will show all IP addresses:

You also need to turn on IP forwarding, run the command below to turn it on:

sysctl -w net.ipv4.ip_forward=1

If you would like an internal network between the VMs/Containers then copy and paste the below code to your /etc/network/interfaces file:

auto vmbr1
iface vmbr1 inet static
    address 10.20.30.1
    netmask 255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up iptables -t nat -A POSTROUTING -s '10.20.30.0/24' -o eth0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.20.30.0/24' -o eth0 -j MASQUERADE

Posted by admin

Using Openvpn Access Server to access your network from a remote location

In order to get Openvpn Access Server to allow ethernet bridging which will allow us to access our home network we must first install Openvpn Access Server. A tutorial by Digital Ocean is posted below that walks you through the Openvpn Access Server Setup. 

https://www.digitalocean.com/community/tutorials/openvpn-access-server-centos

Once the server is installed we want to to enable ethernet bridging on the setup page.

NOTE: In this article we are using port 943, Openvpn Access Server's default port. If you changed your port post installation then adjust accordingly to access the admin panel. 

  1. This can be done by logging into the admin side of the server using the domain/ip extension: mywebsite.com:943/admin
  2. Next we want to change the vpn mode to ethernet bridging which can be done my going to mywebsite.com:943/admin/vpn_mode
  3. The page should have two options, one for ethernet bridging and one for routing. We are going to choose the ethernet bridging option
  4. Once the option is selected press the save settings button at the bottom of the page and your server should be ready to go. 
  5. If your server's hostname is its ip address inside the network one may run into issues so it would probably be a good idea to check the server network settings available at: mywebsite.com:943/admin/server_network_settings
    1. ​For instance if your server's internal ip is 192.168.1.34 the vpn is going to have trouble connecting to it from outside the network. 
    2. Howerver, if your server's hostname/ip is something that is accessible from outside the network such as mywebsite.com or its external ip address the server should be good to go!

I hope this article helps you setup your Openvpn Access Server. If you have any question feel free to contact me at henry@hwrrobotics.com or through the contact section of the website. 

 

 

 

Posted by admin in Linux Tutorials

Spacer Calculator

Need to know the distance between multiple parts over a fixed distance? 

The spreadsheet below should help with those calculations. 

Good Luck!

Posted by admin
Managing a Headless CrashPlan Instance over SSH

Managing a Headless CrashPlan Instance over SSH

CrashPlan is a fantastic online and offline backup program. Although it technically does not have full Linux support it can be installed and run on most Linux systems. If the Linux instance has a monitor or is capable of running VNC or VNC-like services there may not be a need to manage it over SSH with X11 forwarding. However, if this is not the case X11 forwarding over an SSH connection is not only feasible but incredibly easy to setup and use. It should only take a maximum of 10 minutes to implement this feature for the first time. All subsequent connections should only take a couple minutes at most to initiate. Lets get started!

There is also a demonstration video embedded below:

 

1. Make sure that "xterm" is installed. 

a. On Debian based systems such as Ubuntu the command is as follows:

sudo apt-get update

sudo apt-get install xterm​​

2. Download puTTY from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

a. Enable X11 forwarding for the connection

3. Save the connection if you want to reuse it in the future.

4. Download and Install Xming X11 forwarding server

a. https://sourceforge.net/projects/xming/

5. Once Xming is installed and you are ready to run to connect to your remote instance start Xming and XLaunch

a. Double click both icons to run the programs. Note: nothing should appear after the initial configuration until the "CrashPlanDesktop" command is run from the Putty Terminal

6. Once the Xming Server is up and running it time to test the configuration

a. First load up the puTTY instance that has X11 forwarding enabled and connect to the server

b. Next type the command, CrashPlanDesktop and wait for the CrashPlan logo to appear

 

 

 

Useful Links:

http://unix.stackexchange.com/questions/12755/how-to-forward-x-over-ssh-from-ubuntu-machine

https://support.code42.com/CrashPlan/4/Getting_Started/Installing_The_Code42_CrashPlan_App

http://www.benwagner.net/computing/installing-crashplan-on-headless-server-x11-forwarding/

Posted by admin in Crashplan/Code42 Tutorials, Tutorials

Transfer Rate Calculator

 

Enter your transfer rate in kilobytes per second using the blue form below

Final amount transfered in GB shown in box

Posted by admin, 0 comments

Securely Backup Your Linux Server to Google Drive with AES Crypt and Rclone

Backing up to the cloud can be a great option for data backup. However, despite what companies may say about their security policies you are at their mercy once you upload your content to their servers. However, if you encrypt it before you upload it you have double security. It may seem super paranoid but worst case you just have two layers of security and it takes a little bit longer to work with the raw file, but if there is a hack that breaks into Google Drive, or Onedrive, or B2 or whatever cloud service provider you are using you are now at the mercy of the hacker. However, if you encrypt your files before uploading them you are safe as long as the hacker has not broken 256 bit AES which is highly unlikely.

Companies with Google Apps for Work or students with a school Google Drive Account probably have unlimited cloud storage. Now this is great for work or school documents but it is also a fantastic solution for easy to access server backups. Google Drive has very fast upload speeds and it does not charge for bandwidth like it’s bucket storage sibling does. Therefore, you have unlimited storage for free and possibly for life if your institution retains your email account after you graduate. If you keep working for the same company you probably will have access to the account as long as you are an employee for the company. With Logistics aside Google Drive provides a reliable place to store content. In this tutorial we are going to compress a folder with tar and pipe that command to AES Crypt after the file is encrypted we will run Rclone to upload it to Google Drive. We will also write the program for full cron compatibility so that you can run this command every day, every week, or whenever you want to.

Requirements:

  1. A linux machine that is connected to the internet with tar, AES Crypt, and Rclone installed.
  2. If running Ubuntu 14.04: How to install AES Crypt
  3. If running Ubuntu 14.04: How to install Rclone and configure it to use Google Drive
  4. Secure shell access to the server.
  5. 10 minutes to complete the install and modify the script to fit your needs.

If you have AES Crypt and Rclone installed and configured you should be able to modify the script below and have it run successfully.

 cd /opt
    mkdir websitebackup
    cd websitebackup
    rm hwrweb-$(date +%Y-%m-%d).tar.gz
    rm hwrweb-$(date +%Y-%m-%d).tar.gz.aes
    tar -cvf - /var/www/html | aescrypt -e -p plaintextpassword - >hwrweb-$(date +%Y-%m-%d).tar.gz.aes
    sudo /usr/sbin/rclone copy hwrweb-$(date +%Y-%m-%d).tar.gz.aes googledrive-backup:HWR-Robotics-webserver
    rm hwrweb-$(date +%Y-%m-%d).tar.gz
    rm hwrweb-$(date +%Y-%m-%d).tar.gz.aes

Posted by admin, 0 comments

Installing Rclone on Ubuntu 14.04

Backing up is crucial to the success of any website. However, backing up to cloud services such as Amazon S3, Google Drive, Google Bucket Storage, Yandex.Disk, and Backblaze B2 can be difficult and or time consuming to set up. However, Rclone makes that task so much easier. Today we are going to install Rclone on Ubuntu 14.04.

Yandex-Disk1.jpg

 

First you are going to want to go to their download page:

http://rclone.org/downloads/

 

You are going to want to download the appropriate version based on your system and whether it is 32 bit or 64 bit and whether or not it is running on ARM architecture.

We are going to use the 64 bit version which can be downloaded here:

http://downloads.rclone.org/rclone-v1.30-linux-amd64.zip




Once it is downloaded you are going to want to issue the following commands that are published on the Rclone website. Below I have copied and pasted the commands from their webpage, which is available here: http://rclone.org/install/

unzip rclone-v1.17-linux-amd64.zip
cd rclone-v1.17-linux-amd64
#copy binary file
sudo cp rclone /usr/sbin/
sudo chown root:root /usr/sbin/rclone
sudo chmod 755 /usr/sbin/rclone
#install manpage
sudo mkdir -p /usr/local/share/man/man1
sudo cp rclone.1 /usr/local/share/man/man1/
sudo mandb

If you want a complete install script I have composed one below that will download the file using wget and then implement the commands from Rclone’s website. It will also install the unzip package. The script is available below:

sudo apt-get install unzip -y

cd ~

wget http://downloads.rclone.org/rclone-v1.30-linux-amd64.zip

unzip rclone-v1.30-linux-amd64.zip

cd rclone-v1.30-linux-amd64

#copy binary file
sudo cp rclone /usr/sbin/
sudo chown root:root /usr/sbin/rclone
sudo chmod 755 /usr/sbin/rclone
#install manpage
sudo mkdir -p /usr/local/share/man/man1
sudo cp rclone.1 /usr/local/share/man/man1/
sudo mandb

 

Once it is installed you need to configure to work with your cloud storage host. The official configuration page is available here:

http://rclone.org/docs/

 

To setup Rclone for use with Google Drive you are going to want to type:

 

rclone config

n

#name of google drive remote host ex: googledrive-backup

googledrive-backup

6

#client_id leave blank

#client_secret leave blank

#If installing on a headless server type N

n

#you will be given a link to enter into your browser. The link will ask for your permission for Rclone to access your Google Drive account. Once permission is granted it will give you a verification code which you will copy and paste back into your ssh terminal.

#It will then ask you if the current setup is ok or if you want to edit it or delete it. If you are happy with the result press y which will set the remote host.

y

#If there are no more remote hosts that you want to add then you can type q to quit the configuration

q

Once you have your host set you can start copying files to it. For instance if we wanted to copy a file named test.txt into a folder on Google Drive named test (it doesn’t matter if the folder exists or not) we would enter the following:

rclone copy test.txt googledrive-backup:test

Note: If you are running rclone as an automated cron task you are going to need to include the full path to rclone or else it will not copy.

sudo /usr/sbin/rclone copy test.txt googledrive-backup:test

 

Congratulations! Now you are able to copy files from your server or computer to Google Drive with a few simple commands.

If you would like an offline version of this guide I have attached a pdf of it below:

Posted by admin, 0 comments

Installing AES Crypt on Ubuntu 14.04

Encryption is the backbone of modern computing and encrypting your files before uploading them to a cloud storage provider insures a higher level of security. AES Crypt is very simple to install on both Windows and Linux to make encryption very easy. In this tutorial we are going to install AES Crypt on Ubuntu 14.04.

First go to their website and download the appropriate file for your system (32bit or 64 bit) make sure it is the Gui version even though we will not be using the Gui version in this tutorial

https://www.aescrypt.com/download/

32 bit Linux https://www.aescrypt.com/download/v3/linux/AESCrypt-GUI-3.10-Linux-x86-Install.tgz

64 bit Linux https://www.aescrypt.com/download/v3/linux/AESCrypt-GUI-3.10-Linux-x86_64-Install.tgz

 

If you are on a 64 bit system you can use the following command to install AES Crypt:

wget https://www.aescrypt.com/download/v3/linux/AESCrypt-GUI-3.10-Linux-x86_64-Install.tgz

tar -zxf AESCrypt-GUI-3.10-Linux-x86_64-Install.tgz

sudo ./AESCrypt-GUI-3.10-Linux-x86_64-Install

 

You will be prompted with the following questions:

This will install AES Crypt for Linux on your computer.  Continue? [n/Y] Answer Y for the default install

Where do you want to install AES Crypt for Linux? [/usr/share/aescrypt] Press enter to complete the install unless you want to install AES Crypt to a different location.

 

If the installation completes it should display the following:

InInstalling Executables…         

InsIInstallation complete.

 

You can test the program by creating a file named test.txt and then encrypting it with the password of 123abc:

sudo nano test.txt

aescrypt -e -p 123abc test.txt

 

For more commands visit AES Crypt’s Linux usage page:

https://www.aescrypt.com/linux_aes_crypt.html

Below is a pdf of this guide for offline use:

Posted by admin, 0 comments

Free SSL, CDN, and DDoS Protection from Cloudflare!

       Have you ever wanted to secure your website with SSL to protect it and its users from MITM (man in the middle attacks)? Or increase your load times from different parts of the world? Well now you can for a whopping cost of $0. That’s right for free!

       Ok, enough with the sales pitch if you want to do all of the above use cloudflare.com. CloudFlare hosts tons of websites ranging from first websites to websites with millions of users. The free plan provides you with basic DDoS protection, a CDN (content delivery network), a free SSL certificate, and much more. DDos protection will protect your website from crashing due to a high influx of users either created by hackers or innocently by people who just want to view your content.

      The setup is relatively simple as you just have to go to cloudflare.com, create an account, add your website, and then transfer your nameservers for your domain to the ones that will be provided by CloudFlare. It says that it takes a few hours to transfer your domain, but it really only takes a couple on minutes. Once it is all synced up you will be able to configure your server from the control panel. 

Here is a simple tutorial from Digital Ocean on how to integrate CloudFlare with a Digital Ocean Droplet: https://www.digitalocean.com/community/tutorials/how-to-mitigate-ddos-attacks-against-your-website-with-cloudflare 

Posted by admin, 0 comments