rsync with remove source files and directory

Command line to copy files and then remove the source (iow. simulate the mv command with rsync)

Terminal command for hard-restart

rsync -aP --remove-source-files /source/aa/bb/subfolder /destination/subfolder/ && rm -R /source/aa/bb/subfolder

Command line options used:

rsync:

-a  = archive
P   = progress
--remove-source-files  = remove the source files that were transferred successfully (not the source directory)

bash:

&& = will only execute if previous command successful

rm:

-R  = recursive (remove files and directories) that way both the files and directory are removed.

 
 

Terminal commands for hard- reboot & shutdown

So i had the unfortunate scenario where I could not reboot or shutdown a Ubuntu server via the command line and no "out of band" access methods were available.

Luckily the following saved me:

Terminal command for hard-shutdown

sudo sh -c "echo o > /proc/sysrq-trigger"

Terminal command for hard-restart

sudo sh -c "echo b > /proc/sysrq-trigger"

Minecraft Python development environment to live server via Git, ssh and hooks

So I was starting up a Minecraft server for the kids using a Python interface on Github for them to learn and code some of the repetative tasks they complained about.

But as they will make alot of mistakes in the beginning and I don't want to give my 6 year old terminal access with permission to changes files to the server yet ... (as he will most probably keep on messing with his sisters' files) I decided to create a repository on the local pc where they can play/learn then "activate" as I called it or more technical term "commit" the changes and by magic the file they created will be available from within Minecraft.

So I thought I will quickly describe the process as I suspect there might me some more parents out there wanting to do this.

Please note it is a all Linux environment (client and server side).

Server side:

For this spesific use case Minecraft server was loaded on a VPS as well as Forge and mcpipy.

For the automated deployment a repository folder was created on the server

~/repo/mcpipy.git/

With the actual files located at (Production instance):

~/minecraft/mcpipy/

In other words the directory tree would be:

|
|-- minecraft
|           |-- mcpipy
|-- repo
       |-- mcpipy.git
                    |--




Then change into the repository and intitialize it as a git version controlled repository:

cd ~/repo/mcpipy.git/
git init --bare 

--bare does not have a working directory (just the version control) more can be found here and here. Think of --bare as a way to mark a repository as a storage facility, opposed to a development environment.

Git Hooks:

The Git repository you just created will have a folder called hooks and contain files with actions that you can hook and perform as set by you (e.g. 'pre-receive', 'post-receive' and 'update') more to be found in the Git documentation.

Change into the hooks directory and create a post-receive hook (iow. what will be done once we pushed files to the repoitory on the server from the PC and the server received all the files)

 ~/repo/mcpipy.git/hooks/post-receive

#!/bin/sh
git --work-tree=~/minecraft/mcpipy --git-dir=~/repo/mcpipy.git checkout -f

and make it executable

chmod +x post-receive

Then over to the local machine (where the kids will do the coding and push it from to the server)

Client side:

Create a folder for the development and git init it.

cd /kids/mcpipy
git init

Then we need to configure the remote path of our repository.

git remote add live ssh://user@minecraftserver/root/repo/mcpipy.git

Then create your first python file in the repository
And add & commit the files / changes.

git add .
git commit -m "Minecraft Python projects ready" 

then to make it "active" or basically deploy the files to the production environment you need to push to the 'live' 

git push live master

From within Minecraft just type /py and the file name to build/test it e.g.

Google Chromecast and Mikrotik Wireless

So i recently upgraded my wifi at home from a TP-Link to a Mikrotik hAP ac device/s, although the installation of CAPsMAN and provisioning of Configurations were rather quick and easy .. that was not the end.

This post will focus on the wireless connections between the Chromecast and Mikrotik AP.

As in most households these days multiple media devices, wireless speakers/assistants and tablets etc. need the ability to connect ot the wifi.

Below so far my tests and findings with the Google ChromeCast.

Mikrotik Version	6.37.3
Mikrotik Firmware (hAP ac)	3.34
Mikrotik Model	RouterBOARD 962UiGS-5HacT2HnT
ChromeCast Firmware	1.22.78337
ChromeCast Model	Chromecast 2

I have decided to simplify my SSID's and only have 2 available on the network: Private and Guest and have both bands (2ghz & 5ghz) / radios assigned to the same SSID's. The previous wireless installation had seperate SSID's for 2 ghz and 5 ghz.
 
Connecting the Chromecast to the network was a breeze, just launch the Google Home application, select devices > settings > Wi-Fi and enter the new credentials.

and then the fun started, family members start complaining they can't "see" the Chromecasr device to cast to, and constant reboot / power cycles started and I started to keep an eye on the Mikrotik log and wireless registration table (and funnily enough all seemed ok, but the user experience were correct the devices were no longer discoverable)

Initial setup:

I then decided to pidgeon hole the Chromecast onto it's own SSID and assign it only to one spesific radio, that way I can control & troubleshoot. I did so and created a Media SSID and assigned it to a radio. This is done by creating a new  CAP's provisioning with the target Radio MAC.

Provisioned the new configurations and learned lesson 1, for no money would the Chromecast want to join the new SSID, various other devices could connect to it though.

Lesson learned: I performed a factory reset (FDR) on the Chromecast and it seemlessly connected to the new newly created SSID.

I changed the configuration in the following order with all configurations experiencing the same result:

1. Started with  band=2ghz-b/g/n with same behaviour
2. Changed to band=2ghz-b/g as some people / forums suggested issues with n band, with same behaviour
3. Changed to the 5ghz radios with band=5ghz-onlyac, FDR the Chromecast and registered it to the Media SSID.

Current working configuration:


So currently it seem very stable and working without issues.

Media band = 5ghz-onlyac
radio = only the closest 5ghz Radio to the Chromecast




It is running for 3 days without any issues so far.

Inconsistencies and possible Chromecast bugs:

• I have one other Chromecast connected to the "Private" SSID without any issues at the moment, still to investigate further.
• On the Google Splash screen the Wireless SSID disappear sometimes with a "backdrop" change (As I told the kids to look for that to determine if the Chromecast is connected or not it causes false negatives at the moment :-)   )

Next steps:

• Let it run for a while and monitor.
• Change the Media SSID to hidden.




Winbox on OS X

Joshaven have complied “Winbox” by MikroTik with Wine in order to make it usable on Mac.
It can be downloaded from his website here

Below script can be used to manually update to the latest (3.7) Version.

rsync -avloz /Applications/Winbox4Mac.app/ /Applications/Winbox4Mac-3.7.app/

cd /Applications/Winbox4Mac-3.7.app/Contents/Resources/drive_c/Program\ Files/MikroTik

curl http://download2.mikrotik.com/routeros/winbox/3.7/winbox.exe --output winbox.exe

The manual way will keep the router list in tact.



Open Source & Cloud based Application Links

As I am using a lot of open source and cloud based applications I decided to start a list for reference:

https://insightmaker.com/

https://trello.com/

SSH Tunnel

The easiest way to create a socks compliant ssh tunnel:

From the computer inside the network:

ssh -D [port number] -fN user@server

Then you can configure the application or service on the machine to access the remote server via the SSH tunnel and port configured.

e.g.

ssh -D 3333 -fN user@server.com

bfgminer -S opencl:auto --socks-proxy localhost:3333 --debug

Apple OS X - command line network commands

How to show the routing table:

netstat -nr

How to add a route:

sudo route -n add [ip address]/[subnet] [gateway]

e.g.

$ sudo route -n add 10.0.0.0/24 172.16.0.1

List open files (Determine what target IP and application/process is connected to)

$ lsof -i

e.g. To determine what IP a Citrix session is connecting to:

$ lsof -i | grep -E "Citrix"

Favourite Linux Applications

I know what applications you use for day to day tasks is a very personal affair, however i had numerious people asking me what I use daily and sometimes what alternatives to commerial software I am using as such I though I would list them for reference, obviously all of these fit my spesific requirements. Lots of the time the need for me is to have applications that can work cross-platform with some sorts of a mechanism to sync between operating systems / devices.



As operating system for home and general use I am using Kubuntu Xenial.


 Then there is a few out of the box KDE applications I can't live without:
 

For Audio management there is nothing better on this planet than Amarok and an application i will never be without.

  VLC Media player is that swiss army knive for media playback.

Then for IRC I find Konversation the easiest to use.

For that dreaded email world we wlive in nothing do the task better than KMail. It integrates well with Google and for me the use of OpenPGP and encryption from within KMail was very easy to use.

Then to ensure all those Keys as managed easily KGpg is a simple interface for GnuPG.

For backups I found Back In Time to perform the easiest for me nad do have a KDE GUI.


Then more generic applications I use:

Favourite and first choice web browser is FireFox, as i find the web developer tools and Greasemonkey integration valuable.





Second choice for browser is Google Chrome.

Whenever I am not using Dolphins' location bar to perform FTP functionality, I will be using FileZilla client.

The one text editor that stands out head above the rest for me at this moment is Sublime Text. With multi column layout, Syntax Highlighting and the command pallet to perform actions it surely increase the productivity for anything that needs to be done in a text editor.

For all those backups and transfers, nothing better than Rsync, and if I need to use a GUI the best out there is Grsync.

Calibre eBook library management and reader application is another of those applications I will never be without. The ability to tag, and create series for my magamzines as well as convert to the proper format for reading on any of the devices in the household is invaluable.

Another one of those swiss army knives for audio tagging and management is Musicbrainz Picard. Written in Python with the ability to use plugins.

DupeGuru ME (Music Edition) does a great job at identifying duplicates in music collections and can do so based on many criteria.

Puddletag is the best to view & edit custom MP3 tags. As I add things like BPM to my MP3 audio files via a Picard plugin, Puddletag is a great way to confirm the BPM field.

Then for the tagging and library management of multimedia files, I have found the best GUI application to be MediaElch. Ability to sort Movies and Series and spesify source folders is great. It uses multiple scrapers as well.


Then for more work related stuff I do use:

Keepass for those password

I use KRDC for all those remote desktop connectivity (RDP and VNC)

yEd from yWorks is great for BPMN, flowcharts and semantic diagrams with an easy abillity to auto arrange and export.

One of those Gnome apps that I do use frequently as well is Dia.

then there is nothing better for those documents andspreadsheets than LibreOffice.

Graphical disk usage Filelight

Mount linux from OS X

If you need to mount an ssh/sftp volume and make it visible in finder to be able to use it as if it were a local volume you can use.

The easiest way is to install FUSE for macOS.