Linux bash to find files and exlude permission denied

So this is another of those "post to self" notes.
Lots of times you need to search for a file or directory under a user account and get this long list of "Permission Denied" entries back.

The below find command will exclude those.

Terminal command to find a directory with name image: 

find / -name image -type d 2>&1 | grep -v "Permission denied"

rsync with remove source files and directory

Command line to copy files and then remove the source (iow. simulate the mv command with rsync)

Terminal command for hard-restart

rsync -aP --remove-source-files /source/aa/bb/subfolder /destination/subfolder/ && rm -R /source/aa/bb/subfolder

Command line options used:

rsync:

-a  = archive
P   = progress
--remove-source-files  = remove the source files that were transferred successfully (not the source directory)

bash:

&& = will only execute if previous command successful

rm:

-R  = recursive (remove files and directories) that way both the files and directory are removed.

 
 

For when nohup is not available

So I recently had to execute a long running job on a bash shell,
however none of my usual tools like nohup, screen or tmux were available so below is the method I used to make sure I can start the job and disconnect the session.

1. run the job in bash
2. whilst running press ctrl-z (this will suspend the current foreground job (task)  )
3. enter the job control command bg to add it as a background task
4. then remove the job from the shell's job list disown -h 
5. exit

Apple OS X Manual Backup script with notifier

Recently I wanted to add some "user friendliness" to a backup script that do an FTP over TLS upload of multiple files and implemented the following:

• a bash script that iterate through a directory (find) and upload (curl) and send a notification once completed.
• used Automator to create an Application (that a user can run) - The application send a notification that execute when clicked
• Created a "Login item" that execute the Automater application upon login and remind the user to perform backups (and the user can do so by clicking on the notification)

The bash script:

Loading ....

remember to make it executable : chmod u+x Backup_QB.sh
Some gotchas during the script troubleshooting:

• If you have an @ (at sign) in the username or password replace (URL-encode) it with the HTML character code for it namely %40 e.g. username 'man@server' will be 'man%40server'
• find command in the script needs to be completed with ' \;' remember the space.

Automator application:

For the Automator script to work correctly I had to install terminal-notifier from this Github

into the 'Applications' folder and point the script to the binary inside the app bundle for the application to work.
The command:

/Applications/terminal-notifier/terminal-notifier.app/Contents/MacOS/terminal-notifier -message "Time to run your backups" -title "Backup Script" -execute ~/Desktop/Backup_QB.sh

This was picked up during testing as the first run was from the terminal and 'sudo gem install terminal-notifier' and everything worked from the command line.

Terminal-notifier was used as the -execute function can be added that will execute a command if clicked.
save this as something.app in the 'Applications' folder.

At this point you have a fully working application that will create a notification and if the notification is clicked it will execute the backup script and send a second notification when completed.

and in the notification centre if configured.
When you click on the notification it will perform the backups as per the script and send the following notification upon completion:

Then finally to make sure a reminder is send to the user a login item is added to the profile to send this reminder whenever the user log in.

Open the Users & Groups pane of System Preferences and click the Login Items tab then add the 'Application' created in step 1.












Till a next time ....

Delete log files older than x days on QNAP NAS

The easiest way to delete files older than 'x' days via command line (therefore a cron job can be created to delete it daily) would be:

find /share/Download/messages* -mtime +5 | xargs rm -rf

Then crontab -e and add the following to the bottom

1 0 * * * find /share/Download/messages* -mtime +5 | xargs rm -rf

This will then keep the last 5 days worth of logs.

Good article on editing cron located here : http://wiki.qnap.com/wiki/Add_items_to_crontab

Easy way to timestamp date on Ubuntu bash

ts from the moreutils package can be used to timestamp data:

~$ echo my_time_stamp_test | ts

Aug 08 08:09:14 my_time_stamp_test

The timestamp format can be changed as well

:~$ echo my_time_stamp_test | ts [%H.%M.%.S]

[08.12.05.066785] my_time_stamp_test

so to timestamp data collected from a serial port something like below can be used:

ts </dev/ttyS0 >arduino.log

And a true bash version :

<command> | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }'