Friday, May 31, 2013

Using Secure copy to bring all the log files together raspberry pi

So now that I have two RPis doing environmental monitoring, I have these log files on each machine being dumped locally. Some of our clients would like to see the logs so they know that we are keeping the proper temperature outlined in their contracts. I need a repository to dump them to and a way to copy them.

Across the network, we have other devices monitoring temperature, humidity, light, air, etc. I set up a Ubuntu box a long time ago for internal device monitoring before I went to Nagios. During that time, I used "wget" to go across the network and grab the log files from these devices and put them in a shared location on that Ubuntu box every so often which was fine. The log files were always the same name so it was easy.

Now that I have these RPis, it has become apparent that I would like the logs from these devices to go to the same place; this Ubuntu box with the other log files. So... how to get them there.

Using SCP, I set a cron job to copy them securely from one linux distro to another. Ill be referencing the Ubuntu box as well as the RPi but you can use RPis for the whole thing. It is not inclusive to just Ubuntu.

***If this gets technical, that is what we have the Google machine for. I knew nothing about secure copy before embarking on this but after about an hour or so, I was copying files back and forth with success.***

First you need to set up SSH to accept the copy. I found that while trying to copy using SCP, it always wanted a password after running SCP blahblahblah. So I needed to set up Password-less login with SSH.

Following these instructions at:

http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/

I ran (on the local machine, machine where the logs originate)

ssh-keygen

hitting enter all the way through. Then running:

ssh-copy-id -i ~/.ssh/id_rsa.pub remote-host

I was able to find out that I couldnt because the username on the Ubuntu box was not the same and the RPi. Nor the port... ugh. so I had to run this:

ssh-copy-id -P port -i ~/.ssh/id_rsa.pub user@remotehost

Which then worked and I was able to start an ssh session with the remote host without a password. I put this line:

scp -P remote_machine_port local_directory_where_logs_are user@remote_host:remote_directory

and saved it as a executable file that I am calling with THAT USERS CRON. I was trying to call it with roots cron but the keys I made wouldn't allow for a password-less login.

So typing on the local machine where the logs are originating from:

crontab -e

and putting this line in:

0 0 1 * * /home/pi/logcopy

it is successfully copying the contents of the log folder on the FIRST of each month to the remote machines log repository.