Saturday, 18 October 2014

Creating RRD graphs for Raspbery Pi / DS18B20


So I wanted to move the TME sensor I bought in my previous post to the bitcoin mining room, to measure the temperature there.

I still needed to monitor the temp in the comms cupboard where I have 4 HP Microserver NASs, UPS, firewall, HP managed switch etc etc to make sure it isn't going to melt.

I saw this excellent article here that uses simple off the shelf electronics to accomplish this. Following this and buying the parts listed from Maplins and Pimori and ebay.

Maplins parts list:
Resistor pack (you never know when you might need more)

Pimori parts list:
Hook up wire (looked cool with all the colours)
ADA fruit pi - t cobbler
Adafruit raspberry pi perma proto
Raspberry Pi GPIO cable

Ebay parts list:
ds18b20t (waterproof version)
Total cost of parts incl Raspberry pi b model < £90

So I did some basic config following the article there and basically adding the following modules to /etc/modules
w1-gpio
w1_therm
then in /sys/bus/w1/devices/ should have the sensors with their individual serials there

phil@temp-comms /sys/bus/w1/devices $ ls -al
total 0
drwxr-xr-x 2 root root 0 Oct 18 00:13 .
drwxr-xr-x 4 root root 0 Oct 18 00:13 ..
lrwxrwxrwx 1 root root 0 Oct 18 00:13 28-0000062d26d0 -> ../../../devices/w1_bus_master1/28-0000062d26d0
lrwxrwxrwx 1 root root 0 Oct 18 00:13 28-0000062e10ae -> ../../../devices/w1_bus_master1/28-0000062e10ae
lrwxrwxrwx 1 root root 0 Oct 18 00:13 28-0000062e511a -> ../../../devices/w1_bus_master1/28-0000062e511a
lrwxrwxrwx 1 root root 0 Oct 18 00:13 w1_bus_master1 -> ../../../devices/w1_bus_master1
phil@temp-comms /sys/bus/w1/devices $ 



For each sensor there will be a 28-xxx directory, and under each directory 280000062xxxx there is is a file called w1_slave. Cat that and it'll have the temperature
phil@temp-comms /sys/bus/w1/devices/28-0000062d26d0 $ cat w1_slave
c7 01 4b 46 7f ff 09 10 01 : crc=01 YES
c7 01 4b 46 7f ff 09 10 01 t=28437
phil@temp-comms /sys/bus/w1/devices/28-0000062d26d0 $
t=temperature, but it's 1000 higher than it should, so just divide it by 1000 to get the temp in degrees celcius.



and then followed this article to use multiple temp sensors with the wiring like so




So what this article doesn't tell you is how to produce the graphs.
So obviously setting up cron with 1 minute intervals like so in /etc/cron.d/logs
* * * * * root /usr/local/bin/temp.sh 

with temp.sh containing
#!/bin/sh
file=/var/www/temp.txt
echo -n spare \ > $file
cat /sys/bus/w1/devices/28-0000062e511a/w1_slave | grep t | awk '{print $10}' | cut -d = -f 2 >> $file

echo -n comms_top \ >> $file
cat /sys/bus/w1/devices/28-0000062d26d0/w1_slave | grep t | awk '{print $10}' | cut -d = -f 2 >> $file

echo -n comms_bottom \ >> $file
cat /sys/bus/w1/devices/28-0000062e10ae/w1_slave | grep t | awk '{print $10}' | cut -d = -f 2 >> $file
Note the serials for the temp sensors were ascertained by individually plugging them in, making a note and appying labels physically to the device so I knew what I was monitoring.

So after the the data was grabbed and stored in a text file, you have to process it. I decided on PHP, cos it's easy and can be used easily on apache, which I can use to publish the data for view via a browser.
So after installing apache2, libapache2-mod-php5, php5-rrd,  rrdtool etc you need to input the data into an RRD

Using PHP and 
#!/usr/bin/php

 // Fetch current time
    $t = time();
    $filename1="/mnt/ramdisk/temp-spare.txt";
    $filename2="/mnt/ramdisk/temp-bottom.txt";
    $filename3="/mnt/ramdisk/temp-top.txt";

    $handle = fopen($filename1, "r");
    $contents1 = fread($handle, filesize($filename1));
    fclose ($handle);
    $contents1=(float)$contents1/1000;

    $handle = fopen($filename2, "r");
    $contents2 = fread($handle, filesize($filename2));
    fclose ($handle);
    $contents2=(float)$contents2/1000;

    $handle = fopen($filename3, "r");
    $contents3 = fread($handle, filesize($filename3));
    fclose ($handle);
    $contents3=(float)$contents3/1000;

   $option = array ("$t:$contents1:$contents2:$contents3");
        echo "$t ,$contents1, $contents2, $contents3 ";
    $ret = rrd_update("/mnt/ramdisk/temp-comms.rrd", $option);

    if (! $ret) {
              echo "<b>Graph error comms: </b>".rrd_error()."\n";
    }

then you've gotta create the GIFs
index.php
// for comms cupboard
function create_graph_temperature_comms ($output, $start, $title)
{
    $options = array(

    "--slope-mode",
    "--start", $start,
    "--title=$title",
    "--vertical-label=Temperature in oC",
    // oops got the top and bottom values mixed up on input
    "DEF:temp_spare=/mnt/ramdisk/temp-comms.rrd:spare:AVERAGE",
    "DEF:temp_top=/mnt/ramdisk/temp-comms.rrd:temp_bottom:AVERAGE",
    "DEF:temp_bottom=/mnt/ramdisk/temp-comms.rrd:temp_top:AVERAGE",

    "DEF:temp_sparemin=/mnt/ramdisk/temp-comms.rrd:spare:MIN",
    "DEF:temp_sparemax=/mnt/ramdisk/temp-comms.rrd:spare:MAX",
    "DEF:temp_topmin=/mnt/ramdisk/temp-comms.rrd:temp_top:MIN",
    "DEF:temp_topmax=/mnt/ramdisk/temp-comms.rrd:temp_top:MAX",
    "DEF:temp_bottommin=/mnt/ramdisk/temp-comms.rrd:temp_bottom:MIN",
    "DEF:temp_bottommax=/mnt/ramdisk/temp-comms.rrd:temp_bottom:MAX",
   
    "CDEF:ttemp_spare=temp_spare,1,*",
    "CDEF:ttemp_top=temp_top,1,*",
    "CDEF:ttemp_bottom=temp_bottom,1,*",
   
    "VDEF:vtemp_spare=temp_spare,AVERAGE",

    "LINE:ttemp_spare#00aa00:Spare:",
    "GPRINT:temp_spare:AVERAGE:Avg temp \\t %6.2lf",
    "GPRINT:temp_spare:LAST:Last temp \\t %6.2lf \\n",

    "LINE:ttemp_top#cc0000:Top:",
    "GPRINT:temp_top:AVERAGE:Avg temp \\t %6.2lf",
    "GPRINT:temp_top:LAST:Last temp \\t %6.2lf \\n",

    "LINE:ttemp_bottom#000088:Bottom:",

    "COMMENT:\\n",
    "--alt-autoscale",
    "--alt-y-grid",
   

   
  
    "GPRINT:vtemp_spare: test %6.2lf",
    );

    $ret = rrd_graph($output, $options);
        if (! $ret) {
        echo "<b>Graph error for temp comms</b>".rrd_error()."\n";
        }
}
And you'll get something like this:




Tuesday, 16 September 2014

List users on a terminal services server

qwinsta /server:servername1

 SESSIONNAME       USERNAME                 ID  STATE   TYPE        DEVICE
 services                                    0  Disc
 console                                     1  Conn
                   bob5                      2  Disc
 rdp-tcp#1         bob4                      3  Active
                   bob3                      4  Disc
 rdp-tcp#4         bob2                      5  Active
 rdp-tcp#0         bob                       6  Active

Tuesday, 9 September 2014

Disabling clipboard, Drive Mappings, client redirection etc within a RDP session

So you want to disable
* clipboard copy/paste
* Drive mappings
* printer mappings
fDisableClip , fDisableCdm, fDisableCpm, 
and you don't have the PC/Server on the domain?

Easiest thing to do is set it manually using reg query/add like so:
echo off
set server=%1
set passwd=%2
echo =====================================================================
echo %server% %passwd%
psexec \\%server% -u %server%\administrator -p %passwd% reg query "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" | findstr /i fdisable

psexec \\%server% -u %server%\administrator -p %passwd% reg add "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableClip /t REG_DWORD /d 0x1 /f
psexec  \\%server%  -u %server%\administrator -p %passwd% reg add "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableCdm /t REG_DWORD /d 0x1 /f
psexec \\%server% -u %server%\administrator -p %passwd% reg add "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableCpm /t REG_DWORD /d 0x1 /f
psexec \\%server% -u %server%\administrator -p %passwd% reg add "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableLPT /t REG_DWORD /d 0x1 /f
psexec \\%server% -u %server%\administrator -p %passwd% reg add "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisablePNPRedir /t REG_DWORD /d 0x1 /f

psexec \\%server% -u %server%\administrator -p %passwd% reg query "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" | findstr /i fdisable

 And if for some reason, domain PCs don't have it applied:

echo off
set server=%1
echo =====================================================================
echo %server%
psexec \\%server% reg query "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" | findstr /i fdisable

reg add "\\%server%\HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableClip /t REG_DWORD /d 0x1 /f
reg add "\\%server%\HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableCdm /t REG_DWORD /d 0x1 /f
reg add "\\%server%\HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableCpm /t REG_DWORD /d 0x1 /f
reg add "\\%server%\HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisableLPT /t REG_DWORD /d 0x1 /f
reg add "\\%server%\HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" /v fDisablePNPRedir /t REG_DWORD /d 0x1 /f

psexec \\%server% reg query "HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services" | findstr /i fdisable
 
 

Saturday, 30 August 2014

Using google mail servers as a smarthost from a linux server (ubuntu)

So you want to send mail securely using google's mail servers as a smarthost.

The first thing you'll need is an application specific password for exim.

Once that's created, I will assume the following:
  • You are ok with Gmail rewriting your sender address or use this Gmail address.
  • Your account name on your local Debian box: YOUR-USER-NAME
  • Your host name in /etc/hostname: hostname1
  • Your host name in the 127.0.1.1 line of /etc/hosts: hostname1.localdomain hostname1
  • Your envelope address is SMTPAccountName@gmail.com for the outgoing SMTP connection to Gmail's SMTP server via Exim4
  • Your Exim4 relays local SMTP connections coming with the mail envelope address: YOUR-USER-NAME@localhost and YOUR-USER-NAME@localhost.localdomain
  • Your account name for SMTP at Gmail is SMTPAccountName@gmail.com
  • Your password for SMTP at Gmail is y0uRpaSsw0RD
     
# dpkg-reconfigure exim4-config
 
  • Choose "mail sent by smarthost; received via SMTP or fetchmail"
  • Set to "localhost" for "System mail name:".
  • Set to "127.0.0.1" for "IP-addresses to listen on for incoming SMTP connections" to refuse external connections.
  • Leave as empty for "Other destinations for which mail is accepted:".
  • Leave as empty for "Machines to relay mail for:".
  • Set to "smtp.gmail.com::587" for "IP address or host name of the outgoing smarthost:".
  • Choose "NO" for "Hide local mail name in outgoing mail?".
  • Choose "NO" for "Keep number of DNS-queries minimal (Dial-on-Demand)?".
  • Choose "mbox format in /var/mail/" for "Delivery method for local mail".
  • Choose "YES" for "Split configuration into small files?".
vi /etc/exim4/passwd.client
 
 
add the following lines:
*.google.com:SMTPAccountName@gmail.com:y0uRpaSsw0RD
# chown root:Debian-exim /etc/exim4/passwd.client # chmod 640 /etc/exim4/passwd.client
 
Edit your aliases file
# newaliases
 
# update-exim4.conf
# invoke-rc.d exim4 restart
# exim4 -qff
 
 tada
 
mostly grepped from here 
 
 

Tuesday, 26 August 2014

List installed windows programs on a pc remotely via command line


install the sysinternals process tools from here

psexec \\COMPUTERNAME -u administrator -p password reg query "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall" /s | findstr /B ".*DisplayName"
 voila
Note, there are other methds to query via WMI, but this only works with MSI installed packages.
 

Sunday, 24 August 2014

Creating RRD graphs for TME Ethernet thermometer TME_C_EU pt2

 Ok, so last post I created the graphs, but they don't look that good. They don't seem to scale right. They scale from 0 to about 30oC


So it's showing values 0-30oC, which doesn't show the zoomed in detail, as the fluctuations are only 27oC +- 3oC. This is to do with auto-scaling and that stacked area graphs start from 0.
The detail we need is actually in the 25-30 range.

So I've had to change the index.php script so that it auto scales. It doesn't look as fancy, but at least you can see the temperatures better.
    "--slope-mode",
    "--start", $start,
    "--title=$title",
    "--vertical-label=Temperature in oC",
    "DEF:temp=/mnt/ramdisk/temp.rrd:temp:AVERAGE",
    "DEF:tempmin=/mnt/ramdisk/temp.rrd:temp:MIN",
    "CDEF:ttemp=temp,1,*",
    "LINE:ttemp#E54400::",
    "COMMENT:\\n",
    "--alt-autoscale",
    "--alt-y-grid",
    #"--rigid",
    "GPRINT:temp:MAX:Max temp %6.2lf",
    "GPRINT:temp:MIN:Min temp %6.2lf",
    "GPRINT:temp:AVERAGE:Avg temp %6.2lf",

To make this prettier, you could also define the max/min values for the entire graph
$options = array(
    "--slope-mode",
    "--start", $start,
    "--title=$title",
    "--vertical-label=Temperature in oC",
    "DEF:temp=/mnt/ramdisk/temp.rrd:temp:AVERAGE",
    "DEF:tempmin=/mnt/ramdisk/temp.rrd:temp:MIN",
    "DEF:tempmax=/mnt/ramdisk/temp.rrd:temp:MAX",
    "CDEF:ttemp=temp,1,*",
    "VDEF:ttempmin=tempmin,MINIMUM",
    "VDEF:ttempmax=tempmax,MAXIMUM",
    "HRULE:ttempmin#4444FF::dashes=2,2",
    "HRULE:ttempmax#FF5500::dashes=2,2",
    "LINE:ttemp#000000::",
    "COMMENT:\\n",
    "--alt-autoscale",
    "--alt-y-grid",
    #"--rigid",
    "GPRINT:temp:MAX:Max temp %6.2lf",
    "GPRINT:temp:MIN:Min temp %6.2lf",
    "GPRINT:temp:AVERAGE:Avg temp %6.2lf",
        );
This will draw a horizontal ruler at the min and max values with blue and red dashed lines, respectively.

Saturday, 23 August 2014

Creating RRD graphs for TME Ethernet thermometer TME_C_EU pt1

So I decided to buy one of these network temperature sensors from Papouch here to monitor my 19" rack at home. I wanted to graph the temperature variations from it, as it can get quite hot when all the other servers are on.

Setup Apache with mod_php5 + rrd+ php5_cli

There are many articles on setting up apache + mod_php5 + rrdtools + php5_cli, so i won't bother you with the details here.

Create the RRD

One you've installed rrdtool and php5 cli, create the below file and chmod a+x so that it is executable.
Once run, this will create your RRD to input data called temp.rrd
#!/usr/bin/php
<?
echo "Creates blank temp graphs";
$options = array (
 "--step", "60",            // Use a step-size of 1 minute
 "--start", "-1 day",     // this rrd started 6 months ago
 "DS:temp:GAUGE:100:U:U",
"RRA:MIN:0.5:60:8800",
"RRA:MAX:0.5:60:8800",
"RRA:AVERAGE:0.5:2:328800",
);


$ret = rrd_create("./temp.rrd", $options);
if (! $ret) {
 echo "<b>Creation error: </b>".rrd_error()."\n";
}

?>

Create the graphs

This should be your index.php This should be in your /var/www directory
<?

create_graph_temperature("/var/www/temp-hr.gif","-1h","Hourly temperature");
create_graph_temperature("/var/www/temp-2hr.gif","-2h","Bi-Hourly temperature");
create_graph_temperature("/var/www/temp-daily.gif","-23h","Daily temperature");
create_graph_temperature("/var/www/temp-weekly.gif","-1w","Weekly temperature");
create_graph_temperature("/var/www/temp-monthly.gif","-1m","Monthly temperature");
create_graph_temperature("/var/www/temp-yearly.gif","-1y","Yearly temperature");

function create_graph_temperature ($output, $start, $title)
{
    $options = array(
    "--slope-mode",
    "--start", $start,
    "--title=$title",
    "--vertical-label=Temperature in oC",
    "DEF:temp=/mnt/ramdisk/temp.rrd:temp:AVERAGE",
    "CDEF:ttemp=temp,1,*",
    "CDEF:ttempq=temp,6,/",
    "AREA:ttempq#F31616::STACK",
    "AREA:ttempq#F03D11::STACK",
    "AREA:ttempq#ED650D::STACK",
    "AREA:ttempq#EA8D08::STACK",
    "AREA:ttempq#E7B504::STACK",
    "AREA:ttempq#E5DD00::STACK",
    "COMMENT:\\n",
    "GPRINT:temp:MAX:Max temp %6.2lf",
    "GPRINT:temp:MIN:Min temp %6.2lf",
    "GPRINT:temp:AVERAGE:Avg temp %6.2lf",
        );
    $ret = rrd_graph($output, $options);
        if (! $ret) {
        echo "<b>Graph error for temp</b>".rrd_error()."\n";
        }
}


?>

<img src=temp-hr.gif><img src=temp-2hr.gif></br>
<img src=temp-daily.gif><img src=temp-weekly.gif></br>
<img src=temp-monthly.gif><img src=temp-yearly.gif></br>
Note:you may need to chgrp www-data and chmod g+w on the /var/www directory, so that apache can write the GIFs to there. 

Update data into RRD

Do to this, I wrote a small PHP script
#!/usr/bin/php
<?
 // Fetch current time
    $t = time();
    $filename="/mnt/ramdisk/temp.txt";

    $handle = fopen($filename, "r");
    $contents = fread($handle, filesize($filename));
    fclose ($handle);
    $contents=(float)$contents/10;
    $option = array ("$t:$contents");
    echo "$t ,$contents";
    $ret = rrd_update("/mnt/ramdisk/temp.rrd", $option);

    if (! $ret) {
    mail ("blah@gmail.com","temp insert rrd error","trying to insert temp from temp.txt into temp.rrd");
        #echo "<b>Graph error: </b>".rrd_error()."\n";
    }


?>

Crontab

I've set up my TME_C on IP address 192.168.2.120
Note I've saved the output to a ramdisk (as my machine boots from a USB key)
 crontab -e on root
* * * * * /usr/local/bin/update_temp_script.sh
Note you should be calling a script which does the update, then inserts it into the RRD in sequence. If you don't it maybe updating the RRD with blank values.
update_temp_script.sh
/usr/bin/curl -s http://192.168.2.120/fresh.xml | grep sns  | awk '{print $10}' | cut -d "\"" -f 2 > /mnt/ramdisk/temp.txt 
/usr/local/bin/update_temp.php
That's it - it's that simple.
You should have graphs looking similar to this:


Now you can also setup a daily cron job to tar.gz these files to the local file system instead of leaving it on a ramdisk or what not. If you leave it on the ramdisk, you'll lose it if the server powers it off.

part 2 here

NOTE - Have corrected this as i was only storing about 1/5 of the data, so have upped the data storage points on the RR. This was calculated at one data point every 60 seconds. So 60 seconds in 24 hrs = 1440 points.
Hourly points for a year = 366 *24 = 8784 points. I rounded this to 8800