0

I am working on one project required to monitor the UPS using NUT server. For monitoring and data logging I am using the minibian OS (JESSIE). In order to monitor the ETON UPS I have wrote a shell script which inquires data every second. As I am using one 8Gb sd card to store OS and data log, I have partitioned it as root(2Gb) and datalog(4Gb) drives.

The UPS monitor script is launched by Crontab as a background process. The feature included in UPS Monitor scripts are as follows:

  • Fetch UPS data every 1 Second
  • Generate new folder every day
  • Time stamp the log files .csv
  • Search for folder older than 1 day, compress it and delete folder

Issue:

I assumed that the 2 GB space for root would be enough as my OS only takes around 800 Mb. But as I run a test for three days the root partition has consumed more than 100 Mb in three days. In this case, my root partition will be filled within 30 days

I am not sure why running shell script causing too much overhead in memory.

Here is the memory usage by Raspberry Pi OS (without shell script) and UPS Monitor (with shell scripts)

Test Run Time: 3 Days

UPS Monitor Memory Test: START

UPS Monitor Memory: Test Start

UPS Monitor Memory Test: STOP

UPS Monitor Memory: Test Stop

Raspberry PI OS Memory Test: START

enter image description here

Raspberry PI OS Memory Test: STOP

enter image description here

Memory Usage of Scripts

I have found some links online to find the memory usage by script, But not able to make any sense out of it.

Memory Usage

Data Logging Script

#!/bin/bash
gpio -g mode 16 out
gpio -g mode 21 out
gpio -g write 16 0
gpio -g mode 24 in

LOG_PATH='/media/ntfs/logs/'
LOG_HEADER=`date +%H:%M:%S`
LOG_DIR_NAME=`date +%d_%m_%Y`
LOG_FOLDER=`find /media/ntfs/logs -type d -mtime +0`

timeCounter=0
switchRead=0
#  Browse log directory
cd -- $LOG_PATH
LOG_NAME=$LOG_HEADER

while :
do
    switchRead=`gpio -g read 24`
    LOG_DIR_NAME=`date +%d_%m_%Y`
    LOG_DIR=$LOG_PATH$LOG_DIR_NAME
    echo "$LOG_DIR"

    if [ -d "$LOG_DIR" ]
    then
            cd -- $LOG_DIR
            echo "DIRECTORY FOUND"
    else
        mkdir -p $LOG_DIR
        LOG_NAME=`date +%H:%M:%S`
        echo "NEW DIRECTORY CREATED!!!!!"
        cd -- $LOG_DIR
        HEADER='UPS MODEL, UPS STATUS, BATTERY CHARGE, BATTERY RUNTIME,$
        echo $HEADER > $LOG_NAME.csv
        # Calling old folder compression script
        /root/day.sh
    fi

    # executing ups status command and saving result in array
    declare -A status="($(upsc myups | sed 's/\(.*\):\(.*\)/[\1]="\2"/'))"

    if [[ ${status[device.model]} != "" ]] ; then
        echo "WRITIN FILE"
        echo  ${status[device.model]}","${status[ups.status]}"," ${stat$
    fi

    if [[ ${status[ups.status]} = *OB* ]] ; then
        timeCounter=$(( $timeCounter+1))
        #echo $timeCounter  ":("

        if [[ "$timeCounter" -gt 20 ]] ; then
            #echo ":D"
            timeCounter=0
            LOG_NAME=`date +%H:%M:%S`
            echo $HEADER > $LOG_NAME.csv
       fi
    fi

    if [[ "$switchRead" = 1 ]] ; then
        echo "Read Kar liya bhiya"
        gpio -g write  16 1
        `sudo shutdown -h now`
    else
        gpio -g write 16 0
    fi

    gpio -g write 21 0
    sleep 0.5
    gpio -g write 21 1
    sleep 0.5
done
exit 0

Thank you

closed as too broad by Rui F Ribeiro, Thomas Dickey, mdpc, G-Man, Archemar Sep 13 '16 at 10:11

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • Check the logs to see what floods them – Serge Sep 12 '16 at 15:42
  • 2
    You're mixing disk usage and memory usage. The memory card is storage. The screenshots you've posted relate to storage (disk) and memory (RAM). – roaima Sep 12 '16 at 21:37
  • I am interested in finding out the disk space used by the shell script. – Kunal Sonone Sep 14 '16 at 12:40
4

It looks like you are talking about disk usage/storage space rather than memory usage. Memory (ram) is a fast volatile (data is lost when powered off) that is used by programs generally to process data. You are talking about disk usage, which is physically stored on your sd card and is not lost when powered off. For the rest of this I will assume you meant disk usage.

There are many places that your program could be using disk while running, other than your datafiles. Without knowing exactly what your program is doing it will be hard to tell exactly what it is doing to use up the extra space. However some common things to look for are log, cache and temporary files.

Log files are typically written to /var/log, cache files to /var/cache and temporary files to either /tmp or /var/tmp. These are locations that could potentially be growing in size and you should keep your eye on.

You realistically have two options:

  • look at your script to see what other locations it might be writing to, this is highly dependent on what it is doing and without more information it is hard to advise on how to do this.
  • analyse your filesystem to see where the free space is being used.

For the second point the tool du is very handy, it lets you see what files/directories are taking up space. Here is how I typically find files are are consuming a large amount of space.

Start in the root of your file system (or a subdir if you know roughly where the space is being used). Run sudo du -sxh * | sort -h. This will take a while depending on the size of the directory it is analysing (but for 1G it should not take too long). The options mean -s: summary of each listed file/directory. -x: don't cross filesystem boundaries, so you don't get useage of other disks. And -h to print human readable sizes. The | sort -h will sort them in size order.

Once you have found the directories that are using most of the space, or increased in amount of space used you can cd to them and re-run the du command. Repeat this until you find where your space is going.

Note that in your case you are looking for a growth in size, but it will be a small growth rather than absolute sizes.

Not the answer you're looking for? Browse other questions tagged or ask your own question.