Homebrew system monitor software

[RandomJoe]

One of the reasons I purchased the Outback system for my solar system was that they provide the communication specs for the Mate PC interface.  I'm a control freak, and like that sort of thing!  Especially since I don't use Windows, and few companies provide Linux-compatible software...

I installed the Mate a couple weeks ago, and have slowly been working on my software since then.  I'm not a "pro" programmer, it's just a hobby for me, so it's probably not too pretty...  Right now I'm using Python, which nicely means it ought to work on other OSes as well.

At this point, I now have a program that logs the data to CSV files, and I just finished the "live" display.  The Mate provides data from both the FM-80 charge controller and the Flexnet-DC battery monitor, and I've broken that into three chunks - charge controller data, Flexnet "live" data, and Flexnet "totals" data.  Each chunk gets its own CSV file.

The program makes CSV entries under two conditions - at a minimum, every X seconds (I currently have that at 600s / 10min) but for the items I care about it also logs if they change by more than Y (amps, volts, etc) from the last save value.  So overnight I may only have entries every 10 minutes, but during a cloudy day as fast as every two seconds.

Again, I'm a control freak so I like to see all the quick ups and downs in my graphs!

I started fiddling with ncurses in Python a few days ago, and now have a "live" display going.  It updates with every read from the Mate.  Here's a screenshot of what I have so far:

Cool! :-)  Only annoying thing right now is the background - it's supposed to be black, but when I run it in a Gnome terminal it's gray!  Grr...

At the moment, the display code is part of the logging code, so if I want to see it I have to be on that computer or VNC in to it.  My next step will probably be separating the two (they are already implemented as separate classes) so I can use a straight SSH connection to view the data and perhaps even have multiple viewers going at once.  That's also why I used ncurses instead of a GUI - reduces the overhead.

Too much fun!

[commanda]

I like the idea of changing the reporting rate as data changes; might have to add that to my setup.

Display will probably look better with leading zero blanking. Here's a cut'n'paste from my Picaxe basic code;

DisplayByte:

    if b3 < 10 then goto db2        ;double leading zero blanking       

    if b3 < 100 then goto db1          ;leading zero blanking

    b5=b3 DIG 2 + "0"

    @ptrinc = b5

db1:   

    b5=b3 DIG 1 + "0"

    @ptrinc = b5

db2:

    b5=b3 DIG 0 + "0"   

    @ptrinc = b5

return   

I have written a comms package in Python, but found the interpreter to be painfully slow.

Amanda

[TomW]

Joe;

I am using Perl to snag data from my Mate. I process it thru awk and Gnuplot for graphs just so I can see the long term data.

Just switching around some gear here and pondering a different approach. Care to share your code?

I never used Python myself yet so that would be new.

Tom

[RandomJoe]

Yeah, I do intend to drop the leading zeros.  My primary goal at this point was just to make ncurses work! :-)  Years ago I was working in C, and had a devil of a time getting ncurses to do what I want.  The Python library makes it a lot easier - among other things it reduces several semi-redundant slightly-different C functions to a single easier to understand Python function.  Of course, I was also fighting C - I never did enough programming to remember all the details so found it rather frustrating when I spent more time satisfying the compiler than I did improving on my results.

The data comes in as a text string, so to simplify matters that's how I originally kept it.  I also didn't care for my float values turning into "21.19999999999" or such in the CSV files!  So far I just import into a spreadsheet, which handles the conversion just fine.  But the ncurses code just displays the "string" it's given, thus the leading zeros.

I haven't yet tried anything that needed enough speed for Python to be a problem.  This Mate interface is the highest-speed yet, and it just sends a 49-character string ever second so not much to keep up with!  I'm also not usually on anything very low-speed, the computer this is running on is an Atom dual-core (with hyperthreading it looks like 4 CPUs to the OS) - perhaps I'd see some issues if I tried to shove it all on something much slower like an NSLU2, not sure how that would handle Python.

[RandomJoe]

I'll have to keep Gnuplot in mind - I had forgotten about it.  I also have a weather station, and am using a package called 'wview' to collect data from it.  Wview also makes some really nifty web pages with graphs of the data as well, so I intend to look at that as well - I believe it's using 'libgd2' for the graphics portion.

I have somewhat of a lofty goal down the road!  A system I use at work for building control has a gorgeous graphing function for trend data, at the top of the graph it provides a control to dynamically select the time frame you want to see - anything from "today", "yesterday", "past week" to any user-supplied start/end dates.  The weather software doesn't do that, it's just fixed intervals, so I'd also like to add it there as well.  That's all through a web page, too.

Sure, I'll be happy to share the code.  It's pretty much hardwired to my system right now and probably not terribly pretty... ;-)  I'll have to put it somewhere and link to it (way too long to just post) so I'll do that this evening - gotta head off to work now.

[RandomJoe]

Ahhh...  Absolutely gorgeous day! :-)  Just over 80 degrees, lots of sun, barely a cloud in the sky.  Light breeze too, which is nice for me - not so nice for the turbine.  Finally felt good enough to go for a bike ride!  I used to ride 20-30 miles per DAY, probably haven't been out in over a year now...!  I'm whipped, but managed about 8-10 miles.  Not bad...

Anyway...

Here's a link to my program as it stands right now:

http://okcforum.org/~martine/solar_system/logger/

Hopefully that'll work - if so, I'll just toss updates in that directory and they'll show up...

The meat of the code is two classes - one handles collecting the data from the Mate and storing everything in a set of arrays, the other handles the ncurses display.  The logging portion is actually quite simple (simplistic?!), so is just a couple of small functions called straight from the main loop.

The settings for logging time and deltas are clumsy - they're in the main loop toward the bottom.  Ought to move those up to the top...

And there's no error-checking at this point.  I don't validate checksums from the Mate yet, although so far I haven't needed to - I haven't had any data corruption.  Not that it would add much code.  I also don't verify the file save operations, but - again - I wasn't too worried about that at this point.  Eventually...

Hey, I was just happy it worked in the first place without a lot of debugging! ;-)

[frackers]

Ahhh - you've made me go and publish my software now....

See article http://www.fieldlines.com/story/2009/4/22/13826/0795