Any maths and/or data crunchers looking for a challenge?

[frackers]

Once a minute I collect data from my wind turbine. Once per 5 minutes I collect data from my weather station to get wind direction and strength. Anyone fancy crunching the data to correlate the two? Would require a lot of averaging and interpolation to match the timestamps!

The turbine log is currently 112843752 bytes long and contains:

Time in seconds since 1/1/1970

Inverter temperature

Voltage

Current

Charge In Total

Charge Out Total

Power Now

30 second Average power

1 minute max power

1 hour max power

1 day max power

10 minute average power

Load On indicator

The max power values are determined by the sample rate which is twice a second.

The weather station log is currently 24344665 bytes long and contains:

Time in seconds since 1/1/1970

Wind speed in km/hr

Wind direction in 45degree increments

Also lots of other stuff like inside and outside temperature and humidity, air pressure etc

[gizmo]

What is the file format? Text files?

You would need to get them into a database, such as access or dbase. Then its a lot easier to get out the results you want.

Glenn

[TomW]

What he said.

[Rover]

Along with format, you might want to post what you are after out of the Data. I assume there are a few things you want correlated. Are the 1 and 5 minute intervals when it is logged ? (may include contain more data points), or are they the actual sampling intervals?

[cyplesma]

you can get Microsoft express versions of SQL 2005/2008 for free. Don't know if you want to learn SQL.

You can get the .net (VB,C++,C#,Web Developer) for free also.

http://www.microsoft.com/express/Downloads/

[frackers]

Aha - now that the controls section of the old board has appeared I can continue this thread!!

The weather data looks like this (note it has a lot of non-relevant data!!)
20100421215511 2010-Apr-21 21:55:11 20.5 9.7 5.6 41 76 2.5 22.5 NNE 9.7 0.00 0.00 83.39 1015.700 Rising Cloudy
20100421220009 2010-Apr-21 22:00:09 20.5 9.6 5.5 41 76 0.0 45.0 NE 9.6 0.00 0.00 83.39 1015.600 Rising Cloudy
20100421220508 2010-Apr-21 22:05:08 20.4 9.5 5.4 41 76 0.0 45.0 NE 9.5 0.00 0.00 83.39 1015.600 Rising Cloudy
20100421221006 2010-Apr-21 22:10:06 20.4 9.4 5.3 41 76 0.0 22.5 NNE 9.4 0.00 0.00 83.39 1015.600 Rising Cloudy
20100421221514 2010-Apr-21 22:15:14 20.4 9.3 5.2 41 76 0.0 0.0 N 9.3 0.00 0.00 83.39 1015.600 Rising Cloudy
20100421222017 2010-Apr-21 22:20:17 20.4 9.2 5.1 41 76 0.0 45.0 NE 9.2 0.00 0.00 83.39 1015.600 Rising Cloudy
20100421222511 2010-Apr-21 22:25:11 20.4 9.1 5.0 41 76 0.0 0.0 N 9.1 0.00 0.00 83.39 1015.600 Rising Cloudy
20100421223010 2010-Apr-21 22:30:10 20.4 8.9 4.8 41 76 0.0 45.0 NE 8.9 0.00 0.00 83.39 1015.600 Rising Cloudy
where the information is defined by position

and the turbine data looks like this
20100421222619 2010-Apr-21 22:26:19 T:10.53 T1:0.00 T2:0.00 V:25.19 I:-0.24 R:93.26 Ci:19703.13 Co:18062.50 P:-6.15 Pav:-5.94 P1max:0.00 P60max:37.13 P24max:1023.17 P10av:-5.92 L:0 H:97.66
20100421222719 2010-Apr-21 22:27:19 T:10.38 T1:0.00 T2:0.00 V:25.18 I:-0.24 R:93.26 Ci:19703.13 Co:18062.50 P:-6.14 Pav:-6.15 P1max:0.00 P60max:37.13 P24max:1023.17 P10av:-5.86 L:0 H:97.66
20100421222819 2010-Apr-21 22:28:19 T:10.41 T1:0.00 T2:0.00 V:25.19 I:-0.23 R:93.26 Ci:19703.13 Co:18062.50 P:-6.14 Pav:-5.74 P1max:0.00 P60max:37.13 P24max:1023.17 P10av:-5.80 L:0 H:97.66
20100421222919 2010-Apr-21 22:29:19 T:10.38 T1:0.00 T2:0.00 V:25.18 I:-0.24 R:93.26 Ci:19703.13 Co:18062.50 P:-6.15 Pav:-5.94 P1max:0.00 P60max:37.13 P24max:1023.17 P10av:-5.79 L:0 H:97.66
20100421223019 2010-Apr-21 22:30:19 T:10.41 T1:0.00 T2:0.00 V:25.18 I:-0.24 R:93.26 Ci:19703.13 Co:18062.50 P:-6.15 Pav:-5.94 P1max:0.00 P60max:37.13 P24max:1023.17 P10av:-5.80 L:0 H:97.66
20100421223119 2010-Apr-21 22:31:19 T:10.50 T1:0.00 T2:0.00 V:25.18 I:-0.20 R:93.26 Ci:19703.13 Co:18062.50 P:0.00 Pav:-4.92 P1max:0.00 P60max:30.91 P24max:1023.17 P10av:-5.82 L:0 H:97.66
20100421223219 2010-Apr-21 22:32:19 T:10.38 T1:0.00 T2:0.00 V:25.19 I:-0.23 R:93.26 Ci:19703.13 Co:18062.50 P:0.00 Pav:-5.74 P1max:0.00 P60max:30.91 P24max:1023.17 P10av:-5.67 L:0 H:97.66
20100421223319 2010-Apr-21 22:33:19 T:10.41 T1:0.00 T2:0.00 V:25.19 I:-0.20 R:93.26 Ci:19703.13 Co:18062.50 P:-6.15 Pav:-5.12 P1max:18.48 P60max:30.91 P24max:1023.17 P10av:-5.21 L:0 H:97.66
20100421223419 2010-Apr-21 22:34:19 T:10.41 T1:0.00 T2:0.00 V:25.19 I:-0.21 R:93.26 Ci:19703.13 Co:18062.50 P:0.00 Pav:-5.33 P1max:0.00 P60max:18.48 P24max:1023.17 P10av:-5.07 L:0 H:97.66
20100421223519 2010-Apr-21 22:35:19 T:10.34 T1:0.00 T2:0.00 V:25.18 I:-0.22 R:93.26 Ci:19703.13 Co:18062.50 P:-6.14 Pav:-5.53 P1max:0.00 P60max:18.48 P24max:1023.17 P10av:-5.00 L:0 H:97.66

The same seconds count and time but each field is identified by a field name followed by a colon.

I've put the raw files here http://gilks.ath.cx/~g8ecj/maths/ with a couple of filter scripts that reduce the data to the following form:
weather data is only relevant for windspeed and direction
20100421223509 WS:0.0 DIR0:0.0
useful turbine data is volts, amps, power, 1 minute, 1 hour and 1 day maxima and the 10 minute running average power.
20100421223819 V:25.19 I:-0.15 Pav:-3.90 P1max:12.32 P60max:18.48 P24max:1023.17 P10av:-4.65

Note that the times are not synced so I'm not sure how the interleave of them would be solved.

Useful data I think would be:

• Is there more power at a particular wind speed from one direction
• How many times does the cable drop in the tower get wound round in a year
• Is there a prevailing wind direction
• Is there a time of year that has more consistent power generation

[RP]

I don't know if it helps but I had a diary entry a few years ago on creating graphs from NOAA data in Excel.

http://fieldlines.com/board/index.php/topic,127643.0.html

[frackers]

I've got lots of graphs already - they get updated every 5 minutes in fact (http://gilks.ath.cx/cacti/graph_view.php?action=tree&tree_id=3 and at http://gilks.ath.cx/cacti/graph_view.php?action=tree&tree_id=2. What I'm sure is hidden in the data is more general information that might be of use

[SparWeb]

Number crunching is kind of tedious work - not sure if you'll get any volunteers.  Speaking for myself, I'm number crunching all the time!  My datalogger picks up about 8640 data points per day (one per 10 seconds) and I keep a text file on my laptop with a running tally of what experiment or configuration I have set up and when.  Again speaking for myself, I wouldn't expect to get what I want unless I did it for myself, or paid an expert to do it for me (and even so, there are still chances for error and mis-interpretation).

I read a book by Paul Gipe which outlines how the statistics of wind and power should be collected and processed.  All in plain language.  The book is called, of course, Wind Power, you can find links to it easily through his personal website (it think it goes www.wind-works.org) and it's also available at the obvious places like Amazon.  After reading that, you probably won't feel as daunted by the task and set to it yourself.

And if you don't have or want Excel, don't worry there are other spreadsheet programs out there that have the features you need, too, for free download.
Once my spreadsheet format and graphs are set up, I usually just copy it and then "paste" in the new data over top, let it re-calculate, and save the separate file.