Current cost gas smart released

97 replies [Last post]
brett
Offline
Providence, United States
Joined: 9 Jan 2010
Alrightly this is progress.  

Alrightly this is progress.   Now for my next trick.  I"m going to make a change to the WEBUI so that you have the following selection for sensors "Digital/Analogue/Impulse".

If you select Analogue xap-currentcost will simply report the Meter counter number, which is what you are currently seeing.
If you select Impulse it will push a DELTA from the latest meter reading.

With a Delta this will give you a relative consumption reading based on the UNITS settings you have configured on your BASE.

This is a good start.

I see that you have NOT reconfigured your based unit when you paired the gas sensors as its actually reporting as a IMPULSE ELECTRICTY sensor with its default units of 1000.

<msg><src>CC128-v1.29</src><dsb>00013</dsb><time>19:40:30</time><tmpr>20.9</tmpr><sensor>9</sensor><id>02231<
/id><type>2</type><imp>0000003970</imp><ipu>1000</ipu></msg>

The TYPE field is decoded as: 2 Electricity, 3 Gas, 4 Water.

By default the BASE unit is consider to treat all impulse sensors as Electricity (type 2) until reconfigured.
IPU by default is 1000 for 1000Kw/h but for GAS you would nominally have this set to 10dm/3

I'm not sure that this matters as I'm only going to he pushing a DELTA of the <imp> number out but I thought I would let you know all the same.

I'll post when I've published 308.2 with these changes.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
I'll have a go at re-pairing

I'll have a go at re-pairing it and change I did see an option to scroll through some options however the they included with the sensor wasn't great did say will show 1000 (default imp/kWh) however didn't say what to change it to. On my meter it says 1 pulse = 10 dm3 so I'll have to try and figure out how to re-pair with a more meaningful value.

Is there a way to reset the XML in the CC as once I change the data will I guess look odd?

brett
Offline
Providence, United States
Joined: 9 Jan 2010
308.2 published - I made the

308.2 published - I made the following changes.

Added Impulse setting to current cost sensor web page along with backend changes to support it. When a sensor is set to impulse it will emit a DELTA value as the reading from the sensor is a counter value.

You might want to reset your data stream after you switch as it will now be emitting values like 0-10 (or there abouts) instead of 3000-4000 for your counter.   Or with the switch to <insert pachube/cosm's new name here> if you wait 6hrs its flush through.

Update: If you really do want the COUNTER value switch it back to analog and this is what you'll get, I didn't want to take this away completely somebody might want it.  For example if you aren't using COSM and have Cacti setup it can take this value and do its own delta graphing.   So you can choose how to slice and dice it and make julian fries with it.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
OK I've upgraded to 308.2 and

OK I've upgraded to 308.2 and changed the sensor to Impulse however not had time to re-pair the sensor to change to Gas from Electric as yet, will do this later this evening so right now should still be pushing data to Cosm as before.

brett
Offline
Providence, United States
Joined: 9 Jan 2010
I still see your graph

I still see your graph https://xively.com/feeds/56174/ reporting absolute values.  Did you restart the currentcost component (or reboot) after you changed the sensor to impulse mode?

Adjusting the sensor from electricity type to gas isn't necessary its just a "nice to have" I think.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
No I didn't reboot my CC,

No I didn't reboot my CC, just done that now and also rebooted my HAH, does that look any different now?

brett
Offline
Providence, United States
Joined: 9 Jan 2010
See here is the thing when I

See here is the thing when I run with imp.txt which is just a slab of XML that you gave me and I have sensor 9 configured as impulse.

/etc/xap-livebox.ini
[currentcost]
type9=I
desc9=
unit9=
hysteresis9=

I get what I expect to see when I manually push a FILE into currentcost for testing purposes.

# xap-currentcost -s imp.txt -d 7

The first impulse reading will be a delta between meter reading 3970 and 3971.  Its much easier to see all these if you use XfXviewer after you push this file in using debug mode.  NOTE:  Do not do this with xap-pachube running or the data will be pushed up!

xap-header
{
v=12
hop=1
uid=FF00DC2B
class=xAPBSC.event
source=dbzoo.livebox.CurrentCost:sensor.9
}
input.state
{
state=on
text=1
}

So I have no idea why you are still pushing absolute values.  Sorry its a bit hard for me to debug your end I can only tell you what I did and what I expect to see.

Brett

AttachmentSize
imp.txt 3.32 KB
DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Hi Brett, is there anything

Hi Brett, is there anything you want me to do, test, etc?

brett
Offline
Providence, United States
Joined: 9 Jan 2010
Doublesheep in addition to

Doublesheep in addition to what I wrote above there is not much I can do this end.  Sadly the onus is on you a little to double check everything is setup correct.

I'm confused by why its just doing what it did before.  If your xap-livebox.ini file is setup for sensor 9 to be type=I (I for Impule) then you should be getting the DELTA values out that I expect.

You can run the thing in debug mode and see what is going on but I'm not sure this is  going to shed much light for you unless you can read my code and the changes I made and walk it yourself.   I really need a meter to be able to verify all this.   I think Derek was looking in to.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Just doing the repairing of

Just doing the repairing of my Gas Sensor and on my Gas meter it says 1 pulse = 10dm3, when repairing the data channel it shows 1000 (which is the default imp/kWh)

I can switch between 100, 500, 600, 800, 1000, 1250, 1600, 3200 & 10000 does anyone know which if these settings I should use?

garrydwilms
Offline
United Kingdom
Joined: 31 Mar 2011
I think you need to convert

I think you need to convert dm3 (gas volume) to kwh (gas energy). It's a long equation, google will help

brett
Offline
Providence, United States
Joined: 9 Jan 2010
The good new is the changes I

The good new is the changes I made don't care how your sensors is configured.  The fact that your GAS meter thinks its a Electricy meter pulsing every 1000kwH make no difference to xap-currentcost (by design).
All it cares about its that its configured as an Impulse sensor so you get a delta reading in the xap payload.

Brett

AndrewJ
Offline
United Kingdom
Joined: 22 Nov 2012
HI Doublespeed,Not sure if

HI Doublespeed,

Not sure if this helps but a dm3 = decimeter which is basically a cubic litre, i generally work in cubic meters "m3"

1000 dm3 = 1 m3

To covert to kWh see below

You also need your correction Factor which is generally "1.022646"

you will also need your calorific value, mine is taken from a recent gas bill "38.8"

from this you can work out the amount of kWh of gas you are using over a period of time (Billing Quarter)

lets say you used 300000 cubic litres over a quarter, thats equivalent to 300 cubic meters we multiply 300 by the correction factor and again by the calorific value

we then divide this by 3.6 to sum up usage in kWh which is how your bill will be itemised and charged

i.e = 300 * 1.022646 * 38.8 / 3.6 = 3306.55 kWh or "3307" rounded up as it no doubt will be

So based on the default setting of 1000dm3 that will equal 1m3

Hope this helps

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Thanks for the replies on

Thanks for the replies on this, will see what I can do with the data now.

One thing Brett, when should I see the delta values sent I would like to see if I can verify this using xFx viewer?

brett
Offline
Providence, United States
Joined: 9 Jan 2010
You should see xap events

You should see xap events being emitted as soon as the senors is online.   If you can run your xap-currentcost in debug mode and send me some of the logs perhaps I can do some cursory examination of what is going on.

1st make sure its shutdown
# killall xap-currrentcost

Start it up in debug mode manually from the command line.
# xap-currentcost -s /dev/ttyUSB0 -d 7

Let it run for a little while to make sure you get some sensor data.  You can watch with xFxViewer to see when you have something.  Then paste the output into a txt file and attach it to your next post, perhaps I can shed some light.
I've also make some change but they won't affect the mechanics of the version you have, they simply reduce the I/O on the flash filesystem by caching in memory some of the .ini file values, so for now Iwon't bother pushing a new beta for this.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Sorry for delay, tried to run

Sorry for delay, tried to run the commands above and got the following:

 

killall: xap-currrentcost: no process killed

# xap-currentcost -s /dev/ttyUSB0 -d 7

Current Cost Connector for xAP v12

Copyright (C) DBzoo, 2009-2012

[crt][init.c:113:discoverBroadcastNetwork] errno 126 (Cannot assign requested address)                                                      

[crt][init.c:113:discoverBroadcastNetwork] Unable to query capabilities of interface eth0

# xap-currentcost -s /dev/ttyUSB1 -d 7

Current Cost Connector for xAP v12

Copyright (C) DBzoo, 2009-2012

[crt][init.c:113:discoverBroadcastNetwork] errno 126 (Cannot assign requested address)

[crt][init.c:113:discoverBroadcastNetwork] Unable to query capabilities of interface eth0

 

 

So not been able to get any data as yet, can you advise what is wrong?

garrydwilms
Offline
United Kingdom
Joined: 31 Mar 2011
Typo

There are only 2r's in current cost. 

Try the kill all command again with correct spelling

Garry

brett
Offline
Providence, United States
Joined: 9 Jan 2010
I forgot the HAH uses a

I forgot the HAH uses a bridged NIC sorry about that.

xap-currentcost -i br0 -s /dev/ttyUSB0 -d 7

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
OK I think I've got some data

OK I think I've got some data (attached) for sensor 9, I can try and get more, is there a way to send the output to a log file rather than just to screen?

AttachmentSize
sensor-9.txt 6.41 KB
brett
Offline
Providence, United States
Joined: 9 Jan 2010
That log file is a little

That log file is a little thin on detail - its look like there is data overruns on the data being recieved thou but its hard to tell without seeing any more.

I need a capture from start to finish and a little more please.  I see you have xap-serial running on /dev/ttyUSB0 so current cost is probably on usb1

# xap-currentcost -i br0 -s /dev/ttyUSB1 -d 7 > /tmp/currentcost.log

To capture to a file you can do this.

Brett

mark_baldwin
Offline
Blackburn, United Kingdom
Joined: 19 May 2012
Just to muddle the thread a little

When I paired my gas smart I did it twice. Once on channel 9 to set the impulse, then again on channel 0 (it's what I was told to do by CC)

After that I got an incrementing value on channel 9 and a pulse every time the magnet passed the sensor. I know I had this as I compared the graph from my gasnode with the graph on cosm that came from the CC. Unfortunately I can't remember what channel that pulse came in on and my HAH kit is currently boxed pending a house move.

I'll look through my saved documents to see if I can get some more info.

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
OK much better logging to a

OK much better logging to a file, attached lots of data hopefully useful?

AttachmentSize
currentcost.zip 43.2 KB
brett
Offline
Providence, United States
Joined: 9 Jan 2010
Recall when I asked you to

Recall when I asked you to check your xap-livebox.ini file to make sure that type9 was set to I (for impulse) well the debug tells me its set to A - analogue which means it will not report DELTA readings.  That certainly explains it.

[dbg][bsc.c:138:bscFindEndpoint] name sensor subaddr 9
[inf][currentcost.c:235:updateSensor] Adding new sensor uid=2b
[inf][currentcost.c:107:loadSensorINI] type9=A

There is some strange data overruns being seen in this file as well.  Almost as if there is no flow control on the serial interface and its sending faster than the code can service it or the data is being sent but the program isn't noticing its there for reading... (very odd) and impossible for me to trouble shoot without a CC unit and a Serial tether cable.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
OK we'll it defo says impulse

OK we'll it defo says impulse in the Web UI where is the .ini file in the folder structure I will manually edit it?

derek
Offline
Glasgow, United Kingdom
Joined: 26 Oct 2009
.ini file is at

/etc/xap-livebox.ini

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
OK in both the ini and the

OK in both the ini and the web ui it already says impulse I've not altered it please see...

 

I've also attached the full ini file?

AttachmentSize
type.PNG 5.39 KB
webui.PNG 63.89 KB
xap-livebox.txt 1.75 KB
brett
Offline
Providence, United States
Joined: 9 Jan 2010
I thought long and hard as to

I thought long and hard as to why you have configured sensor 5 as your Impulse sensor and could not come up with a resonable explaination.

  • You've corrected configured pachube to upload sensor 9.  (good)
  • All the data you've supplied since I started working on this has been from sensor 9.
  • XfxViewer would have also should you meter reading coming from sensor 9.

But still you setup sensor 5 - I'll let you explain to me why you did this.

Needless to say this is your problem.

So there is no confusion about what you need to do I've included a screenshot of how your WEB gui should be setup.

Configure for impulse on sensor 9

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Just being dumb I guess

Just being dumb I guess I didn't realise that there was any correlation between the ID in the Web UI and the CC sensor so just picked the next available ID in the list. I've amended now so hopefully it will now push the correct data.

Updated...

Not sure what's being output now?

brett
Offline
Providence, United States
Joined: 9 Jan 2010
Whats being output now is a

Whats being output now is a delta value.  Its the difference between each METER reading.

As for what this means I don't know either as I don't know what UNITS this value is and how many TICKS are requires before one meter numbers clicks over.

I'm just writing the code I know it will do what I want but interpretation of what this means is a different story.
Your graphs should start to look a little more sane thou.  Usage will show rising lines and no usage will be zero.

Derek assures me he included a USB dongle with the CC unit he sent me.  I'll dig in my cupboard and pull it out and see if I get the data overruns I saw.  However I never touched this part of the code and last time I looked it was stable.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Don't seem to be getting any

Don't seem to be getting any data to Cosm/Xively now, just in case I remapped the Gas to a new stream '9' (was 5) in Cosm/Xively still nothing. I've attached a new log, can you advise what is going on?

AttachmentSize
currentcost.zip 14.45 KB
brett
Offline
Providence, United States
Joined: 9 Jan 2010
This is because every bit of

This is because every bit of XML data that is being fed to the HAH from the currentcost unit is corrupt, that is not a welformed, so the XML parser is throwing it away.  Which mean no data is being emitted and there is nothing for the pachube engine to pick up.

This gets back to what I said before - there is a data overrun issue happening.  I've seen the EDF unit do this before but not a CC128 unit.

What's more strange is that I never touched this part of the code either.

If you look for the string "parseXml" you will see that in all cases for the log you supplied the XML is garbage.

HANG ON...

In the currentcost logs I also see its receiving a xAP packet with from currentcost data before this current program has recieved any XML from the serial port.... this is IMPOSSIBLE unless.

Do you have TWO HAH units?   Are you running xap-currentcost TWICE on the SAME unit?

Because I never see the CURRENT program parsing data for this a transmission is RECEIVED.... yes not sent. but RECEIVED.

[dbg][rx.c:23:readXapData] Rx xAP packet
xap-header
{
v=12
hop=1
uid=FF00DC2B
class=xAPBSC.event
source=dbzoo.livebox.CurrentCost:sensor.9
}
input.state
{
state=on
text=1
}

and yet there it is in the debug output !

And its doing exactly what it shoud be doing as well... showing a text=1 which is a DELTA value !?

I'm just really confused by what you are doing ... it should not be this hard.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
I only have one HAH, one CC,

I only have one HAH, one CC, I do have one RF room node which I am using to monitor the hot water temperature of my water tank and that's pretty much it.

I've run top on the HAH and its only displaying one process for currentcost

I guess I must still have something misconfigured somewhere.

AttachmentSize
top.PNG 121.97 KB
brett
Offline
Providence, United States
Joined: 9 Jan 2010
Everything looks correct...

Everything looks correct... I'm guessing you are not stopping the running xap-currentcost program before you start up another when you run it in debug mode and the two start competeing for data down the serial port which could explain dropped data etc..

I didn't think this was possible thou... Is this what you do?

However from the packets being received, that I could see, its working just fine.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
I thought that I had killed

I thought that I had killed the process however I've just gone thought the steps again and generated a further log file which should now be with the cc process killed for sure...

AttachmentSize
currentcost.zip 41.38 KB
brett
Offline
Providence, United States
Joined: 9 Jan 2010
That's better yes the data is

That's better yes the data is not being truncated now !
This is working, where working is defined as "my code is doing what I expected it to do".

As for interpretting the output this is a different problem.
Because I'm always looking at your graph on a 6 hour windows (thanks xively) and during this time there is no has consumption for me the graph always looks flat for your feed.

However I am seeing values range from 1-3  which show a steady usage of gas with occasional increases every 2mins.

You can always flip sensor.9 back to ANALOGE and revert to graphing the meter reading if the delta's aren't making anymore sense than just the absolute reading.

I'm not sure there is much more I can do on this - I'm coding blind.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
OK great I did turn on the

OK great I did turn on the central heating whilst running the logging so I was expecting to see some usage.  Can you just clarify what I should be seeing output via sensor 9? Part of the pairing process required a double pairing of a second sensor I'm not too sure what this was needed for just said one for data the other for display although nothing is displayed on the CC is the delta & usage all pushed through the one sensor?

brett
Offline
Providence, United States
Joined: 9 Jan 2010
Sorry your asking questions

Sorry your asking questions about a device I've never seen.  All i know is that the CC base unit sends serial XML data to the xap-currentcost program.  I slurp this and output a xAP packet which is picked up by xap-pachube (soon to be xap-xively).

I would not worry about nothing displayed on the CC base unit (for now) if you fire up xFxViewer you should see data being emitted as per my changes. I know it was inbound as the debug trace you sent me had data values in it.

As Isaid before at this point I've almost done all I can to support this.

Brett

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Thanks for your help

Thanks for your help Brett.

I'm confused as to what is going on now though basically now having used xFx viewer this evening with the central heating on I am now only seeing text=0 or text=1 or text=2 from sensor 9 which was the 'data' sensor whereas now on sensor 1 I'm now seeing all kinds of values anything from text=0 to numbers like text=509, text=411, text=312. I've not configured sensor 1 as anything in my HAH so just unsure/confused as to what this data is now. Can anyone throw any light on this?

I've attached an export from my xFx viewer in case.

AttachmentSize
xAPMessageHistory.zip 12.4 KB
garrydwilms
Offline
United Kingdom
Joined: 31 Mar 2011
My take on it

Hi,

l'm no expert but here's my take on it for what it's worth.

your screen shot earlier shows you registered sensors 1-4 for what I think should be ch1-3 and temp for the main house (sensor 0)

Your good xml data capture suggests this is the case. I think you need to remove these. Sensor 0 channels and temp are registered automatically (I think).

you should only have sensor 9 from your impulse sensor registered i think. If you are seeing data on sensor 1( which looks like kw data to me) it may be you've accidentally paired something to it. Try resetting the envir totally and starting again as per the manual. Main house and sensor 9.

now, about the impulse data. Reread the links to xml output earlier in the thread. The xml output from the sensor is in two parts. The first is the pulse counter within the <IMP> tag. This is the counter from your meter pulses. I think Brett's code can output either the total pulse count or the difference between counts. Either way this is purely meter pulse counting, and it appears to be working as you are getting text=1, text=2 on sensor 9, which i beleive are the delta counts.

however, to make this correspond to KwH of energy then the <IPU> tag needs to be used, this appears to be the number of pulses from the meter that equates to one kWH and you should set this value on your envir unit (somehow).

Therefore each pulse equates to 1/<IPU> kwH. To chart this i think you will need a lua script to do the calc and squirt to xively.

Brett, if this info is correct, could xap-current cost divide the delta value by the <IPU> figure and report a kWH value as a display text field as well? Just a thought

Garry

ps this might all be rubbish as I don't own one and am guessing ;)

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
After I found out that the ID

After I found out that the ID in the Current Cost settings matches the sensor number I removed all my old settings as the first 4 all related to the mains channels and the temp so there were irrelevant settings


In the pairing process of the GasSensor the instructions were to pair it twice one being the data the other said to use another spare sensor which I chose sensor 1 for the second pairing. 

With sensor 9 set as impulse I am seeing text=0 or 1 or 2 and with old settings removed I now see larger numbers pushed via the sensor 1 which I guess being skewed previously due to the old settings I removed.

 

So now my issue is I don’t really know what the data is that’s being pushed so don't know how to use it.

AttachmentSize
CCS.PNG 65.48 KB
garrydwilms
Offline
United Kingdom
Joined: 31 Mar 2011
I've read

I've read the instructions and I believe the second pairing should be on the main House not sensor 1. 

I would suggest that you start from scratch and follow instructions to the letter.

even so, you're only going to get pulse data off the HAH. Therefore some other calc will be required to convert to kwh etc.

 

garry

DoubleSpeed
Offline
Bristol, United Kingdom
Joined: 31 Mar 2012
Thanks for the feedback Gary,

Thanks for the feedback Gary, I queried this with CC's support at the time and they said that if you are using their OptiSmart sensor for your whole house reading then you needed to do what you've said however the GasSmart needs to be paired twice to appliance sensors, so unless they've given me incorrect information as far as I know I've got it paired correctly.

andy_godber
Offline
Joined: 13 Sep 2011
Beta firmware

Chaps - can you remind me, how do I put the beta fimrware on with Brett's Current Cost gas code?

garrydwilms
Offline
United Kingdom
Joined: 31 Mar 2011
Here you go #

Here you go

 

# /etc_ro_fs/update-dev hah-beta.dbzoo.com

 

andy_godber
Offline
Joined: 13 Sep 2011
Thx

Thanks Gary

andy_godber
Offline
Joined: 13 Sep 2011
Working - Nice Job

Brett - et al, this is working for me - nice job and thanks.

 

Ive actually got it on my water meter to log litres etc. I dismantled the reed switch on the CC unit, and attached to a similar switch on the meter.

My gas meter is really old, and I cant find any sign of it having any form of output or magnet :(

brett
Offline
Providence, United States
Joined: 9 Jan 2010
Andy thanks for the positive

Andy thanks for the positive feedback on the water meter this is great confirmation I'm not going insane.  Pretty cool I can code all this up not having any of these devices myself  :(

Brett

Hardware Info