Android

This is really interesting. Lots of folks that I talk to about the HAH project are surprised that we don't have mobile 'apps'. I can run the native HAH browser UI very nicely on my HTC One X, but it does seem that people want a dedicated app. 

Brett would indeed know how best to architect this. I'm definately thinking of a Lua script that is watching for a UDP message from the Android app and re-broadcasting a xAP message on the home LAN based on what is needing to be controlled. Authentication might need a bit of thinking about - but this needn't be hugely complex.

I don't know enough about mobile apps to know about how any 'incoming' message might be handled, but I'm betting that Lua can send a response to the socket that carries the original message.

With apps, the feel of the UI is everything. http://www.vivint.com/
Hmm, how about $0/day?

Derek.

There are two aspect to an xAP client - if you just want to send information its pretty easy you just contruct an xAP payload and broadcast it on port 3639 job done.  There are python samples for you to look at for a high level overview of the most basic client you can construct.   Easily doable in most languages really.

If you want to deal with incoming information its only slightly more tricky.  There are two sub-aspects to this.  If you run a single xap application on the android then you can simply listen for UDP broadcast packets on 3639.  If you want to run multiple xap applications as the same time then you need a hub.

I'm not sure what language you program up on the android is this C, Java, what?

I've already written logic in C and LUA for decoding xAP packets and I'm sure this would be easy enough to port into yet another language if required.  I have an android tablet - well its my wifes and she is very protective of it however I'm sure if I indicated that it was in her interest I might be able to use it.  I could always flash up my webos tablet for android and use that I guess - I've been keen to look into this but I'm short of time.

Let me know how I can help out and I'll be happy to provide information where I can.

Brett

John,

If your IP is 10.0.0.30, and we assume a subnet of 255.255.255.0, you're broadcast address will be 10.0.0.255.  If your subnet is 255.0.0.0, your boardcast address will be 10.255.255.255.  xFxViewer picks up the broadcast messages.

Karl

Andy, this sounds really interesting. Excellent work!

Actually makes me a little bit gutted I have an iPhone :( which I must say is a first for me 

would love to help with the decoding but I'm in the same boat as you regarding a scope. Have tried the audio jack on laptop method in the past but I get way to much noise and not had much success ( actually No success!)

Keep up the good work and I may consider switching to android ( did I actually just say that!)

Garry

Andy,

Great to hear about your Android work. My HTC One X is sooo ready for this!

I did make a start on the Lightwave RF protocol. Turning a bulb on and off is totally solid. I need to do more research if we are to be able to do smarter stuff. I'm very impressed with the quality of the Lightwave kit.

Paul did some work too and was good enuff to let me publish this. All we have is on the wiki.

Derek.

Derek,

Paul's been in touch, and am hopeful his work will provide a needed boost on this front. B.O. has also been intouch and i'll see if we can get some sort of research/verification process underway with them both. I really like the LWRF & Siemens branded kit, its very modern and up to date, and my wife/kids really likes how the dimer kit works, both the wallplates (siemens) and the CFL (lwrf).

I used both of your wiki input, but it quickly took a back seat to the above project, but now I am getting things running, I have found that I want to integrate more of my lwrf kit into the HAH to keep it all central (and use plugboard) - my aim is to ditch my lwrf iLink (or keep as backup). Now I have some new data from Paul, I can move on this front also - hopefully there will not be too many obstacles. I know I should buy a o-scope (perhaps USB) (and learn to use it!!) - but I promisedd my O.H that I would keep costs 'reasonable' and £200+ mark is a stiff drink away. I know I could get a cheaper scope but would that be money well spent in long run.?

Anyhow, I also now just won a Joggler on fleabay so that should also help with some compatability testing.

--andy

The bit that I stopped at was getting the HAH to 'train' a CFL bulb. I used a lightwave remote to do this piece. 

As for a USB 'scope - it needn't cost the earth. Brett picked up one from eBay - kinda basic, but certainly up for the job. Perhaps he can provide a link to the one that he has?

Derek.

Thanks for the testing. I just didn't get to trying this.

Further, I hadn't thought of the 'multiple transmitter' support that is built into the Lightwave units. This makes dual HAH/handset control easy to do, with a 'standard' set of codes from the HAH.

My Lightwave bulb control from the HAH is very solid. I really must add Lightwave support to the RF Rx unit.

Meanwhile, the LightwaveRF prices at B&Q still seem a lot better than most anywhere else (eBay included).

Derek.

Shall we start a new thread under Hardware for LightwaveRF.

The multiple transmitter support has come in handy, my HAH is now mimicking 2 * 4 button remotes already.

@Derek - my CFL's control is rock solid, just experimenting with @paulsbiz data, and looking ok. Will share my new info next few days.

--andy

John,

UDP is a fire and forget protocol, with any ack needing to be built in at the protocol level. Imagine a fast two wheel drive car.

It cannot go over ditches and banks (its not routable in the normal sense)

Whereas TCP has lots of goodies, and built in support for ack and routing etc. Imagine a 4 wheel drive with bullbars, high ground clearance and lots oftorque. Its can traverse most anywhere.

At a basic level, your (smart)phone operates in two environments, over cell network, and over Wifi. Cell best using TCP, Wifi can be TCP OR UDP. You need to sit a 'high level protocol' on either. For UDP you would use xAP for obvious reasons. HAH listens for xAP over UDP by default so support is out-of-box. For TCP you must have a listener (just like a webserver listens for HTTP/HTTPS, something needs to listen (and reply) over the TCP network to communicate with your phone and broker for the HAH. The iServer operates as a TCP->UDP(xAP) broker/proxy/router (call it what you will).

So, if you want to talk back to your house in UK from spain using your phone, you really need TCP. If you want to talk to HAH from phone only in your own home via your android mobile, the UDP is good choice.

I have reasons to using both protocols, but also plan on using other listeners as well as iServer - I am experimenting. For example you could try looking at a message queueing system, like AMQP or XMPP etc. You could try implementing a 'restful' interface and you could stick to HTTP over TCP. There are lots of options, and lost of considerations, like resources on LB/HAH (or offload onto other CPU), battery on mobiles etc, responsiveness etc etc.

--andy

John,

(I am sure you googled) But this small page sums up some of the pro's/con's of tcp/udp and may serve to help make a refined choice as to how to proceed.

btw: I have chosen initially to use both here, but may not always finish up with using both.

http://tunnel.mrq3.com/explain/node2.html

There are lots of other pro's/con's and security of data could be another that may be important to you (especially outside your homes).

--andy

A dual topic thread ... :-(

Although iServer Windows  is closed source the protocol that is used for connections is pretty simple and I'm happy to disclose any aspects of it for people wanting TCP connectivity onto xAP. Unfortunately xServer was closed source so my iServer fork of that app has to remain that way.  The protocol isn't documented as such but I know it should be.

Aside from the guaranteed delivery and sequence ordering that TCP offers there are other aspects that iServer assists with.   Notably a device attaching to iServer can create filters such that only traffic of relevance to the device (perhaps the xAP devices that match buttons on the device or certain schema) traverses the socket.  This filtering helps significantly with lighter devices in terms of their parsing load.   The iServer also automatically synchs (queries) all registered BSC devices when a client connects and handles pacing of traffic.  There are additional aspects in the Windows iServer that add a security level to restrict a connected user to certain devices.

On the upcoming iOS application I have implemented xAP using both UDP and TCP / iServer and my experience is that TCP is preferable and recommended.  UDP works fine with strong WiFi signals but in weak situations I see packet loss.  Because there is no pre filtering with UDP the app has to work significantly harder parsing all incoming traffic.   Also as you switch an app in and out of background you can easily lose traffic meaning your state model is inconsistent, this might be avoidable with TCP depending on how background buffering is implemented in the OS.

Just to clarify a xAP connection via TCP to iServer maintains the full broadcast nature of xAP where all devices hear all messages from everyone.  However filtering , if used, can reduce this at the iServer client.  Messages specifically targeted at all devices e.g. target=>:> can be configured to traverse a Windows iServer link regardless of filtering as can certain schema e.g. message.display used in CallerID display.  xAP wildcards can also be used in filters.

The features differ between the various iServer's that exist although a core feature set provides compatibility.    Windows iServer derived from an application designed to bridge xAP networks and so has better security and device protection as well as supporting several different client connection protocols.  HAH's iServer has slightly better device filter wildcarding. 

In order to use iServer you have to connect via TCP , logon and then setup filters for the types of xAP messages that you want iServer to pass back to you from the xAP network. If you use the Windows iServer then these filters can be associated with a profile set up in iServer but the HAH iServer requires they be setup each time a connection is made.  During a session you can create filters on an ad hoc basis (if your profile permits this).

Messages you send to iServer are of two types 'command' and 'xAP' and each are enclosed in XML type tags .   Control messages use <cmd>......</cmd>  and xAP messages <xap>... </xap>.   The data between the <xap> tags is exactly the same as a normal xAP message and all such messages sent to iServer will be forwarded onto the xAP UDP network verbatim.    xAP messages received by iServer will be compared against the filters in place for each attached client and forwarded to you when matched,  enclosed within <xap>...</xap> tags.   Bear in mind that in your TCP listener because of buffering, multiple xAP messages may be concatenated and also truncated, any such partial messages should be retained and prepended to the next data arrival.  No data will ever be lost but you lose the natural behaviour of UDP where one datagram arrival always represents one complete xAP message.

Where there are differences between HAH and Windows iServer I have tried to mention them.

On opening a socket to iServer (TCP port 9996) it will respond

<ACL></ACL>

You need to logon using log<code>  and sending a password that matches a profile configured in iServer - which could be blank if passwords are not enabled.  On Windows iServer the password is really just a secondary code that determines the priveledges and preset filters associated with a profile .  HAH iServer just implements the password as a logon credential granting full access. There is a more advanced challenge response authentication but that is not used here.

<cmd>log<code>password</cmd>

 A successful logon is not acknowledged although some versions of Windows iServer may return the name of the profile in use.  <ACL>Joggler</ACL>

You should now immediately send a heartbeat message from your application which iServer uses to establish your source address.

<xap>

xap-hbeat
{
v=13
hop=1
uid=FF.2122:00
class=xAP-HBeat.Alive
source=ERSP.Barix.Barionet
interval=60
}

</xap>

Windows iServer has additional security to restrict messages to avoid clients spoofing source addresses and can also restrict classes e.g. to block incoming xAPBSC.cmd messages meaning a client can monitor but not control certain devices on your netowrk. HAH iServer does not implement this additional security as it is intended for intranet use or password protected (full access) security only.

Now you need to create your filters using <flt+> to tell iServer which xAP messages to pass to you. As soon as you create a filter any matching xAP messages will be forwarded to you. Typically this involves adding the specific source address for each device you wish to hear messages from. This creates the most efficient traffic flow between iServer and your client and is the recommended approach. Filters are case insensitive.

<cmd><flt+>acme.controller.house:heating.override</cmd>

<cmd><flt+>acme.controller.security:PIR.Hall</cmd>

However you can use wildcards too e.g.

<cmd><flt+>acme.controller.house:*</cmd>

this latter one would pass all sub addressed messages from acme.controller.house to your application, but not heartbeats for example as they do not originate from a sub address.  If you were passing all endpoints from one specific device this is an efficient way of setting the filters up.

You can also create a filter based on the xAP class using <cflt+>.

<cmd><cflt+>xapbsc.*</cmd>

There are also the reverse <flt-> and <cflt-> options to create filters blocking device messages from being passed.  These are really only useful when you have other previous wildcard filters in place.

BSC specific feature:   When you create a filter using <flt+> iServer will automatically on your behalf issue a xAPBSC.query command to that device address and pass any xAPBSC.info response back to you . This ensures that your application is immediately synched with the current status value of the device.  Although the HAH iServer usage is typically passing xAPBSC class messages it is not in any way restricted to this and will pass / filter any class of message. Windows iServer has a 'BSC only' configuration option.

iServers  buffer/pace messages to avoid a sudden burst of messages flooding the network when filters are created.  This can be most useful to avoid overflowing buffers in limited memory devices.  

  K

.

Writing an xAP parser is pretty simple there are two available in source format for you to examine.  One written in C inside xaplib2 and one in LUA both are written from scratch by me.

I've not looked at a Java xAP parser but based on either of the code samples from above it wouldn't take long to sling one together.

Patrick has a java parser: http://patrick.lidstone.net/html/dev_tools.html

Having said that I rewrote his C parser so it would work on the livebox as his has LARGE memory requirements and was not suitable for embedded devices - HAH nor Arduino.  So your mileage may vary on his Java parser too. I've not looked as his source so I can't say for sure but if its anything like his C parser its  in need of some love.

I really have to get around to writing some Android xAP stuff.  Its definately on my do to list.

Brett

Nice screenshot John, looks good i hope we can all make use of an Android app someday.

What version of the SDK did you create in?