Well yes. Looks like I had a case of the diarrhea yesterday!

I have just been looking at ancient code I have for two instances of drivers I have written.

One is in Java for the HAI OmniPro. It uses comm.jar for all the really low level stuff. That is used by a class

public class HaiConnection {
	public boolean open(String comport, int baudRate);
        public boolean isConnected();
	public boolean login();
	public boolean getResponse(MessageInterface outMsg,MessageInterface response);
	public boolean receiveMsg(MessageInterface response);
	private void sendMsg(MessageInterface outMsg);
  }

bytesRead = inputStream.read(response.getBytes(), offset,bytesToRead);
	outputStream.write(outMsg.getBytes());

Higher level classes create messages, such as

public class RequestThermostatStatus extends Message {
    byte[] thisMsg = { 0x5A, 0x03, 0x1E, 0, 0, 0, 0 };
	
   public RequestThermostatStatus(int firstZone, int lastZone){
	msg = thisMsg;
	msg[3] = (byte)firstZone;
	msg[4] = (byte)lastZone;
	calcCRC();
	setCRC();
}	
}

		if (!hai.getResponse(new RequestZoneStatus(first, last), zoneStatus)) {
			System.out.printf("\nNo or bad response from HAI");
			return false;
		}

To me it is proper that all this logic is in Java and platform independent. I don't know where I got comm.jar from, but hopefully there should be a Linux and a Windows and a Mac version.

The other driver is for a Denon AVR4306. It is written in Lua and uses the Girder gip class for low level access. gip stands for generic IP and comes with girder as a library. Its code looks something like this (I will edit it a bit to make it more concise)

--[[ Denon 4306 AVR --]]
avr ={}

--[[
AVR callback function.  Handles unsolicited events from the Denon
Callbacks on the Socket object are made when connections are made
or broken, when data is received or when errors occur. 
Usually these mean someone is fiddling with the knobs and selecting
a new mode.
--]]
function avr.CallBack(data,code)
	if ( code == gip.CONNECTIONCLOSED ) then
		print("Connection Closed")
		return
	end
	if ( code == gip.CONNECTIONESTABLISHED ) then
		if ( data == 0 ) then
			print("New IP connection to AVR 4306")
			avr.state = "open"
		else -- data ~=0
			print("New Connection Failed. Data(error?): ", data)
		end
		return
	end
	--print("AVR callback data:",data,"   Code:",code)
end
--[[
Open the socket in the AVR on the Telnet port (23) & register a callback
--]]
function avr.init()
	print("Open socket to avr")
	avr.state = nil
	count = 0
	socket = gip.Open('192.168.002.7',23)
	socket:Callback(2, '\r', 1000, avr.CallBack)-- 1s timeout
	while avr.state == nil do
		win.Sleep(50)-- ms wait for callback - does not work??
	end	
	print("avr socket:",avr.state)
end

function avr.write(something)
	win.Sleep(500)
	if avr.state ~= "open" then
		avr.init()
	end
	--print("Writing to AVR socket:", something)
	socket:Write(something)
end


function avr.on()
	avr.write("PWON\r")
	win.Sleep(1000)
	print("AVR on")
end

function avr.off()
	avr.write("PWSTANDBY\r")
	print("AVR off")
end

-- avr input source
--available args: PHONO, CD, TUNER, DVD, VDP, TV, DBS, VCR-1,
--                VCR-2, V-AUX, CDR/TAPE, AUXNET, AUXIPOD, ?
function avr.selectSource(src)
	avr.write("SI"..src.."\r")
end

-- avr sound mode
--available args: DIRECT, PURE DIRECT, STEREO, MULTICH IN, MULTI CH DIRECT
-- MULTI CH PURE, DOLBY PRO LOGIC,DOLBY PS2, DOLBY PL2X, DOLBY DIGITAL,
-- DOLBY D EX, DTS NEO:6, DTS SURROUND,ES DSCRT6.1, DTS ES MTRX6.1, WIDE SCREEN,
-- 5CH STEREO, 7CH STEREO, SUPER STADIUM, ROCK ARENA, JAZZ CLUB, CLASSIC CONCERT,
-- MONO MOVIE,MATRIX, VIDEO GAME, VIRTUAL,, MPEG2 AAC, AAC+DOLBY EX                
function avr.soundMode(mode)
	avr.write("MS"..mode.."\r")
end

-- avr master volume increase
function avr.masterVolumeUp()
	avr.write("MVUP\r")
end
end

You don't need to now too much about Lua to follow that. The functionality provided by gip is already in Java. Open a socket. Listen on it...

Back to philosophy.

(IMHO) This is the kind of 'driver' code that you want lots of for lots of devices. A good point was made elsewhere that you won't own all of these potential devices but you do want a library of implementations. How to standardise it a bit? Good base classes or interfaces.

I can envisage a generic IR device class that could initialise itself by reading a csv or JSON file of codes and interface with USB-UIRT or Global Cache or any IR hardware using JNI (isn't it). Such code could handle a huge range of IR devices. Or you could download the codes as xml (I suppose) from the IR database.

An issue would be what is the interface of a driver. What methods does it have? How does it expose what it can do? Is a driver really a bean?

Creating the gui is then a case of mapping gui widgets to driver (bean?) actions. Not too difficult if actions have no arguments.

Hibernate does some of this mapping stuff in xml. It is readable, but not an exciting read.....

Enough. - Mapping should be in a different topic.

Marc Fleury wrote on Jul 28, 2008 06:58:

Robin H wrote on Jul 28, 2008 03:27:

... [ snip 8< ]

Is this for a select group of Linux/iPhone I-will-solder-it geeks or aiming at the bigger issue of better HA control?

Well yes and yes, we are actually quite snobbish in who we like to hang around with. That being said, democratized HA control, integrated HA, easy to program HA, easy to install and maintain HA are all valid goals. We are trying to figure out what it means to us exactly and your feedback is appreciated.

... [ snip 8< ]

Welcome to OR.

PS: do check out the help button on the top right it does give you great tips for formatting your post. A tip is to turn preview on so you know when you went wrong. It is a bit stiff at first but it works well and it looks good.

I decided to wait to post, how dare Robin denegrate my Linux! ;-) (See my sig if you don't quite get the humor, if it still doesn't make sense, ignore it).

I don't know, there seem to be only two of us (MarkS and I) who can wield a solder iron properly. The rest are just programmers (that's going to get me in trouble ;-). I don't know about MarkS but I can code too (which is not the same ting as programming).

All kidding aside, the reason for Linux on the reference implementation (the ORC) was it was cheap, enough of us are familiar with it and it fits well on smaller systems such as the controller.

On the Java side I think we settled on the network daemon type interface. OS differences are now gone. I was thinking about that last night. Such a daemon could be written for any OS to hide it's device interface details from the Java programmer. One thing that doesn't translate well between OS's is the device interface (not everything is a serial port). Windows has the HID on the USB (for some devices), Linux kind of has it but it's different (but Unix folks prefer /dev/xxx devices) and on other OS implementations, ..., well I don't know. At the moment we're not that far along to actually have a lot of code written. Once a daemon is written the Java remains the same. The daemon also allows for devices that are not directly connected to the host controller but are reachable over a route-able network (they could be worlds away).

I see being cross platform as being a good thing. It was one of the things that worked well in the Misterhouse project. I see no problem with that. As for the starting off with the Geek crowd well that's a necessity. If the average folks could do this already the project would have done. But as we are just starting out we have geeks with egos running the asylum. ;-) Eventually the normal folks will get to running things and we'll either be put back in our padding rooms with full length formal wear, crayons, soft food and a keyboard or we'll run another asylum. :-)

Robin H wrote on Jul 28, 2008 16:16:
Well yes. Looks like I had a case of the diarrhea yesterday! I have just been looking at ancient code I have for two instances of drivers I have written. One is in Java for the HAI OmniPro. It uses comm.jar for all the really low level stuff. That is used by a class

public class HaiConnection {
	public boolean open(String comport, int baudRate);
        public boolean isConnected();
	public boolean login();
	public boolean getResponse(MessageInterface outMsg,MessageInterface response);
	public boolean receiveMsg(MessageInterface response);
	private void sendMsg(MessageInterface outMsg);
  }

Within this code, two key lines (for the discussion here) are:

bytesRead = inputStream.read(response.getBytes(), offset,bytesToRead);
	outputStream.write(outMsg.getBytes());

The tricky thing in this code is sorting out the solicited input from the unsolicited input noting that the unsolicited stuff is not well-documented.

Well congratulations on mastering the markup language so fast, you are a quick study.

Higher level classes create messages, such as

public class RequestThermostatStatus extends Message {
    byte[] thisMsg = { 0x5A, 0x03, 0x1E, 0, 0, 0, 0 };
	
   public RequestThermostatStatus(int firstZone, int lastZone){
	msg = thisMsg;
	msg[3] = (byte)firstZone;
	msg[4] = (byte)lastZone;
	calcCRC();
	setCRC();
}

GAH! HEX CODES!!

pant! pant!

Ideally we would externalize these in machine readable format hence the xml in Hibernate for example. You may want to read up the notes by CB on Beehive, a lot of work and thinking is needed there to achieve the database you refer to (see under Misc).

function avr.CallBack(data,code)
Open the socket in the AVR on the Telnet port (23) & register a callback

Neat!

-- avr master volume increase
function avr.masterVolumeUp()
	avr.write("MVUP\r")
end

GAH! NON-STANDARD ABBREVIATIONS!!

See point about externalizing the proprietary lingo so we can map a more standard message format like KNX. Then you can isolate all the local stuff in the java driver, makes sense to me.

Back to philosophy. (IMHO) This is the kind of 'driver' code that you want lots of for lots of devices. A good point was made elsewhere that you won't own all of these potential devices but you do want a library of implementations. How to standardise it a bit? Good base classes or interfaces.

Well you may want to check out the discussions with Neil and Juha, they are both going through the moves of defining a driver for X10, CM15A (reference implementation on Linux :). I will go through the moves of a KNX driver. We are thinking about an INSTEON driver etc. Maybe an OmniPro driver integration would help us? You obviously have been through the moves of writing your own driver. Maybe you can help them.

Also they defining a packaging for such a driver. Once you define you have an omnipro installation, we would indeed download a bean that is a java representation of your device. The point is standardizing packaging so we can simplify installation and maintenance.

I can envisage a generic IR device class that could initialise itself by reading a csv or JSON file of codes and interface with USB-UIRT or Global Cache or any IR hardware using JNI (isn't it). Such code could handle a huge range of IR devices. Or you could download the codes as xml (I suppose) from the IR database.

yes, see Beehive, you may want to help there as well. Stewart Allen from Tonto has offered to help us if we want to take the project. We will be interested in the CCF data as a populator for Beehive. Defining the format, as you point out doesn't make for an exciting read but is an important machine step.

An issue would be what is the interface of a driver. What methods does it have? How does it expose what it can do? Is a driver really a bean?

Could very well be a bean. Not truly relevant as long as it is a dynamic profile and this is a function of the classloaders not of packaging. We are using a JBoss base to have access to these distributed classloaders (so you can load a class from a URL, in this case www.openremote.org. You instantiate the classloader with the URL and you can remote load, as simple as that. We will want to explore the implications for installation and maintenance of the controllers.

Creating the gui is then a case of mapping gui widgets to driver (bean?) actions. Not too difficult if actions have no arguments.

See UI discussions. There is a layer of abstraction here, so we can have custom remotes, even my grandma remote profiles that are really a selection of a subset of the commands externalized by the device. The idea is that the cocoa application reads an XML file that describes the UI, we would generate these from a web application of drag and drop, we plan on using GWT for that.

Robin H wrote on Jul 28, 2008 16:16:
To me it is proper that all this logic is in Java and platform independent. I don't know where I got comm.jar from, but hopefully there should be a Linux and a Windows and a Mac version.

Right now we are steering away from comms.jar to increase the reliability of the system. The assumption here is that Neil can code a daemon that doesn't crash left and right ;-) But the point is that even if it occasionally trips up, we'll just restart it without affecting the rest of the controller functionality or responsiveness. We get subsystem isolation via OS process spaces.

Should that plan fail, we can always fall back to comms.jar. And if somebody does provide a driver to a device we don't provide an integration for, and it does rely on comms.jar, we can still use it. Either short or long term. Depending on the level use, reliability and the comfort level of the community in case of a failure.

Robin H wrote on Jul 28, 2008 16:16:
(IMHO) This is the kind of 'driver' code that you want lots of for lots of devices. A good point was made elsewhere that you won't own all of these potential devices but you do want a library of implementations. How to standardise it a bit? Good base classes or interfaces.

Java frameworks have evolved far enough that we don't necessarily need to lock things down with base classes or interfaces. Introspection of class members and annotations or XML configuration enables flexibility for implementors to follow their own programming models. I'm leery of locking things down at this point when it is not clear how people will want to use this thing we are building.

Robin H wrote on Jul 28, 2008 16:16:
I can envisage a generic IR device class that could initialise itself by reading a csv or JSON file of codes and interface with USB-UIRT or Global Cache or any IR hardware using JNI (isn't it). Such code could handle a huge range of IR devices. Or you could download the codes as xml (I suppose) from the IR database.

The goal is to have a normalized device database, as was mentioned elsewhere.

In the interim, csv or other formats for bean class to initialize itself is quite plausible. The middleware is generic enough to handle such a case.

Robin H wrote on Jul 28, 2008 16:16:
An issue would be what is the interface of a driver. What methods does it have?

Doesn't really matter. It's a component with an arbitrary interface. We've supported this in middleware for quite some time.

What matters is the metadata that allows input commands (HTTP, IR) to be mapped to the driver methods.

Robin H wrote on Jul 28, 2008 16:16:
How does it expose what it can do?

This is metadata. Several specifications and implementations have solved attaching arbitrary metadata to a class.

Robin H wrote on Jul 28, 2008 16:16:
Is a driver really a bean?

To an extent that it wants to integrate with the middleware, yes it is.

Right now we are steering away from comms.jar to increase the reliability of the system. The assumption here is that Neil can code a daemon that doesn't crash left and right ;-) But the point is that even if it occasionally trips up, we'll just restart it without affecting the rest of the controller functionality or responsiveness.

If comms.jar has bugs (I don't know), isn't the right approach to write a new comms2.jar that doesn't have bugs. Device driver bugs can crash the whole OS, so the device driver ought to be small tight code that just connects the hardware to the app and not much more. In this case (comms.jar) an RS232 driver.

Above this layer is the packetisation, hex codes and asynchronous stuff that is different for every device. Given that this will have more bugs than the former, it should run in user space where it can exception rather than crash.

Custom drivers should only be needed when you have a custom hardware interface, for example a parallel port that drives relays or an embedded sub-micro controller where you really need to know the hardware.

Anything IP or RS232 should use heavily reused code that really works. Comm.jar or a derivative or replacement.

I liked the BT Serial dongle that was showcased here the other day. This way we can put a dongle in the back of a TV and BT to it from the ORC without any other physical connection. If that is what you want to program, just start coding it will control your TV... methinks you got your itch...

we got a winner!

My Panasonic plasma TVs have a pretty dumb IR interface. There are some discrete codes though for on/off.

If you did into the menus, deep down, there is a Linux acknowledgment. My TV runs Linux!

I'll wager that one could hack into that. I'll bet service guys have an IP header that gives access to that.

Don't look at me!

Robin H wrote on Jul 29, 2008 00:15:
My Panasonic plasma TVs have a pretty dumb IR interface. There are some discrete codes though for on/off. If you did into the menus, deep down, there is a Linux acknowledgment. My TV runs Linux!

Many Panny's have RS-232 ports, which would make it much easier to integrate, what is the model number of your plasma?

Juha Lindfors wrote on Jul 29, 2008 00:41:

Robin H wrote on Jul 28, 2008 23:30:
Device driver bugs can crash the whole OS

And that's also a problem. So I'm counting on a modern Linux kernel having a sufficient protection from a user process reading and writing bytes. Linux can do this, right? Right?

RIGHT! :-) I think that Linux has had such protection since early on (same thing with Unix and jsut about every modern OS). I've written all sorts of bad code while experimenting and I haven't crashed the system. I'm pretty good at crashing things, that's why I was hired at the labs.

It's not like we're writing a real driver here in the kernel space, just a process that reads and writes thru the actual kernel driver? Anyway, we're getting into Neil's domain here, he can correct me if I am wrong.

Robin H wrote on Jul 28, 2008 23:30:
so the device driver ought to be small tight code that just connects the hardware to the app and not much more. In this case (comms.jar) an RS232 driver.

the daemon doesn't do anything other than pipe bytes through for us. In that sense it's no different from Java Comms. There's no logic in it. Just that we partition the system to two processes. Instead of JNI we use sockets.

Robin H wrote on Jul 28, 2008 23:30:
Custom drivers should only be needed when you have a custom hardware interface, for example a parallel port that drives relays or an embedded sub-micro controller where you really need to know the hardware.

We're not writing kernel space drivers.

Sorry for the late replies, had a very bad day at work. I lost two Ethernet switches and an ancient server (Fedora 4) due to power problems. They were bringing in a new 13Kv into the complex and some of the old equipment did not survive power up. I'm still trying to rebuild my test bed. I was at work until 8PM, long day.

Hey, I'm writing kernel space drivers. :-) But as Juha has said the drivers are just juggling bytes as quickly as possible and don't do a lot. That's the way all device drivers should work, don't do more than the absolute minimum. Slightest slip and poof you're rebooting everything (this is true of any OS). I don't know why everyone freaks with C, it's a lot better than assembly language. ;-) My object are made up of bits. Most of my work is at the lower levels.

The daemons should be very stable. I've run most of this code on my Fedora server for a long time. Other than power problems in my home and bad hardware my systems stay up for months. In the past I've used CGYWIN to compile the Unix code to operate under Windows. It's worked well but I'd prefer native Windows code. I'm not a Windows person so we'll probably someone to help me with that. But that's later.

Robin H wrote on Jul 28, 2008 23:30:
If comms.jar has bugs (I don't know), isn't the right approach to write a new comms2.jar that doesn't have bugs.

All software has bugs :-)

The problem with JNI bugs is that they have potential to bring down the JVM with them. I'd like try and keep the JVM around even if it can't tell the client anything other than oops, something went wrong, want to reboot?

Robin H wrote on Jul 28, 2008 23:30:
Device driver bugs can crash the whole OS

And that's also a problem. So I'm counting on a modern Linux kernel having a sufficient protection from a user process reading and writing bytes. Linux can do this, right? Right? It's not like we're writing a real driver here in the kernel space, just a process that reads and writes thru the actual kernel driver? Anyway, we're getting into Neil's domain here, he can correct me if I am wrong.

Robin H wrote on Jul 28, 2008 23:30:
so the device driver ought to be small tight code that just connects the hardware to the app and not much more. In this case (comms.jar) an RS232 driver.

the daemon doesn't do anything other than pipe bytes through for us. In that sense it's no different from Java Comms. There's no logic in it. Just that we partition the system to two processes. Instead of JNI we use sockets.

Robin H wrote on Jul 28, 2008 23:30:
Custom drivers should only be needed when you have a custom hardware interface, for example a parallel port that drives relays or an embedded sub-micro controller where you really need to know the hardware.

We're not writing kernel space drivers.

Robin H wrote on Jul 28, 2008 23:30:
Right now we are steering away from comms.jar to increase the reliability of the system. The assumption here is that Neil can code a daemon that doesn't crash left and right ;-) But the point is that even if it occasionally trips up, we'll just restart it without affecting the rest of the controller functionality or responsiveness.

If comms.jar has bugs (I don't know), isn't the right approach to write a new comms2.jar that doesn't have bugs. Device driver bugs can crash the whole OS, so the device driver ought to be small tight code that just connects the hardware to the app and not much more. In this case (comms.jar) an RS232 driver.

Above this layer is the packetisation, hex codes and asynchronous stuff that is different for every device. Given that this will have more bugs than the former, it should run in user space where it can exception rather than crash.

Custom drivers should only be needed when you have a custom hardware interface, for example a parallel port that drives relays or an embedded sub-micro controller where you really need to know the hardware.

Anything IP or RS232 should use heavily reused code that really works. Comm.jar or a derivative or replacement.

When I speak of drivers I'm speaking of code that interfaces to the operating system. This can't be avoided if the device is connected to the box then a device driver is required. Some USB devices are HID but even these devices depend on the USB device driver. I'm not a Java coder, I learned a little in school but I've quickly learned that a lot of what folks say here is well beyond what I know.

The comm.jar doesn't handle anything more than the parallel port and the serial port. A daemon can hide the fact that you're talking a a fancy dll, device driver or peeking/poking (sorry couldn't resist) the bytes into place.

Robin H wrote on Jul 28, 2008 23:30:
If comms.jar has bugs (I don't know), isn't the right approach to write a new comms2.jar that doesn't have bugs. Device driver bugs can crash the whole OS, so the device driver ought to be small tight code that just connects the hardware to the app and not much more. In this case (comms.jar) an RS232 driver.

comm.jar doesn't have any issues that I'm aware of (I wrote the how-to afterall!)... if you all want to steer clear of it, that's fine, but it doesn't have any issues that I'm aware of... all of my stuff runs fine without issues using Trent Jarvi's version (as referenced by my how-to: http://www.agaveblue.org/howtos/Comm_How-To.shtml).

However, I still highly recommend partitioning anything that talks to hardware from the main JVM that way it is easier to re-start individual things as needed as opposed to having to restart the entire JVM...

As I stated early on, we can always install two JVM's, one full-featured JVM for the main JBOSS stuff, and then one scaled-way-back jvm for dealing with hardware interactions (all we need is some basic stuff with networking... an old java 1.1 or scaled back 1.2 vm would work fine for that stuff).

I know that this may sound hackish, but it may be worth doing.

Robin, don't know how I missed this thread. Regarding your Omni pro Java code, I just started writing the same thing. My idea was to pair it with Apache Axis and have a soap/rest/json interface to the omni (how cool to have a web services enable home!), I'm tired of trying to port it to this language or that language, having a web service enabled server would be heaven. I was planning on releasing this as a GPL'd project. Is your source code available for such a project? I have a C implementation I wrote for my iPhone app H@me which I hope to release as open source one day (its rather customized for my app), I also have a ruby proof of concept I'm currently writing for inclusion into the LinuxMCE project so I have some experience with both the original omnilink protocol (serial,udp up to 2.16) and the new omnilink II spec (2.16 +, uses tcp and event driven notification). I would be willing to do the work to UDP/TCP enable your stack as well as port it to the new omni spec.

It would save me alot of time to start with your code, I'm spread thin as it is between a couple HA projects.

Thanks! Dan-

Robin and Dan,

We would welcome such a project. I hear everyone is spread thin but starting a project and getting it some visibility is not something that will cost you a lot more compared to what you are doing already. Open Source can help as long as someone is the driver.