Juha Lindfors wrote on Jul 03, 2008 11:33:
Frankly, for device drivers you might be better off with native C. Then just provide an API for Java services to use it through socket/file/other io

Actually I was thinking about having this handled by the Java server software but IO daemon works too. Alright a daemon it is then, it will be written in C. So how many C coders do we have? Also do you have any ideas for primitives to talk to the daemon? Example: A1 On, or maybe 02.01.53 dim 50%. I won't be keeping a database in the daemon (i.e.: A1 is the kitchen light) and I won't be keeping states of the device though I could poll some devices. What do you think?

MarcF, what was the outcome of talking to Dan Lanciani and his daemon code. Can we use it or do I need to start writing a new daemon, which might not be a bad idea, threading this time.

Neil Cherry wrote on Jul 03, 2008 15:32:
Actually I was thinking about having this handled by the Java server software but IO daemon works too.

I have trust issues.

I don't trust anything in the server to talk to hardware directly ;-)

However, if writing a new daemon is a significant effort for you, I can reconsider.

Neil Cherry wrote on Jul 03, 2008 15:32:
Alright a daemon it is then, it will be written in C. So how many C coders do we have?

You :-) I'm too lazy to go back to C.

Neil Cherry wrote on Jul 03, 2008 15:32:
Also do you have any ideas for primitives to talk to the daemon? Example: A1 On, or maybe 02.01.53 dim 50%.

I'm all ok as long as I can write simple strings to it.

Neil Cherry wrote on Jul 03, 2008 15:32:
I won't be keeping a database in the daemon (i.e.: A1 is the kitchen light) and I won't be keeping states of the device though I could poll some devices. What do you think?

Yes, state and mappings belong to the server (compiled by the user on the website - A1 is my kitchen light) and then downloaded to server (still assuming no offline mode yet).

State of A1 as well on the server. A1 is ON, UI will reflect that state, physical state change (light switch manually off) we can update the state on the server with polling.

Polling I suppose you can also leave for the server to initiate? Unless there's a benefit in having the driver do constant polling?

Neil Cherry wrote on Jul 03, 2008 15:32:

Juha Lindfors wrote on Jul 03, 2008 11:33:
Frankly, for device drivers you might be better off with native C. Then just provide an API for Java services to use it through socket/file/other io

Actually I was thinking about having this handled by the Java server software but IO daemon works too. Alright a daemon it is then, it will be written in C. So how many C coders do we have? Also do you have any ideas for primitives to talk to the daemon? Example: A1 On, or maybe 02.01.53 dim 50%. I won't be keeping a database in the daemon (i.e.: A1 is the kitchen light) and I won't be keeping states of the device though I could poll some devices. What do you think? MarcF, what was the outcome of talking to Dan Lanciani and his daemon code. Can we use it or do I need to start writing a new daemon, which might not be a bad idea, threading this time.

MarkS is a great C coder as well :)

I haven't talked to Dan here. He didn't really respond to my overture. if it is a bunch of work, maybe we can license the original work. He said he was open to CC re-licensing which may be good for our short term. Long term having a bunch of people hold copyright is not a problem, where it gets messy is on the re-licensing...

Ben Drawbaugh wrote on Jul 03, 2008 14:29:
From a usability standpoint, I'd just like to point out (again) that however it is designed that a user should be able to edit a config file (xml maybe) that maps strings (hex, ascii or a combination of the two) to commands and maybe even pre-defined buttons for the UI.

It seems like not everyone as seen this. :)

Ben Drawbaugh wrote on Jul 03, 2008 14:29:
...

To test I've configured the appropriate settings (usually 8-n-1) and connected with a cable and copied and pasted ASCII commands into the terminal screen.

It would be line oriented, basically what you do is type in an ASCII string: A1 on or 02.03.45 dim 50% or something similar and the daemon would send the command to the device.

What I'm trying to say is that I'd be very surprised if the built in API for an OS wouldn't already have most of the functionality required.

I'm not aware of any OS with this kind of API. This is something I've written in my daemons (based on Dan Lanciani code).

I think Juha might be on to something though, all that may be required is for a Java API to be written so you can access the OS level device (assuming one doesn't exist already). From a usability standpoint, I'd just like to point out (again) that however it is designed that a user should be able to edit a config file (xml maybe) that maps strings (hex, ascii or a combination of the two) to commands and maybe even pre-defined buttons for the UI.

Users do not speak to the daemons. The server speaks to the daemons. That way only standard commands can be issued. The standard is something we need to work on. I like to call these primitives, these primitive would be also used for status of the devices.

• all commands are strings, terminated with a LF and CR (Internet standard) this could be called a sentence
• device cmd where cmd is a string of one or more verbs and adverbs
• On/Off would be for simple things like lights and bit manipulation
• numbers would be ascii strings so you could send 50 500 or 5000000000
• Strings would be for extreme examples and I'd probably like to avoid those.

Hmm, this sounds like I need a daemon tool kit. I guess I'll rewrite the daemon to not use Dan's code.

Thoughts, comments? I will need help coming up with some kind of standard language to have a conversation. I also think the Java code will need a way of asking what the capabilities (what makes up the language) of the daemon.

Neil Cherry wrote on Jul 03, 2008 16:01:
Thoughts, comments? I will need help coming up with some kind of standard language to have a conversation. I also think the Java code will need a way of asking what the capabilities (what makes up the language) of the daemon.

I think you might be pushing too much logic into the driver.

The language is needed only between the UI and server to translate a command like ON/OFF to whatever bits needs to be written to serial.

What I was thinking about is basically Java Communications API but behind an IPC. So we can partition the java services from the native layers without having to run separate JVMs.

so basically just this to write:

outputStream = serialPort.getOutputStream();
serialPort.setSerialPortParams(9600, 
				SerialPort.DATABITS_8, 
				SerialPort.STOPBITS_1, 
				SerialPort.PARITY_NONE);
outputStream.write(messageString.getBytes());

Check out the Java Comms API, that's probably it, if it looks like too much work, we can just use it direct (but I still have trust issues with allowing native code to crash the VM)

Juha Lindfors wrote on Jul 03, 2008 23:52:

Neil Cherry wrote on Jul 03, 2008 16:01:
Thoughts, comments? I will need help coming up with some kind of standard language to have a conversation. I also think the Java code will need a way of asking what the capabilities (what makes up the language) of the daemon.

I think you might be pushing too much logic into the driver. The language is needed only between the UI and server to translate a command like ON/OFF to whatever bits needs to be written to serial. What I was thinking about is basically Java Communications API but behind an IPC. So we can partition the java services from the native layers without having to run separate JVMs. so basically just this to write:

outputStream = serialPort.getOutputStream();
serialPort.setSerialPortParams(9600, 
				SerialPort.DATABITS_8, 
				SerialPort.STOPBITS_1, 
				SerialPort.PARITY_NONE);
outputStream.write(messageString.getBytes());

Check out the Java Comms API, that's probably it, if it looks like too much work, we can just use it direct (but I still have trust issues with allowing native code to crash the VM)

Okay, I'm confused, when I was talking about a 'device driver' what I meant was software that talks to the hardware and translates and handles the protocols. The CM11A is relatively easy to deal with, the CM15A a little more complicated, the Insteon controllers a lot more complicated. Here's an example:

• Get INSTEON Engine Version
• ping
• ID Request
• Light On
• light Off
• Lamp status request
• Light On ramp rate
• Sprinkler valve on
• Sprinkler valve off
• Sprinkler Program on
• Sprinkler Program off
• Pool device on
• Pool device off
• Window covering (on, off, stop, same command different parameters)
• Thermostat up
• Thermostat down
• ... (there are hundreds more)

A lot of there seem stupid but they are useful also the format of the command depends on the device type (sorry this section is 35 pages long). The responses are dependent also on the device. According to the document (Dev Guide) there is supposed to be a response to manual changes (turn the lamp switch on or press the module button). Then there is the database linking. You can't control a device unless it's registered with the controllers and modules. These are the kinds of things that needs to be hidden in the daemon but some agreement between the server and daemon is needed (how does the server learn about a new module that the daemon has registered)

Okay so the daemon is then to receive bytes and transmit them to the port (it may not just be the serial port but USB also).

So am I receiving bytes not lines (lines are terminated by <CR><LF>)?

Juha, I do have one question. Why would the writeBytes hang on talking directly to the serial port and not to the daemon?

MarcF, I'd say that this pretty much says no to the iPhone talking directly to the PODs as the heavy logic is in the server. I doubt you want to replicate that logic in the iPhone.

Neil Cherry wrote on Jul 04, 2008 16:38:
Okay so the daemon is then to receive bytes and transmit them to the port (it may not just be the serial port but USB also).

Yes, or anything else we might come up with down the road.

Neil Cherry wrote on Jul 04, 2008 16:38:
So am I receiving bytes not lines (lines are terminated by <CR><LF>)?

Yep :-)

Neil Cherry wrote on Jul 04, 2008 16:38:
Juha, I do have one question. Why would the writeBytes hang on talking directly to the serial port and not to the daemon?

What I want to avoid is a bad implementation crashing the JVM, like this:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0x4c94986c, pid=2673, tid=2979568528
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0_12-b04 mixed mode, sharing)
# Problematic frame:
# C [libc.so.6+0x6886c]
#

There's no difference in this regard with libc crashing (what Java Comms uses via JNI to talk to serial) or your daemon crashing. Bad code is bad code.

The difference is I am separating your daemon code to another process space, your daemon crashes, it won't bring down the server with it. If we use JNI direct from Java to serial then you can see what might happen above, ideally it wouldn't but there's no such thing as perfect code.

Juha, when you're doing the socket programming are you going to use ip:port (I don't remember the Java format)? Using this makes it easy to put the daemon code on the POD, though I could listen on a local socket and IP port. Also since the daemon code is much lighter in weight now, the POD can be 'smaller' (less MHz and ram/rom). One last advantage which is an odd one; I can setup a daemon and the person doing the server programming can use my devices here at home until they get their own device (we'll work out connectivity details). The last part is not a big deal but a cool side effect. I'll have 2 NSLU2s (RS232 for now, USB drivers a little later) setup as PODs here for whatever we need them for.

Hmm, this simple daemon turns out to be a good thing. Sorry I was so obstinate about this subject (won't be the last I'm sure).

I'm writing the daemon code right now and I'll let everyone know when it's done. I'm using the userver code (GPL2, so we have no license issues). I don't think MarcF will have a problem using GPL code here as it is isolated from the rest of the code (no cross license issues) and won't be a problem as it's a tiny part of the over all code.

Neil Cherry wrote on Jul 04, 2008 18:43:
Juha, when you're doing the socket programming are you going to use ip:port

Yes.

One last advantage which is an odd one; I can setup a daemon and the person doing the server programming can use my devices here at home until they get their own device (we'll work out connectivity details).

Cool, remote test lab should be useful.

Neil Cherry wrote on Jul 04, 2008 18:43:
Hmm, this simple daemon turns out to be a good thing. Sorry I was so obstinate about this subject (won't be the last I'm sure).

In the history of technical arguments on open source mailing lists you gave in easily, in a day. Had this gone on for two weeks, unresolved and then resurface every two months or so, now that gets frustrating ;-)

Juha Lindfors wrote on Jul 04, 2008 19:03:

Neil Cherry wrote on Jul 04, 2008 18:43:
Juha, when you're doing the socket programming are you going to use ip:port

Yes.

One last advantage which is an odd one; I can setup a daemon and the person doing the server programming can use my devices here at home until they get their own device (we'll work out connectivity details).

Cool, remote test lab should be useful.

Let's face it I've got enough of this junk, might as well make use of it. :-)

Neil Cherry wrote on Jul 04, 2008 18:43:
Hmm, this simple daemon turns out to be a good thing. Sorry I was so obstinate about this subject (won't be the last I'm sure).

In the history of technical arguments on open source mailing lists you gave in easily, in a day. Had this gone on for two weeks, unresolved and then resurface every two months or so, now that gets frustrating ;-)

Technically it was two days, two days is long enough. This is internet time, but two weeks is intolerable. I'm not very patient (I want answers and I want them now. Where's that search engine?

g answer to everything

There were two things I could do, first was to continue to argue and eventually go off and create my own software (hard to do and I've made no progress with that so far) and the other was to listen to the arguments, follow along and see what progress we make. At some point a decision needs to be made to move forward. Even a bad decision (I'm not saying this was) is better than no decision. I need code and I need it now!

The reason I 'gave' in was that I'm trying to be flexible. I tried it my way in the past (fancy daemon) and got nowhere. Time to try something else. I can't think of everything but as a team we can. I've written lots of software but it's only pieces. I'm tired of pieces I want a complete extensible system. This way seems more likely to succeed.

Neil Cherry wrote on Jul 04, 2008 18:43:
Juha, when you're doing the socket programming are you going to use ip:port (I don't remember the Java format)? Using this makes it easy to put the daemon code on the POD, though I could listen on a local socket and IP port. Also since the daemon code is much lighter in weight now, the POD can be 'smaller' (less MHz and ram/rom). One last advantage which is an odd one; I can setup a daemon and the person doing the server programming can use my devices here at home until they get their own device (we'll work out connectivity details). The last part is not a big deal but a cool side effect. I'll have 2 NSLU2s (RS232 for now, USB drivers a little later) setup as PODs here for whatever we need them for. Hmm, this simple daemon turns out to be a good thing. Sorry I was so obstinate about this subject (won't be the last I'm sure).

This thread is getting a little long in the tooth but I am glad you are liking the simplicity! There is nothing in it and it crashes less! How about that! I know juha had you at less work for you. Java is good at mapping sets, so let us juggle the symbolic crap, it is easy in java. Just write bytes to a port we know nothing about!

I'm writing the daemon code right now and I'll let everyone know when it's done. I'm using the userver code (GPL2, so we have no license issues). I don't think MarcF will have a problem using GPL code here as it is isolated from the rest of the code (no cross license issues) and won't be a problem as it's a tiny part of the over all code.

OK.

Neil Cherry wrote on Jul 03, 2008 16:01:
It would be line oriented, basically what you do is type in an ASCII string: A1 on or 02.03.45 dim 50% or something similar and the daemon would send the command to the device.

But this would mean that we'd need to push down the mappings of A1 -> command into your daemon.

That's not what we need the daemon for. We already have the server for it.

Daemon exists just to separate the server and native layers into different process spaces.

Ben is correct, this should really be just pushing the commands through from the server to serial.

Juha Lindfors wrote on Jul 03, 2008 23:59:

Neil Cherry wrote on Jul 03, 2008 16:01:
It would be line oriented, basically what you do is type in an ASCII string: A1 on or 02.03.45 dim 50% or something similar and the daemon would send the command to the device.

But this would mean that we'd need to push down the mappings of A1 -> command into your daemon. That's not what we need the daemon for. We already have the server for it. Daemon exists just to separate the server and native layers into different process spaces.

Right, he should just be writing bytes as is without caring for mappings that is why we are writeByte() to it. Let the mappings go in the server.

All juha is really saying is that he doesn't want writeByte() to hang on him or crash on him. Also writing bytes should be an easier C implementation than dealing with mappings. Let java do what java does: mappings.

Asynchronous calls, fire and callbacks and all interesting semantics may apply to different hardware devices.

marc fleury wrote on Jul 04, 2008 04:40:

Juha Lindfors wrote on Jul 03, 2008 23:59:

Neil Cherry wrote on Jul 03, 2008 16:01:
It would be line oriented, basically what you do is type in an ASCII string: A1 on or 02.03.45 dim 50% or something similar and the daemon would send the command to the device.

But this would mean that we'd need to push down the mappings of A1 -> command into your daemon. That's not what we need the daemon for. We already have the server for it. Daemon exists just to separate the server and native layers into different process spaces.

Right, he should just be writing bytes as is without caring for mappings that is why we are writeByte() to it. Let the mappings go in the server. All juha is really saying is that he doesn't want writeByte() to hang on him or crash on him. Also writing bytes should be an easier C implementation than dealing with mappings. Let java do what java does: mappings. Asynchronous calls, fire and callbacks and all interesting semantics may apply to different hardware devices.

Hmm, I understand what you are saying but I view the daemon as a layer of the OSI model, it should hide the interface details (as well as keeping things from hanging). You need to know the module details not the interface details. Unfortunately that includes translating string A1 to 66 then send the CM15A the bytes 04 66 and handle the ACK (0x55) then give the server some acknowledgment and so on. The Insteon, Z-Wave and UPB protocols are much more complicated.

I'm not saying it can not be done, you can move the interface layer back to the server. As my friend likes to say, you can move complexity but you can not remove it.

Christian Bauer wrote on Jul 03, 2008 15:45:

Ben Drawbaugh wrote on Jul 03, 2008 14:29:
From a usability standpoint, I'd just like to point out (again) that however it is designed that a user should be able to edit a config file (xml maybe) that maps strings (hex, ascii or a combination of the two) to commands and maybe even pre-defined buttons for the UI.

It seems like not everyone as seen this. :)

A User editing an XML file? No way, that has to be hidden by the Java server/UI.

Christian, I'm not sure how that's going to apply to the daemon. Do we start another daemon? Does the daemon dynamically add the device and it manages all the devices? I'm not sure how to approach this yet. For now let me write a daemon to handle just one device and we'll go from there.

Neil Cherry wrote on Jul 03, 2008 16:06:
A User editing an XML file? No way, that has to be hidden by the Java server/UI.

Of course, that's the RC Builder we keep talking about. There isn't any page yet describing it because it only now becomes clear what it exactly needs to do: Spit out an XML file like this.

Christian, I'm not sure how that's going to apply to the daemon.

Uh, I don't think it is related at all? I was replying to Ben who was saying we need a UI that maps buttons to some commands that are then passed around and transformed until they end up on the destination device - through whatever protocols and layers and transports we create.

Christian Bauer wrote on Jul 03, 2008 16:45:

Neil Cherry wrote on Jul 03, 2008 16:06:
A User editing an XML file? No way, that has to be hidden by the Java server/UI.

Of course, that's the RC Builder we keep talking about. There isn't any page yet describing it because it only now becomes clear what it exactly needs to do: Spit out an XML file like this.

Christian, I'm not sure how that's going to apply to the daemon.

Uh, I don't think it is related at all? I was replying to Ben who was saying we need a UI that maps buttons to some commands that are then passed around and transformed until they end up on the destination device - through whatever protocols and layers and transports we create.

Sorry I was thinking how a user goes about adding a new device to the system. There would have to be some kind of UI to add the device but the inner working become 'interesting'. That's the part where I wasn't sure how it would be applied to the daemon (DeviceManager?).

This is why I try to stay away from the UI. I haven't a clue how to do it. Now the device interface that I can handle.

Neil Cherry wrote on Jul 03, 2008 16:06:
A User editing an XML file? No way, that has to be hidden by the Java server/UI.

No, not a user per se. But an integrator or smart DIY'er who wants to use the product but doesn't want to wait around for someone else to add support for their device.

I have done this for every RS-232 device I have with the current HA software I'm using. There are a few reasons for this.

1. It is nearly impossible to add support for every device known. 2. Testing a new device without owning the device can be impossible and who wants to buy the latest $6k HDTV just to test to see if you got the protocols right.

If everyone shares their XML config files then they will be there to use and to include with the software.

Ben Drawbaugh wrote on Jul 03, 2008 17:36:
No, not a user per se. But an integrator or smart DIY'er who wants to use the product but doesn't want to wait around for someone else to add support for their device. I have done this for every RS-232 device I have with the current HA software I'm using. There are a few reasons for this. 1. It is nearly impossible to add support for every device known. 2. Testing a new device without owning the device can be impossible and who wants to buy the latest $6k HDTV just to test to see if you got the protocols right. If everyone shares their XML config files then they will be there to use and to include with the software.

yep, you got it ben. This needs to be done once by someone who owns the device, after that it will go into our database and the rest of the users can choose it from the pretty web UI, never needing to see XML unless they want to fix it or learn from it.

Neil Cherry wrote on Jul 03, 2008 16:06:

Christian Bauer wrote on Jul 03, 2008 15:45:

Ben Drawbaugh wrote on Jul 03, 2008 14:29:
From a usability standpoint, I'd just like to point out (again) that however it is designed that a user should be able to edit a config file (xml maybe) that maps strings (hex, ascii or a combination of the two) to commands and maybe even pre-defined buttons for the UI.

It seems like not everyone as seen this. :)

A User editing an XML file? No way, that has to be hidden by the Java server/UI. Christian, I'm not sure how that's going to apply to the daemon. Do we start another daemon? Does the daemon dynamically add the device and it manages all the devices? I'm not sure how to approach this yet. For now let me write a daemon to handle just one device and we'll go from there.

he he Neil has discovered java and already the idea of a user interacting with an XML file gives him the creeps. Good, good!

For sure this is where a good internet app can help.

Christian Bauer wrote on Jul 03, 2008 15:45:
It seems like not everyone as seen this. :)

I don't understand what that link has to do with this discussion.

Ben Drawbaugh wrote on Jul 03, 2008 14:29:
From a usability standpoint, I'd just like to point out (again) that however it is designed that a user should be able to edit a config file (xml maybe) that maps strings (hex, ascii or a combination of the two) to commands and maybe even pre-defined buttons for the UI.

Right, which is why I don't want the daemon to be doing the mapping cause then we need to be pushing this stuff all the way down to native layers which seems unnecessary to me.