Hi.

Sorry this is probably a better link. They have had 3 code releases this year, the last was about 2 months ago. The server has X10 support and a model for adding additional drivers.

Paul

Hello Paul,

Thanks for the pointer to Obix.

I took a very quick glance at the material. It looks like Obix is attempting to target a higher level integration than what we are currently working on. Right now we are investigating which control protocols to integrate with, KNX, INSTEON, X10, etc.

Obix is addressing how to integrate building control with enterprise applications. So it defines a HTTP interface for this.

An example workflow could be something like this: an office manager books a conference space using their calendaring application, this schedules the room controls for proper HVAC for the event, perhaps it's off the regular office hours so also adjusts the security system accordingly. By exposing the building controls as a web service the workflow can integrate with the facilities system.

So in order for Obix to make sense for OR we need to have the control protocol integration first.

The good news is Obix defines a REST interface and our controller is already going to expose its various parts via HTTP. So that's a fit there if there's a demand. Given the HTTP access point however most enterprise apps will be able to integrate regardless of a specification. Once you're dealing with multiple vendor system though, the spec saves you the integration effort. So for integrator space it might eventually make sense. Don't know if that demand exists today, the demand might emerge sometime in the future though. Let me know if you think otherwise or if I misunderstood.

Paul Wilkinson wrote on Aug 07, 2008 03:57:
While the example you copied above is quite advanced a simpler example could be An X10 powerflash module sends a command which results in the ObixServer issuing a status change to a process that is watching the point represented by that X10 address. The process could then issue a command to an object which is located on a KNX system to change state (eg turn on a light).

What if the ObixServer is down? I still want my lights to work. This requires broadcasting of messages at the transport layer level, which is why xAP, xPL (and probably KNX somewhere) use their own application protocol on top of UDP. I'm not sure why/how I would build a mesh messaging network with TCP and HTTP. This is not a client/server problem.

TCP and HTTP are great for higher level control when you really have a client/server situation, e.g. from a touch panel to a controller, as Juha said. This is where xPL for example gets it wrong, with their own TCP + xPLHACFOOBAR protocol.

I think part of the problem in the XAP/XPL world is that it was designed from low-level protocols up rather than high-level functionality down.

I don't see why that would be bad, you have certain low-level problems to solve first.

Christian Bauer wrote on Aug 07, 2008 05:32:
What if the ObixServer is down? I still want my lights to work. This requires broadcasting of messages at the transport layer level, which is why xAP, xPL (and probably KNX somewhere) use their own application protocol on top of UDP. I'm not sure why/how I would build a mesh messaging network with TCP and HTTP. This is not a client/server problem. TCP and HTTP are great for higher level control when you really have a client/server situation, e.g. from a touch panel to a controller, as Juha said. This is where xPL for example gets it wrong, with their own TCP + xPLHACFOOBAR protocol.

I don't disagree, but you are going to have this problem if you use any software to enable light switching. Yes, embedded hardware can be more reliable; you can have no moving parts but software is still there.

In my home I have Clipsal C-Bus which uses a proprietary Cat-5 network to connect the switches to the relays and dimmers. As this system is used in thousands of domestic and commercial premises I am reasonable confident that it will keep working, but it is still possible that there is some obscure bug in their embedded code that will cause a failure.

The basic functionality of being able to walk in to a room and turn on a light using the switch on the wall shouldn't depend on any external software.

This is where we need to understand the role of OpenRemote. What I personally am looking for is a platform on which to build the automation smarts.

OpenRemote allows you to turn on a light from an iPhone, or a remote web browser or because an RSS feed notifies you that a total eclipse is happening right now and so on. OpenRemote turns off all of the lights because the security system was just armed. While you want this functionality to be as reliable as possible it shouldn't be the end of the world if the server/software dies.

To do this I want some abstraction from the underlying hardware/protocols.

I think part of the problem in the XAP/XPL world is that it was designed from low-level protocols up rather than high-level functionality down.

I don't see why that would be bad, you have certain low-level problems to solve first.

I don't see why you need to solve the low level problems. The low level problems have been solved. X10 (well, some would say X10 is a problem in itself ;) ) or UPB or KNX or C-Bus already solve these low level problems.

What the industry is not good at solving is the integration problem. Each vendor has their own protocols and there may be gateways between some of these (for example, my alarm panel can talk to both X10 and C-Bus but there is no gateway between X10 and C-Bus).

Consider OO software design - You start with an abstract model and then implement concrete code under the model. I think the same process should apply here. Build an abstract model and then plug protocols into it. An X10 library for Java already exists. Assume that a similar library can be written for KNX.

Once the abstract model is defined someone needs to build a wrapper that connects these to OpenRemote. Someone can connect C-Bus or UPB or a bunch of relays connected to their parallel port.

From a user interface point of view I want to achieve a task like turning on a light. I suck at UI design and graphics while others are going to be able to come up with the coolest visual stuff while not knowing anything about serial ports or soldering. I want the UI designers to be able to create UIs without having to worry about what the ultimate message is that will turn on the light. I want people to be able to use the UI whether they have X10 or KNX or UPB.

Paul

Click HELP for text formatting instructions. Then edit this text and check the preview.

Paul, just so we start talking about the same thing: How would you replace the messages and actors (apps, hubs, hal) in this scenario with something based on TCP and HTTP?

I actually think that we should adapt this picture to what we want to do with OR as soon as possible.

Christian Bauer wrote on Aug 07, 2008 07:20:
Paul, just so we start talking about the same thing: How would you replace the messages and actors (apps, hubs, hal) in this scenario with something based on TCP and HTTP?

Hi,

I don't really want to think about TCP and HTTP at this point. They are low level details. The way I would describe that scenario is:

1. Movement sensor detects motion and the data point in the obix server that represents this object is updated to reflect this event
2. A process that is watching this data point is notified of the change and issues a set command to modify the data point that represents the light
3. The interface between the Obix server receive and the lighting control network receives this command and issues an on command.

Step 2 could be achieved in a couple of ways. One is for some form of program or script to perform the action, the other concept I have is that you have some form of gui plugboard where you can connect data points together so you could simply connect the status of the movement sensor to the status of the light.

There will need to be a number of on-the-wire transmissions take place to make this happen and they could involve X10 over wireless, KNX on twisted pair and REST via HTTP. The point is that the definition of the schenario should be independent of these underlying communications and events.

Paul

Paul Wilkinson wrote on Aug 07, 2008 08:08:
I don't really want to think about TCP and HTTP at this point. They are low level details.

Now that's a convenient thing to do :)

Movement sensor detects motion and the data point in the obix server that represents this object is updated to reflect this event

How is the obix server updated? The event needs to get there somehow from the sensor and ideally there should be no targeted addressing in the message - the obix server should not be named as the destination of the message. We want transparent failover to a hot standby obix server if the first one goes down.

Step 2 could be achieved in a couple of ways. One is for some form of program or script to perform the action, the other concept I have is that you have some form of gui plugboard where you can connect data points together so you could simply connect the status of the movement sensor to the status of the light.

In the enterprise software world we use rules and workflow engines for that and we plan to do the same with OpenRemote. If you are familiar with that: Draw a statechart that contains your states (scenes or device states) and possible transitions (actions or triggers). Write rules for more complex transition decisions.

There will need to be a number of on-the-wire transmissions take place to make this happen and they could involve X10 over wireless, KNX on twisted pair and REST via HTTP. The point is that the definition of the schenario should be independent of these underlying communications and events.

Agreed but as I said, this is a problem that is easy to solve because we have done it before and we have a good idea how to approach that. (Some of us work at a company that builds rules and workflow engines, among many other enterprisey tools.)

Speaking probably not only for myself, what we want to figure out is:

- What is the ideal topology of a home network? Looks like everyone is doing mesh and broadcast and I can see why.

- What are the typical actors in such a network and what pieces of software do we need to provide? I like the xPL approach with Apps, Hubs, and HALs - although I would pick different names.

- What messaging protocols exist (X10, KNX) and what wrapper protocols (xPL again, Orbix) exist and which would should we work with?

The Obix standard defines the addressing. Each item is named via a URL hierarchy. Each item in the URL hierarchy is an XML encoded object that has one or more properties.

What is the ideal topology of a home network? Looks like everyone is doing mesh and broadcast and I can see why.

Agreed. Do we have a broadcasting system where a mesh of systems receive messages and act upon them (This is the XPL model) or a system where there is effectively a central co-ordinator - This is the Obix server model.

I don't really want to think about TCP and HTTP at this point. They are low level details.

Now that's a convenient thing to do :)

I don't mean to understate the important of dealing with the bits on the wire. This is very important and I am sure for some people pulling the protocol apart is exactly what they want to do, but I think it is important that we define a target architecture and object model for these people to code to. If we design bottom up then we risk designing elements of a particular protocol into the solution.

Paul

Paul Wilkinson wrote on Aug 07, 2008 11:33:
Consider OO software design - You start with an abstract model and then implement concrete code under the model. I think the same process should apply here. Build an abstract model and then plug protocols into it. An X10 library for Java already exists. Assume that a similar library can be written for KNX.

... but I think it is important that we define a target architecture and object model for these people to code to. If we design bottom up then we risk designing elements of a particular protocol into the solution. Paul

Paul, I agree with this philosophy whole-heartedly. While I'm not 100% sure about Obix, I'm already working on a 50,000 foot view Object Model for the Controller portion (bridge between O.R. and Controller protocols/devices). I'm hoping to publish something on the Controllers page later this week, or sometime this weekend. In no way will what I publish be final... it will be subject to change, but it will be good to get feedback on it, and figure out if we are abstracting things at the right level.

I don't think we're ready to provide anything concrete yet that we want people to begin programming to, but we definitely need to start thinking in this way and put it out there and let the dogs chew it apart.

Looking forward to seeing your object model. It seems that one area we need to make progress on is the integration tool with the programming model. If we can do it for IR and X10 alone for the first version I would be reasonably happy. We can always pick an abstract schema for the generalization. A/V (LIRC) and lights (X10) are what we want for Version 0, that and the basic tools to DO something with it. (assembly the UI, when I press a button, I send the command to IR).

Marc Fleury wrote on Aug 07, 2008 16:48:
Looking forward to seeing your object model.

I'm looking forward to the feedback :)

It seems that one area we need to make progress on is the integration tool with the programming model.

What do you mean by 'integration tool with the programming model'? I don't follow. (please excuse my ignorance)

If we can do it for IR and X10 alone for the first version I would be reasonably happy. We can always pick an abstract schema for the generalization. A/V (LIRC) and lights (X10) are what we want for Version 0, that and the basic tools to DO something with it. (assembly the UI, when I press a button, I send the command to IR).

This doesn't seem too far-fetched to me... especially on the X10-side (if I can just resolve my X10 issues, we'll be golden).

Paul,

Paul Wilkinson wrote on Aug 07, 2008 06:55:
I don't see why you need to solve the low level problems. The low level problems have been solved. X10 (well, some would say X10 is a problem in itself ;) ) or UPB or KNX or C-Bus already solve these low level problems.

We are not trying to 'solve' control protocols in a sense that we would want to create a new one. At least not yet. What we currently want to do is to support these various control protocols.

I think there's a couple of different ways we can do that. Note that I'm not at all sure which is the 'right' approach so I welcome the discussion there.

1) We provide integration by way of 'tunnelling'. This allows us to use a single protocol 'packet' as the common format which we can use to wrap messages with in all the various control protocols across a mesh network of controllers and 'pods' (or hubs or however you want to call them). Think of HTTP tunnelling here, same idea. Currently looking at if KNX telegram could be that universal 'packet'.

So marshalling occurs at the originating controller/hub/pod (A) and unmarshalling at the receiving controller/hub/pod (B). We get from A to B over powerline, radio or ethernet as long as the generic packet protocol defines these datalink/physical protocols. We don't need to care about the semantics of the 'real' payload message during transit. We may need to translate the protocol at the receiving controller/hub/pod.

2) Integration by way of protocol translation. From your description this is how I understood Obix approaching the problem, by way of translating protocols via a HTTP/XML/REST interface. This could however also be a direct translation, essentially a bridge between networks (e.g. KNX-NXB or KNX-BACnet).

Paul Wilkinson wrote on Aug 07, 2008 06:55:
What the industry is not good at solving is the integration problem.

I am glad you said this ;-) This is what we've been hearing from the community so far and you are enforcing that message. This is the problem we want to be solving.

Paul Wilkinson wrote on Aug 07, 2008 06:55:
there may be gateways between some of these (for example, my alarm panel can talk to both X10 and C-Bus but there is no gateway between X10 and C-Bus).

Yes that's the issue with protocol translation is that you do need the right combinations.

Paul Wilkinson wrote on Aug 07, 2008 06:55:
Consider OO software design - You start with an abstract model and then implement concrete code under the model. I think the same process should apply here. Build an abstract model and then plug protocols into it. An X10 library for Java already exists. Assume that a similar library can be written for KNX. Once the abstract model is defined someone needs to build a wrapper that connects these to OpenRemote. Someone can connect C-Bus or UPB or a bunch of relays connected to their parallel port.

Personally I have a dislike for early abstraction. Software engineers love to 'model the universe', usually way before they even know how the world they're trying to model works.

I'd rather see an OR release that actually does something first, and only then worry about abstractions or interfaces. It helps to understand the thing that you are trying to generalize.

Juha Lindfors wrote on Aug 07, 2008 19:33:
Personally I have a dislike for early abstraction. Software engineers love to 'model the universe', usually way before they even know how the world they're trying to model works. I'd rather see an OR release that actually does something first, and only then worry about abstractions or interfaces.

Ye of little faith :P

As a Software Engineer, yes, admittedly, I do love to 'model the universe', however as what I consider to be a very good Software Engineer, I won't allow myself to be boxed into a corner by over designing either. You have to balance it with being flexible as well.

And while my experience in this field is limited mostly to the X10 subset, I'd like to believe that I have enough control over my mind to think more abstractly. After all, I do have some previous experience with this... you won't find any spring chicken at my KFC.

It helps to understand the thing that you are trying to generalize.

Agreed, but I'd like to think that I do have a pretty good grasp of what it is that I'm trying to generalize here. The trick is to generalize it at the right level of abstraction (which I believe to be the point you are trying to make), but that's why we need to have a starting point, and then be flexible to change/shift of that abstraction. I'm just afraid that if we don't do it, we're going to end up with an inordinate amount of re-work later, and even worse, the possibility a crappy product.

I've already conceded to not produce actual Java code... but a little diagram never hurt anyone...

Paul Wilkinson wrote on Aug 07, 2008 03:57:
I think that the answer to the question which protocols should OpenRemote integrate with? is All of them. I think the value is in providing a platform that makes it easy to implement protocols and then make use of their protocols. If we base design platform design decisions around low-level protocol choices we risk losing flexibility in the architecture.

Hello Paul,

I think everyone here agrees with your statement above. We are all about creating a platform that can support multiple control protocols. We are currently targeting several control protocols which should ensure that no one implementation leaks details towards higher levels. As for the architecture's flexibility, a lot of it is based on an existing middleware architecture which we know as flexible enough to be customized for many different use cases.