A BACnet Gateway using BAC0 (Part 3)

Until now, we discussed the hardware and what would be the orientation of the software. It’s now time to describe how the code is.

Apologies… This post is full of code extracts and weird bash commands

You’ve been warned :0P

As you probably already know, BAC0 is based on bacpypes. So coding with BAC0 is very similar in concept to what it would be using bacpypes, except that, there is a lot of work already done. Before going too far, I always use berryconda to run Python on a Pi. I find this distribution easy to install. You get Python3 with everything you need like Numpy or Pandas… no efforts.

When you initialize your BAC0 app

import BAC0 
new_device = BAC0.lite(
            ip=ip, deviceId=deviceId, localObjName="BAC0_Fireplace"
        )

every details about BACnet basis are taken care of. Just doing that, you could find your implementation on the network already. “App” could read or write to others devices, it has a device ID, a vendor ID, am object name. It respond to Who-Is requests by a I-Am. You would have to think about all those details if starting from scratch.

So, knowing that, what do we need to create for our app to work ? I see 4 parts :

  1. A serial driver that will take care of all communication with the fireplace
  2. A local variable definition file that will add objects to the “app”
  3. An application file that will serve as bridge between BACnet and serial
  4. A main file that will be used to initialize everything (and will be defined as a service) We’ll see this later.

Serial communication

I used pyserial to implement serial communication with the fireplace. It is the “de facto” module to work with serial ports in Python.As usual, I didn’t want to reinvent the wheel so I stick to the examples I found in the documentation and it served me well.

It even provide a little terminal tool to make simple tests :

python -m serial.tools.miniterm /dev/ttyS0 115200

The terminal picture from last post is showing the output of this terminal app.

# Really simple example to open a serial port with pyserial
def open_serial_port():     
    print('Opening COM port 1')     
    # open serial port
    ser = serial.Serial('/dev/ttyS0', 115200, timeout=1) 
    # check which port was really used        
    print(ser.name)              
    return ser  

The first version of my driver was working fine. It was based on a simple “on-demand” type of requests. I mean that I was issuing requests, and waited for a response. Each time.

# You write to the serial port
response = ser.write(b'GET LED')
# You get an answer
b'ON'

Problem came when I tried to integrate the touch screen interface to the driver. My driver was not constantly listening on the serial port for requests that would be issued by something else than the “app”. This was a big flaw.

I’ve been really inspired by this thread when defining my reading thread.

I also used some regex to process the responses I got.

class ReadLine:
    def __init__(self, s):
        self.buf = bytearray()
        self.s = s
        self.need_to_write = False
        self.can_write = False
        self.write_responses = deque()
        self.display_commands = deque()

    def readline(self):
        i = self.buf.find(b"\n")
        if i >= 0:
            r = self.buf[: i + 1]
            self.buf = self.buf[i + 1 :]
            self.process_data(r)
            self.buf = bytearray()
            return
        while True:
            i = max(1, min(2048, self.s.in_waiting))
            data = self.s.read(i)
            i = data.find(b"\n")
            if i >= 0:
                r = self.buf + data[: i + 1]
                self.buf[0:] = data[i + 1 :]
                self.process_data(r)
                self.buf = bytearray()
                return
            else:
                self.buf.extend(data)

    def process_data(self, response):
        if response == b"\n":
            return
        response_to_write = re.compile(r"(\r)([\w  :.]*)(\n)")
        display_write = re.compile(r"(HEY )([\w  :.]*)(\n)?")
        if response:
            data = response.decode()
            if response_to_write.search(data):
                data = response_to_write.search(data)[2]
                self.write_responses.append(data)

            elif display_write.search(data):
                data = display_write.search(data)[2]
                self.display_commands.append(data)

This have been put into a thread running readline all the time. The heart of the requests handling is

    def _request(self, req):
        self.ser.write(bytes("{}\r\n".format(req), "UTF-8"))  # write a string
        self._log.debug("{} sent to COM1".format(req))
        time.sleep(0.2)
        try:
            while True:
                try:
                    _result = self.rl.write_responses.pop()
                    result = self.decode_result(_result)
                    break
                except IndexError:
                    time.sleep(0.1)
        except EmptyResult:
            self._log.warning("No result to request {}".format(req))
            result = None
        except ErrorResponse:
            raise ErrorResponse("Error received for request {}".format(req))
        self._log.debug("Response : {}".format(result))
        if not result:
            raise CommunicationFailure("No response to request : {}".format(req))
        return result

When it is needed, we send something on the serial interface, then we wait for something appearing in the “deque” of write responses. We can then decode and process to the next thing.

Using this, I can also monitor ser.rl.display_commands deque and see what is going on with the remote display. Case closed.

Defining BACnet objects

I’ve already said BAC0 made things easy. This is the file named device.py that creates all the objects I needed to define the BACnet side of the object. An important thing is to declare the class with your vendor id (see lines 27-30)

import BAC0

from BAC0.core.utils.notes import note_and_log

from bacpypes.local.object import (
    AnalogOutputCmdObject,
    AnalogValueCmdObject,
    BinaryOutputCmdObject,
    BinaryValueCmdObject,
)
from bacpypes.object import AnalogInputObject, BinaryInputObject, register_object_type
from bacpypes.basetypes import EngineeringUnits
from bacpypes.primitivedata import CharacterString

import time

def start_device(ip, deviceId):
    try:
        new_device = BAC0.lite(
            ip=ip, deviceId=deviceId, localObjName="BAC0_Fireplace"
        )
    except Exception:
        new_device = BAC0.lite(deviceId=deviceId, localObjName="BAC0_Fireplace")
    time.sleep(1)

    # Register class to activate behaviours
    register_object_type(AnalogOutputCmdObject, vendor_id=842)
    register_object_type(AnalogValueCmdObject, vendor_id=842)
    register_object_type(BinaryOutputCmdObject, vendor_id=842)
    register_object_type(BinaryValueCmdObject, vendor_id=842)

    online = BinaryValueCmdObject(
        objectIdentifier=("binaryValue", 1),
        objectName="Fireplace Online",
        presentValue="inactive",
        description=CharacterString("Communication status on the serial interface"),
    )
    ledpulse_bv = BinaryValueCmdObject(
        objectIdentifier=("binaryValue", 2),
        objectName="LEDPULSE",
        presentValue="inactive",
        description=CharacterString(
            "Set Led to pulse between current color and color set by LEDCOLOR after turning on LEDPULSE on"
        ),
    )
    flameenable_bv = BinaryValueCmdObject(
        objectIdentifier=("binaryValue", 3),
        objectName="FLAME-EN",
        presentValue="inactive",
        description=CharacterString(
            "If set to false, will prevent FLAME to be turned ON"
        ),
    )
    flamealarm_bv = BinaryValueCmdObject(
        objectIdentifier=("binaryValue", 4),
        objectName="FLAME-ALARM",
        presentValue="inactive",
        minimumOnTime=300,
        description=CharacterString(
            "Will be On if there is a discrepancy between command and status of flame"
        ),
    )

    led_bo = BinaryOutputCmdObject(
        objectIdentifier=("binaryOutput", 1),
        objectName="LED",
        presentValue="inactive",
        description=CharacterString("Turns the LED on or off"),
    )
    flame_bo = BinaryOutputCmdObject(
        objectIdentifier=("binaryOutput", 2),
        objectName="FLAME",
        presentValue="inactive",
        description=CharacterString("Turns flame relay on or off"),
    )
    heatfan_bo = BinaryOutputCmdObject(
        objectIdentifier=("binaryOutput", 3),
        objectName="HEATFAN",
        presentValue="inactive",
        description=CharacterString(
            "Turns the heat exchanger blower on or off, if present"
        ),
    )
    lamp_bo = BinaryOutputCmdObject(
        objectIdentifier=("binaryOutput", 4),
        objectName="LAMP",
        presentValue="inactive",
        description=CharacterString("Lamp on or off (non LED)"),
    )
    auxburner_bo = BinaryOutputCmdObject(
        objectIdentifier=("binaryOutput", 5),
        objectName="AUXBURNER",
        presentValue="inactive",
        description=CharacterString("Auxiliary burner or valve, if present"),
    )

    ledfadetime_av = AnalogValueCmdObject(
        objectIdentifier=("analogValue", 1),
        objectName="LEDFADETIME",
        presentValue=0,
        units=EngineeringUnits("milliseconds"),
        description=CharacterString("Sets fade time between led colors (0-32767)"),
    )
    leddwelltime_av = AnalogValueCmdObject(
        objectIdentifier=("analogValue", 2),
        objectName="LEDDWELLTIME",
        presentValue=0,
        units=EngineeringUnits("milliseconds"),
        description=CharacterString(
            "Sets how long a color will remain before it transitions to the newer color"
        ),
    )
    ledcolorR_av = AnalogValueCmdObject(
        objectIdentifier=("analogValue", 3),
        objectName="LEDCOLOR_R",
        relinquishDefault=100,
        presentValue=100,
        description=CharacterString("RED value of LEDCOLOR (0-255)"),
    )
    ledcolorG_av = AnalogValueCmdObject(
        objectIdentifier=("analogValue", 4),
        objectName="LEDCOLOR_G",
        relinquishDefault=100,
        presentValue=100,
        description=CharacterString("GREEN value of LEDCOLOR (0-255)"),
    )

    ledcolorB_av = AnalogValueCmdObject(
        objectIdentifier=("analogValue", 5),
        objectName="LEDCOLOR_B",
        relinquishDefault=100,
        presentValue=100,
        description=CharacterString("BLUE value of LEDCOLOR (0-255)"),
    )
    ledcolorW_av = AnalogValueCmdObject(
        objectIdentifier=("analogValue", 6),
        objectName="LEDCOLOR_W",
        relinquishDefault=100,
        presentValue=100,
        description=CharacterString("WHITE value of LEDCOLOR (0-255)"),
    )
    humidity_ai = AnalogInputObject(
        objectIdentifier=("analogInput", 1),
        objectName="HUMIDITY",
        presentValue=0,
        units=EngineeringUnits("percentRelativeHumidity"),
        description=CharacterString("Reading of humidity sensor"),
    )
    temperature_ai = AnalogInputObject(
        objectIdentifier=("analogInput", 2),
        objectName="TEMPERATURE",
        presentValue=0,
        units=EngineeringUnits("degreesCelsius"),
        description=CharacterString("Reading of temperature sensor"),
    )
    dewpoint_ai = AnalogInputObject(
        objectIdentifier=("analogInput", 3),
        objectName="DEWPOINT",
        presentValue=0,
        units=EngineeringUnits("degreesCelsius"),
        description=CharacterString(
            "Calculated dewpoint based on temperature and humidity readings"
        ),
    )

    heatfanspeed_ao = AnalogOutputCmdObject(
        objectIdentifier=("analogOutput", 1),
        objectName="HEATFANSPEED",
        presentValue=0,
        description=CharacterString("Sets speed of heat exchanger"),
    )
    flamelevel_ao = AnalogOutputCmdObject(
        objectIdentifier=("analogOutput", 2),
        objectName="FLAMELEVEL",
        presentValue=0,
        description=CharacterString("Level of flame, if present (1-10)"),
    )
    lamplevel_ao = AnalogOutputCmdObject(
        objectIdentifier=("analogOutput", 3),
        objectName="LAMPLEVEL",
        presentValue=6,
        description=CharacterString("Level of lamp (1-10)"),
    )

    # BV
    new_device.this_application.add_object(online)
    new_device.this_application.add_object(ledpulse_bv)
    new_device.this_application.add_object(flameenable_bv)
    new_device.this_application.add_object(flamealarm_bv)

    # BO
    new_device.this_application.add_object(led_bo)
    new_device.this_application.add_object(flame_bo)
    new_device.this_application.add_object(heatfan_bo)
    new_device.this_application.add_object(lamp_bo)
    new_device.this_application.add_object(auxburner_bo)

    # AI
    new_device.this_application.add_object(humidity_ai)
    new_device.this_application.add_object(temperature_ai)
    new_device.this_application.add_object(dewpoint_ai)

    # AO
    new_device.this_application.add_object(heatfanspeed_ao)
    new_device.this_application.add_object(flamelevel_ao)
    new_device.this_application.add_object(lamplevel_ao)

    # AV
    new_device.this_application.add_object(ledfadetime_av)
    new_device.this_application.add_object(leddwelltime_av)
    new_device.this_application.add_object(ledcolorR_av)
    new_device.this_application.add_object(ledcolorG_av)
    new_device.this_application.add_object(ledcolorB_av)
    new_device.this_application.add_object(ledcolorW_av)

    return new_device

The application itself

What I call application here is the part of the code that glue the BACnet side and the serial driver. By the name of the functions, I think you will understand what is going on with this part of the code. I removed a lot of code to emphasize the flow of the application code. I kept parts that were directly tight to BAC0 itself and how I write to BACnet objects for example.

from BAC0.tasks.RecurringTask import RecurringTask
from BAC0.tasks.DoOnce import DoOnce
from BAC0.tasks.TaskManager import Manager
from BAC0.core.utils.notes import note_and_log

from bacpypes.primitivedata import Real, Null


import time

from beat import led_beat
from serial_driver import Fireplace
from device import start_device


@note_and_log
class App(object):
    def __init__(self, ip, deviceId):
        self.dev = start_device(ip=ip, deviceId=deviceId)
        self.fireplace = Fireplace()
        self.process_permission = True

    def update_online_status(self):
        online = self.dev.this_application.get_object_name("Fireplace Online")
        if self.fireplace.online:
            online.presentValue = "active"
        else:
            online.presentValue = "inactive"

    def ping_serial(self):
        """
        This function ask the serial driver to ping the device.
        Depending on the response, update the bacnet object Fireplace Online
        """
        self._log.debug(
            "Validating Online Status of Fireplace : {}".format(self.fireplace.online)
        )
        self.fireplace.driver.ping()
        self.update_online_status()

    def process(self):
        while self.process_permission:
            try:
                if self.fireplace.online:
                    self.update_present_values()
                    self.make_display_write_to_priority_15()
                    self.process_write_requests_to_device()
                else:
                    self.ping_serial()
                    if not self.fireplace.online:
                        self._log.warning("Device offline... will pause for 5 seconds")
                        time.sleep(5)
                        self.ping_serial()
            except Exception as e:
                self._log.error(
                    "Error happend, trying to recover in 1 min.\n{}".format(e),
                    exc_info=True,
                )
                time.sleep(60)

    def update_present_values(self):
        """
        Read all variables from the serial interface and update bacnet presentValues 
        accordingly
        """

    def decode_ledcolor(self, result):
        # You get a list of R G B W... decode and return values
        return (r, g, b, w)

    def make_display_write_to_priority_15(self):
        """
        If something happen on the display, make that change appear on the priority 
        array of the corresponding BACnet object... and update the present value as it 
        may have changed.
        """
    def process_write_requests_to_device(self):
        """
        Read all variable from the serial interface and update bacnet presentValues 
        accordingly
        """
        # Write if necessary (if priority array != presentValue)

And at the end, a main file

The main file is calling everything, and dealing with exceptions… because I don’t want the app to stop… just try to reset.

def main():
    # BAC0.log_level("debug")
    app = App(ip="172.16.3.246/22", deviceId=3246)
    beat_task = DoOnce(led_beat)
    process = DoOnce(app.process)
    ping = RecurringTask(app.ping_serial, name="ping", delay=300)

    app._log.debug("Starting leds blinking")
    beat_task.start()
    app._log.debug("Starting processing values task")
    process.start()

    while True:
        time.sleep(10)
        if not process.is_alive():
            app._log.error("Probleme with process thread...trying to join")
            process.join()
            process = DoOnce(app.process)
            app._log.error("Process thread...restarting")
            process.start()

if __name__ == "__main__":
    main()

Let see what will happen from there. As usual, it will probably have some bugs to fix…

Running as a service ?

[ref : https://tecadmin.net/setup-autorun-python-script-using-systemd/ ]

To run the python script as a service (each time the Pi restart) I’ve used systemd and had to create one file and put it under

sudo nano /lib/systemd/system/fireplace.service
[Unit]
Description=BACnet Gateway
After=multi-user.target
[email protected]

[Service]
Type=Simple
ExecStart=/home/pi/berryconda3/bin/python /home/pi/dev/Fireplace/main.py
StandardInput=tty-force
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

Once it’s done, make systemd find the new file by reloading, enable the service to make it “enabled” on start…. and start the service.

sudo systemctl daemon-reload
sudo systemctl enable fireplace.service
sudo systemctl start fireplace.service

That’s it. Now the python script will run as soon as the Pi is alive. You can check the status of the service, like any other

sudo systemctl status fireplace.service

Now, what do you get ? I mean, does it work in Niagara ? We’ll see that in the next post.

A BACnet Gateway using BAC0 (Part 2)

The making of a BACnet device

Those of you working with BACnet devices, have you ever thought about how the stack interact with the physical part of the controller ?

When you write to a variable and you see the present value of the object changing, is this value representing what you want the output to be ? Or is this an image of what exactly is the output ?

How can you deal with communication problems between the BACnet stack and the physical device itself ?

All those questions suddenly came to me. All those concepts are taken for granted when you use a BACnet controller.

When you think about it, having a BACnet device reporting some read-only points is an easy thing. But building a device that deals with points that could be written to is different.

I mean, in the first case you update some values coming from somewhere (IO for example) and you update the present value of some BACnet objects. Problem solved.

But if you work with objects that you will write to. And/or objects that something else could write to. It brings a bunch of questions.

Implementation means taking decisions

Asking here and there, I concluded that there was not only one way to implement a device. I had to decide, make my own choices for the best.

For now, and I will be experimenting… I decided to :

  1. Constantly update the present value of the objects with the information read on the serial device.
  2. Evaluate the priority array of writable points (compared to the present value) to handle write requests.
  3. Play with the priority array to deal with the touch screen remote controller of the fireplace.
  4. Use the simple AT command to implement a serial “ping” function that will help me handle cases where the fireplace will not respond. This value being linked to a BACnet object informing the user on the status of the fireplace.

Let’s explain those 4 points.

The idea behind point #1 is that I consider the present value of the point being the actual physical state of a IO/point/thing on the device.

The serial device have a feature using a “get” request on the serial interface that returns the value of the variable. I will implement a function that will constantly read everything in the fireplace and will update the present value of the BACnet objects in the gateway accordingly.

Knowing this, it leads to point #2. BACnet objects that are commandable implements a priority array on which you can write from somewhere else. So the gateway will constantly look at every BACnet objects and look for the priority array and get the highest priority value and compare that value to the present value. If there is a difference, a write action will be needed on the serial interface to modify the present value. If both are the same, no need to do anything.

If there was only BACnet, things would be easy… But life isn’t easy. The fireplace comes with a Touch Screen remote command interface. It can be used to turn on the flame, the LEDs, change the colours of the LEDs… Fine. How will I tell the gateway that something else changed the state of the fireplace. I don’t want to fall into an infinite loop where the gateway fights the remote command.

The chance I have with this device is that, each time somebody touch the remote command, there is a serial message on the line. Something like “HEY SET LED ON”. This is nice. Because I will be able to store those “external” requests and process them.

At this point, I’m not yet convinced of the right way to do this. The choice I’ve made is that each time the gateway will see something coming from the remote display, it will write to the BACnet object at priority 15. Technically, the remote display will have changed the fireplace object. Next update of present values will see the new value. The highest priority value will be the write at priority 15 and everything will be ok.

Some caveats. If an override was present, the present value will be reverted to the override value. So there will be a glitch in the command. I also need to find a way to set the priority 15 value to null sometime to give back control to Niagara4.

I’m evaluating the option of writing at priority 16. Which would mean that Niagara4 and the remote display would write at same priority. Last one writing would win.

But the Niagara4 driver is built in a way where the Niagara objects priority inputs are not synchronized with BACnet priority arrays. This lead to inconsistencies where I could see a False on priority 16 in Niagara4… but a True have been written by the remote display. And yes, the present value of the point is true.

We will see what we get… and anyway, if it’s not ok… I will change the code.

.

A BACnet Gateway using BAC0 (part 1)

Recently, we have been asked if we were able to integrate a fireplace in our Niagara4 system.

A fireplace… well maybe. Why ? you will ask. Because, it’s using propane and the thing is installed on the top of a ski hill. Would be nice if we could shut it off by night to be sure there will be enough propane for a all winter.

Now can we buy a BACnet card for the fireplace ? Seems not. But there is a “domotic” option card ! Great ! You need to connect with a serial cable and use Putty…

What ? Like… using old AT commands ? Hum. Pretty far from what we are used to.

I won’t lie, I’ve always wondered if it would be possible to build my own BACnet gateway using BAC0. A few months ago, I played a little and came to some Weather connected device. I also went to the BACnet PlugFest and met a lot of people working on BACnet devices and I came to the conclusion that I could, too, build something real.

First challenge is to talk to the device. This is pretty simple. A serial interface, accepting AT commands… let see what can be done.

Choosing the platform

I really like Raspberry Pi. Those little cheap computers are well done. Price is low. They run a usable Linux version by default. Python runs smooth on them. And they provide GPIO pins which can be used for anything.

I already used them a lot with OpenVPN. Used with a LTE modem, they are allowing us to have remotely accessible devices on in-progress jobs. Where network isn’t ready.

So, as I’m used to RaspberryPi, I decided to use one to build the gateway. I knew that on the GPIO, I could access a UART port and make serial communication. After some researches, I understood that this UART was using 3.3V (CMOS) voltage levels. In the documentation of the fireplace, 5V (TTL) levels were requested. First challenge, make voltage level fit.

What is a level shifter ?

Fireplace documentation was referring to a level shifter “interface” to be used to connect the PC to the control card. So the 12V from the computer RS-232 port would be compatible with the 5V of the fireplace. Having never really worked with that kind of things, I made some researches and found that the MAX3232 could be used to convert different levels from TTL to CMOS.

My first attempt was bad though as I used only one chip which created a RS-232 signal on the output (which is 12V). First time I tried to connect to the fireplace, I’ve been flooded by zeros ! Back to the designing phase. I must have done something wrong.

Yeah, I forgot that RS-232 was 12V levels. Big fail. But what if I would use another MAX3232 and take this new 12V and bring this back to 5V ? This would lead to a pretty complicated level shifter… yes I know. But I’m having fun right now, let’s try.

Ok, I already hear some people out there saying : “Why haven’t you just used a simple voltage divider ?”

(In fact, my brother told me something just like that… I think it was finding my efforts… amusing.)

The answer is simple : because I didn’t know it would be sufficient ! I’m so used to buy some “cards” or “controllers” that already have all that is needed to communicate. My background in electronic is also really old. I was sure I needed something…more. It was my first attempt to build something from scratch. So, at the end, I gave myself the right to be “over-engineering”. And anyway, what is the most important thing ?

Will it work ?

This is the Super-too-complicated level shifter I created. I used Perma-Proto Hat cards to make it. It is connected on the top of the RaspberryPi. I’ve also tried to recycle some old pieces of hardware. Like the 2 little surface mount LEDs which are coming from and old dead Jace (FX60). Some of you will also recognize the serial connector took from a broken PCG from Johnson Controls. The two surface mount LEDs are connected to GPIO pins that will be used in the software to give some feedback on the application. The two red LEDs are connected to Rx and Tx because I wanted to see something when it’s communicating. Those have been stolen from an old Automated Logic board.

After having removed some plastic parts from the case, everything fits and give a nice looking device.

Getting responses

At this time of the project, I haven’t been able to communicate with the fireplace. Remember, it is installed on the top of a ski hill. Impossible to put my hand on the control card. Work is done on an act of faith, with the strong feeling than making serial communication should not be so hard. I have to climb up the hill when I’m ready to test.

So I’m here now, at the top of the hill, connecting my device to the fireplace for the second time (remember the first attempt was outputting 12V… it was not working) and bang !

Serial communication works !

I can send requests to the fireplace and it responds.

Real work could start.

To be continued…

New version for BAC0 / 19.4.28

Update your version !

I’ve just released a new version of BAC0 on Pypi. Thanks to many users who identified bugs and suggested improvements.

As I waited a little too long to publish to Pypi, there has been a lot of little improvements that were only in the develop branch for almost a year… and recently, I began to push more regularly to the master branch (which trigger new Pypi releases) because there is no reason to hold back fixes and new features.

Here is a quick list of the things you may like in the new version. (There may be “not so new” items here…but it’s been so long since an official release, let’s put them here anyway)

Calendar versioning

It seems like I would never ever be ready enough to present version 1. And in fact, it’s probably the same with all of software outside. They could always be better. For this reason and for you to know easily the release date of the version, I switched to calendar versioning.

I don’t plan to offer any guaranteed backward compatibility.  I want this library to evolve, I want to add features. I’ll do my best to “not break” everything… that is the best I can do. So… no more 0 versioning.

BAC0 Lite version

BAC0 can come with a lot of options which are sometimes too much. The lite version will load just the minimum and fits perfectly for little scripts that do not require the web interface. It was there in last version but it is worth repeating this.


import BAC0

bacnet = BAC0.lite()

Support for trendLogs

To be honest, working with JCI devices in the office and Niagara Jaces, makes a world without the immediate need to support trendLogs most of the time. But there is more and more devices out there providing them so it was more than time that BAC0 support theses. The implementation is yet to be improved but the biggest work has been done and we can easily recover trendLogs and output them with pandas format.

Super easy to do analytics then.

To support trendLogs, BAC0 now implement the Read Range function. If you need it.


import BAC0

bacnet = BAC0.lite()
dev_with_trendLogs = BAC0.device('2:5', 5, bacnet)

dev_with_trendLogs.trendlogs

# will output a dict with keys being

# objectType_address_property

Name_of_log, Av1_trendLog = dev_with_trendLogs['analogValue_1_presentValue']

Act as a Foreign Device

You can now use BAC0 as a foreign device on a network and cross subnets. Pretty useful when you are using VPN with TUN mode… or any other situation where you would need the feature. Just provide bbmdAddress and bbmdTTL.


import BAC0

bacnet = BAC0.lite(bbmdAddress='192.168.33.12',bbmdTTL=900)

A web interface

Based on Flask and using a bootstrap dashboard theme, the new Web Interface opens the door to a lot of future features.

BAC0 dashboard view
BAC0 dashboard view

The web interface contains the new Trend page with the live trending feature provided by Bokeh. The implementation has been improved to use callbacks and BAC0 will be ready for Bokeh version 1 coming soon. Some work need to be done to improve readability but having the trends live is pretty cool when testing your device !

Live trending
Bokeh server running in the Flask Dashboard of BAC0

I also included a simple table presenting the result of a global Who Is on the network.Device Table

Devices found on network

Logging

To keep track of bug and internal messages, I’ve configured a logging module with a specific text format that will allow a better reading of what’s happening in the app, what bugs occurred… but also, the possibility to add your own messages in your script which will result in your notes being “saved” in the log file.
Logging

Logging text format

Supporting new bacpypes versions

Bacpypes is also evolving fast and I try to keep in phase with bacpypes. BAC0 is nothing without bacpypes so better keep up pace ! Actual bacpypes version is 0.17.6. Stay tuned as there is some nice improvements coming.

Documentation improvement

I’ve added a few sections and tried to be more precise on other. Regarding this specific topic, don’t be shy ! If you want to help, improving documentation is really appreciated. BAC0’s documentation

Couple new built-in functions added

Dealing with overrides

I use BAC0 in my day-to-day work and sometimes, I add some features to ease my work. I also try to listen to users and some features come from their suggestions.

During startup of big BMS jobs, we often find ourselves dealing with a lot of overrides made by anyone in the team while doing there work… But what if somebody forgot to remove the overrides in one controller ?

PAUSE. In my world, overrides are made by writing @ priority 8 on points. In the documentation we can see that called Manual Operator writing… To know if a point is “overridden”… aka commanded @ priority 8, you need to read a special property on the point (priority array)… which require a specific reading as usually, BAC0 will read the present value of the point. And in real life, it can get really hard to tell if all 200 thermostats that have been installed are free of overrides.

BAC0 have a special function that allow the detection of overrides in a device… and another function that allow to release those overrides if needed. Just use :


device.find_overrides()

# or

device.release_all_overrides()

# or

device['point'].is_overridden

The two functions running on device (find_overrides and release_all_overrides) will start a thread to do all the work they have to and you can continue to work with BAC0 without being blocked. The logging function will inform you when everything is done.

Overriddes functions
Example of running the overrides functions

Finding points

If you don’t know the name of a point, but you know its object type and address, you can get it using :


device.find_point('analogValue',1)

Proprietary object support

In one big project, I worked with the TEC3000 thermostat from Johnson Controls. To be sure everything was ok with the communication, we thought using the “Supervisor Online” property which is a proprietary object in the device. To make it work, I had to include a few things in BAC0 and now, it’s possible for you, following what’s been done for this thermostat, to add new proprietary object to your script if you need.

If you work with those devices, you’ll see I also provide a basic custom object list to accelerate the loading of the devices. Have a look to the documentation.

New tests suite

It’s been a while I was looking for a way to test BAC0 and I could not find something I liked. Until I discovered (yeah, sometimes I’m slow learner)… that I could create 2 instances of BAC0 and make them talk together. This way I could tests a lot of functions easily. It’s not perfect, I know. Mostly because I’m actually testing using the socket for real… and it slows down the tests. But it allows me to make tests easily and for a lot of situation so I made the choice to be patient when the tests suite runs.

To do that, I also started to implement a feature I had not touched with BAC0, which is adding local object to the instance (like an analog value for example). This is a completely new field that open a lot of possibilities… and a few question on bacpypes side. But some users did ask for that and I think they will like the fact that it’s possible.

Improve SQLite saving support

SQLite support is there from a long time but it was not working every times. I put some effort to that and now it is super easy to save your device, and connect to a SQL backup after that.

import BAC0

bacnet = BAC0.lite()
offline_device = BAC0.device(from_backup="backup.db")

Over for now

I hope you’ll like this new version. Don’t hesitate to drop a comment on Gitter or in Github, I always appreciate your feedback.

You are doing your sideloops wrong

You work with Johnson Control PCT or CCT, I’m pretty sure you are doing your sideloops wrong !

Ok but what is the link to Pyhon ? Well… this post is a little off-topic but I thought I could bring that here anyway. And to be honest, I found out this while working with BAC0 to test my sequences of operation. I’ll release something soon related to that but let stick to sideloops for now.

For those who don’t work with Johnson Controls FEC or FX-PC products, you could think you won’t learn anything… I’m sure it could help with other products. And by the way, it still relates to building automation !

For outsiders to JCI stuff, you must know that all PIDs, by default, will be configured to use their auto-tuning features and building a “program” for a controller is easier when you use the selection tree which wil build for you the basic sequences you will need to modify. No more blank page. Some like it, some not. But let’s put aside our feelings for now.

Sideloops

When in a sequence, you need to add a PID loop of some kind, you can use what’s called “Sideloop” using a friendly button called… yeah “Sideloop”.

This button will ask some questions and you will end with a PID loop and everything else you need (like interlocks, override detection, entries in the state table (ok for those who don’t know the product, the state table is a kind of coordinator for your different blocks. It is really close to a Finite-State-Machine algorithm). This little button is kinda sharp and creates for you everything you need is much less time than if you would create everything by hand.

Everything ?

Nope. And this is where we (yes I include myself here) make a gigantic mistake.

Remember I told you about the auto-tuning feature of JCI devices ? This feature is really interesting. It works really well in “mostly” all situations. But ! If you ever leave a PID loop active for a long period of time when it should not be active (like a heating loop when it’s cooling time), you will end over-tuning the loop… Always. It will result in PID loop acting really bad. So bad that you will get call back, you will get angry against the tool, argue about the auto-tune and try by any mean to deactivate this stupid feature that never work…

Sounds familiar ? Don’t lie.

The problem

Problem is that we missed a step. An important step. If you study how things are done in JCI applications, you will find for every PID loop, a state generation block attached. Let’s take a simple example of a Fan Coil application with Cooling and Heating.
There is a Output Control Block for the Cooling PID Loop. There is another one for the Heating PID loop.

In the state table, we can see that they are not let in control everytime. Forced in Hold for PID tuning, in Hold when an override is detected, OFF when the Fan is not running… And in T Control only when the “Occupied Zone Sequencing” state generation block tells to the state table that we are in Cooling mode or in Heating mode (depending on the loop you are looking at).

This state generation block will assure you that the PID loop will run only when it’s needed. It will never run forever with an output to 0%.

But why there is no such block for Sideloop ?

It would probably be hard to cover every use cases. But what I know is that you need one. So build it ! A simple addition that will save you time and make your app stronger.

How ?

You first need to know what your sideloop is controlling. For now, let’s say we need a heating loop for an auxiliary room. You created the sideloop and got all the common stuff.

You now need to add in the state generation column, something like a “Occupied Zone Sequencing” block. This block will need some inputs and output to work. The same room temperature than the one used by the sideloop. The setpoints used (cooling and heating). And the control status of the heating sideloop.

What do I take for control status ?

You will need a Last Value Enum block that you will add to the Output Control colum. No efforts needed here the default value of those blocks is Control Status. And you will choose the control status of the PID Loop related to your process that will get to a maximum output.

More details about control status

I’m going a little further than our case here. But let’s say that we would be cascading a PID loop calculating a setpoint. Setpoint used by a second PID loop. When the setpoint loop reaches its maximum, it is possible (and mostly sure) that the output of the second loop won’t need to get to a max position. A control status will notify the app by telling if the control is

  • Overridden (typically not in control)
  • Low (at 0% since a certain delay, you could turn this off)
  • Timing Low (just reach 0%… counting)
  • Normal (working)
  • Timing High (just reach 100%… counting)
  • High (at 100% since a certain delay, need backup!)

The control status is used to switch the mode of the state generation block. Here is what could be in the mind of a simple heating app :

  • I’m satisfied, temperature reached setpoint
  • No, now I need heating, start heating loop
  • Control status of heating is normal, continue
  • Control Status of heating is now High, we reached maximum. Switch state to more heat (a second stage for example)
  • Second stage of heating is normal, modulating…
  • Second stage control status is now Low…. I can get back to my first heating stage…
  • Heating stage is now Low… I’m back to Satisfied

So what would happen in the case of cascaded PIDs if I would use the heating valve PID loop control status instead of the setpoint loop control status ?

  • I’m satisfied
  • No, I need heating
  • Control Status of heating is normal, it modulates.
  • Setpoint has reached max value
  • Control Status of heating is normal, it modulates. Yeah it is a very powerful heating valve, I’m at 25% modulation
  • But room temp is now OK
  • I don’t care, I’m still modulating and I didn’t reached my control status “Low State”
  • I really chose the wrong status right ?
  • You bet ! So I keep heating.

Pretty subtile difference but really important. In our case though, no cascade. So we just link the Sideloop Control Status to the last value block… and we link the output of the Last Value block to the Occupied Zone Sequencing State Generation block. Now the block will know when the Sideloop starve or when it is no more needed.

Modifying the state table

Last step, you must create a new table in the State Table that will use the same output than the Occupied Zone Sequencing. You will create it after all the table holding the Control state for the Sideloop (typically the override check table or the passtrough).

The sideloop will then be configured to be in control only when needed (in heating mode). Will be Off in all other cases. Don’t forget to replace the old Control state in the preceding table by a star.

That’s it

You now have a robust Sideloop that won’t over-tune and stop working for no reason… in fact, reasons you didn’t know. Because now you know why it was not working correctly.

You know why you did your Sideloop wrong.

 

pyhaystack 0.92 / The Haystack Connect 2017 Release

Intro

I created pyhaystack a few years ago. I had discovered python and I thought I could get access to the data stored in our Jaces more easily if I could connect to them using some kind of script. I was actively participating in project-haystack as I was sure it was a very good idea. Tagging data instead of relying on point names made a lot of sense to me.

I had a lot of chance when Stuart from VRT offered to bonified the project. He brought a lot of his knowledge and made the library more robust and very flexible. In fact, we changed everything, but keeping the same idea behind.

Now we reached a point where it is well documented and it is working with the major platforms we know.  Also, in the field of data analysis, python is one of the most used language competing with R, Julia and the others. I feel pyhaystack is really on its spot.

New authentication schemes supported

This new release (version 0.92) allows python to connect to haystack server running Skyspark, Skyspark v3+, Widesky, Niagara AX and Niagara4 !

Thanks to a new contributor (Pierre Sigwalt), we have been able to make pyhaystack compatible with SCRAM authentication. This was needed for Skyspark v3+ and for the new Niagara4 devices.

It’s been hard, but we succeeded !

This update also brings improvements to the syntax so it’s easier to connect and explore your haystack server.

Syntax improvements

It’s now easier to connect to a haystack server using pyhaystack.connect(args). You just provide the implementation you need and login info. You can also use a text file to store the login info. See the docs for details.

You can also use the pythonic square bracket feature over objects to make searches.

my_equip = my_session.site['my_equip_dis'] # dis of an equip
znt = my_equip['ZN~2dT'] # a point name
temp_sensors = my_equip['sensor and air and temp'] # a filter expression

histories

Histories are fundamental to analysis. Pyhaystack uses Pandas series and dataframes when dealing with histories.
You then gain access to a lot of nice features right out of the box. Statistical functions, model fitting, removing nan values or
filling them using last good values…

temp_sensors_his = session.his_read_frame(temp_sensors,
rng='2017-04-01,2017-04-30')
temp_sensors.wait()
temp_sensors.result

A great tool

Those features make pyhaystack a great tool for “on the spot” analysis but also as a robust module to build global distributed analysis
application (Widesky).

More informations

pyhaystack is a 100% open source project supported by SERVISYS inc. and VRT Systems

Come and join us on https://github.com/ChristianTremblay/pyhaystack and chat with us on https://gitter.im/ChristianTremblay/pyhaystack

You can also read the docs on http://pyhaystack.readthedocs.io

You can read more about project-haystack here

New release for BAC0 / 0.99.100

Recent updates by Joel Bender on its very cool bacpypes module needed from me some modifications on BAC0.

I’ve also received a lot of help from a new contributor : kjlockhart regarding documentation. Numerous typo and some weird sentences (yeah… being French Canadian implies some weirdness. But hey, it’s charming weirdness…)

This new release is now better documented and work well with the latest bacpypes module (0.16 coming!!!)

Just pip install BAC0 and enjoy !

I’ve worked a lot on some new features to test sequences of operation. I will keep you informed of the next steps !

I’ve also been really busy on the new release of pyhaystack which will be sent to the wild very soon ! A lot of new stuff !!! Be aware !

How much fresh air ?

When we talk about analytics, we often refer to really big systems and software running on the cloud and giving full graphical features to the user… But analytic is also the possibility to get answer to simple questions in our day to day job.

Today, I’ve run into a situation where I wanted to know what was the quantity of fresh air that was provided to a system.

Luckily, outside air temperature was pretty cold so the difference between the return air and outside air made sense to use the well known formulae :

(Mixed Air – Return Air) / (Outdoor Air – Return Air)

It is not 100% exact but gives a pretty nice approximation.

Getting a result based on a specific time (let say at this moment) is easy. Just do the math… but what if I would like to know what was the proportion for the day… or yesterday ?

That is where pyhaystack can help.

First we need to establish a connection with a Niagara platform running nHaystack. Please note that by default, every histories will be available… no tagging necessary !

from pyhaystack.client.niagara import NiagaraHaystackSession
import logging
import pandas as pd
logging.root.setLevel(logging.DEBUG)
session = NiagaraHaystackSession(uri='http://ip.ip.ip.ip', username='admin', password='reallyhardpassword', pint=True)

As no point were tagged in this station, I used this simple request to find every points having a history… then I searched the ID of the points I were interested in (mixed air, outside air, return air).

history = session.find_entity(filter_expr='his').result
history

Once the ID were found… just retrieve the data from today

MAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.MA~2dT', rng='today').result
OAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.OA~2dT', rng='today').result
RAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.RA~2dT', rng='today').result
DAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.DA~2dT', rng='today').result

Using the data retrieved, create a pandas DataFrame so we can do some math on the columns…

df_ac1 = pd.DataFrame({'mat' : MAT,
                       'rat' : RAT,
                       'oat' : OAT,
                       'dat' : DAT})

df_ac1 = df_ac1.fillna(method = 'ffill').fillna(method = 'bfill').between_time('08:00','17:00')
df_ac1['prop'] = ((df_ac1['mat'] - df_ac1['rat']) / (df_ac1['oat'] - df_ac1['rat']))*100

See the last line, I created a supplemental column, and the result of that column is based on a calculation made on the other columns…

reading the tail we get this table

freshair_2

Now it would be more visual to see it on a chart

import matplotlib.pyplot as plt
plt.plot(df_ac1['prop'])
plt.title("Proportion d'air frais \n 30 janvier 2017")
plt.show()

freshair_1

A few lines of code (around 20)… a really visual answer to the question asked.

pyhaystack getting visibility

I’m pleased to announce (a little late I admit) that pyhaystack is now listed as an official python library for project-haystack.
http://project-haystack.org/download

We also have been mentionned in Haystack Connect Issue #2

Click to access Haystack-Connections-Magazine-Issue-2-Jan-2017.pdf

pyhaystack is compatible with Python 3 and support Python 2.7

Give it a try !

ControlTalks…. about BAC0 !

Just wanted to let you know that guys from ControlTalks (http://controltrends.org/) have been kind enough to talk about BAC0 in their last podcast.

I had the chance to meet Eric Stromquist and Ken Smyers in Nashville while attending the Johnson Controls CBC16.

Here’s our first contact in Nashville

And here’s the podcast 201 !

Thanks ControlTalk !

Design a site like this with WordPress.com
Get started