pyhaystack 0.92 / The Haystack Connect 2017 Release

Intro

I created pyhaystack a few years ago. I had discovered python and I thought I could get access to the data stored in our Jaces more easily if I could connect to them using some kind of script. I was actively participating in project-haystack as I was sure it was a very good idea. Tagging data instead of relying on point names made a lot of sense to me.

I had a lot of chance when Stuart from VRT offered to bonified the project. He brought a lot of his knowledge and made the library more robust and very flexible. In fact, we changed everything, but keeping the same idea behind.

Now we reached a point where it is well documented and it is working with the major platforms we know. Also, in the field of data analysis, python is one of the most used language competing with R, Julia and the others. I feel pyhaystack is really on its spot.

New authentication schemes supported

This new release (version 0.92) allows python to connect to haystack server running Skyspark, Skyspark v3+, Widesky, Niagara AX and Niagara4 !

Thanks to a new contributor (Pierre Sigwalt), we have been able to make pyhaystack compatible with SCRAM authentication. This was needed for Skyspark v3+ and for the new Niagara4 devices.

It’s been hard, but we succeeded !

This update also brings improvements to the syntax so it’s easier to connect and explore your haystack server.

Syntax improvements

It’s now easier to connect to a haystack server using pyhaystack.connect(args). You just provide the implementation you need and login info. You can also use a text file to store the login info. See the docs for details.

You can also use the pythonic square bracket feature over objects to make searches.

my_equip = my_session.site['my_equip_dis'] # dis of an equip
znt = my_equip['ZN~2dT'] # a point name
temp_sensors = my_equip['sensor and air and temp'] # a filter expression

histories

Histories are fundamental to analysis. Pyhaystack uses Pandas series and dataframes when dealing with histories.
You then gain access to a lot of nice features right out of the box. Statistical functions, model fitting, removing nan values or
filling them using last good values…

temp_sensors_his = session.his_read_frame(temp_sensors,
rng='2017-04-01,2017-04-30')
temp_sensors.wait()
temp_sensors.result

A great tool

Those features make pyhaystack a great tool for “on the spot” analysis but also as a robust module to build global distributed analysis
application (Widesky).

More informations

pyhaystack is a 100% open source project supported by SERVISYS inc. and VRT Systems

Come and join us on https://github.com/ChristianTremblay/pyhaystack and chat with us on https://gitter.im/ChristianTremblay/pyhaystack

You can also read the docs on http://pyhaystack.readthedocs.io

You can read more about project-haystack here

How much fresh air ?

When we talk about analytics, we often refer to really big systems and software running on the cloud and giving full graphical features to the user… But analytic is also the possibility to get answer to simple questions in our day to day job.

Today, I’ve run into a situation where I wanted to know what was the quantity of fresh air that was provided to a system.

Luckily, outside air temperature was pretty cold so the difference between the return air and outside air made sense to use the well known formulae :

(Mixed Air – Return Air) / (Outdoor Air – Return Air)

It is not 100% exact but gives a pretty nice approximation.

Getting a result based on a specific time (let say at this moment) is easy. Just do the math… but what if I would like to know what was the proportion for the day… or yesterday ?

That is where pyhaystack can help.

First we need to establish a connection with a Niagara platform running nHaystack. Please note that by default, every histories will be available… no tagging necessary !

from pyhaystack.client.niagara import NiagaraHaystackSession
import logging
import pandas as pd
logging.root.setLevel(logging.DEBUG)
session = NiagaraHaystackSession(uri='http://ip.ip.ip.ip', username='admin', password='reallyhardpassword', pint=True)

As no point were tagged in this station, I used this simple request to find every points having a history… then I searched the ID of the points I were interested in (mixed air, outside air, return air).

history = session.find_entity(filter_expr='his').result
history

Once the ID were found… just retrieve the data from today

MAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.MA~2dT', rng='today').result
OAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.OA~2dT', rng='today').result
RAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.RA~2dT', rng='today').result
DAT = session.his_read_series('C.Drivers.BacnetNetwork.PCA~2d2~2d004.points.DA~2dT', rng='today').result

Using the data retrieved, create a pandas DataFrame so we can do some math on the columns…

df_ac1 = pd.DataFrame({'mat' : MAT,
                       'rat' : RAT,
                       'oat' : OAT,
                       'dat' : DAT})

df_ac1 = df_ac1.fillna(method = 'ffill').fillna(method = 'bfill').between_time('08:00','17:00')
df_ac1['prop'] = ((df_ac1['mat'] - df_ac1['rat']) / (df_ac1['oat'] - df_ac1['rat']))*100

See the last line, I created a supplemental column, and the result of that column is based on a calculation made on the other columns…

reading the tail we get this table

freshair_2

Now it would be more visual to see it on a chart

import matplotlib.pyplot as plt
plt.plot(df_ac1['prop'])
plt.title("Proportion d'air frais \n 30 janvier 2017")
plt.show()

freshair_1

A few lines of code (around 20)… a really visual answer to the question asked.

pyhaystack getting visibility

I’m pleased to announce (a little late I admit) that pyhaystack is now listed as an official python library for project-haystack.
http://project-haystack.org/download

We also have been mentionned in Haystack Connect Issue #2

Click to access Haystack-Connections-Magazine-Issue-2-Jan-2017.pdf

pyhaystack is compatible with Python 3 and support Python 2.7

Give it a try !

pyhaystack

A few years ago, I found out project-haystack. I knew that this was going to be a game changer.

Haystack brings a way to find what you need in your BAS, abstracting variables names.

What if I could use Python to do some analysis ? I could use Pandas, numpy, scikit-learn ?

There’s been plenty of versions but the latest brings stability and efficiency, thanks to Stuart J. Longland from VRT.

We now have another reason to bring Python in the building !

See it on Github !

Read the docs here !