Effective monitoring of thousands of IT Services requires you to transform realtime data streams into actionable alerts and high level situation overviews. In this blog post I will describe how to create reliable alerts and reduce the problem of alert storms by clustering.
This blog is part of a series of blogs on setting up a Smart monitoring system.
In my previous post on monitoring I wrote about setting up customer activity monitoring supported by anomaly detection. Now I will show how to monitor hundreds of services without drowning in data. The solution i present processes raw monitoring metrics and turns that data into actionable alerts and usefull insights.
In this article I’ll describe how I implemented customer activity monitoring and anomaly detection. If you are a service provider that provide services to a group of large accounts its vital to know that your customers can do their business.
Monitoring customer behavior is not only required for managing IT operations its also vital to know from a business point of view. Nowadays customers get smart too. High tech customers such as Netflix and Airbnb use data analytics to monitor the results they get from their payment providers. Based on this data they make real-time decisions to switch to another supplier when that better suits their needs.
Microservices promise to deliver new business services faster and at lower cost, but it comes at a price: increased operational complexity. I share my vision on managing microservices on the enterprise level:
This article is part of a series on service monitoring.
During my period at ING Bank, I was part of the forward-thinking 4WRD DevOps team, focusing on the Basel II reporting system. This project involved transitioning a mainframe application to the ING Data Lake, utilizing Netezza Data Warehouse and IBM Infosphere Datastage for improved data management and reporting.
There where two major challenges:
At first sight this looked overwelming for our small team of six. We took a practical approach of learning on the job. We started with the easiest looking datastream and implement a full end to end data flow. We learned and gradually improved our methods for documenting, coding and testing.
My role was pivotal in establishing a robust deployment pipeline, crafting ETL (Extract, Transform, Load) data transformations, executing monthly Basel reporting batches, and offering comprehensive technical support to our team. In overseeing operations, I ensured the system’s reliability and efficiency. Moreover, as a developer of team tools, I innovated automated deployment strategies using a suite of technologies including Gitlab, Jenkins, Artifactory, Ansible, and Bash scripting, coupled with IBM Datastage for deployment tool sets.
Our team’s accomplishments are notable. We successfully processed monthly data for over 8 million customers regarding unsecured loans and credit cards, using IBM Datastage ETL jobs to handle the data and storing the outcomes in a Netezza warehouse. We pioneered a continuous deployment process, integrated FitnesseDB for regression testing, and meticulously stored every code asset on Gitlab. This comprehensive approach not only facilitated seamless deployment but also provided a solid framework for recovery from any setbacks.
In an effort to share our journey and methodologies with others, I created an engaging animation depicting Team 4WRD’s development of the Basel II reporting system. This animation showcased our dynamic workflow and updates to various assets, produced using the open-source visualization tool Gource.
Tauvic's Blog 