https://dmsimard.com/ Recent content on System Administration, Hosting, Cloud and technologies in between Hugo -- gohugo.io en-us Fri, 14 Apr 2017 02:20:11 +0000 https://dmsimard.com/2018/10/08/ansiblefest-2018-community-project-highlights/ Mon, 08 Oct 2018 00:00:00 +0000 https://dmsimard.com/2018/10/08/ansiblefest-2018-community-project-highlights/ With two days of AnsibleFest instead of one this time around, we had 100% more time to talk about Ansible things ! I got to attend great sessions, learn a bunch of things, chat and exchange war stories about Ansible, ARA, Zuul, Tower and many other things. It was awesome and I wanted to take the time to share a bit about some of the great Ansible community projects that were featured during the event. https://dmsimard.com/2018/05/03/ara-records-ansible-0.15-has-been-released/ Thu, 03 May 2018 00:00:00 +0000 https://dmsimard.com/2018/05/03/ara-records-ansible-0.15-has-been-released/ I was recently writing that ARA was open to limited development for the stable release in order to improve the performance for larger scale users. This limited development is the result of this 0.15.0 release. The #OpenStack community runs over 300,000 continuous integration jobs with #Ansible every month with the help of the awesome Zuul. Learn more about scaling ARA reports with @dmsimard https://t.co/l8zFXHqhhc — OpenStack (@OpenStack) April 18, 2018 Changelog for ARA Records Ansible 0. https://dmsimard.com/2018/04/09/scaling-ara-to-a-million-ansible-playbooks-a-month/ Mon, 09 Apr 2018 00:00:00 +0000 https://dmsimard.com/2018/04/09/scaling-ara-to-a-million-ansible-playbooks-a-month/ The OpenStack community runs over 300 000 CI jobs with Ansible every month with the help of the awesome Zuul. It even provides ARA reports for ARA’s integration test jobs in a sort-of nested way. Zuul’s Ansible ends up installing Ansible and ARA. It makes my brain hurt sometimes… but in an awesome way. As a core contributor of the infrastructure team there, I get to witness issues and get a lot of feedback directly from the users. https://dmsimard.com/2018/02/24/awesome-things-in-software-engineering-open-source/ Sat, 24 Feb 2018 00:00:00 +0000 https://dmsimard.com/2018/02/24/awesome-things-in-software-engineering-open-source/ This is part of a blog series highlighting awesome things in software engineering because not everything has to be depressing, about bugs, vulnerabilities, outages or deadlines. If you’d like to collaborate and write about awesome things in software engineering too, let’s chat: reach out on Twitter or LinkedIn. What’s this blog series about ? Between you and me, software engineering isn’t always fun. You’re not always working on what you like. https://dmsimard.com/2018/02/23/rebranding-ansible-run-analysis-to-ara-records-ansible/ Fri, 23 Feb 2018 00:00:00 +0000 https://dmsimard.com/2018/02/23/rebranding-ansible-run-analysis-to-ara-records-ansible/ So I got an idea recently… Let’s rebrand Ansible Run Analysis to ARA records Ansible. If you’d like to review and comment on the code change, you can do so here: https://review.openstack.org/#/c/547245/. Why ? I watched the last season of Sillicon Valley recently. The series, while exaggerated, provides a humorous look at the world of startups. I don’t have any plans on creating a startup but I love that it makes you think about things like needing a clever name or how you would do a proper “elevator” pitch to get funding. https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ Wed, 29 Nov 2017 00:00:00 +0000 https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ The RDO project community provides vanilla RPM packages and mirrors for deploying OpenStack on the CentOS or RHEL linux distributions. The packages provided by RDO can be deployed manually or through different OpenStack installers such as TripleO, Kolla, Packstack and Puppet-OpenStack. OpenStack-Ansible also relies on RDO for dependencies although it currently installs OpenStack projects from source. At each OpenStack development cycle milestone, the RDO community holds a test day. This gives the opportunity to the greater community of OpenStack users, developers and operators to try out the latest and the greatest of OpenStack with people around to help on IRC in the #rdo channel. https://dmsimard.com/2017/11/22/status-update-ara-1.0/ Wed, 22 Nov 2017 00:00:00 +0000 https://dmsimard.com/2017/11/22/status-update-ara-1.0/ Back in August, I posted about what was the roadmap for ARA 1.0 and why it was a very important milestone for the project. We’re now almost in December and I said there would likely be a beta version out by September. “What’s going on ?", one might ask… A fair question. There’s definitely been progress and I could’ve been doing a better job at communicating updates other than the tweet from time to time. https://dmsimard.com/2017/08/27/new-ways-of-reaching-the-ara-community/ Sun, 27 Aug 2017 00:00:00 +0000 https://dmsimard.com/2017/08/27/new-ways-of-reaching-the-ara-community/ More and more users requested other ways of reaching the ARA community and I’ve finally given in ! Until now, the only way of getting in touch was through IRC and I understand that, in 2017, IRC is not for everyone. Seamless communication across IRC, Slack and Discord That’s right, you can now reach us through Slack and Discord. Both are linked to IRC so messages sent to one will be relayed automatically to the others. https://dmsimard.com/2017/08/16/whats-coming-in-ara-1.0/ Wed, 16 Aug 2017 00:00:00 +0000 https://dmsimard.com/2017/08/16/whats-coming-in-ara-1.0/ Not long ago, I wrote that ARA: Ansible Run Analysis had it’s first birthday. It was an important milestone and it was a great opportunity to reflect back on where the project was coming from and think about what we needed to do in the future. Just for fun, let’s look at what I had written back in May to summarize what was probably coming: Python 3 compatibility This is done and was shipped in ARA 0. https://dmsimard.com/2017/07/18/patching-ansible-with-ansible/ Tue, 18 Jul 2017 00:00:00 +0000 https://dmsimard.com/2017/07/18/patching-ansible-with-ansible/ (Edit: See at the end) I can’t get enough of this Ansible thing, it’s great and makes my life easier. Sometimes there’s this little awesome feature that is in a pull request or has already landed in the development branch. You can’t wait to use but it won’t be shipped until the next release of Ansible.. and sometimes that takes a while. What do you do in the meantime ? https://dmsimard.com/2017/05/10/limiting-the-amount-of-concurrent-asynchronous-tasks-in-ansible/ Wed, 10 May 2017 00:00:00 +0000 https://dmsimard.com/2017/05/10/limiting-the-amount-of-concurrent-asynchronous-tasks-in-ansible/ Ansible is known to be good at running things in the order you write them and that’s why it’s awesome for orchestration. However, I have a use case where I have several similar and long-running tasks to run that do not need to run sequentially. Ansible provides a way to run tasks asynchronously and later recover their result. The problem The problem is that Ansible doesn’t provide a way to limit the amount of concurrent tasks run asynchronously. https://dmsimard.com/2017/05/08/ara-is-one-year-old-a-look-back-at-the-past-year/ Mon, 08 May 2017 00:00:00 +0000 https://dmsimard.com/2017/05/08/ara-is-one-year-old-a-look-back-at-the-past-year/ ARA is one year old, happy birthday ARA ! ARA’s come a long way since the early prototypes. The latest version, 0.13, looks pretty awesome. It even caught the eye of Michael DeHaan, the author of Ansible ! This looks pretty awesome (maybe run on a Jenkins box?) https://t.co/qvd3aYE7n9 — laserllama (@laserllama) May 6, 2017 Let’s go back in time to look at the humble beginnings of the project and some of the important milestones that marked it’s history this past year. https://dmsimard.com/2017/05/05/ara-ansible-run-analysis-0-13-is-out-and-its-awesome/ Fri, 05 May 2017 00:00:00 +0000 https://dmsimard.com/2017/05/05/ara-ansible-run-analysis-0-13-is-out-and-its-awesome/ I’m excited to announce the release of ARA: Ansible Run Analysis 0.13.0! ARA 0.13.0 is available on PyPi or from source on GitHub. I’m also happy to announce that ARA 0.13.0 will be the first version of ARA packaged for Fedora and CentOS EPEL. Stay tuned in the near future to hear when the packages will be available. Wait, what’s ARA ? ARA is an Ansible callback plugin that records your playbook runs, wherever it is. https://dmsimard.com/about/ Fri, 14 Apr 2017 02:20:11 +0000 https://dmsimard.com/about/ $ whoami Hi o/ I’m David or otherwise known online as dmsimard. I don’t blog much here anymore for various reasons but feel free to reach out and chat ! If you’re interested about Ansible, you might find my blog posts about ara interesting: https://ara.recordsansible.org/blog/. What’s this blog about I learn new stuff every day. Sometimes I write about it. Sometimes I rant about it. The topics you’ll read about here involve system administration, hosting, cloud and technologies in between. https://dmsimard.com/2017/03/12/an-even-better-ansible-reporting-interface-with-ara-0-12/ Sun, 12 Mar 2017 00:00:00 +0000 https://dmsimard.com/2017/03/12/an-even-better-ansible-reporting-interface-with-ara-0-12/ Not even a month ago, I announced the release of ARA 0.11 with a bunch of new features and improvements. Today, I’m back with some more great news and an awesome new release, ARA 0.12(.3) ! That’s right, 0.12.3! Due to the nature of this new release, I wanted to be sure to get feedback from the users before getting the word out. We got a lot of great input! This allowed us to fix some bugs and significantly improve the performance. https://dmsimard.com/2017/02/13/announcing-the-ara-0.11-release/ Mon, 13 Feb 2017 00:00:00 +0000 https://dmsimard.com/2017/02/13/announcing-the-ara-0.11-release/ We’re on the road to version 1.0.0 and we’re getting closer: introducing the release of version 0.11! Four new contributors (!), 55 commits since 0.10 and 112 files changed for a total of 2,247 additions and 939 deletions. New features, more stability, better documentation and better test coverage. The changelog since 0.10.5 New feature: ARA UI and Ansible version (ARA UI is running with) are now shown at the top right New feature: The Ansible version a playbook was run is now stored and displayed in the playbook reports New feature: New command: “ara generate junit”: generates a junit xml stream of all task results New feature: ara_record now supports two new types: “list” and “dict”, each rendered appropriately in the UI UI: Add ARA logo and favicon UI: Left navigation bar was removed (top navigation bar will be further improved in future versions) Bugfix: CLI commands could sometimes fail when trying to format as JSON or YAML Bugfix: Database and logs now properly default to ARA_DIR if ARA_DIR is changed Bugfix: When using non-ascii characters (ex: äëö) in playbook files, web application or static generation could fail Bugfix: Trying to use ara_record to record non strings (ex: lists or dicts) could fail Bugfix: Ansible config: ‘tmppath’ is now a ‘type_value’ instead of a boolean Deprecation: The “ara generate” command was deprecated and moved to “ara generate html” Deprecation: The deprecated callback location, ara/callback has been removed. https://dmsimard.com/2016/12/12/updates-to-fedora-account-system-and-impacts-to-fedpkg-koji/ Mon, 12 Dec 2016 00:00:00 +0000 https://dmsimard.com/2016/12/12/updates-to-fedora-account-system-and-impacts-to-fedpkg-koji/ Today when trying to do my Fedora packaging stuff I encountered problems with the fedpkg and koji commands – they would just hang without any output. It turns out the Fedora Infrastructure folks are having a “Flag Day” which impacts some of the packaging workflow. After talking with some helpful individuals on #fedora-devel and #fedora-admin, here’s what I had to do on my Fedora 25 installation to get things working again: https://dmsimard.com/2016/12/01/ara-0.10-the-biggest-release-yet-is-out/ Thu, 01 Dec 2016 00:00:00 +0000 https://dmsimard.com/2016/12/01/ara-0.10-the-biggest-release-yet-is-out/ 19 commits, 59 changed files, 2,404 additions and 588 deletions… and more than a month’s on and off work. 0.10 is out ! Where to get it ? Get started easily by installing and configuring ARA. I’m excited to tell you about this new release ! Here’s a few highlights ! An improved web application browsing experience A lot of work has gone into the browsing experience: less clicks, more information, faster. https://dmsimard.com/2016/11/29/recorded-presentations-from-openstack-canada-day/ Tue, 29 Nov 2016 00:00:00 +0000 https://dmsimard.com/2016/11/29/recorded-presentations-from-openstack-canada-day/ OpenStack Days are sort of local mini one-day OpenStack summits. Not everyone can travel to the OpenStack summits for a number of reasons so OpenStack days are great for that since there are many across a wide range of countries. The first one in Canada was in Montreal, at an awesome venue: the Notman house. There’s a lot of history about the place, definitely look it up - it’s very interesting ! https://dmsimard.com/2016/11/09/visualizing-kolla-ansible-playbooks-with-ara/ Wed, 09 Nov 2016 00:00:00 +0000 https://dmsimard.com/2016/11/09/visualizing-kolla-ansible-playbooks-with-ara/ Kolla is an OpenStack deployment tool that’s growing in popularity right now. An OpenStack installation by Kolla was even showcased by Chris Hoge and Mark Collier in one of the main keynotes at the recent Barcelona OpenStack Summit. Installing OpenStack is complex Installing and configuring OpenStack is no easy task – Kolla tackles this challenge with the help of Docker containers that are deployed with Ansible. This translates into quite a few playbooks, lots of plays, many more tasks and especially lots of data to parse through when trying to understand what is going on. https://dmsimard.com/2016/08/04/openstack-operators-developers-users-its-your-summit-vote/ Thu, 04 Aug 2016 00:00:00 +0000 https://dmsimard.com/2016/08/04/openstack-operators-developers-users-its-your-summit-vote/ Once again, the OpenStack Summit is nigh and this time it’ll be in Barcelona. The OpenStack Summit event is an opportunity for Operators, Developers and Users alike to gather, discuss and learn about OpenStack. What we know is that there’s going to be keynotes, design sessions for developers to hack on things and operator sessions for discussing and exchanging around the challenges of operating OpenStack. We also know there’s going to be a bunch of presentations on a wide range of topics from the OpenStack community. https://dmsimard.com/2016/07/13/improving-rdo-packaging-testing-coverage/ Wed, 13 Jul 2016 00:00:00 +0000 https://dmsimard.com/2016/07/13/improving-rdo-packaging-testing-coverage/ DLRN builds packages and generates repositories in which these packages will be hosted. It is the tool that is developed and used by the RDO community to provide the repositories on trunk.rdoproject.org. It continuously builds packages for every commit for projects packaged in RDO. RDO is completely open source and community driven: anyone can submit patches to improve RDO packaging. When someone submits a patch for review on review.rdoproject.org, we have a gate job called “DLRN-rpmbuild” that will run DLRN to see if the package builds successfully. https://dmsimard.com/2016/06/07/ara-one-month-and-200-commits-later/ Tue, 07 Jun 2016 00:00:00 +0000 https://dmsimard.com/2016/06/07/ara-one-month-and-200-commits-later/ On May 21st, I wrote a blog post about ARA, an idea to store, browse and troubleshoot Ansible playbook runs. Let’s rewind a bit further back in time. On May 6th, I got tired of trying to make our human_log callback write user friendly HTML files. I simply wasn’t happy with my attempts… I’m a big fan of the UNIX philosophy: Do one thing and do it well. Trying to hack HTML writing into human_log didn’t feel like that at all. https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/ Sat, 21 May 2016 00:00:00 +0000 https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/ The context Ansible can be used for a lot of things and it’s grown pretty popular for managing servers and their configuration. In the RDO and OpenStack communities, Ansible is heavily used to deploy or test OpenStack through Continuous Integration (CI). Projects like TripleO-Quickstart, WeIRDO, OpenStack-Ansible or Zuul v3 are completely driven by Ansible. In the world of automated continuous integration, it’s not uncommon to have hundreds, if not thousands of jobs running every day for testing, building, compiling, deploying and so on. https://dmsimard.com/2016/05/15/what-did-everyone-do-for-the-mitaka-release-of-openstack/ Sun, 15 May 2016 00:00:00 +0000 https://dmsimard.com/2016/05/15/what-did-everyone-do-for-the-mitaka-release-of-openstack/ Just what did everyone do for the Mitaka OpenStack release ? RDO community liaison Rich Bowen went to find out. He interviewed some developers and engineers that worked on OpenStack and RDO throughout the Mitaka cycle and asked them what they did and what they were up to for the Newton cycle. It was definitely a good idea and I hope we do the same retrospective for Newton ! Here’s a quick summary of who participated and what they talked about. https://dmsimard.com/2016/03/21/tips-on-controlling-the-flow-of-notifications-in-sensu/ Mon, 21 Mar 2016 00:00:00 +0000 https://dmsimard.com/2016/03/21/tips-on-controlling-the-flow-of-notifications-in-sensu/ I had been wanting to try Sensu out for quite a while. If I’m not mistaken, I think the first time I heard about someone using it in production was back at the 2014 OpenStack summit in Paris where fellow OpenStack Operator Joe Topjian told me he had been using Sensu. A few weeks back, the RDO community formalized that we needed to improve the monitoring for our build, packaging, mirror and CI infrastructure. https://dmsimard.com/2016/03/15/changing-the-ssh-port-with-ansible/ Tue, 15 Mar 2016 00:00:00 +0000 https://dmsimard.com/2016/03/15/changing-the-ssh-port-with-ansible/ One of the first things I do when I get my hands on a new linux server is to change the SSH port. It’s a basic, easy and efficient way of warding most brute force attempts. In a nutshell, you edit the Port parameter of /etc/ssh/sshd_config, restart sshd and you’re done. If you have Selinux enabled, you’ll have to allow sshd to listen on that port too with semanage port -a -t ssh_port_t -p tcp <<port>> https://dmsimard.com/2016/03/02/openstack-montreal-a-talk-about-ci-in-openstack-and-rdo/ Wed, 02 Mar 2016 00:00:00 +0000 https://dmsimard.com/2016/03/02/openstack-montreal-a-talk-about-ci-in-openstack-and-rdo/ OpenStack Montreal is a meetup that happens around every 2 months in Montreal. We talk and discuss about different OpenStack topics with interests for developers, users and operators. If you’d like to keep up with the event, attend (or present a topic!), please subscribe to the mailing list to be notified when something’s going on. Meanwhile, I presented a talk at the last event in February to talk about CI in and RDO, it was tons of fun. https://dmsimard.com/2016/03/02/packstack-gates-against-itself/ Wed, 02 Mar 2016 00:00:00 +0000 https://dmsimard.com/2016/03/02/packstack-gates-against-itself/ TL;DR Today I made an announcement on both openstack-dev and rdo-list that Packstack was going to gate against itself. Until now Packstack essentially only gated against pep8. From now on it will test itself with a self-installed Tempest with three different configuration scenarios in the upstream OpenStack gate. Additionally, we will use these new tests in the RDO CI pipeline to improve our test coverage. Pretty exciting. There is no long version, that was a joke. https://dmsimard.com/2016/01/08/selinux-python-virtualenv-chroot-and-ansible-dont-play-nice/ Fri, 08 Jan 2016 00:00:00 +0000 https://dmsimard.com/2016/01/08/selinux-python-virtualenv-chroot-and-ansible-dont-play-nice/ The problem Aborting, target uses selinux but python bindings (libselinux-python) aren't installed! Uh oh, have you seen that before ? I bumped into an interesting issue recently when running Ansible from a RHEL derivative host. It turns out that Ansible requires the package libselinux-python to be installed on hosts where Ansible is running file modules. If you have SELinux enabled on remote nodes, you will also want to install libselinux-python on them before using any copy/file/template related functions in Ansible. https://dmsimard.com/2015/12/07/thinking-outside-the-box-and-outside-the-gate-to-improve-openstack-and-rdo/ Mon, 07 Dec 2015 00:00:00 +0000 https://dmsimard.com/2015/12/07/thinking-outside-the-box-and-outside-the-gate-to-improve-openstack-and-rdo/ I recently explained how testing everything in OpenStack is hard. The reality is that the RDO community has finite resources. Testing everything is not just hard, it’s time consuming and expensive. I’ve been thinking a lot about how to tackle this and experimented around ways to improve our test coverage in this context. How can we improve testing coverage for RDO ? Upstream OpenStack does a good job of testing things as much as it can. https://dmsimard.com/2015/12/04/testing-openstack-is-hard/ Fri, 04 Dec 2015 00:00:00 +0000 https://dmsimard.com/2015/12/04/testing-openstack-is-hard/ I joined Red Hat last September to help the RDO community ship a quality OpenStack package distribution faster. Previously, I worked for Internap, where I helped operate their cloud. There’s a lot of challenges involved in deploying and supporting a large scale OpenStack public cloud. At Internap, we did extensive integration testing with a representation of our production cloud in development environments. Looking back, we were really just testing limited configurations: we only tested what we were deploying. https://dmsimard.com/2015/07/18/migrating-glance-images-to-a-different-backend/ Sat, 18 Jul 2015 00:00:00 +0000 https://dmsimard.com/2015/07/18/migrating-glance-images-to-a-different-backend/ The use case This isn’t exactly far-fetched but nothing seems to do it out of the box: migrate Glance images from a backend to the other. We’re in the process of moving our Glance default store to Swift, it simply scales infinitely better than the file backend. Now, changing the default store does just that: change the default store. It makes it so the newly created images will be uploaded to the new store - it doesn’t do anything for the existing images. https://dmsimard.com/2015/06/28/openstackclient-is-better-than-i-thought/ Sun, 28 Jun 2015 00:00:00 +0000 https://dmsimard.com/2015/06/28/openstackclient-is-better-than-i-thought/ When openstackclient came out, I was not a believer. I thought.. Great, a new standard (cue XKCD), and I joked about it. I had been using the ordinary CLI clients like novaclient, keystoneclient and so on. Over time, I guess I got used to their strengths, weaknesses and their quirks for better or for worse. We’ve started wrapping around Openstackclient in the different puppet modules for Openstack. Since Openstackclient provides CSV output, it makes it easy for us to parse the command outputs. https://dmsimard.com/2015/03/27/ceph-erasure-coding-overhead-in-a-nutshell/ Fri, 27 Mar 2015 00:00:00 +0000 https://dmsimard.com/2015/03/27/ceph-erasure-coding-overhead-in-a-nutshell/ Calculating the storage overhead of a replicated pool in Ceph is easy. You divide the amount of space you have by the “size” (amount of replicas) parameter of your storage pool. Let’s work with some rough numbers: 64 OSDs of 4TB each. Raw size: 64 * 4 = 256TB Size 2 : 128 / 2 = 128TB Size 3 : 128 / 3 = 85.33TB Replicated pools are expensive in terms of overhead: Size 2 provides the same resilience and overhead as RAID-1. https://dmsimard.com/2015/03/14/harness-the-power-of-puppetdb-with-ansible-dynamic-inventory/ Sat, 14 Mar 2015 00:00:00 +0000 https://dmsimard.com/2015/03/14/harness-the-power-of-puppetdb-with-ansible-dynamic-inventory/ People out there are wondering what orchestration and configuration tool to use. Rightfully so because it gets confusing quickly. Puppet ? Ansible ? Chef ? Salt ? The myriad of other tools ? In the age of Infrastructure as Code, have you made your choice yet ? My answer: Why limit yourself to just one option ? I started getting involved with Puppet a long time ago and, to this day, I don’t regret that choice. https://dmsimard.com/2015/03/11/new-release-of-python-cephclient/ Wed, 11 Mar 2015 00:00:00 +0000 https://dmsimard.com/2015/03/11/new-release-of-python-cephclient/ I’ve just drafted a new release of python-cephclient on PyPi: v0.1.0.5. After learning about the ceph-rest-api I just had to do something fun with it. In fact, it’s going to become very handy for me as I might start to develop with it for things like nagios monitoring scripts. The changelog: dmsimard: Add missing dependency on the requests library Some PEP8 and code standardization cleanup Add root “PUT” methods Add mon “PUT” methods Add mds “PUT” methods Add auth “PUT” methods Donald Talton: https://dmsimard.com/2015/02/06/how-to-disable-head-request-caching-with-nginx-and-tengine/ Fri, 06 Feb 2015 00:00:00 +0000 https://dmsimard.com/2015/02/06/how-to-disable-head-request-caching-with-nginx-and-tengine/ Using tengine (an awesome fork of Nginx, check it out), I found that it would cache HEAD requests even when I thought I explicitly told it not to. Here’s the long story short. What I expected to work In the http section of /etc/nginx/nginx.conf: proxy_cache_min_uses 3; proxy_cache_methods GET; proxy_cache_valid 200 10m; proxy_cache_valid 403 404 1m; proxy_cache_path /var/cache/nginx/website/ levels=1:2 keys_zone=website:16m loader_threshold=300 loader_files=200 max_size=20g; Reading the above, I would expect nginx to only cache GET requests, as per configured through proxy_cache_methods after the third hit. https://dmsimard.com/2014/10/09/a-rant-about-supermicro-ipmi/ Thu, 09 Oct 2014 00:00:00 +0000 https://dmsimard.com/2014/10/09/a-rant-about-supermicro-ipmi/ Our OS installations are automated for the most part so we don’t need to use IPMI all that much if not for troubleshooting over the KVM virtual console or rebooting servers. However, I’ve had to use Supermicro’s IPMI a LOT in the last couple days… for a mix of good and bad reasons. I’ll let the video talk for itself. https://dmsimard.com/2014/09/29/s3ql-a-filesystem-over-http-with-openstack-swift/ Mon, 29 Sep 2014 00:00:00 +0000 https://dmsimard.com/2014/09/29/s3ql-a-filesystem-over-http-with-openstack-swift/ Swift and Object Storage Swift is Openstack‘s Object Storage project. At a very high level, I like to present Object Storage as a filesystem accessible through a set of APIs, often directly by HTTP. Object Storage backends are usually built from the ground up to be resilient, failure tolerant, highly available and provide mechanisms to ensure data redundancy and security. Object Storage is the secret sauce that hides behind interfaces such as Dropbox, Google Drive or Microsoft OneDrive. https://dmsimard.com/2014/09/28/finding-the-right-openstack-provider/ Sun, 28 Sep 2014 00:00:00 +0000 https://dmsimard.com/2014/09/28/finding-the-right-openstack-provider/ How to find a great Openstack provider. Isn’t that a good question ? The Openstack foundation thinks so, too. That’s why it announced the release of the Openstack Marketplace at the last Openstack summit in Atlanta. It’s definitely a step in the right direction towards helping users find trustworthy providers of Openstack services. To be listed, providers must meet basic requirements that are verified by the foundation. The great thing is that the marketplace is not limited to public or private cloud offerings, you’ll also be able to find offers for training or consulting, too. https://dmsimard.com/2014/09/25/the-bash-cve-2014-6271-shellshock-vulnerability/ Thu, 25 Sep 2014 00:00:00 +0000 https://dmsimard.com/2014/09/25/the-bash-cve-2014-6271-shellshock-vulnerability/ The vulnerability A bash vulnerability ? Talk about a lot of servers affected. It’s been all over the internet since yesterday: CVE-2014-6271 on National Vulnerability Database CVE-2014-6271: Remote code execution through bash on /r/netsec CVE-2014-6271: Remote code execution through bash on Hacker News CVE-2014-7169 on National Vulnerability Database CVE-2014-7169: Bash Fix Incomplete, Still Exploitable on /r/netsec CVE-2014-7169: Bash Fix Incomplete, Still Exploitable on Hacker News CVE-2014-7186 on National Vulnerability Database CVE-2014-7187 on National Vulnerability Database Red hat article on the issue Wikipedia article on Shellshock Lots of good reading above into what the vulnerability is and what is the impact. https://dmsimard.com/2014/08/23/ssl-google-cloudflare-webmastertools-and-you/ Sat, 23 Aug 2014 00:00:00 +0000 https://dmsimard.com/2014/08/23/ssl-google-cloudflare-webmastertools-and-you/ Google recently made waves on the internet when they announced they would now rank favorably sites that were secured by SSL. You could see the tears of joy of the SSL business. Google just gave people an additional incentive to purchase SSL certificates but don’t get me wrong - this is very much a good thing for most sites and the internet in general. Giving in Now, on the technical side of things, it’s not like this blog could benefit from SSL encryption all that much: https://dmsimard.com/2014/08/12/send-your-encrypted-duplicity-backups-to-a-swift-object-storage/ Tue, 12 Aug 2014 00:00:00 +0000 https://dmsimard.com/2014/08/12/send-your-encrypted-duplicity-backups-to-a-swift-object-storage/ Duplicity is a Python utility that backups your stuff, encrypts it and allows you to upload them away using as little bandwidth as possible using the rsync algorithm. Cost effectiveness and security aren’t things that especially come to mind when thinking about sending your backups to the cloud. However, as the cost of mass storage and bandwidth goes down, so does the pricing of cloud Object Storage. As far as security is concerned, your backups are encrypted locally by Duplicity before being uploaded. https://dmsimard.com/2014/08/09/a-new-ceph-mirror-on-the-east-coast/ Sat, 09 Aug 2014 00:00:00 +0000 https://dmsimard.com/2014/08/09/a-new-ceph-mirror-on-the-east-coast/ I’m glad to announce that Ceph is now part of the mirrors iWeb provides. It is available in both IPv4 and IPv6 by: http on http://mirror.iweb.ca/ or directly on http://ceph.mirror.iweb.ca/ rsync on ceph.mirror.iweb.ca::ceph The mirror provides 4 Gbps of connectivity and is located on the eastern coast of Canada, more precisely in Montreal, Quebec. We feel this complements very well the principal ceph mirror at ceph. https://dmsimard.com/2014/06/28/deploy-virtual-machines-in-openstack-with-vagrant/ Sat, 28 Jun 2014 00:00:00 +0000 https://dmsimard.com/2014/06/28/deploy-virtual-machines-in-openstack-with-vagrant/ Vagrant is awesome as part of a development workflow. It allows you to leverage virtualization to create automated, repeatable environments. If you find yourself doing the same bootstrapping/deployments often, you’re probably using shell scripts or configuration management tools like Puppet or Chef already. That’s great, but that doesn’t take care of actually creating/installing your server - and that’s where Vagrant comes in the picture. Oh, and you get to keep your current shell scripts or other configuration means because once Vagrant is done creating your server, it can launch them. https://dmsimard.com/2014/06/21/a-use-case-of-tengine-a-drop-in-replacement-and-fork-of-nginx/ Sat, 21 Jun 2014 00:00:00 +0000 https://dmsimard.com/2014/06/21/a-use-case-of-tengine-a-drop-in-replacement-and-fork-of-nginx/ I’ve always been a fan of nginx, it was love at first sight. I tend to use nginx first and foremost as a reverse proxy server for web content and applications. This means that nginx sends your request to backend servers and forwards you their response. Some examples of backend servers I use: php5-fpm for PHP gunicorn or wsgi for Python PSGI/Plack or fastcgi for Perl Now, the cool thing is that these backend servers are good at what they do: serve code and applications written in specific languages. https://dmsimard.com/2014/05/18/hours-of-free-ccna-training-video-material/ Sun, 18 May 2014 00:00:00 +0000 https://dmsimard.com/2014/05/18/hours-of-free-ccna-training-video-material/ About a year ago, Andrew Crouthamel launched a kickstarter to produce a free series of videos about the CCNA Cisco certification. Andrew is a Cisco Certified Academy Instructor and he’s actually been teaching CCNA courses so he’s knowledgeable about the training material. I remember the kickstarter pretty well because the pilot video reminded me a bit of the video series about Reddit’s architecture. I didn’t think much of it at the time. https://dmsimard.com/2014/05/03/inktank-redhat-to-open-source-calamari-the-ceph-web-interface/ Sat, 03 May 2014 00:00:00 +0000 https://dmsimard.com/2014/05/03/inktank-redhat-to-open-source-calamari-the-ceph-web-interface/ You might have heard this already but Redhat made an annoucement last week that they will be acquiring Inktank, the company behind Ceph. Inktank steered Ceph’s development, offered training and provided support through an entreprise package which included Calamari: a web interface to have insight on what is going on inside your cluster. You can have a peek at what Calamari looks like here - in a session from Portland’s Openstack Summit. https://dmsimard.com/2014/05/02/no-network-on-ubuntu-14-04-cloud-image-with-cloud-init/ Fri, 02 May 2014 00:00:00 +0000 https://dmsimard.com/2014/05/02/no-network-on-ubuntu-14-04-cloud-image-with-cloud-init/ I had this weird problem while testing the brand new Ubuntu 14.04 Trusty in my Openstack environment. Same image, same flavors, same specifications - sometimes even the same nova-compute host. I spawn 20 VMs and maybe 50% of the VMs spawned wouldn’t ping. And now, it’s not like I tinkered a lot with the image at all. It’s provided directly from Ubuntu from http://cloud-images.ubuntu.com/trusty/. I started investigating… No relevant information provided by logs of nova-api, neutron, nova-compute. https://dmsimard.com/2014/04/17/puppet-started-by-default-in-ubuntu-1404/ Thu, 17 Apr 2014 00:00:00 +0000 https://dmsimard.com/2014/04/17/puppet-started-by-default-in-ubuntu-1404/ With Ubuntu’s Trusty Tahr imminent release, I’ve been poking around to see what changes there are and how they could have an impact on the work I do. A change I quickly noticed is that when you install the puppet package, the puppet agent now starts by default but will be administratively disabled. This comes from a change in upstream with a commit by the Debian maintainers. You can see that in the postinst, they disable the agent: https://dmsimard.com/2014/03/23/tac-reverse-output-in-linux/ Sun, 23 Mar 2014 00:00:00 +0000 https://dmsimard.com/2014/03/23/tac-reverse-output-in-linux/ You probably use cat everyday. Did you know that tac existed ? It’s the reverse of cat! (no, really). It allows to reverse the contents of a file or, if used with a pipe, the output of the initial command. Let’s see how it works by sorting a directory listing by timestamp… # ls -alt total 4492 drwxrwsr-t 2 owner group 712704 Mar 23 10:32 . -rw-r--r-- 1 owner group 225 Mar 21 16:26 tmp. https://dmsimard.com/2014/03/15/openstack-swift-ceph-openstack-montreal/ Sat, 15 Mar 2014 00:00:00 +0000 https://dmsimard.com/2014/03/15/openstack-swift-ceph-openstack-montreal/ The second meetup of Openstack Montreal, in collaboration with iWeb, Enovance and Savoir-faire Linux, will happen at the Université du Québec à Montréal (UQAM) monday march 17th. It’s with great pleasure that I accepted an invitation from my colleague Rafael Rosa (@rafaelrosafu) to talk about Ceph in the context of Openstack. Our friends at Enovance will be talking about Swift, the object storage project in Openstack. Should definitely be fun, it will in fact be my first public speech ever :D https://dmsimard.com/2014/02/15/quick-hiera-tips/ Sat, 15 Feb 2014 00:00:00 +0000 https://dmsimard.com/2014/02/15/quick-hiera-tips/ If you use Puppet at all, you should be using Hiera if you’re not already ! Hiera is a powerful key/value lookup tool for configuration data for puppet. Here’s some quick tips on how you can use it. Puppet 3 ships with Hiera by default so that’s a good start… Using Hiera For the first example I’ll be showing in this post, I’ll be using the following simplistic files and configuration: https://dmsimard.com/2014/01/18/python-cephclient-now-on-pypi/ Sat, 18 Jan 2014 00:00:00 +0000 https://dmsimard.com/2014/01/18/python-cephclient-now-on-pypi/ Back on January 1st, I wrote about my initiative regarding a client for the Ceph REST API made in python. I’m glad to announce that the client is now available on PyPi and I have just drafted the second release: v0.1.0.2. Check it out and let me know what you think ! https://dmsimard.com/2014/01/07/from-chrome-to-opera/ Tue, 07 Jan 2014 00:00:00 +0000 https://dmsimard.com/2014/01/07/from-chrome-to-opera/ I’ve been using Google Chrome for a couple years, before that I was using Firefox 3. My main reason for switching over at the time was the huge difference in browsing speed. The resource usage of the browsers never really bothered me all that much because I upgraded my hardware frequently. So, anyway, when I heard that Opera was faster than Chrome - I had to take a look. I went a bit on the bleeding edge and tried Opera Next (version 19. https://dmsimard.com/2014/01/01/a-python-client-for-ceph-rest-api/ Wed, 01 Jan 2014 00:00:00 +0000 https://dmsimard.com/2014/01/01/a-python-client-for-ceph-rest-api/ After learning there was an API for Ceph, it was clear to me that I was going to write a client to wrap around it and use it for various purposes. It is still a work in progress and I feel it is not complete and clean enough to publish on pypi yet but I invite you to take a look and tell me what you think ! The client is available on Github under the Apache v2 license here: https://github. https://dmsimard.com/2014/01/01/documentation-for-ceph-rest-api/ Wed, 01 Jan 2014 00:00:00 +0000 https://dmsimard.com/2014/01/01/documentation-for-ceph-rest-api/ I learned that there was a Ceph REST API and I experimented with it a bit. I said the documentation was lacking and I take that back, I didn’t catch on that the API documentation was built into the application. I opened a pull request to make the documentation a bit more explicit about that: https://github.com/ceph/ceph/pull/1026 Here’s what the API documentation currently looks like: Enjoy ! https://dmsimard.com/2013/12/26/updating-xenserver-eject-all-the-dvds/ Thu, 26 Dec 2013 00:00:00 +0000 https://dmsimard.com/2013/12/26/updating-xenserver-eject-all-the-dvds/ I was preparing an update for a Xenserver pool with over 100 VMs on it and I forgot that when you apply patches on Xenserver nodes, you need to eject images/ISO/DVDs from the virtual machines. I start looking through some of the VMs, eject a few manually through Xencenter and think “There must be an easier way..". Thankfully, the folks at Citrix made a simple and efficient bash script that loops through the VMs, verifies if there is a mounted media and allows you to eject it. https://dmsimard.com/2013/12/21/experimenting-with-the-ceph-rest-api/ Sat, 21 Dec 2013 00:00:00 +0000 https://dmsimard.com/2013/12/21/experimenting-with-the-ceph-rest-api/ Like I mentioned in my previous post, Ceph has a REST API now. That opens a lot of possibilities. The Ceph REST API is a WSGI application and it listens on port 5000 by default. This means you can query it directly but you probably want to put a webserver/proxy such a Apache or nginx in front of it. For high availability, you could run ceph-rest-api on several servers and have redundant load balancers pointing to the API endpoints. https://dmsimard.com/2013/12/06/ceph-has-a-rest-api/ Fri, 06 Dec 2013 00:00:00 +0000 https://dmsimard.com/2013/12/06/ceph-has-a-rest-api/ Ceph is a distributed object store and file system designed to provide excellent performance, reliability and scalability. It’s a technology I’ve been following and working with for the past couple months, especially around deploying it with puppet, I really have a feeling it is going to revolutionize the world of storage. I just realized Ceph has a REST API since the Dumpling (0.67) release. This API essentially wraps around the command line tools allowing you to monitor and manage your cluster. https://dmsimard.com/2013/12/02/fixing-mysql-fatal-error-1236/ Mon, 02 Dec 2013 00:00:00 +0000 https://dmsimard.com/2013/12/02/fixing-mysql-fatal-error-1236/ It happens. You have a Master -> Slave MySQL replication and your master crashes for a reason or another. You’re worried about bringing the master back online as soon as possible. When you finally get it back up, you notice the slave is stopped, complaining about a dreaded error that looks a bit like this: The binlog file got corrupted when the master database server crashed. Restoring a backup won’t save you, nor will re-importing the database. https://dmsimard.com/2013/11/29/capture-output-from-parallel-execution-with-fabric/ Fri, 29 Nov 2013 00:00:00 +0000 https://dmsimard.com/2013/11/29/capture-output-from-parallel-execution-with-fabric/ I had this use case where I needed to retrieve the output of a command on multiple servers as part of a python library. At first, I did it with paramiko and it worked well but as the amount of servers grew, it took too much time to run. I knew about fabric and read the docs in hope of finding a solution which I did: the @parallel decorator. I didn’t want to deal with a fabfile and the fab binary, I wanted to include this in my library - fabric can be used as a library but it’s not extremely well documented. https://dmsimard.com/2013/11/26/hello-world/ Tue, 26 Nov 2013 00:00:00 +0000 https://dmsimard.com/2013/11/26/hello-world/ I’m going to try this blog thingy again. I’ve done this in the past and never really got around to it. Let’s see if I’ll be a recidivist. https://dmsimard.com/2013/11/26/how-to-contribute-to-puppet-openstack/ Tue, 26 Nov 2013 00:00:00 +0000 https://dmsimard.com/2013/11/26/how-to-contribute-to-puppet-openstack/ Thanks to the hard work of the puppet-openstack community, Puppet was the preferred method of deployment for Openstack in the latest Openstack User Survey. If you’d like to join in on the fun and contribute, read on ! First things first, a bit of context: Openstack is a modular cloud orchestration platform, self-described as “Open source software for building private and public clouds”. puppet-openstack is a Stackforge project that centralizes the development of puppet modules related to Openstack.