Installing Rally is easy if you're familiar with Python. If you're not, things can get confusing quickly. Many of our users, including Kibana developers, could not care less about Python, and would prefer having a simple executable that embeds CPython and all dependencies. The gold standard here is Go, where binaries are how you ship your code. But we're not going to rewrite Rally in Go :) Instead, we want an installer like the one provided by awscli v2: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html. You just have to unzip and run a simple install script to use it.
Sharp edges
I asked a friend that maintains such an installer. He prefers not to be named, but here's what he had to share (thanks!):
Yeah, it's pyinstaller under the hood. That's been the most straightforward option.
I can give you the sharp edges we've found so far to hopefully save yourself some pain.
- AV Vendors do not like when you upgrade PyInstaller. Every time we've done that, it's required going to Microsoft and other vendors to have the allowlist the new binary. So pick a recent version and plan to stay on it for a while.
- Given that you're bundling a Python interpreter, you may need to upgrade and backport changes in the event of a CPython CVE. We learned the hard way you want this to be fast and easy.
- Linux and Windows are pretty straight forward, but distributing the macOS binary had a somewhat steep learning curve. Mostly around signing, using the right base OS. Elastic may already have those pieces setup though.
(I think only 2 and 3 apply to us here.)
Another thing to watch out for: to provide both esrally and esrallyd, we'll need to use multipackage support: https://pyinstaller.readthedocs.io/en/stable/spec-files.html#multipackage-bundles.
Platforms
We don't support Windows, but we want to support developers on Linux and macOS, as well as running on Linux servers (including ARM). This means we would ideally like to provide packages for:
- Linux x86_64
- Linux aarch64
- macOS x86_64
- macOS arm64
However, since pyinstaller is not a cross-compiler, this means we'll need a macOS environment. A good way to automate this is to use GitHub Actions. Since GitHub Actions do not currently provide ARM-based Macs, that means we would only be providing a x86_64 executable. This could be good enough, since ARM-based Macs can run x86_64 executables with Rosetta 2. Not sure how we should go about signing the executable. We can probably start with ad-hoc signing.
We'll have a similar problem for Linux ARM, but at least it's easy to get our hands on a Graviton instance.
Installing Rally is easy if you're familiar with Python. If you're not, things can get confusing quickly. Many of our users, including Kibana developers, could not care less about Python, and would prefer having a simple executable that embeds CPython and all dependencies. The gold standard here is Go, where binaries are how you ship your code. But we're not going to rewrite Rally in Go :) Instead, we want an installer like the one provided by awscli v2: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html. You just have to unzip and run a simple install script to use it.
Sharp edges
I asked a friend that maintains such an installer. He prefers not to be named, but here's what he had to share (thanks!):
(I think only 2 and 3 apply to us here.)
Another thing to watch out for: to provide both esrally and esrallyd, we'll need to use multipackage support: https://pyinstaller.readthedocs.io/en/stable/spec-files.html#multipackage-bundles.
Platforms
We don't support Windows, but we want to support developers on Linux and macOS, as well as running on Linux servers (including ARM). This means we would ideally like to provide packages for:
However, since pyinstaller is not a cross-compiler, this means we'll need a macOS environment. A good way to automate this is to use GitHub Actions. Since GitHub Actions do not currently provide ARM-based Macs, that means we would only be providing a x86_64 executable. This could be good enough, since ARM-based Macs can run x86_64 executables with Rosetta 2. Not sure how we should go about signing the executable. We can probably start with ad-hoc signing.
We'll have a similar problem for Linux ARM, but at least it's easy to get our hands on a Graviton instance.