Setting up Intel SGX

2023-09-29T00:00:00+00:00

Setting up Intel SGX</h1>
Since my research partially focuses on Intel SGX, I often find myself having to go through the process of setting up Intel SGX on various machines. Unfortunately, this process consists of multiple steps and has changed quite a bit over the years. In addition, most documentation focuses on specific Linux distributions, usually older versions, such as Ubuntu 20.04 LTS, and many resources handwave the process by providing a Docker image instead. In the meanwhile the Intel SGX driver has finally been mainlined into the Linux kernel ^{[1], and as such the linux-sgx-driver</a> is no longer necessary.
As a result, my notes focus on setting up Intel SGX on a system with a Linux kernel that ships the Intel SGX driver (Linux 5.11 and later).}
Furthermore, there are currently two different implementations of Intel SGX: EPID (Enhanced Privacy ID) and DCAP (Data Center Attestation Primitives). I won't go into the details of the differences between these two implementations, but you can read more about these in the EPID whitepaper</a> and the DCAP whitepaper</a>. While developers should aim to use DCAP-based Intel SGX rather than EPID-based Intel SGX, as the latter is slowly being phased out (the Intel Xeon E-23xx series is the latest platform to support EPID-based Intel SGX), my notes will focus on setting up Intel SGX on EPID-based platforms as that is what I have mostly been working with. However, I might update these notes in the future for Intel DCAP.

First, we will need to find a platform that offers Intel SGX. This depends on both the processor and the motherboard, and more specifically the UEFI implementation provided by the OEM.

To determine if a given processor supports Intel SGX or not, you can look up the specification of the processor on https://ark.intel.com</a>. If Intel SGX is supported by the processor, there will be an entry for Intel Software Guard Extensions (Intel SGX) with either Yes with Intel ME or Yes with Intel SPS.

Here is a list of processor families with some processors in each family supporting the EPID-based version of Intel SGX:

Skylake</a> (e.g. Intel Core i7-6700K)</li>
Kaby Lake</a> (e.g. Intel Core i7-7700K)</li>
Coffee Lake</a> (e.g. Intel Core i7-8700K)</li>
Coffee Lake Refresh</a> (e.g. Intel Core i9-9900K)</li>
Comet Lake</a> (e.g. Intel Core i9-10900K)</li>
Rocket Lake (Intel Xeon E-23xx series)</a> (e.g. Intel Xeon E-2314)</li>
Ice Lake</a> (e.g. Intel Core i7-1065G7)</li>
Tiger Lake (Intel Core 11th gen B series)</a> (e.g. Intel Core i9-11900KB)</li>
Gemini Lake</a> (e.g. Intel Celeron J4105)</li>
Gemini Lake Refresh</a> (e.g. Intel Celeron J4125)</li> </ul>
Here is a similar list for processors supporting the DCAP-based Intel SGX:

Ice Lake SP</a> (e.g. Intel Xeon Silver 4314)</li>
Sapphire Rapids SP</a> (e.g. Intel Xeon Silver 4416+)</li> </ul>
Determining whether the motherboard itself supports Intel SGX beforehand is a little bit more difficult, but it usually helps to check the motherboard manual. Usually the manual will have an overview of all the options provided by the BIOS/UEFI, including an option for Software Guard eXtensions (SGX) which can be set to Disabled, Software-Controlled or Enabled. If such an option is not available, I recommend looking for a different motherboard/platform instead.

Depending on the platform, it may not be possible to set the Software Guard eXtensions (SGX) setting to Enabled. Instead some platforms only allow you to configure the setting to Software-Controlled or Disabled. This is perfectly fine, as we can still enable Intel SGX from the operating system when the setting is set to Software-Controlled. </blockquote>
Enabling Intel SGX</h1>
Enter the UEFI setup menu to enable Intel SGX. A quick way of entering the UEFI setup menu on Ubuntu, is by invoking the following command from a terminal:
sudo systemctl reboot --firmware-setup </code></pre> Alternatively, you can enter the UEFI setup menu by pressing a specific key immediately after the machine is powered on (usually F2 or Del). In my case the option to enable Intel SGX can be accessed by selecting the Advanced tab, followed by the CPU Configuration menu, where it is available as Software Guard Extensions (SGX). However, the option may be available elsewhere in your UEFI setup menu depending on the exact platform. Make sure that the option is set to either Enabled or Software-Controlled as shown in Figure 1. Figure 1: The Software Guard Extensions (SGX) setting in the UEFI setup menu. </figcaption> </figure> </center> On some platforms, there may also be a setting named SGX Launch Control Policy with the options Intel Locked, Locked and Unlocked. Ensure that this setting is set to Unlocked, otherwise the SGX kernel driver will fail to load. Software-Controlled SGX</h2> If the option is set to Software-Controlled, then you will need to clone, build and run the sgx-software-enable tool</a> to enable Intel SGX. Make sure to install the required build dependencies: sudo apt -y install build-essential git </code></pre> Clone the repository as follows: git clone https://github.com/intel/sgx-software-enable </code></pre> Then cd</code> into the directory: cd sgx-software-enable </code></pre> Run make</code> to build the tool: make </code></pre> Run the following command to enable Intel SGX: sudo ./sgx-enable </code></pre> Finally reboot the system for the setting to take effect: sudo reboot </code></pre> At this point you should have a platform with support for Intel SGX and where the feature has been enabled. Checking your Intel SGX Setup</h1> Since setting up Intel SGX consists of multiple steps, it is useful if we can check this using a diagnostics tool. Fortunately, Fortanix</a> provides such a tool called sgx-detect</code>. However, to install the tool, we will first need to install Rust. First, make sure to install curl</code> (not the snap</code> version): sudo apt -y install curl </code></pre> Then we can install rustup</code> to manage our Rust toolchain(s): curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh </code></pre> Source the newly added environment variables: source "$HOME/.cargo/env" </code></pre> To build the sgx-detect</code> tool, we will need to install the nightly version of the Rust toolchain: rustup install nightly </code></pre> In addition, we will also need to add the x86_64-fortanix-unknown-sgx</code> target: rustup target add x86_64-fortanix-unknown-sgx --toolchain nightly </code></pre> We will also need to install some build dependencies to the build the sgx-detect</code> tool: sudo apt -y install build-essential pkg-config libssl-dev protobuf-compiler </code></pre> Finally, we can use Cargo to install the Fortanix tools, including sgx-detect</code>: cargo install fortanix-sgx-tools sgxs-tools </code></pre> To run the tool with the appropriate runner, we will need to add the following to .cargo/config</code>: [target.x86_64-fortanix-unknown-sgx] runner = "ftxsgx-runner-cargo" </code></pre> Then we should be able to run sgx-detect</code>, which should output something like the following: ✔ SGX instruction set ✔ CPU support ✔ CPU configuration ✔ Enclave attributes ✔ Enclave Page Cache SGX features ✔ SGX2 ✔ EXINFO ✔ ENCLV ✔ OVERSUB ✔ KSS Total EPC size: 188.0MiB ？ Flexible launch control ✔ CPU support ？ CPU configuration ？ Able to launch production mode enclave ✘ SGX system software ✘ SGX kernel device ✘ libsgx_enclave_common ✘ AESM service 🕮 Flexible launch control > CPU configuration Your hardware supports Flexible Launch Control, but whether it's enabled could not be determined. More information might be available by re-running this program with sudo. Would you like to do that? (not supported yet) (run with `--verbose` for more details) More information: https://edp.fortanix.com/docs/installation/help/#flc-cpu-configuration 🕮 SGX system software > SGX kernel device Permission denied while opening the SGX device (/dev/sgx/enclave, /dev/sgx or /dev/isgx). Make sure you have the necessary permissions to create SGX enclaves. If you are running in a container, make sure the device permissions are correctly set on the container. (run with `--verbose` for more details) More information: https://edp.fortanix.com/docs/installation/help/#sgx-driver 🕮 SGX system software > AESM service AESM could not be contacted. AESM is needed for launching enclaves and generating attestations. Please check your AESM installation. (run with `--verbose` for more details) More information: https://edp.fortanix.com/docs/installation/help/#aesm-service </code></pre> As we can see, we are still missing parts of the software stack, such as the AESM service and the driver. Intel SGX SDK</h1> Up next, we will build and install the Intel SGX SDK for Linux. First, we will have to install a number of build dependencies to build the Intel SGX SDK for Linux: sudo apt -y install build-essential ocaml ocamlbuild automake autoconf libtool wget python-is-python3 libssl-dev git cmake perl libcurl4-openssl-dev protobuf-compiler libprotobuf-dev debhelper cmake reprepro unzip pkgconf libboost-dev libboost-system-dev libboost-thread-dev lsb-release libsystemd0 </code></pre> To build the Intel SGX SDK for Linux, I recommend using one of the reproducible tags of the linux-sgx repository</a>, as these are used to build the Docker images. As of writing, the latest version is sgx_2.21_reproducible. We can clone that tag as follows: git clone https://github.com/intel/linux-sgx.git -b sgx_2.21_reproducible </code></pre> Then cd</code> into the repository: cd linux-sgx </code></pre> To download the submodules and the prebuilt binaries, we need to invoke make preparation</code> as follows: make preparation </code></pre> Then we can build the installer for the Intel SGX SDK for Linux as follows: make sdk_install_pkg </code></pre> If the installer built successfully, you should be able to run it as follows: ./linux/installer/bin/sgx_linux_x64_sdk_2.21.100.1.bin </code></pre> It will prompt you whether you want to install it in the current directory: Do you want to install in current directory? [yes/no] : </code></pre> Type no, followed by Enter. Instead type your home directory (e.g. /home/user</code>), such that it will be installed as /home/user/sgxsdk</code>. It should output something like the following: Unpacking Intel SGX SDK ... done. Verifying the integrity of the install package ... done. Installing Intel SGX SDK ... done. /tmp/sgx-sdk-72hs80 ~ ~ uninstall.sh script generated in /home/user/sgxsdk Installation is successful! The SDK package can be found in /home/user/sgxsdk Please set the environment variables with below command: source /home/user/sgxsdk/environment </code></pre> Run the following to source the newly created environment variables: source ~/sgxsdk/environment </code></pre> Finally, add the following to the bottom of your ~/.bashrc</code>: . "$HOME/sgxsdk/environment" </code></pre> Intel SGX PSW</h1> Now that we have the Intel SGX SDK installed, we can build the Intel SGX PSW which contains the platform software, i.e. the AESM service that needs to be installed. First, make sure your terminal is pointing to the linux-sgx</code> repository: cd linux-sgx </code></pre> Then we can simply build the PSW installer package as follows: make psw_install_pkg </code></pre> If everything went well, you should be able to run the installer as root: sudo ./linux/installer/bin/sgx_linux_x64_psw_2.21.100.1.bin </code></pre> After installing the Intel SGX PSW package, we should have the AESM service up and running. Invoking the sgx-detect</code> command should now tell us that we have a SGX kernel device, libsgx_enclave_common as well as the AESM service: Detecting SGX, this may take a minute... ✔ SGX instruction set ✔ CPU support ✔ CPU configuration ✔ Enclave attributes ✔ Enclave Page Cache SGX features ✔ SGX2 ✔ EXINFO ✔ ENCLV ✔ OVERSUB ✔ KSS Total EPC size: 188.0MiB ✔ Flexible launch control ✔ CPU support ？ CPU configuration ✔ Able to launch production mode enclave ✘ SGX system software ✔ SGX kernel device (/dev/sgx_enclave) ✔ libsgx_enclave_common ✔ AESM service ✘ Able to launch enclaves ✘ Debug mode ✔ Production mode 🕮 SGX system software > Able to launch enclaves > Debug mode The enclave could not be launched. (run with `--verbose` for more details) More information: https://edp.fortanix.com/docs/installation/help/#run-enclave-debug </code></pre> If you see the above output where you may not be able to launch (debug) enclaves, it may help to reboot your system: sudo reboot </code></pre> Invoking the sgx-detect</code> command should output the following: Detecting SGX, this may take a minute... ✔ SGX instruction set ✔ CPU support ✔ CPU configuration ✔ Enclave attributes ✔ Enclave Page Cache SGX features ✔ SGX2 ✔ EXINFO ✔ ENCLV ✔ OVERSUB ✔ KSS Total EPC size: 188.0MiB ✔ Flexible launch control ✔ CPU support ？ CPU configuration ✔ Able to launch production mode enclave ✔ SGX system software ✔ SGX kernel device (/dev/sgx_enclave) ✔ libsgx_enclave_common ✔ AESM service ✔ Able to launch enclaves ✔ Debug mode ✔ Production mode ✔ Production mode (Intel whitelisted) You're all set to start running SGX programs! </code></pre> At this point you should be able to build and run any of the examples in ~/sgxsdk/SampleCode</code>. References</h1> https://www.phoronix.com/news/Intel-SGX-Linux-5.11</a></li> </ol>

Building the Linux Asahi kernel 2023-03-20T00:00:00+00:00 Introduction</h1> Since the announcement of the Asahi Linux Alpha Release</a>, the team behind Asahi Linux provides a very simple script that you can run from a terminal on MacOS to install AsahiLinux: curl https://alx.sh | sh </code></pre> If you carefully follow the instructions while running the shell script, you should end up with a functional ArchLinux-based Linux system (depending on the option you chose during the installation, of course). Since support for Apple Silicon is still being upstreamed to the Linux kernel, ArchLinux does not yet support Apple Silicon out of the box. Instead Asahi Linux provides its own packages for ArchLinux to support Apple Silicon hardware, including their fork of the Linux kernel through the linux-asahi</code> package. These PKGBUILDs can be found on Github</a>. As such, if you are trying to build a Linux kernel module, for example, you may need to install the linux-asahi-headers</code> package (not the linux-aarch64-headers</code> package): sudo pacman -Syu linux-asahi-headers </code></pre> Sometimes it might be interesting to try out a newer version of the Linux kernel, or to run the Linux kernel with custom patches. This guide shows how to build m1n1 and Linux Asahi from their respective source code using the provided PKGBUILDs, as well as how to modify the PKGBUILDs. Building m1n1 and the Linux Asahi kernel from source</h1> First, we clone the PKGBUILDs repository: git clone https://github.com/AsahiLinux/PKGBUILDs.git </code></pre> Then we point the terminal to the m1n1 directory: pushd PKGBUILDs/m1n1 </code></pre> Run makepkg -s</code> to build the m1n1 package: makepkg -s </code></pre> If everything went well, you should now be able to install the m1n1 package using pacman</code>: sudo pacman -U m1n1-1.2.3-1-aarch64.pkg.tar.xz </code></pre> Finally, run popd</code> to go back to the original directory. To build the Linux kernel, we need to install version 1.62 of the Rust toolchain (at the time of writing). We first install rustup</code> through pacman</code>: sudo pacman -Syu rustup </code></pre> Then we use rustup to install the toolchain: rustup toolchain install 1.62.0 </code></pre> Similar to building the m1n1 package, we point the terminal to the linux-asahi directory: pushd PKGBUILDs/linux-asahi </code></pre> Run makepkg -s</code> with MAKEFLAGS</code> set to -j8</code> (where 8 is the number of CPU cores): MAKEFLAGS="-j8" makepkg -s </code></pre> Install the linux-asahi package using pacman</code>: sudo pacman -U linux-asahi-6.1.asahi3-1-aarch64.pkg.tar.xz linux-asahi-headers-6.1.asahi3-1-aarch64.pkg.tar.xz </code></pre> You may also want to install the linux-asahi-edge if you are normally running linux-asahi-edge (make sure you have mesa-asahi-edge</code> installed as well): sudo pacman -U linux-asahi-edge-6.1.asahi3-1-aarch64.pkg.tar.xz </code></pre> Update the GRUB configuration file to reference the newly installed Linux kernel: sudo update-grub </code></pre> Finally, run popd</code> to go back to the original directory. If everything went well, you should be able to boot the system with the new Linux kernel. Updating m1n1 and Linux Asahi</h1> The released versions of m1n1 can be found on Github</a>. For instance, to install m1n1 1.2.6, we can simply edit pkgver</code> in PKGBUILDs/m1n1/PKGBUILD</code>: pkgname=m1n1 pkgver=1.2.6 pkgrel=1 </code></pre> Then we need to run updpkgsums</code> to update the checksums and we should be able to install the package as before: pushd PKGBUILDs/m1n1 updpkgsums makepkg -s sudo pacman -U m1n1-v1.2.6-1-aarch64.pkg.tar.xz popd </code></pre> Similarly, the released versions of Linux Asahi can be found on Github</a> too. To install Linux Asahi 6.2.11, we can set _rcver</code> to 6.2 and _asahirel</code> to 11 in PKGBUILDs/linux-asahi/PKGBUILD</code>: _rcver=6.2 #_rcrel= _asahirel=11 pkgrel=1 </code></pre> Also update _m1n1_version</code>, if you changed the version of m1n1: _m1n1_version=1.2.6 </code></pre> Update the package checksums and install the new package as follows: pushd PKGBUILDs/linux-asahi updpkgsums MAKEFLAGS="-j8" makepkg -s sudo pacman -U linux-asahi-6.2.asahi11-1-aarch64.pkg.tar.xz linux-asahi-edge-6.2.asahi11-1-aarch64.pkg.tar.xz linux-asahi-headers-6.2.asahi11-1-aarch64.pkg.tar.xz popd sudo update-grub </code></pre> If everything went well, you should now be able to boot the system with linux-asahi-6.2-11. In addition, running uname -a</code> should show something like the following: Linux asahi 6.2.0-asahi-11-1-edge-ARCH #2 SMP PREEMPT_DYNAMIC Mon, 20 Mar 2023 07:17:32 +0000 aarch64 GNU/Linux </code></pre> Patching the Linux kernel</h1> To start writing and applying patches for the Linux kernel, we need to clone the Linux source code for the version that we installed using the PKGBUILDs (e.g. the tag for linux-asahi-6.2-11</code> is asahi-6.2-11</code>). We will make sure to use --depth=1</code> to minimize the amount of disk space used: git clone https://github.com/AsahiLinux/linux -b asahi-6.2-11 --depth=1 </code></pre> After modifying the Linux source code in the linux</code> directory, we can simply create a patch using git diff</code>: git diff > ../PKGBUILDs/asahi-linux/0001-my-patch.patch </code></pre> Add the patch(es) to source</code> in PKGBUILDs/linux-asahi/PKGBUILD</code>: source=( https://github.com/AsahiLinux/linux/archive/${_commit_id}.tar.gz config # the main kernel config file config.edge # overrides for linux-asahi-edge 0001-my-patch.patch </code></pre> Update the package checksums: pushd PKGBUILDs/linux-asahi updpkgsums </code></pre> Make sure the patch</code> command is available: sudo pacman -Syu patch </code></pre> Build the package (you may need to pass -f</code> if you already had the package built); MAKEFLAGS="-j8" makepkg -s </code></pre> Install the patched kernel using pacman</code>: sudo pacman -U linux-asahi-6.2.asahi11-1-aarch64.pkg.tar.xz linux-asahi-edge-6.2.asahi11-1-aarch64.pkg.tar.xz linux-asahi-headers-6.2.asahi11-1-aarch64.pkg.tar.xz </code></pre> Update the GRUB configuration file to reference the newly installed Linux kernel: sudo update-grub </code></pre> Finally, run popd</code> to go back to the original directory. If everything went well, you should be able to boot the system with the patched Linux kernel. Building Linux kernel modules</h1> If you are building Linux kernel modules for Asahi Linux, you need to make sure the build directory is present in /lib/modules/$(uname -r)/build</code>. For the base config, you can simply copy the build directory from the PKGBUILD you built locally: sudo cp -r ~/PKGBUILDs/linux-asahi/src/linux-asahi-6.2-11/build/base /lib/modules/6.2.0-asahi-11-1-ARCH/build </code></pre> For the edge config, the command would be as follows: sudo cp -r ~/PKGBUILDs/linux-asahi/src/linux-asahi-6.2-11/build/edge /lib/modules/6.2.0-asahi-11-1-edge-ARCH/build </code></pre> You can also create symbolic links if you prefer that instead. With the build</code> directory available, you should be able to simply build external Linux kernel modules. Setting up Prosody 2023-03-04T00:00:00+00:00 Installation</h1> Install Prosody: apk add prosody </code></pre> We also need to install lua-unbound to use prosodyctl</code>: apk add lua-unbound@testing </code></pre> Configuration</h1> The configuration file for Prosody can be found in /etc/prosody/prosody.cnf.lua</code>. Getting started with Prosody is relatively straight-forward. Just comment out the VirtualHost</code> directive for "localhost", and add one for your domain instead: --VirtualHost "localhost" VirtualHost "example.com" </code></pre> It is also possible to add more than one domain to your configuration. If we already set up Let's Encrypt certificates, we can simply run the following command to import them: /usr/bin/prosodyctl --root cert import /etc/letsencrypt/live </code></pre> However, when renewing the certificates, it is a good idea to add this as a --deploy-hook</code> to your renewal command, e.g.: certbow renew --deply-hook "prosodyctl --root cert import /etc/letsencrypt/live" </code></pre> The Prosody documentation has further information about setting up certificates</a> as well as setting up Let's Encrypt with Prosody</a>. Start the service: /etc/init.d/prosody start rc-update add prosody </code></pre> Create an XMPP account</a>: prosodyctl adduser [email protected] </code></pre> Port forwarding</h1> The Prosody documentation provides a list of ports</a> used by Prosody that we need to set up using ufw</code>: ufw allow 5000/tcp ufw allow 5222/tcp ufw allow 5269/tcp ufw allow 5280/tcp ufw allow 5281/tcp ufw allow 5347/tcp ufw allow 5582/tcp </code></pre> Setting up file sharing</h1> Setting up file sharing is as simple as enabling the http_file_share</code> component: Component "xmpp-serve.example.com" "http_file_share" http_file_share_expires_after = 3 * 24 * 60 * 60 </code></pre> In addition, you can set a number of parameters such as when the file expires (e.g. after three days). Here is a list of the available parameters</a>. If you have other domains available, then make sure to add the following to each of them below their VirtualHost</code> directive to enable file sharing: disco_items = { { "xmpp-serve.exmaple.com", "file sharing service" } } </code></pre> Building a Linux kernel with Rust support on Gentoo 2023-01-25T00:00:00+00:00 Some background</h1> After watching Alex Gaynor and Geoffrey Thomas their presentation</a> and video</a> about writing Linux kernel modules, I have been trying to keep a close eye on their progress. Especially since I have been playing around with the now archived linux-kernel-module-rust</a> repository, to port over one of my toy Linux kernel modules to fiddle around with physical memory and page tables from C to Rust. Three years after the presentation we are now looking at initial Rust support being part of the Linux 6.1 kernel</a>. Shortly after that Raphael Nestler</a> wrote some interesting blog posts about building out-of-tree Rust kernel modules on ArchLinux (you can read part one</a> and part two</a> here). These articles got me wondering about whether this is possible on Gentoo. Of course, while we focus on Gentoo here, most of this knowledge is also applicable to other Linux distributions. Installing the Linux kernel</h1> As not everyone is familiar with Gentoo, we will first take a look at the various ways of installing a Linux kernel. In fact, many people, including many Gentoo users, do not seem to be aware that Gentoo ships sys-kernel/gentoo-kernel-bin</a>, which is a precompiled binary of the Linux kernel with Gentoo-specific patches which has been available since January 10, 2020</a>. However, while a precompiled binary of the Linux kernel exists, many users instead compile the Linux kernel from source. While Gentoo offers many flavors of the kernel source, the typical ones are sys-kernel/vanilla-sources</a>, which provides the Linux kernel source with no modifications, sys-kernel/gentoo-sources</a>, which provides the Linux kernel source with Gentoo-specific patches and sys-kernel/git-sources</a>, which provides the Linux kernel source for current release candidates. Once installed, you would use eselect kernel list</code> to list the installed kernel sources and eselect kernel set 1</code> to set the first entry in the list to be the default kernel source to use. More specifically, eselect kernel set 1</code> creates a symbolic link from /usr/src/linux</code> to the directory containing that specific kernel source. After setting up the kernel source, the next step is to get a working kernel configuration. A relatively straightforward to obtain a working kernel configuration is from an existing Linux kernel that can properly boot your system, such as the kernel from another distribution like Ubuntu or ArchLinux, or from sys-kernel/gentoo-kernel-bin</a>. Once you having a working kernel configuration, you would typically copy it over using zcat /proc/config.gz > .config</code> or from your /boot</code> directory, and run make olddefconfig</code> before building the Linux kernel. Finally, you can perform further tweaking of the kernel configuration using make menuconfig</code>. If you want to manually build and install the Linux kernel and the kernel modules, you can do so using the following commands: make -j16 make INSTALL_MOD_STRIP=1 modules_install make install </code></pre> To build your own initramfs, you can look into tools like dracut</a>. Alternatively, you can use genkernel</a> to build a Linux kernel and initramfs by simply running something like the following command: genkernel --clean --menuconfig --install all </code></pre> After building and installing the Linux kernel using either way, we need to run the following command to update GRUB's configuration file: grub-mkconfig -o /boot/grub/grub.cfg </code></pre> What are the prerequisites to enable Rust in the Linux kernel?</h1> The first part</a> of Raphael's blog post provides some good pointers of where to start. More specifically, it points us to a basic template</a> for an out-of-tree Linux kernel module in Rust that we can use. One of the requirements listed there is that our Linux kernel has to be built with CONFIG_RUST=y</code>. To check this we can simply use grep</code> on our kernel configuration: grep CONFIG_RUST .config </code></pre> If we run the above command, there is simply no output, which indicates that the option is not set. So let's run make menuconfig</code> and press /, type in Rust and press Enter to look for CONFIG_RUST</code>. It should look something like the image below: Figure 1: make menuconfig</code> showing the CONFIG_RUST</code> option and its dependencies. </figcaption> </figure> </center> More specifically, it shows the following text: Symbol: RUST [=n] Type : bool Defined at init/Kconfig:1913 Prompt: Rust support Depends on: HAVE_RUST [=y] && RUST_IS_AVAILABLE [=n] && !MODVERSIONS [=y] && !GCC_PLUGINS [=y] && !RANDSTRUCT [=n] && !DEBUG_INFO_BTF [=n] Location: (1) -> General setup -> Rust support (RUST [=n]) Selects: CONSTRUCTORS [=n] </code></pre> Looking more closely at Depends on:</code>, we can see the other configuration options that need to be set or not for CONFIG_RUST=y</code>. In our case, RUST_IS_AVAILABLE</code> is set to n</code>, while MODVERSIONS</code> and GCC_PLUGINS</code> are set to y</code>. So let's tackle each of these issues, one at a time. Ensuring a proper Rust toolchain is available</h1> The Linux documentation</a> provides some helpful instructions to figure out how to check if a proper Rust toolchain is available or not, among other things. In fact, we can run make rustavailable</code> to check if our system is set up accordingly: make rustavailable *** *** Rust bindings generator 'bindgen' could not be found. *** make: *** [Makefile:1800: rustavailable] Error 1 </code></pre> It turns out that we don't have the dev-util/bindgen</a> package installed. So let's unmask it, since there is no stable version available yet: echo "dev-util/bindgen ~amd64" >> /etc/portage/package.accept_keywords/rust </code></pre> Then we can install it by running emerge bindgen</code>. From looking at the output, it turns out we also need to enable the rustfmt</code> USE-flag: Calculating dependencies... done! [ebuild R ] dev-lang/rust-bin-1.66.1 USE="rustfmt*" [ebuild R ] virtual/rust-1.66.1 USE="rustfmt*" [ebuild N ~] dev-util/bindgen-0.62.0 USE="-debug" ABI_X86="32 (64) (-x32)" The following USE changes are necessary to proceed: (see "package.use" in the portage(5) man page for more details) # required by dev-util/bindgen-0.62.0::gentoo # required by bindgen (argument) >=virtual/rust-1.66.1 rustfmt # required by virtual/rust-1.66.1::gentoo # required by dev-util/bindgen-0.62.0::gentoo # required by bindgen (argument) >=dev-lang/rust-bin-1.66.1 rustfmt </code></pre> So let's enable that USE-flag: echo "virtual/rust rustfmt" >> /etc/portage/package.use/rust echo "dev-lang/rust-bin rustfmt" >> /etc/portage/package.use/rust </code></pre> Now we run emerge bindgen</code> again, and it succeeds, and from running make rustavailable</code> again, we can see that we now have a working Rust compiler as well as the bindgen crate installed: *** *** Rust compiler 'rustc' is too new. This may or may not work. *** Your version: 1.66.1 *** Expected version: 1.62.0 *** *** *** Rust bindings generator 'bindgen' is too new. This may or may not work. *** Your version: 0.62.0 *** Expected version: 0.56.0 *** *** *** Source code for the 'core' standard library could not be found *** at '/opt/rust-bin-1.66.1/lib/rustlib/src/rust/library/core/src/lib.rs'. *** Rust is available! </code></pre> Apparently, we should enable the rust-src</code> USE-flag for dev-lang/rust</a>: echo "dev-lang/rust-bin rust-src" >> /etc/portage/package.use/rust </code></pre> In addition, the versions of the Rust compiler and the bindgen are a bit new compared to what make rustavailable</code> expects. This makes sense, since the Linux kernel has to support distributions that are not rolling release distributions like Debian</a>, which currently ships Rust 1.63 and 1.64, and Ubuntu</a>, which currently ships Rust 1.61. Unfortunately, if we try to build the Linux kernel with Rust 1.65 or 1.66.1 However, if we run eix dev-lang/rust-bin</code> and eix virtual/rust</code>, we see that Gentoo only provides 1.65.0 and 1.66.1 at the time of writing. Fortunately, Rust 1.62.0 used to be available at some point</a>, thus we can simply run a local ebuild repository and grab the old ebuilds. Setting up a local repository</h1> The Gentoo wiki</a> already provides instructions on how to set up a local repository, but it boils down to running emerge pkgdev</code> to make sure that we have pkgdev</code> installed, and running eselect repository create local</code> to set up the skeleton directory structure for our ebuild repository. Our ebuild repository can then be found at /var/db/repos/local</code>: cd /var/db/repos/local </code></pre> We then need to create the appropriate directories for dev-lang/rust-bin</code> and virtual/rust</code>: mkdir -p dev-lang/rust-bin mkdir -p virtual/rust </code></pre> Then we can download the old ebuilds from their respective commits for dev-lang/rust-bin</a> and virtual/rust</a> as follows: wget https://gitweb.gentoo.org/repo/gentoo.git/plain/dev-lang/rust-bin/rust-bin-1.62.0.ebuild?id=0d7d7b32c9b472a25fbe998ed6ad196501d383a1 -O dev-lang/rust-bin/rust-bin-1.62.0.ebuild wget https://gitweb.gentoo.org/repo/gentoo.git/plain/virtual/rust/rust-1.62.0.ebuild?id=6302c360cd3bbf54c716d1642e329d14046fd722 -O virtual/rust/rust-1.62.0.ebuild </code></pre> We then need to create a package manifest for both packages: pushd dev-lang/rust-bin pkgdev manifest popd pushd virtual/rust pkgdev manifest popd </code></pre> Make sure to run cp /usr/share/portage/config/repos.conf /etc/portage/repos.conf</code> if the following error appears: pkgdev: error: repos.conf: default repo 'gentoo' is undefined or invalid </code></pre> Finally, we can run eix-update</code> to update our eix database, and the packages should be available to install. Since bindgen-0.56.0 is still available at the time of writing, we don't have to create our own ebuild for bindgen. Now to ensure that we have the appropriate versions installed, we can pin the versions we want by writing the following to /etc/portage/package.mask/kernel</code>: >dev-lang/rust-bin-1.62.0 >dev-util/bindgen-0.56.0 >virtual/rust-1.62.0 </code></pre> We should also make sure to unmask our ebuilds: echo "dev-lang/rust-bin ~amd64" >> /etc/portage/package.accept_keywords/rust echo "virtual/rust ~amd64" >> /etc/portage/package.accept_keywords/rust </code></pre> Then we can simply install Rust and bindgen as follows: emerge rust-bin bindgen virtual/rust </code></pre> Now if we run make rustavailable</code> again, it should output the following: Rust is available! </code></pre> Building the kernel with Clang</h1> We can try to set CONFIG_GCC_PLUGINS=n</code> by deselecting the following option in make menuconfig</code> to set CONFIG_RUST=y</code>: General architecture-dependent options ---> [ ] GCC plugins </code></pre> However, building the Linux kernel with gcc seems to result in the following error: arch/x86/tools/insn_decoder_test: error: malformed line 1318434: tBb_+0x102> </code></pre> Let's try building the kernel (and our initramfs) with Clang instead, as that is the recommended way of building the Linux kernel to enable Rust support. Especially since the bindgen crate relies on libclang, thus LLVM and Clang to produce Rust bindings from C source code, anyway. We can simply achieve this by changing the /etc/genkernel.conf</code> file. First, we will add the LLVM=1 LLVM_IAS=1</code> options to MAKEOPTS</code>: MAKEOPTS="$(portageq envvar MAKEOPTS) LLVM=1 LLVM_IAS=1" </code></pre> Then we make sure to point the toolchain used to build the kernel to LLVM/Clang: KERNEL_AS="llvm-as" KERNEL_AR="llvm-ar" KERNEL_CC="clang" KERNEL_LD="ld.lld" KERNEL_NM="llvm-nm" KERNEL_OBJCOPY="llvm-objcopy" KERNEL_OBJDUMP="llvm-objdump" KERNEL_READELF="llvm-readelf" KERNEL_STRIP="llvm-strip" KERNEL_RANLIB="llvm-ranlib" </code></pre> We also do the same for building the initramfs: UTILS_AS="llvm-as" UTILS_AR="llvm-ar" UTILS_CC="clang" UTILS_CXX="clang++" UTILS_LD="ld.lld" UTILS_NM="llvm-nm" UTILS_OBJCOPY="llvm-objcopy" UTILS_OBJDUMP="llvm-objdump" UTILS_READELF="llvm-readelf" UTILS_RANLIB="llvm-ranlib" </code></pre> Note that we do not set the UTILS_STRIP="llvm-strip"</code> as this currently results in the following error: llvm-strip: error: Link field value 34 in section .rela.plt is not a symbol table * ERROR: Failed to strip '/var/tmp/genkernel/gk_dZRzC8UH/util-linux/image/sbin/blkid'! * ERROR: create_initramfs(): append_data(): append_util-linux(): populate_binpkg(): gkbuild(): Failed to create binpkg of util-linux-2.37.2! * Please consult '/var/log/genkernel.log' for more information and any * errors that were reported above. </code></pre> As genkernel</code> typically caches some of the build artifacts, we should also make sure to clear the cache to enforce genkernel</code> to rebuild the kernel and initramfs. We can do this by setting the following option to yes</code>: CLEAR_CACHEDIR="yes" </code></pre> Now we should be able to run genkernel</code> as follows: genkernel --clean --menuconfig --install all </code></pre> Building the kernel with Rust support</h1> Finally, we need to make sure that MODVERSIONS=n</code>, thus we need to deselect the following option in make menuconfig</code>: Enable loading module support ---> [ ] Module versioning support </code></pre> Now we should be able to set CONFIG_RUST=y</code>: General setup ---> [*] Rust support </code></pre> Then we can simply save the kernel configuration and let genkernel</code> build and install the kernel. Kernel panic?</h1> After building and installing both the kernel and our initramfs using genkernel</code>, we should now be able to reboot into our kernel. Unfortunately, it seems to result in a kernel panic because busybox sh</code> seems to run into segmentation fault. What is going on? One quick way of testing what is going on with busybox sh</code>, we can build and install sys-apps/busybox</a> using Clang and then run it with dev-util/valgrind</a>. Fortunately, the Gentoo wiki</a> explains how to set up Portage to compile and build busybox with Clang. To do so, we first need to create /etc/portage/env/compiler-clang</code> with the following contents: COMMON_FLAGS="-O2 -march=native" CFLAGS="${COMMON_FLAGS}" CXXFLAGS="${COMMON_FLAGS}" CC="clang" CXX="clang++" AR="llvm-ar" NM="llvm-nm" RANLIB="llvm-ranlib" LDFLAGS="${LDFLAGS} -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind -Wl,--as-needed" </code></pre> Then we can tell Portage to build busybox with Clang: echo "sys-apps/busybox compiler-clang" >> /etc/portage/package.env/compiler </code></pre> Then we install both busybox and valgrind using emerge</code>. Running busybox sh</code> seems to result in the same Segmentation fault.</code> If we run valgrind busybox sh</code> instead, we get a better idea of what is going on: valgrind: mmap(0x143000, 1093632) failed in UME with error 22 (Invalid argument). valgrind: this can be caused by executables with very large text, data or bss segments. </code></pre> Fortunately, the following bug report</a> seems to describe a similar issue. Let's try installing busybox without PIE set (as Clang 15 now defaults to using PIE): LDFLAGS="-no-pie" emerge busybox </code></pre> Now if we run valgrind busybox sh</code>, we seem to run into another issue resulting in a segmentation fault: ==1795== Invalid write of size 4 ==1795== at 0x47F0E0: ash_main (in /bin/busybox) ==1795== Address 0x4 is not stack'd, malloc'd or (recently) free'd </code></pre> After trying to experiment a bit by copying over the ebuild and modifying it a bit to enable the USE_PORTABLE_CODE</code> and DEBUG</code> configuration options when building busybox: busybox_config_option y USE_PORTABLE_CODE busybox_config_option y DEBUG </code></pre> We do get a working busybox sh</code> that no longer results in a segmentation fault. After inspecting eix busybox</code>, it seems that version 1.35.0-r1 is available too. Let's unmask it and install it to see what happens: echo "sys-apps/busybox **" >> /etc/portage/package.accept_keywords/busybox emerge busybox </code></pre> All right, busybox sh</code> seems to work fine for version 1.35.0-r1 without it resulting in any segmentation faults when built with Clang. So how do we tell genkernel</code> to use this version of busybox instead? Well, genkernel</code> relies on a shell script to install the software /usr/share/genkernel/defaults/software.sh</code>. More specifically, we can change the version of sys-apps/busybox</a> that genkernel</code> should build and install (1.34.1-r1 at the time of writing): GKPKG_BUSYBOX_PV="${GKPKG_BUSYBOX_PV:-1.34.1}" </code></pre> We can simply change the above line to: GKPKG_BUSYBOX_PV="${GKPKG_BUSYBOX_PV:-1.35.0}" </code></pre> Then we can copy over the busybox-1.35.0.tar.bz</code> archive from our distfiles to the distfiles directory used by genkernel: cp /var/cache/distfiles/busybox-1.35.0.tar.bz2 /usr/share/genkernel/distfiles </code></pre> Now we run genkernel</code> again to rebuild and install the kernel: genkernel --clean --no-menuconfig --install all </code></pre> Testing our kernel</h1> After booting into our new Linux kernel with Rust support, we can run zgrep RUST /proc/config.gz</code> to confirm that Rust support is enabled as shown in Figure 2. Figure 2: the result of running zgrep RUST /proc/config.gz</code> and uname -a</code> after booting our Linux kernel with Rust support enabled. </figcaption> </figure> </center> Now that we have Rust support enabled in our Linux kernel, we can clone the basic template</a> from before and try to compile and load it: git clone https://github.com/Rust-for-Linux/rust-out-of-tree-module </code></pre> Since we need to use the same Rust compiler that we used to build the Linux kernel, we might have to specify PATH="/usr/bin:$PATH"</code> in case the user has their own Rust compiler installed locally through rustup</a>. We should then be able to build the Linux kernel module as follows: PATH="/usr/bin:$PATH" make </code></pre> After building the kernel module, we should then be able to load and unload it as follows: sudo insmod rust_out_of_tree.ko sudo rmmod rust_out_of_tree.ko </code></pre> If everything worked correctly, running dmesg</code> should show like the following: [54488.118458] rust_out_of_tree: loading out-of-tree module taints kernel. [54488.118500] rust_out_of_tree: module verification failed: signature and/or required key missing - tainting kernel [54488.119702] rust_out_of_tree: Rust out-of-tree sample (init) [54506.127820] rust_out_of_tree: My numbers are [72, 108, 200] [54506.127825] rust_out_of_tree: Rust out-of-tree sample (exit) </code></pre> Conclusion</h1> We had to jump through quite some hoops, but in the end we managed to build and install a Linux kernel with Rust support enabled and build and load a basic kernel module written in Rust. It turns out that we needed an older version of the Rust compiler and bindgen installed to build the Linux kernel with Rust support. Hopefully, Gentoo will make these ebuilds available again for a more streamlined experience. Furthermore, we had to use the Clang toolchain to build the Linux kernel, and optionally our initramfs. While genkernel seems to have proper support for this, we ran into an issue where version 1.34.1 of busybox results in a segmentation fault when compiled with Clang. Fortunately, this issue seems to be solved with version 1.35.0 and it was pretty straightforward to have genkernel use this version of busybox instead. Maybe the sys-kernel/gentoo-kernel-bin</a> package will come with Rust support too some day. Accessing physical memory from userspace on Linux 2023-01-10T00:00:00+00:00 Why access physical memory?</h1> The ability to read from and write to arbitrary physical address is a very useful and powerful tool. It can give us some insight into what the operating system is doing under the hood, it can be used to interface with devices that use memory-mapped I/O, such as PCI devices, from userspace and it is in general incredibly useful for (hardware) reverse engineering purposes. Thus, we are going to look at how to access arbitrary physical memory on Linux using the /dev/mem</code> interface. Of course, with great power comes great responsibility, and as full access to /dev/mem</code> poses a huge security risk, you should not be doing this on a production system. Furthermore, since we are trying to access physical memory, we also need to know where the memory of our process is located in physical memory. So we will first be looking at how to map a given virtual address to its physical counterpart. Looking up physical addresses</h1> Before we can actually use the interface to access physical memory, we first need to look at how to look up the physical address for any given virtual address. Fortunately, since version 2.6.25 the Linux kernel provides such an interface through the /proc/<pid>/pagemap</code> file. This file consists of a 64-bit entry for each virtual page describing attributes such as whether the page is present, swapped, etc. as well as the physical frame number if the page is present but not swapped as described in the documentation</a>. Thus, to obtain the physical address for any given virtual address, we first need to open this file. Then we need to divide the virtual address by the page size of the platform, which is typically but not necessarily 4 kiB, and multiply it by 8 to get the file offset to seek to. Finally, we can read the 64-bit entry corresponding to that virtual address. Let's first start with an abstraction for this entry. Since the page can be swapped out, present or neither, we are going to abstract the entry with an enum: use bitflags::bitflags; use simple_bits::BitsExt as _; #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)] pub struct SwapType(pub u8); const SWAPPED: u64 = 1 << 62; const PRESENT: u64 = 1 << 63; bitflags! { pub struct PageEntryFlags: u64 { const SOFT_DIRTY = 1 << 55; const EXCLUSIVE = 1 << 56; const SHARED = 1 << 61; } } #[derive(Clone, Debug)] pub enum PageEntry { Unmapped, Present { pfn: u64, flags: PageEntryFlags, }, Swapped { ty: SwapType, offset: u64, flags: PageEntryFlags, }, } </code></pre> Now given the 64-bit entry, we need to map the 64-bit value to the appropriate enum variant and extract the various fields. To achieve that, we implement the From<u64></code> trait for our enum: impl From<u64> for PageEntry { fn from(value: u64) -> Self { let flags = PageEntryFlags::from_bits_truncate(value); if value & SWAPPED == SWAPPED { Self::Swapped { ty: SwapType(value.extract_bits(0..4)), offset: value.extract_bits(4..55), flags, } } else if value & PRESENT == PRESENT { Self::Present { pfn: value.extract_bits(0..55), flags, } } else { Self::Unmapped } } } </code></pre> In addition, since we get the physical page frame number, we implement a helper function to query the page size and use it to calculate the actual physical address: use nix::unistd::{sysconf, SysconfVar::PAGE_SIZE}; impl PageEntry { fn physical_address(&self) -> Option<u64> { match self { Self::Present { pfn, .. } => { let page_size = sysconf(PAGE_SIZE) .unwrap_or(None) .unwrap_or(4096) as u64; Some(pfn * page_size) } _ => None, } } } </code></pre> Finally, instead of opening the pagemap interface ourselves and using the pread</a> system call, we can abstract the pagemap interface with functions to open it for the current process or a given process ID as well as a function to read the entry for a given virtual address as follows: use libc::off_t; use nix::sys::uio::pread; use std::fs::File; use std::os::unix::io::AsRawFd; #[derive(Debug)] pub struct PageMap(File); impl PageMap { fn with_self() -> Result<Self, std::io::Error> { Ok(Self(File::open("/proc/self/pagemap")?)) } fn with_pid(pid: u32) -> Result<Self, std::io::Error> { Ok(Self(File::open(format!("/proc/{pid}/pagemap"))?)) } fn read_entry(&self, address: off_t) -> Result<PageEntry, std::io::Error> { let page_size = sysconf(PAGE_SIZE) .unwrap_or(None) .unwrap_or(4096) as off_t; let mut bytes = [0u8; 8]; pread(self.0.as_raw_fd(), &mut bytes, 8 * address / page_size)?; Ok(PageEntry::from(u64::from_ne_bytes(bytes))) } } </code></pre> To showcase our Rust implementation, we use the mmap-rs</a> crate to map in a page and use the pagemap interface to look up its physical address. use mmap_rs::{MmapFlags, MmapOptions}; fn main() -> Result<(), Box<dyn std::error::Error>> { let mut mapping = MmapOptions::new(MmapOptions::page_size().0) .with_flags(MmapFlags::COPY_ON_WRITE) .map_mut()?; mapping.fill(0xff); let pagemap = PageMap::with_self()?; let entry = pagemap.read_entry(mapping.as_ptr() as _)?; println!("entry: {:x?}", entry.physical_address()); Ok(()) } </code></pre> If we just try to run the example using cargo r</code>, we see the following output: entry: Some(0) </code></pre> This is because we need the SYS_CAP_ADMIN</code> capability to actually observe the physical addresses, as having information about the physical addresses is useful for Rowhammer attacks. We thus need to build the above example and run it as the root user to see the physical address: cargo b sudo ./target/debug/pagemap </code></pre> Accessing physical memory</h1> To access physical memory, we can use the /dev/mem</code> interface provided by the Linux kernel. To read from or write to physical memory, we simply use the physical address as the file offset to seek to and then perform a read/write. More specifically, we can again use the pread</a> and pwrite</a> offsets to read/write without having to seek. In fact, we can provide a similar abstraction as we did for the pagemap interface in Rust: use libc::off_t; use nix::sys::uio::{pread, pwrite}; use std::fs::{File, OpenOptions}; use std::os::unix::io::AsRawFd; #[derive(Debug)] pub struct PhysicalMemory(File); impl PhysicalMemory { fn open() -> Result<Self, std::io::Error> { let file = OpenOptions::new() .read(true) .write(true) .create(false) .open("/dev/mem")?; Ok(Self(file)) } fn read(&self, bytes: &mut [u8], address: u64) -> Result<usize, std::io::Error> { let n = pread(self.0.as_raw_fd(), bytes, address as _)?; Ok(n) } fn write(&self, bytes: &[u8], address: u64) -> Result<usize, std::io::Error> { let n = pwrite(self.0.as_raw_fd(), bytes, address as _)?; Ok(n) } } </code></pre> We can then extend our example from before such that we first look up the physical address of our mapped page through the pagemap interface, and then use that address to read from physical memory. This should hopefully give us the same bytes we wrote to our page. The example now looks like this: use mmap_rs::{MmapFlags, MmapOptions}; fn main() -> Result<(), Box<dyn std::error::Error>> { let mut mapping = MmapOptions::new(MmapOptions::page_size().0) .with_flags(MmapFlags::COPY_ON_WRITE) .map_mut()?; mapping.fill(0xff); mapping[0..4].copy_from_slice(b"test"); let pagemap = PageMap::with_self()?; let entry = pagemap.read_entry(mapping.as_ptr() as _)?; if let Some(physical_address) = entry.physical_address() { let physical_memory = PhysicalMemory::open()?; let mut bytes = [0u8; 4]; let _ = physical_memory.read(&mut bytes, physical_address)?; println!("{:x?}", bytes); } Ok(()) } </code></pre> We then run the above example, and... we get a permission denied error: Error: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } </code></pre> Let's have a look at the Linux kernel source code to see what is actually going on there. In particular, when we try to read from /dev/mem</code>, we end up calling the read_mem()</code></a> function. More specifically, it checks whether we are allowed to access the page(s) we are trying to read at lines 152-154. allowed = page_is_allowed(p >> PAGE_SHIFT); if (!allowed) goto failed; </code></pre> There are two possible definitions for the page_is_allowed()</code></a> function depending on whether the Linux kernel has been compiled with CONFIG_STRICT_DEVMEM</code> or not. If not, then this function always returns 1 and as a result we can access any physical address. Otherwise, this function ends up calling the devmem_is_allowed()</code></a> function which is defined differently depending on the platform (the link points to the implementation for x86). On x86 the first megabyte of our physical memory is always accessible, as that traditionally contains the BIOS code and data, which some applications on Linux need access to for emulation purposes. However, since the first megabyte may contain actual free physical memory that can be used by the Linux kernel and/or userspace, accessing any such region simply results in zero filling the buffer for reads. In addition, not all physical memory is backed by DRAM, but may instead be backed by memory-mapped I/O resources. Thus, the /dev/mem</code> interface also provides access to memory-mapped I/O regions, ACPI regions and BIOS regions that live above the first megabyte depending on whether iomem=relaxed</code> or iomem=strict</code> is passed as a kernel command line argument. Unrestricted access to /dev/mem</h1> Warning: unrestricted access to /dev/mem</code> is potentially dangerous and poses a huge security risk as you can access all physical memory in the system, including physical memory in use by the operating system as well as other applications, as well as to all PCI devices. Be aware of these risks and don't run this on a production system. First, to make sure we can access any I/O resources, we can simply add the iomem=relaxed</code> kernel command line argument to our GRUB configuration in /etc/default/grub</code>: GRUB_CMDLINE_LINUX="iomem=relaxed" </code></pre> Run sudo grub-mkconfig -o /boot/grub/grub.cfg</code> and reboot the system. This is sometimes necessary for userspace applications to access I/O devices (e.g. flashrom</a>). However, the above setting doesn't give us access to any physical memory backed by DRAM. One option is to build our own Linux kernel</a> with CONFIG_STRICT_DEVMEM=n</code>. However, we can also write a Linux kernel module that uses kernel probes</a> to alter the behaviour of the devmem_is_allowed()</code> function to always return 1. Since the page_is_allowed()</code> and range_is_allowed()</code> functions are typically inlined, we cannot rely on kernel probes to hook these functions. Before we can build a Linux kernel module, we need to make sure that we have the appropriate build tools and kernel header files installed: sudo apt install linux-headers-$(uname -r) build-essential </code></pre> Then we can simply use the following Makefile</code> to build our kernel module: obj-m += mem_bypass.o mem_bypass-objs += source/main.o all: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules clean: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean </code></pre> The actual kernel module code in source/main.c</code> looks like this: #include <linux/init.h> #include <linux/module.h> #include <linux/kernel.h> #include <linux/kprobes.h> MODULE_LICENSE("GPL"); MODULE_AUTHOR("Stephan van Schaik"); MODULE_DESCRIPTION("Bypass /dev/mem checks."); MODULE_VERSION("1.0"); static int bypass(struct kretprobe_instance *probe, struct pt_regs *regs) { regs->ax = 1; return 0; } static struct kretprobe probe = { .handler = bypass, .maxactive = 20, }; static int __init mem_bypass_init(void) { probe.kp.symbol_name = "devmem_is_allowed"; register_kretprobe(&probe); return 0; } static void __exit mem_bypass_exit(void) { unregister_kretprobe(&probe); } module_init(mem_bypass_init); module_exit(mem_bypass_exit); </code></pre> Since we simply want to override the return values of the function to always return 1, we register a kretprobe. Whenever the function is about to return, the kernel invokes our bypass()</code> function which simply overrides the return value, which on x86 is passed through the ax</code> register. This way we are always allowed to access any physical address we want. We compile and load the above kernel module as follows: make sudo insmod mem_bypass.ko </code></pre> Then we can run our example from before as the root user. If everything worked, then we should see the following output: [74, 65, 73, 74] </code></pre> Of course, if you are done playing with /dev/mem</code>, then make sure to remove the kernel module: sudo rmmod mem_bypass.ko </code></pre> Upgrading and downgrading CPU microcode 2023-01-09T00:00:00+00:00 Introduction</h1> As I often have to go through the process of downgrading CPU microcode to test and reproduce various speculative and transient-execution attacks, I am using this as an opportunity to document the process of checking what CPU microcode version is actually running, as well as how to downgrade/upgrade it from the operating system on both Linux and Microsoft Windows. It is important to note that in addition to the operating system loading CPU microcode, both the CPU and the BIOS/UEFI also provide their own CPU microcode blobs. Since microcode updates can only be installed if the version is newer than what is currently running, we can only downgrade to the version fused into the CPU or provided by the BIOS/UEFI at best, if we are not resorting to more drastic measures such as modifying the BIOS/UEFI or flashing an older version of the BIOS/UEFI firmware. Thus, in this case "downgrading" means reverting to the version provided by the BIOS/UEFI on the system by ensuring that the operating system does not load any newer CPU microcode upon booting. If instead you just want to get your performance back, you may want to look at the instructions on how to disable the mitigations instead. Linux</h1> Disabling Mitigations</h2> In /etc/default/grub</code>, add mitigations=off</code> to GRUB_CMDLINE_LINUX</code>: GRUB_CMDLINE_LINUX="mitigations=off" </code></pre> Run sudo grub-mkconfig -o /boot/grub/grub.cfg</code> and reboot. If you want to re-enable the mitigations, remove mitigations=off</code> and run grub-mkconfig</code> again. Checking the Version</h2> Run cat /proc/pcuinfo</code>. The line that starts with microcode should indicate the microcode version that is currently running. Programmatically, the microcode version that is currently running can be retrieved as follows on Intel CPUs: Write 0 to the IA32_BIOS_SIGN_ID</code> (0x8B) MSR.</li> Issue cpuid</code>.</li> Read the IA32_BIOS_SIGN_ID</code> (0x8B) MSR. The upper 32 bits contains the microcode version.</li> </ol> This can be done through the msr</code> kernel module. Downgrading</h2> Run cat /proc/cpuinfo</code>. Check the lines that start with cpu_family, model and stepping. Convert the decimal to hexadecimal, then the filename of your microcode blob would be something like: 06-5e-03 (Intel Core i7-6700K). Remove or rename the file for your CPU in /lib/firmware/intel-ucode. Then run update-initramfs -u</code> to generate a new initramfs. The microcode is normally loaded by the Linux kernel during boot and the initramfs simply provides the microcode blobs for Linux to load. Upgrading</h2> Write the microcode blob to /lib/firmware/intel-ucode/06-5e-03 (replace with the correct family-model-stepping). Run echo 1 > /sys/devices/system/cpu/microcode/reload</code>. You should now be running the new microcode version. To make this change "persistent", you can run update-initramfs -u</code> to generate a new initramfs. Microsoft Windows</h1> Disabling Mitigations</h2> To disable the mitigations on Microsoft Windows, you need to add the following two registry keys: reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverride /t REG_DWORD /d 0 /f reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f </code></pre> If you want to enable the mitigations again, but leave Intel Hyper-Threading enabled, you can add the following registry keys: reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverride /t REG_DWORD /d 72 /f reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f </code></pre> If you also want to disable Intel Hyper-Threading, then you can add the following registry keys: reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverride /t REG_DWORD /d 8246 /f reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f </code></pre> See also KB4072698: Windows Server and Azure Stack HCI guidance to protect against silicon-based microarchitectural and speculative execution side-channel vulnerabilities</a> for more information. Checking the Version</h2> The Intel Processor Identification Utility</a> is one tool that you can use to check the microcode version that is currently running. Programmatically, the microcode version that is currently running can be retrieved as follows on Intel CPUs: Write 0 to the IA32_BIOS_SIGN_ID</code> (0x8B) MSR.</li> Issue cpuid</code>.</li> Read the IA32_BIOS_SIGN_ID</code> (0x8B) MSR. The upper 32 bits contains the microcode version.</li> </ol> Downgrading</h1> To downgrade the microcode you will have to follow the following instructions to ensure you have the right permissions to change the filename of the DLLs: Open File Explorer.</li> Navigate to C:\Windows\System32.</li> Find mcupdate_GenuineIntel.dll or mcupdate_AuthenticAMD.dll.</li> Right-click the DLL.</li> Select Properties.</li> In the Properties window, go to Advanced</li> Click Change next to Owner: TrustedInstaller.</li> In the text area named Enter the object name to select, write your username.</li> Press OK.</li> In the Properties window, click the Add button.</li> Next to Principal click on Select a principal.</li> In the text area named Enter the object name to select, write your username.</li> Press OK.</li> Check the Full control permission.</li> Press OK.</li> Press OK.</li> </ol> If everything went well, you should now have the permission to change the filenames. Make sure the DLLs are not named mcupdate_GenuineIntel.dll and mcupdate_AuthenticAMD.dll. For instance, you can name them mcupdate_GenuineIntel.dll.bak and mcupdate_AuthenticAMD.dll.bak. Reboot Microsoft Windows to ensure that the microcode does not get loaded. Note that CPU microcode cannot be downgraded at run-time. The CPU has old microcode fused into the CPU, and other stages of the boot process have the opportunity to upgrade the CPU microcode to something more recent. These stages are the BIOS/UEFI and the operating system. To revert to the most recent version, you can simply change the filename back to the original name. Note: Microsoft Windows may install new versions of these DLLs during an update. You will have to go through this process again in that case. Updating Alpine Linux 2023-01-08T00:00:00+00:00 Exporting PostgreSQL</h1> If you have installed PostgreSQL</a> on your server, then you may need to migrate your existing PostgreSQL database to the new version. This happens with every major version increase, as PostgreSQL may decide to modify the database format. As such, we first export the contents of our PostgreSQL database to a series of SQL queries that we can then execute to import the data into the new database. Thus, we run the following commands to export the database: FILENAME=dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql pg_dumpall -c -U postgres > $FILENAME </code></pre> The above command is also useful if you want to regularly make backups of your PostgreSQL database. Updating Alpine</h1> To upgrade Alpine we simply need to modify the versions in the /etc/apk/repositories</code> file. For instance, to upgrade from Alpine Linux 3.16 to 3.17, we would change v3.16</code> to v3.17</code> in the following two lines in the /etc/apk/repositories</code> file: http://dl-cdn.alpinelinux.org/alpine/v3.16/main http://dl-cdn.alpinelinux.org/alpine/v3.16/community </code></pre> It's also possible to use the sed</code> tool to simply update the version numbers: sed -i -e 's/v3\.16/v3.17/g' /etc/apk/repositories </code></pre> After updating the repositories, we can simply update the package lists by running the following command: apk update </code></pre> While not always necessary, it is recommend to update Alpine's package manager first: apk add --upgrade apk-tools </code></pre> Then we can upgrade all installed packages, including packages that have the same version numbers using the --available</code> switch (in case uClibc requires this): apk upgrade --available </code></pre> If the Linux kernel has been updated, you may want to reboot the machine as follows: sync reboot </code></pre> Otherwise you can simply update any of the updated services. Importing PostgreSQL</h1> After updating Alpine Linux, we can restart the PostgreSQL service: /etc/init.d/postgresql restart </code></pre> If the major version increased in between updates, the above command will create a new database from scratch. Thus, we have to import the contents of our previous PostgreSQL database by running the following command: cat $FILENAME | psql -U postgres -d postgres </code></pre> Building the Linux Kernel (Ubuntu) 2022-09-18T00:00:00+00:00 Introduction</h1> These are my notes on how to build your own Linux kernel for Ubuntu, as the information online is scattered all over the place and dated. Fortunately, Canonical recently followed suit and it is now possible to simply build a mainline Linux kernel, rather than having to build Canonical's fork. While this may be useful if you need more recents drivers, because drivers such as amdgpu are typically behind on Ubuntu, this guide is targeted at users who want to either write their own drivers or modify existing drivers in the Linux kernel. Installing build dependencies</h1> There are number of dependencies that need to be installed before we can build the Linux kernel. They can be installed by running the following command on Ubuntu: sudo apt-get install libncurses-dev gawk flex bison openssl libssl-dev dkms libelf-dev libudev-dev libpci-dev libiberty-dev autoconf llvm </code></pre> Building the kernel</h1> Now that we have all the dependencies required to build the kernel, we can either download a tarball with the Linux source code from https://kernel.org</a> or clone the appropriate tag from the Linux kernel repository. For instance, we can clone the source code of the Linux kernel for version 6.5 as follows: git clone -b v6.5 --depth=1 https://github.com/torvalds/linux </code></pre> Once we have downloaded the source code, we can simply cd</code> into the directory as follows: cd linux </code></pre> Before we can build the Linux kernel, we need to configure the kernel. Fortunately, we can simply copy the kernel configuration from an existing installation: cp -v /boot/config-$(uname -r) .config </code></pre> However, we have to tune a few configuration options before we can actually build the kernel 1</a>: scripts/config --disable SYSTEM_TRUSTED_KEYS scripts/config --disable SYSTEM_REVOCATION_KEYS </code></pre> In addition, the kernel configuration we copied may be from an older version than the kernel version we are trying to compile. We can update the kernel configuration to be compatible with the version of the forked Linux kernel tree as follows: make olddefconfig </code></pre> The above command will simply pick the default option for any newly introduced configuration option. To build the Linux kernel and the kernel modules we can simply invoke make</code>. However, make</code> will by default only use one CPU core/thread, so we have to specify the jobs option or -j</code> followed by the number of threads that we want to use. For example, to use 16 CPU cores/threads, we can invoke make</code> as follows: make -j16 </code></pre> If the above command was successful, we can now install the kernel modules: sudo make INSTALL_MOD_STRIP=1 modules_install </code></pre> As well as the Linux kernel: sudo make install </code></pre> The above command will also update the initramfs and regenerate the GRUB configuration file automatically on Ubuntu. However, it is also possible to update the initramfs or regenerate the GRUB configuration file yourself. The commands that get passed to the kernel by GRUB during boot are located in /etc/default/grub</code>. You can modify the kernel command line in /etc/default/grub</code>, then you can run the following to update the GRUB configuration file: sudo update-grub </code></pre> If you want to rebuild the initramfs, you can run the following: sudo update-initramfs -u </code></pre> Finally, after installing the Linux kernel, the kernel modules and updating the initramfs and GRUB configuration, you can reboot your system: sudo reboot </code></pre> GRUB should now present you with the option to boot the newly installed kernel. However, depending on your installation, GRUB may not present you with a boot menu at all, e.g. on Ubuntu this is the default behavior. To change this, you can edit the /etc/default/grub</code> file. More specifically, comment out the GRUB_TIMEOUT_STYLE</code> variable and set GRUB_TIMEOUT</code> to a non-zero value as follows: #GRUB_TIMEOUT_STYLE=hidden GRUB_TIMEOUT=10 </code></pre> Then run the following command to regenerate the GRUB configuration file: sudo update-grub </code></pre> Building the Linux Kernel (Ubuntu) 2022-09-18T00:00:00+00:00 This article has been moved here</a>. Steamcmd 2021-11-28T00:00:00+00:00 Installation</h1> Make sure you have set up the VoidLinux chroot</a> first. Since Steamcmd comes with 32-bit binaries, we first have to enable 32-bit compatibility as follows: ./root-voidlinux.sh "xbps-install -Sy void-repo-multilib" ./root-voidlinux.sh "xbps-install -S" ./root-voidlinux.sh "xbps-install libgcc-32bit libstdc++-32bit" </code></pre> Download the tarball containing Steamcmd outside of the chroot and extract it into the home directory in our chroot: wget http://media.steampowered.com/installer/steamcmd_linux.tar.gz tar xf steamcmd_linux.tar.gz -C ~/chroot/voidlinux/home/$USER </code></pre> Update steamcmd: ./voidlinux.sh "/home/$USER/steamcmd.sh +quit" </code></pre> Setting up a Valheim Server 2021-11-28T00:00:00+00:00 Installation</h1> Make sure you have set up the VoidLinux chroot</a> and installed Steamcmd</a> first. We can then simply create a script as follows to install/update the Valheim server: cat >~/chroot/update-valheim.sh <<'EOF' #!/bin/sh ./voidlinux.sh "/home/$USER/steamcmd.sh +@sSteamCmdForcePlatformType linux +force_install_dir /home/$USER/valheim +login anonymous +app_update 896660 validate +quit" EOF </code></pre> Ensure the script is executable: chmod +x ~/chroot/update-valheim.sh </code></pre> #!/bin/sh SERVER_NAME="My server" PORT=2456 WORLD_NAME="Dedicated" PASSWORD="secret" ./voidlinux.sh "/home/$USER/valheim/valheim_server.x86_64 -name $SERVER_NAME -port $PORT -world $WORLD_NAME -password $PASSWORD" </code></pre> Setting up a VoidLinux chroot 2021-11-28T00:00:00+00:00 Installation</h1> We will first create a directory for our VoidLinux installation: mkdir -p ~/chroot/voidlinux </code></pre> Then download the glibc flavour of the rootfs tarball containing the VoidLinux root filesystem for the x86-64 architecture (check the download page</a> for the download link): wget https://alpha.de.repo.voidlinux.org/live/current/void-x86_64-ROOTFS-20210930.tar.xz </code></pre> Extract the tarball: tar xf void-x86_64-ROOTFS-20210930.tar.xz -C ~/chroot/voidlinux </code></pre> chroot scripts</h1> We will now set up a script that mounts /dev/</code>, /proc/</code>, /sys</code> and /tmp</code>, copies resolv.conf</code> for networking and then either executes the command that we pass as the first argument or otherwise defaults to /bin/bash</code>: cat >~/chroot/root-voidlinux.sh << 'EOF' #!/bin/sh CHROOT_PATH="/home/$USER/chroot/voidlinux" cd $CHROOT_PATH mount | grep $CHROOT_PATH/dev >/dev/null || sudo mount --rbind /dev $CHROOT_PATH/dev && sudo mount --make-rslave $CHROOT_PATH/dev mount | grep $CHROOT_PATH/proc >/dev/null || sudo mount -t proc /proc $CHROOT_PATH/proc mount | grep $CHROOT_PATH/sys >/dev/null || sudo mount --rbind /sys $CHROOT_PATH/sys && sudo mount --make-rslave $CHROOT_PATH/sys mount | grep $CHROOT_PATH/tmp >/dev/null || sudo mount --rbind /tmp $CHROOT_PATH/tmp cp /etc/resolv.conf $CHROOT_PATH/etc/resolv.conf sudo chroot $CHROOT_PATH ${1:-/bin/bash} EOF </code></pre> In addition, we will set up a script that simply lets us run commands as the same user as on our host system: cat >~/chroot/voidlinux.sh << 'EOF' #!/bin/sh CHROOT_PATH="/home/$USER/chroot/voidlinux" cd $CHROOT_PATH mount | grep $CHROOT_PATH/dev >/dev/null || sudo mount --rbind /dev $CHROOT_PATH/dev && sudo mount --make-rslave $CHROOT_PATH/dev mount | grep $CHROOT_PATH/proc >/dev/null || sudo mount -t proc /proc $CHROOT_PATH/proc mount | grep $CHROOT_PATH/sys >/dev/null || sudo mount --rbind /sys $CHROOT_PATH/sys && sudo mount --make-rslave $CHROOT_PATH/sys mount | grep $CHROOT_PATH/tmp >/dev/null || sudo mount --rbind /tmp $CHROOT_PATH/tmp cp /etc/resolv.conf $CHROOT_PATH/etc/resolv.conf sudo chroot --userspec=$USER:users $CHROOT_PATH ${1:-/bin/bash} EOF </code></pre> In addition, we will set up a script to undo the mounts: cat >~/chroot/cleanup-voidlinux.sh << 'EOF' #!/bin/sh CHROOT_PATH="/home/$USER/chroot/voidlinux" mount | grep $CHROOT_PATH/dev >/dev/null && sudo umount -R $CHROOT_PATH/dev mount | grep $CHROOT_PATH/proc >/dev/null && sudo umount -R $CHROOT_PATH/proc mount | grep $CHROOT_PATH/sys >/dev/null && sudo umount -R $CHROOT_PATH/sys mount | grep $CHROOT_PATH/tmp >/dev/null && sudo umount -R $CHROOT_PATH/tmp EOF </code></pre> Ensure the scripts are executable: chmod +x ~/chroot/voidlinux.sh ~/chroot/root-voidlinux.sh ~/chroot/cleanup-voidlinux.sh </code></pre> We can then add a user with the same name, uid and gid as the user on our host system as follows: ./root-voidlinux.sh "useradd -mU $USER -u $UID -g $GID" </code></pre> We can now simply test our chroot setup by entering the VoidLinux chroot as follows: ./voidlinux.sh </code></pre> Exit the shell and run the cleanup script as follows: ./cleanup-voidlinux.sh </code></pre> Updating VoidLinux</h1> Enter the VoidLinux chroot and run the following commands to update the system: ./root-voidlinux.sh "xbps-install -Su xbps" ./root-voidlinux.sh "xbps-install -u" ./root-voidlinux.sh "xbps-install base-system" ./root-voidlinux.sh "xbps-remove base-voidstrap" </code></pre> You can also create a script to update the VoidLinux installation: cat >~/chroot/update-voidlinux.sh << 'EOF' #!/bin/sh CHROOT_PATH="/home/$USER/chroot/voidlinux" cd $CHROOT_PATH mount | grep $CHROOT_PATH/dev >/dev/null || sudo mount --rbind /dev $CHROOT_PATH/dev && sudo mount --make-rslave $CHROOT_PATH/dev mount | grep $CHROOT_PATH/proc >/dev/null || sudo mount -t proc /proc $CHROOT_PATH/proc mount | grep $CHROOT_PATH/sys >/dev/null || sudo mount --rbind /sys $CHROOT_PATH/sys && sudo mount --make-rslave $CHROOT_PATH/sys mount | grep $CHROOT_PATH/tmp >/dev/null || sudo mount --rbind /tmp $CHROOT_PATH/tmp cp /etc/resolv.conf $CHROOT_PATH/etc/resolv.conf sudo chroot $CHROOT_PATH xbps-install -Su xbps sudo chroot $CHROOT_PATH xbps-install -u sudo chroot $CHROOT_PATH xbps-install base-system sudo chroot $CHROOT_PATH xbps-remove base-voidstrap EOF </code></pre> Ensure the script is executable: chmod +x ~/chroot/update-voidlinux.sh </code></pre> Edit ~/chroot/voidlinux/etc/default/libc-locales</code> and uncomment the line containing "#en_US.UTF-8 UTF-8" by removing the "#", if it is not already uncommented. Update the locales as follows: ./root-voidlinux.sh "xbps-reconfigure -f glibc-locales" </code></pre> Windows Drivers in Rust: I/O Controls 2021-10-07T00:00:00+00:00 Introduction</h1> In the previous article</a> we extended our device file to support reading and writing from userspace as a way of interacting with our driver. In this article we are going to be looking at extending our interface to be able to handle device I/O controls or ioctls</code>, which we pronounce as I/O controls, which is another way to let userspace interace with our driver. What is an ioctl?</h1> Since our userspace process runs with fewer privileges than the operating system kernel, we have to rely on the services provided by the kernel to us. For instance, our userspace process does not have access to all the physical memory of the system, but we can ask the kernel to allocate some memory for us and map it into our address space using VirtualAlloc</code></a>. Under the hood, a call to VirtualAlloc</code></a> will end up calling NtAllocateVirtualMemory</code></a> in ntdll.dll</code>, which is responsible for actually invoking the system call that jumps from userspace to the system call handler in the kernel. So how does the system call handler know that we called NtAllocateVirtualMemory</code>? Well, each service routine that is available to userspace does have its own unique number that ntdll.dll</code> passes along to the system call handler, together with the arguments for that service routine. Similarly, we have been using ReadFile</code></a> to read from our device file in the previous article. This function ends up calling NtReadFile</code></a> in ntdll.dll</code>. However, sometimes we want to provide service routines specific to our driver that do not map well to any of the existing service routines, such as reading from or writing to our device file. Fortunately, userspace has access to DeviceIoControl</code></a> which accepts a control code and optionally an input and/or an output buffer. Invoking DeviceIoControl</code></a> causes the I/O manager to create an IRP_MJ_DEVICE_CONTROL</code></a> and send it to the topmost driver in the driver stac, which may be our driver, or it may eventually be dispatched to our driver. The driver then receives the control code and optionally the input buffer and/or the output buffer along with the IRP, and uses the control code to perform the corresponding operation. Dispatching the Callbacks</h1> The dispatch code is similar to the one from the previous article. We first extend the dispatch_callback</code> function to handle the IRP_MJ_DEVICE_CONTROL</code> major function: IRP_MJ_DEVICE_CONTROL => { let control_request = IoControlRequest { inner: request, }; let status = data.ioctl(&device, &control_request); request = control_request.inner; status } </code></pre> We add the ioctl</code> callback to the DeviceOperations</code> trait with a default implementation: fn ioctl(&mut self, _device: &Device, request: &IoControlRequest) -> Result<(), Error> { request.complete(Ok(0)); Ok(()) } </code></pre> Finally, we implemented our IoControlRequest</code> by wrapping the IoRequest</code>: pub struct IoControlRequest { pub(crate) inner: IoRequest, } impl Deref for IoControlRequest { type Target = IoRequest; fn deref(&self) -> &Self::Target { &self.inner } } </code></pre> Our IoControlRequest</code> provides a control_code</code> function to access the control code of the request: impl IoControlRequest { pub fn control_code(&self) -> ControlCode { let stack_location = self.stack_location(); unsafe { stack_location.Parameters.DeviceIoControl.IoControlCode.into() } } </code></pre> Dissecting Control Codes</h1> While the control codes are 32-bit values, they consist of several fields. The "Defining I/O Control Codes" - MSDN</a> article explains how the control codes are structured. Essentially, we have four fields: 0:2 - Transfer Type: whether the I/O control is buffered I/O, direct I/O or neither.</li> 2:14 - Function Code: the function code that corresponds to the operation performed by our driver.</li> 14:16 - Required Access: whether the file needs to be opened with any access, read access, write access or both read and write access.</li> 16:32 - Device Type: the device type we passed to the create_device</code> function.</li> </ul> Let's first define the RequiredAccess</code> and TransferMethod</code> types, for which we just have to use the constants provided by the windows-kernel-sys</code> crate: use windows_kernel_sys::base::{ FILE_ANY_ACCESS, FILE_READ_DATA, FILE_WRITE_DATA, METHOD_NEITHER, METHOD_IN_DIRECT, METHOD_OUT_DIRECT, METHOD_BUFFERED }; bitflags! { pub struct RequiredAccess: u32 { const ANY_ACCESS = FILE_ANY_ACCESS; const READ_DATA = FILE_READ_DATA; const WRITE_DATA = FILE_WRITE_DATA; const READ_WRITE_DATA = FILE_READ_DATA | FILE_WRITE_DATA; } } #[derive(Clone, Copy, Debug, PartialEq, Eq)] #[repr(u32)] pub enum TransferMethod { Neither = METHOD_NEITHER, InputDirect = METHOD_IN_DIRECT, OutputDirect = METHOD_OUT_DIRECT, Buffered = METHOD_BUFFERED, } </code></pre> We can then create a nice struct for the ControlCode</code> with the four fields: #[derive(Clone, Debug, PartialEq, Eq)] pub struct ControlCode(pub DeviceType, pub RequiredAccess, pub u32, pub TransferMethod); </code></pre> Then we will define all the constants we will need to extract these fields from the 32-bit value, or vice versa to pack them into a 32-bit value: impl ControlCode { const METHOD_BITS: usize = 2; const NUM_BITS: usize = 12; const ACCESS_BITS: usize = 2; const TYPE_BITS: usize = 16; const METHOD_SHIFT: usize = 0; const NUM_SHIFT: usize = Self::METHOD_SHIFT + Self::METHOD_BITS; const ACCESS_SHIFT: usize = Self::NUM_SHIFT + Self::NUM_BITS; const TYPE_SHIFT: usize = Self::ACCESS_SHIFT + Self::ACCESS_BITS; const METHOD_MASK: u32 = (1 << Self::METHOD_BITS) - 1; const NUM_MASK: u32 = (1 << Self::NUM_BITS) - 1; const ACCESS_MASK: u32 = (1 << Self::ACCESS_BITS) - 1; const TYPE_MASK: u32 = (1 << Self::TYPE_BITS) - 1; </code></pre> In addition, we will add some convenience functions to access the fields individually: pub fn device_type(&self) -> DeviceType { self.0 } pub fn required_access(&self) -> RequiredAccess { self.1 } pub fn number(&self) -> u32 { self.2 } pub fn transfer_method(&self) -> TransferMethod { self.3 } } </code></pre> Finally, our code to pack and unpack the fields into a 32-bit value and from a 32-bit value looks as follows: impl From<u32> for ControlCode { fn from(value: u32) -> Self { let method = (value >> Self::METHOD_SHIFT) & Self::METHOD_MASK; let num = (value >> Self::NUM_SHIFT) & Self::NUM_MASK; let access = (value >> Self::ACCESS_SHIFT) & Self::ACCESS_MASK; let ty = (value >> Self::TYPE_SHIFT) & Self::TYPE_MASK; Self( ty.into(), RequiredAccess::from_bits(access).unwrap_or(RequiredAccess::READ_DATA), num, method.into() ) } } impl Into<u32> for ControlCode { fn into(self) -> u32 { let method = Into::<u32>::into(self.3) << Self::METHOD_SHIFT; let num = self.2 << Self::NUM_SHIFT; let access = self.1.bits() << Self::ACCESS_SHIFT; let ty = Into::<u32>::into(self.0) << Self::TYPE_SHIFT; ty | access | num | method } } </code></pre> Extending UserPtr</h1> Similar to how it is possible to specify whether you want to use direct I/O or buffered I/O for reads and writes, it is also possible to specify the transfer method for your ioctl as we have seen when dissecting the control codes. The different transfer methods are discussed in "Buffer Descriptions for I/O Control Codes" - MSDN</a>. We will first start by turning our UserPtr</code> into an enum to handle the different situations: pub enum UserPtr { Buffered { ptr: *mut cty::c_void, read_size: usize, write_size: usize, }, Direct { read_ptr: *const cty::c_void, write_ptr: *mut cty::c_void, read_size: usize, write_size: usize, }, Neither, } </code></pre> We then add some convenience functions to construct the different variants of our enum: impl UserPtr { pub unsafe fn new_buffered( ptr: *mut cty::c_void, read_size: usize, write_size: usize, ) -> Self { Self::Buffered{ ptr, read_size, write_size, } } pub unsafe fn new_direct( read_ptr: *const cty::c_void, write_ptr: *mut cty::c_void, read_size: usize, write_size: usize, ) -> Self { Self::Direct { read_ptr, write_ptr, read_size, write_size, } } pub unsafe fn new_neither() -> Self { Self::Neither } </code></pre> We then have to modify the rest of our code to grab the fields from the various variants that we need. For the read_size</code> and write_size</code> functions that looks as follows: pub fn read_size(&self) -> usize { match self { Self::Buffered { read_size, .. } => *read_size, Self::Direct { read_size, .. } => *read_size, Self::Neither => 0, } } pub fn write_size(&self) -> usize { match self { Self::Buffered { write_size, .. } => *write_size, Self::Direct { write_size, .. } => *write_size, Self::Neither => 0, } } </code></pre> For the as_slice</code> and as_mut_slice</code> functions, we also have to extract the pointer of the appropriate buffer: pub fn as_slice(&self) -> &[u8] { let (ptr, size) = match self { Self::Buffered { ptr, read_size, .. } => (*ptr as _, *read_size), Self::Direct { read_ptr, read_size, .. } => (*read_ptr, *read_size), Self::Neither => (core::ptr::null(), 0), }; if ptr.is_null() || size == 0 { &[] } else { unsafe { core::slice::from_raw_parts(ptr as *const u8, size) } } } pub fn as_mut_slice(&mut self) -> &mut [u8] { let (ptr, size) = match self { Self::Buffered { ptr, write_size, .. } => (*ptr, *write_size), Self::Direct { write_ptr, write_size, .. } => (*write_ptr, *write_size), Self::Neither => (core::ptr::null_mut(), 0), }; if ptr.is_null() || size == 0 { &mut [] } else { unsafe { core::slice::from_raw_parts_mut(ptr as *mut u8, size) } } } </code></pre> Finally, we have to do the same for our read</code> and write</code> functions: pub fn read<T: Default>(&self) -> Result<T, Error> { let (ptr, size) = match self { Self::Buffered { ptr, read_size, .. } => (*ptr as _, *read_size), Self::Direct { read_ptr, read_size, .. } => (*read_ptr, *read_size), Self::Neither => (core::ptr::null(), 0), }; if ptr.is_null() || size == 0 { return Err(Error::INVALID_PARAMETER); } if core::mem::size_of::<T>() > size { return Err(Error::INVALID_USER_BUFFER); } let mut obj = T::default(); unsafe { core::ptr::copy_nonoverlapping( ptr as _, &mut obj, core::mem::size_of::<T>(), ); } Ok(obj) } pub fn write<T>(&mut self, obj: &T) -> Result<(), Error> { let (ptr, size) = match self { Self::Buffered { ptr, write_size, .. } => (*ptr, *write_size), Self::Direct { write_ptr, write_size, .. } => (*write_ptr, *write_size), Self::Neither => (core::ptr::null_mut(), 0), }; if ptr.is_null() || size == 0 { return Err(Error::INVALID_PARAMETER); } if core::mem::size_of::<T>() > size { return Err(Error::INVALID_USER_BUFFER); } unsafe { core::ptr::copy_nonoverlapping( obj, ptr as _, core::mem::size_of::<T>(), ); } Ok(()) } </code></pre> Then in the user_ptr</code> function of the IoControlRequest</code> type, we first concern ourselves with grabbing all the information we need: the MDL address if available, the system buffer and the input and output size. The code looks as follows: pub fn user_ptr(&self) -> IoControlBuffers { let stack_location = self.stack_location(); let irp = self.irp(); let system_buffer = unsafe { irp.AssociatedIrp.SystemBuffer }; let mdl_address = if !irp.MdlAddress.is_null() { unsafe { MmGetSystemAddressForMdlSafe(irp.MdlAddress, MM_PAGE_PRIORITY::HighPagePriority as _) } } else { core::ptr::null_mut() }; let input_size = unsafe { stack_location.Parameters.DeviceIoControl.InputBufferLength } as usize; let output_size = unsafe { stack_location.Parameters.DeviceIoControl.OutputBufferLength } as usize; </code></pre> The rest of our code depends on the transfer method specified in the control code: match self.control_code().transfer_method() { TransferMethod::Buffered => unsafe { UserPtr::new_buffered(system_buffer, input_size, output_size) }, TransferMethod::InputDirect => unsafe { UserPtr::new_direct(mdl_address, system_buffer, output_size, input_size) }, TransferMethod::OutputDirect => unsafe { UserPtr::new_direct(system_buffer, mdl_address, input_size, output_size) }, TransferMethod::Neither => unsafe { UserPtr::new_neither() }, } </code></pre> While this is a lot of code that we had to implement to deal with the various transfer methods, this implementation greatly improves the ergonomics of our user buffers when dealing with them in our ioctl</code> handler. Matching Control Code</h1> While we already greatly improved the ergonomics of dealing with user buffers, there is another issue we want to tackle. Currently in our ioctl</code> handler, we would have to match on the ControlCode</code>, which is rather inconvenient. We could already perform some checks beforehand, and provide a function that just return the RequiredAccess</code> and the function number of the ioctl instead of all four fields. Let's first extend our DeviceExtension</code> to keep track of the DeviceType</code> we specified when creating our device: #[repr(C)] pub struct DeviceExtension { pub(crate) vtable: *const device_operations, pub(crate) data: *mut cty::c_void, pub(crate) device_type: DeviceType, } </code></pre> We can then add the following convenience function to the Device</code> type: pub(crate) fn device_type(&self) -> DeviceType { self.extension().device_type } </code></pre> Then in the create_device</code> function, we can simply keep track of the DeviceType</code> that the user specified when creating the device: let extension = device.extension_mut(); extension.device_type = device_type; extension.vtable = &DeviceOperationsVtable::<T>::VTABLE; extension.data = Box::into_raw(data) as *mut cty::c_void; </code></pre> Now we can check when handling IRP_MJ_DEVICE_CONTROL</code>, whether the DeviceType</code> matches or not: IRP_MJ_DEVICE_CONTROL => { let control_request = IoControlRequest { inner: request, }; let status = if device.device_type() == control_request.control_code().device_type() { data.ioctl(&device, &control_request) } else { Err(Error::INVALID_PARAMETER) }; </code></pre> Since our UserPtr</code> type already handles the transfer method, we generally don't need to deal with that field either. So we can implement the following function called function()</code> that returns the required access and function number fields instead for IoControlRequest</code>: pub fn function(&self) -> (RequiredAccess, u32) { let code = self.control_code(); (code.required_access(), code.number()) } </code></pre> Example</h1> The code for our example driver can be found on Github</a>. For this example, we will be creating a driver that stores a 32-bit value as part of the device and provides three operations that allows userspace to print the value, to retrieve the value and to set the value. Our MyDevice</code> struct looks as follows: struct MyDevice { value: u32, } </code></pre> Then we will define the function numbers for our operations, which have to start at 0x800 since any numbers below are Microsoft-specific: const IOCTL_PRINT_VALUE: u32 = 0x800; const IOCTL_READ_VALUE: u32 = 0x801; const IOCTL_WRITE_VALUE: u32 = 0x802; </code></pre> Then we can implement the ioctl</code> callback for MyDevice</code>: impl DeviceOperations for MyDevice { fn ioctl(&mut self, _device: &Device, request: &IoControlRequest) -> Result<(), Error> { match request.function() { </code></pre> For the first operation, we don't have to process any input or output buffers as we just need to print the value we currently have stored: (_, IOCTL_PRINT_VALUE) => { println!("value: {}", self.value); request.complete(Ok(0)); } </code></pre> For the other operations, we will assume buffered I/O and only handle the buffered case of IoControlBuffers</code> to grab the user pointer. In any other case we return the Error::INVALID_PARAMETER</code> error instead. Then we can use the user pointer to read or write the value respectively: (RequiredAccess::READ_DATA, IOCTL_READ_VALUE) => { let mut user_ptr = request.user_ptr(); user_ptr.write(&self.value)?; request.complete(Ok(core::mem::size_of::<u32>() as u32)) } (RequiredAccess::WRITE_DATA, IOCTL_WRITE_VALUE) => { let user_ptr = request.user_ptr(); self.value = user_ptr.read()?; request.complete(Ok(0)) } </code></pre> In any other case, we don't know how to handle the request from userspace, so we simply return the Error::INVALID_PARAMETER</code> error: _ => { return Err(Error::INVALID_PARAMETER); } } Ok(()) } } </code></pre> That concludes our driver. As usual, we can load our driver as follows: sc create example binPath=$(realpath target/x86_64-pc-windows-msvc/debug/driver.sys) type=kernel sc start example </code></pre> Userspace</h1> While we could just invoke DeviceIoControl</code></a> from userspace, it has a low-level interface that requires us to write unsafe code to pass the buffers as *mut std::ffi::c_void</code>. It would be nice if we could easily define the proper function prototypes for the control codes of our driver. We will use a similar approach as the approach used nix</a> crate and define macros that help us define convenience wrappers to perform ioctls from userspace. To get an idea of how this works, we will look at the ioctl_read</code> macro that we define for Microsoft Windows: #[macro_export] macro_rules! ioctl_read { ($(#[$attr:meta])* $name:ident, $dev_ty:expr, $nr:expr, $ty:ty) => { $(#[$attr])* pub unsafe fn $name(handle: *mut std::ffi::c_void, data: *mut $ty) -> Result<u32, $crate::Error> { </code></pre> The code above generates an unsafe function that accepts a file and a pointer to the data of the type specified when using the macro. Then in this function, we take the device type and function number specified to the macro, and use the ControlCode</code> type to pack everything into 32-bit value: let code = $crate::ControlCode( $dev_ty, $crate::RequiredAccess::READ_DATA, $nr, $crate::TransferMethod::Buffered, ).into(); let mut return_value = 0; </code></pre> Then we invoke the DeviceIoControl</code></a>: let status = $crate::DeviceIoControl( handle as _, code, std::ptr::null_mut(), 0, data as _, std::mem::size_of::<$ty>() as _, &mut return_value, std::ptr::null_mut(), ) != 0; </code></pre> Finally, we deal with the error handling: match status { true => Ok(return_value), _ => Err(std::io::Error::last_os_error())?, } } } } </code></pre> For our userspace example, we can then use the macros as follows: use winioctl::{DeviceType, Error}; use winioctl::{ioctl_none, ioctl_read, ioctl_write}; const IOCTL_PRINT_VALUE: u32 = 0x800; const IOCTL_READ_VALUE: u32 = 0x801; const IOCTL_WRITE_VALUE: u32 = 0x802; ioctl_none!(ioctl_print_value, DeviceType::Unknown, IOCTL_PRINT_VALUE); ioctl_read!(ioctl_read_value, DeviceType::Unknown, IOCTL_READ_VALUE, i32); ioctl_write!(ioctl_write_value, DeviceType::Unknown, IOCTL_WRITE_VALUE, i32); </code></pre> For our example, we read the value from the kernel driver, increment it by one and then store the value. Then we will ask the driver to print the current value. Thus, every time we run our example, it will increment the value stored by the driver. The code looks as follows: fn main() -> Result<(), Error> { let file = OpenOptions::new() .read(true) .write(true) .create(false) .open("\\??\\Example")?; let mut value = 0; unsafe { ioctl_read_value(file.as_raw_handle(), &mut value)?; } value += 1; unsafe { ioctl_write_value(file.as_raw_handle(), &value)?; } unsafe { ioctl_print_value(file.as_raw_handle())?; } Ok(()) } </code></pre> The winioctl</a> crate and this example</a> are both available on Github. What's Next?</h1> TODO References</h1> VirtualAlloc</code> - MSDN</a></li> NtAllocateVirtualMemory</code> - MSDN</a></li> ReadFile</code> - MSDN</a></li> NtReadFile</code> - MSDN</a></li> DeviceIoControl</code> - MSDN</a></li> IRP_MJ_DEVICE_CONTROL</code> - MSDN</a></li> "Introduction to I/O Control Codes" - MSDN</a></li> "Buffer Descriptions for I/O Control Codes" - MSDN</a></li> "Defining I/O Control Codes" - MSDN</a></li> nix</code> crate</a></li> </ol> Windows Drivers in Rust: Reading and Writing 2021-10-06T00:00:00+00:00 Introduction</h1> In the previous article</a> we took the first step towards an API to create devices with user data and our own callbacks. In this article we will be looking at implementing callbacks for reading from and writing to our device file. Dispatching the Callbacks</h1> Similar to how we handled the IRP_MJ_CREATE</code>, IRP_MJ_CLOSE</code> and IRP_MJ_CLEANUP</code> major functions in the dispatch_callback</code> function in the previous article, we will now have to add code to dispatch the IRP_MJ_READ</code> and IRP_MJ_WRITE</code> functions: IRP_MJ_READ => { let read_request = ReadRequest { inner: request, }; let status = data.read(&device, &read_request); request = read_request.inner; status } IRP_MJ_WRITE => { let write_request = WriteRequest { inner: request, }; let status = data.write(&device, &write_request); request = write_request.inner; status } </code></pre> The code above is slightly different from the code we wrote before. While it still calls into the read</code> and write</code> functions that we will define in DeviceOperations</code>, it wraps the IoRequest</code> into a ReadRequest</code> and WriteRequest</code> respectively before passing the requests to these functions. That is because the I/O request of the ReadFile</code></a> and WriteFile</code></a> functions will provide us access to the data buffer that userspace supplied to these functions. We now proceed to extend the DeviceOperations</code> trait with the read</code> and write</code> callbacks, as well as some default implementation: fn read(&mut self, _device: &Device, request: &ReadRequest) -> Result<(), Error> { request.complete(Ok(0)); Ok(()) } fn write(&mut self, _device: &Device, request: &WriteRequest) -> Result<(), Error> { request.complete(Ok(0)); Ok(()) } </code></pre> Extending the Requests</h1> When implementing the ReadRequest</code> and WriteRequest</code> types, we first want to make sure that we can still use all the functions that we implemented for IoRequest</code>. Fortunately, we can wrap the IoRequest</code> into our ReadRequest</code> and WriteRequest</code> and then implement core::ops::Deref</code></a>: pub struct ReadRequest { pub(crate) inner: IoRequest, } impl Deref for ReadRequest { type Target = IoRequest; fn deref(&self) -> &Self::Target { &self.inner } } </code></pre> With ReadRequest</code> and WriteRequest</code> implementing Deref</code> for IoRequest</code>, we can now access all the functions available for IoRequest</code>, but at the same time we can extend ReadRequest</code> and WriteRequest</code> with functions specific to those requests. First, we will implement the function that gives us access to the user buffer that has been supplied to us from userspace for these requests. Fortunately, thanks to Deref</code> we have access to the IRP and the current I/O stack location: impl ReadRequest { pub fn user_ptr(&self) -> UserPtr { let stack_location = self.stack_location(); let irp = self.irp(); </code></pre> We need these to tell apart buffered I/O from direct I/O. For direct I/O, we will get a Memory Descriptor List (MDL)</a> instead. A MDL is a set of one or more physical pages that may be discontiguous, that is they are not necessarily neighboring pages in the physical address space. However, we can use MmGetSystemAddressForMdlSafe</code></a> to map these physical pages to a contiguous virtual area, such that our driver can access the user buffer. Similarly, MmGetMdlByteCount</code></a> gives us the size of the MDL in bytes. The function looks like this: if !irp.MdlAddress.is_null() { let ptr = unsafe { MmGetSystemAddressForMdlSafe(irp.MdlAddress, MM_PAGE_PRIORITY::HighPagePriority as _) }; let size = unsafe { MmGetMdlByteCount(irp.MdlAddress) } as usize; unsafe { UserPtr::new(ptr, 0, size) } </code></pre> For buffered I/O, we will get a SystemBuffer</code> and the length of the buffer as a parameter: } else if !unsafe { irp.AssociatedIrp.SystemBuffer }.is_null() { let ptr = unsafe { irp.AssociatedIrp.SystemBuffer }; let size = unsafe { stack_location.Parameters.Read }.Length as usize; unsafe { UserPtr::new(ptr, 0, size) } </code></pre> Alternatively, if the user did not set the DO_BUFFERED_IO</code> or DO_DIRECT_IO</code> flag, we would get NULL</code> pointers in both cases, so we will just return a UserPtr</code> that does not do anything: } else { unsafe { UserPtr::new(core::ptr::null_mut(), 0, 0) } } } </code></pre> Similarly, we will implement an offset</code> function that provides us the byte offset in the file that userspace is trying to read from or write to: pub fn offset(&self) -> i64 { let stack_location = self.stack_location(); let irp = self.irp(); if !irp.MdlAddress.is_null() { (unsafe { MmGetMdlByteOffset(irp.MdlAddress) }) as i64 } else if !unsafe { irp.AssociatedIrp.SystemBuffer }.is_null() { unsafe { stack_location.Parameters.Read.ByteOffset.QuadPart } } else { 0 } } } </code></pre> The implementation for WriteRequest</code> is pretty much the same. User Pointers</h1> If you paid close attention, you may have noticed that we have been wrapping the pointer to the user buffer and the size of the user buffer into a UserPtr</code> struct. We don't want to provide access to the raw pointer itself, but preferably a byte slice or some type instead Therefore, we will be providing our own interface UserPtr</code> that owns the pointer and the size, and gives us access to a byte slice or that lets us read the data into some type T</code>. The basic implementation looks as follows: impl UserPtr { pub unsafe fn new( ptr: *mut cty::c_void, read_size: usize, write_size: usize, ) -> Self { Self { ptr, read_size, write_size, } } pub fn read_size(&self) -> usize { self.read_size } pub fn write_size(&self) -> usize { self.write_size } </code></pre> To convert the pointer and the size to an immutable or mutable slice, we can use the core::slice::from_raw_parts</code></a> and core::slice::from_raw_parts_mut</code></a> functions: pub fn as_slice(&self) -> &[u8] { unsafe { core::slice::from_raw_parts(self.ptr as *const u8, self.read_size) } } pub fn as_mut_slice(&mut self) -> &mut [u8] { unsafe { core::slice::from_raw_parts_mut(self.ptr as *mut u8, self.write_size) } } </code></pre> In addition, we also provide functions to read the buffer into a type T</code> or write a type T</code> into the buffer, as long as the size of T</code> does not exceed the size of the user buffer. We check the size of T</code> and if it exceeds the size of the user buffer, we return Error::INVALID_USER_BUFFER</code>. Otherwise, we use core::ptr::copy_nonoverlapping</code></a> to copy over the data between T</code> and the user buffer. The code for that looks as follows: pub fn read<T: Default>(&self, obj: &mut T) -> Result<T, Error> { if core::mem::size_of::<T>() > self.read_size { return Err(Error::INVALID_USER_BUFFER); } let mut obj = T::default(); unsafe { core::ptr::copy_nonoverlapping( self.ptr as _, &mut obj, core::mem::size_of::<T>(), ); } Ok(obj) } pub fn write<T>(&mut self, obj: &T) -> Result<(), Error> { if core::mem::size_of::<T>() > self.write_size { return Err(Error::INVALID_USER_BUFFER); } unsafe { core::ptr::copy_nonoverlapping( obj, self.ptr as _, core::mem::size_of::<T>(), ); } Ok(()) } } </code></pre> In the implementation above we also differentiate the read_size</code> from the write_size</code>. We won't need that right now, but it will be useful in the next article when we implement support for I/O controls. Example</h1> The code for this example can be found on Github</a>. For this example, we will be implementing the read</code> and write</code> callbacks of our DeviceOperations</code> trait to read the user data into a Vec<u8></code> and to write whatever data is stored inside a Vec<u8></code> to the user buffer: First, we extend our MyDevice</code> struct to store the user data: struct MyDevice { data: Vec<u8>, } </code></pre> In the read</code> callback, we will first make sure the user specified offset does not execeed the size of our buffer. Then we make sure the user specified size does not exceed the size of our buffer minus the offset. After limiting both the offset and the size, we copy over the bytes and return how many bytes we copied over to userspace. The code looks like this: impl DeviceOperations for MyDevice { fn read(&mut self, _device: &Device, request: &ReadRequest) -> Result<(), Error> { let mut user_ptr = request.user_ptr(); let slice = user_ptr.as_mut_slice(); let offset = (request.offset() as usize).min(self.data.len()); let size = slice.len().min(self.data.len() - offset); slice[0..size].copy_from_slice(&self.data[offset..offset + size]); request.complete(Ok(size as u32)); Ok(()) } </code></pre> For our write</code> callback, we simply assume the offset is always 0 and limit the user input to no more than 4096 bytes. If both constraints are met, we simply take the slice of user data and turn it into a Vec<u8></code> and store it: fn write(&mut self, _device: &Device, request: &WriteRequest) -> Result<(), Error> { let user_ptr = request.user_ptr(); if request.offset() > 0 { return Err(Error::END_OF_FILE)?; } let slice = user_ptr.as_slice(); let size = slice.len().min(4096); self.data = slice[0..size].to_vec(); request.complete(Ok(size as u32)); Ok(()) } } </code></pre> That concludes the changes to our driver. We can load our driver as follows: sc create example binPath=$(realpath target/x86_64-pc-windows-msvc/debug/driver.sys) type=kernel sc start example </code></pre> Userspace</h1> The code can be found on Github</a>. To test our driver, we will open our device file with read and write permissions. Then we write a string into the device file, which our driver will store. Up next we will allocate a buffer and try to read data from the device file into that buffer. Finally, we try to decode it as an UTF-8 string and print it if it succeeds, otherwise we simply display the data in hex: use std::fs::OpenOptions; use std::io::{Read, Write}; fn main() -> Result<(), std::io::Error> { let mut file = OpenOptions::new() .read(true) .write(true) .create(false) .open("\\??\\Example")?; file.write_all("Hello, world!".as_bytes())?; let mut data = vec![0u8; 4096]; let size = file.read(&mut data)?; match std::str::from_utf8(&data[..size]) { Ok(s) => println!("read {} bytes: \"{}\"", size, s), _ => println!("read {} bytes: {:x?}", size, &data[..size]), } Ok(()) } </code></pre> What's Next?</h1> In the next article</a>, we will look at I/O controls which provide us with an alternative way to interact with our driver. References</h1> ReadFile</code> - MSDN</a></li> WriteFile</code> - MSDN</a></li> core::ops::Deref</code> - Rust documentation</a></li> "Using MDLs" - MSDN</a></li> MmGetSystemAddressForMdlSafe</code> - MSDN</a></li> MmGetMdlByteCount</code> - MSDN</a></li> core::slice::from_raw_parts</code> - Rust documentation</a></li> core::slice::from_raw_parts_mut</code> - Rust documentation</a></li> core::ptr::copy_nonoverlapping</code> - Rust documentation</a></li> </ol> Windows Drivers in Rust: Creating Devices 2021-09-27T00:00:00+00:00 Introduction</h1> In the previous article</a> we created a safe framework in Rust for writing Windows kernel drivers with a global allocator, print!</code> and println!</code> macros and boilerplate code to set up the driver_entry</code> and driver_exit</code> functions. However, up to now our driver has only been printing "Hello, world!" and "Bye bye!" whenever the driver gets loaded and unloaded. It would be more interesting if we could interact with our driver from userspace. Therefore, we are going to look at how to create device files in this article and implement a safe abstraction for setting up device files in Rust. Creating Devices</h1> To create and delete a device we can use the IoCreateDevice</code></a> and IoDeleteDevice</code></a> functions respectively. We will first start with declaring the Device</code> struct wrapping the DEVICE_OBJECT</code> pointer: pub struct Device { pub(crate) raw: *mut DEVICE_OBJECT, } </code></pre> Since we want to be able to store the Device</code> as part of the struct implementing the KernelModule</code>, we ideally want it to be thread-safe. Therefore, we will implement both Send</code> and Sync</code> for the Device</code>, but as we have to use an unsafe impl</code> we have to make sure that our implementation is guaranteed to be thread-safe: unsafe impl Send for Device {} unsafe impl Sync for Device {} </code></pre> Then we can implement the Drop</code> trait for our Device</code> which simply calls IoDeleteDevice</code> on the pointer. In addition, we will add a check to see if the pointer is not null. This way we can take ownership of the pointer without it being dropped, which will be useful later on: impl Drop for Device { fn drop(&mut self) { if self.raw.is_null() { return; } unsafe { IoDeleteDevice(self.raw); } } } </code></pre> It is also helpful to implement some functions to get the raw pointer or turn a raw pointer to a DEVICE_OBJECT</code> into a Device</code>: impl Device { pub unsafe fn from_raw(raw: *mut DEVICE_OBJECT) -> Self { Self { raw, } } pub unsafe fn as_raw(&self) -> *const DEVICE_OBJECT { self.raw as *const _ } pub unsafe fn as_raw_mut(&self) -> *mut DEVICE_OBJECT { self.raw } pub fn into_raw(mut self) -> *mut DEVICE_OBJECT { core::mem::replace(&mut self.raw, core::ptr::null_mut()) } } </code></pre> Before we extend our Driver</code> type with a create_device</code> function to create the Device</code>, we will have a look at the arguments of IoCreateDevice</code></a>. We can skip over the first two arguments, as the first is the pointer of our Driver</code> type and the second one is the amount of size IoCreateDevice</code></a> should allocate for user-specific data associated with the Device</code>. The third argument is a pointer to a UNICODE_STRING</code> containing the device path, so let's create a function that conveniently creates a UNICODE_STRING</code> from a slice of UTF-16 data or &[u16]</code>: pub fn create_unicode_string(s: &[u16]) -> UNICODE_STRING { let len = s.len(); let n = if len > 0 && s[len - 1] == 0 { len - 1 } else { len }; UNICODE_STRING { Length: (n * 2) as u16, MaximumLength: (len * 2) as u16, Buffer: s.as_ptr() as _, } } </code></pre> The fourth argument is the device type</a>. For the device type we will implement our own enum in Rust, and then implement Into<u32></code> to map the enum values into the appropriate constants: #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum DeviceType { Port8042, Acpi, Battery, ... } impl Into<u32> for DeviceType { fn into(self) -> u32 { match self { DeviceType::Port8042 => windows_kernel_sys::base::FILE_DEVICE_8042_PORT, DeviceType::Acpi => windows_kernel_sys::base::FILE_DEVICE_ACPI, DeviceType::Battery => windows_kernel_sys::base::FILE_DEVICE_BATTERY, ... } } } </code></pre> The fifth argument is a bitwise concatenation of flags that provide information about the driver's device and of which the most common one is FILE_DEVICE_SECURE_OPEN</code>. We will simply use the bitflags!</code> macro from the bitflags</code> crate to create a DeviceFlags</code> type: bitflags! { pub struct DeviceFlags { const SECURE_OPEN = windows_kernel_sys::base::FILE_DEVICE_SECURE_OPEN; ... } } </code></pre> The sixth argument specifies whether access to the device is exclusive or not. Instead of using a boolean, we will implement an enum to clearly differentiate between Access::NonExclusive</code> and Access::Exclusive</code>: #[derive(Copy, Clone, Debug, PartialEq, Eq)] pub enum Access { NonExclusive, Exclusive, } impl Access { pub fn is_exclusive(&self) -> bool { match *self { Access::Exclusive => true, _ => false, } } } </code></pre> Finally, the seventh argument is a pointer to a pointer of the DEVICE_OBJECT</code>, such that the function can give us a pointer to the allocated DEVICE_OBJECT</code>. Now that we have gone over all the arguments and provided safe Rust abstractions for them, we can start implementing Driver::create_device()</code>: impl Driver { pub fn create_device( &mut self, name: &str, device_type: DeviceType, device_flags: DeviceFlags, access: Access, ) -> Result<Device, Error> { // Convert the name to UTF-16 and then create a UNICODE_STRING. let name = U16CString::from_str(name).unwrap(); let mut name_ptr = create_unicode_string(name.as_slice()); // Create the device. let mut device = core::ptr::null_mut(); let status = unsafe { IoCreateDevice( self.raw, 0, &mut name_ptr, device_type.into(), device_flags.bits(), access.is_exclusive() as _, &mut device, } }; if status != STATUS_SUCCESS { return Err(error::from_kernel_errno(status)); } Ok(Device { raw: device, }); } } </code></pre> Associating User Data</h1> While we now have an API that allows us to create and drop device files, there is not much useful we can do with them yet. However, we will first have a look at associating user data with the device before we can actually deal with providing our own implementation for the device file operations. Ideally, we want to be able to define our own type that implements a trait and associate that with our device. We will call this trait DeviceOperations</code> for reasons that will be more obvious later on, but for now we will leave it empty other than requiring it to be Sync</code> and Sized</code>: pub trait DeviceOperations: Sync + Sized { } </code></pre> If we look back at IoCreateDevice</code></a>, then we can see that the second argument allows us to specify an arbitrary size for IoCreateDevice</code></a> to allocate extra storage for user-specific data. We will be using this to store the data that the user wants to associate with the device. More specifically, we will use this extra storage to store a virtual table, i.e. a reference to a struct of function pointers, and a void pointer that points to the user data: #[repr(C)] pub struct DeviceExtension { pub(crate) vtable: *const device_operations, pub(crate) data: *mut cty::c_void, } </code></pre> In addition, we will extend impl Device</code> with functions to access the DeviceExtension</code> as well as the vtable</code> and the user data: pub(crate) fn extension(&self) -> &DeviceExtension { unsafe { &*((*self.raw).DeviceExtension as *const DeviceExtension) } } pub(crate) fn extension_mut(&self) -> &mut DeviceExtension { unsafe { &mut *((*self.raw).DeviceExtension as *mut DeviceExtension) } } pub(crate) fn vtable(&self) -> &device_operations { unsafe { &*(self.extension().vtable as *const _) } } pub fn data<T: DeviceOperations>(&self) -> &T { unsafe { &*(self.extension().data as *const T) } } pub fn data_mut<T: DeviceOperations>(&self) -> &mut T { unsafe { &mut *(self.extension().data as *mut T) } } </code></pre> We can then extend Driver::create_device</code> to accept user data of type T</code> that must implement the DeviceOperations</code> trait. Since we want to associate it with the device, we must ensure that it will live at least as long as the device itself. Therefore we will box it up using Box::new</code> from alloc::boxed::Box</code>. Then we can turn the Box</code> into a raw pointer and store that pointer into the extra allocated storage for DeviceExtension</code>. With these changes, our Driver::create_device</code> function looks a bit like the following: impl Driver { pub fn create_device<T>( &mut self, name: &str, device_type: DeviceType, device_flags: DeviceFlags, access: Access, data: T, ) -> Result<Device, Error> where T: DeviceOperations { // Box the data. let data = Box::new(data); ... let status = unsafe { IoCreateDevice( self.raw, core::mem::size_of::<DeviceExtension>() as u32, &mut name_ptr, device_type.into(), device_flags.bits(), access.is_exclusive() as _, &mut device, } }; if status != STATUS_SUCCESS { return Err(error::from_kernel_errno(status)); } let device = unsafe { Device::from_raw(device) }; // Store the boxed data and vtable. let extension = device.extension_mut(); extension.vtable = &DeviceOperationsVtable::<T>::VTABLE; extension.data = Box::into_raw(data) as *mut cty::c_void; Ok(device) } } </code></pre> In the above code you may notice that we are already setting up the vtable, but what is the purpose of this vtable? Well, we will need it later on to handle any of the operations on the device file, but there is another issue we have to solve. The Driver::create_device</code> function currently takes an argument called data</code> of type T</code> and then decides to Box</code> it and store the raw pointer to Box<T></code> as part of the device extension data, but when we drop the Device</code> we also have to drop this boxed data. Unfortunately, the Drop</code> implementation of Device</code> does not know what the original type T</code> is, but it has to know that in order to call the appropriate Drop</code> implementation. However, we can generate a unique vtable for each type T</code> which means that the function pointers in our vtable point to functions that are aware of what T</code> is. We will first declare device_operations</code> with a release</code> function pointer that accepts a pointer to the DEVICE_OBJECT</code>: #[repr(C)] pub struct device_operations { release: Option<extern "C" fn (*mut DEVICE_OBJECT)>, } </code></pre> Then we will create our release_callback</code> function which accesses the device extension of the DEVICE_OBJECT</code>, swaps out the pointer with a NULL</code> pointer and converts the swapped out raw pointer back into the original Box<T></code>. Then as the function goes out of scope, the Box<T></code> will be dropped, invoking the right Drop</code> implementation for our user data: extern fn release_callback<T: DeviceOperations>( device: *mut DEVICE_OBJECT, ) { unsafe { let extension = (*device).DeviceExtension as *mut DeviceExtension; let ptr = core::mem::replace(&mut (*extension).data, core::ptr::null_mut()); Box::from_raw(ptr as *mut T); } } </code></pre> Then we declare the DeviceOperationsVtable<T></code> struct, which is simply an empty struct, except that if we are not using T</code> in any of the struct members, we have to use a PhantomData</code> marker: pub(crate) struct DeviceOperationsVtable<T>(core::marker::PhantomData<T>); </code></pre> We can then define a VTABLE</code> constant with the corresponding device_operations</code> struct for T</code>. impl <T: DeviceOperations> DeviceOperationsVtable<T> { pub(crate) const VTABLE: device_operations = device_operations { release: Some(release_callback::<T>), }; } </code></pre> As we can see in the code above, we set the release</code> function pointer to release_callback::<T></code>, which means the function pointer now points to the release_callback</code> operating on T</code>. In our Driver::create_device</code> function, we set the vtable</code> to &DeviceOperatonsVtable::<T>::VTABLE</code>. In other words, we point the vtable of the device to the vtable that operates on T</code>. Since the user calls the Driver::create_device</code> function with an actual user defined type such as MyDevice</code>, the vtable will end up pointing to a VTABLE that operates on MyDevice</code> and thus the release</code> function pointer will point to release_callback::<MyDevice></code>. That essentially means that the release_callback</code> function will cast the raw pointer back into a Box<MyDevice></code> and invoke the Drop</code> implementation of MyDevice</code>. All that is left to do for us to complete this implementation, is to take the vtable in the Drop</code> implementation of Device</code> and then invoke the release</code> function pointer before calling IoDeleteDevice</code></a>: impl Drop for Device { fn drop(&mut self) { if self.raw.is_null() { return; } unsafe { let extension = (*self.raw).DeviceExtension as *mut DeviceExtension; let vtable = (*extension).vtable; if let Some(release) = (*vtable).release { release(self.raw); } IoDeleteDevice(self.raw); } } } </code></pre> Handling Userspace Operations</h1> The entire point of creating a device file is such that userspace can interact with our device. For an application to actually do anything with our device file, it will have to call CreateFile</code></a> and DeleteFile</code></a>, which we then have to handle from our driver. On the driver side, these functions will ultimately invoke the function pointers in MajorFunction[IRP_MJ_CREATE]</code> and MajorFunction[IRP_MJ_CLOSE]</code> that are part of the DRIVER_OBJECT</code>. That means we first have to change the driver_entry</code> function in our kernel_module!</code> macro to set up these function pointers. We will simply point all possible major functions to the same function, which we will call dispatch_device</code>: for i in 0..$crate::IRP_MJ_MAXIMUM_FUNCTION { driver.MajorFunction[i as usize] = Some($crate::dispatch_device); } </code></pre> The dispatch_device</code> has two arguments, of which the first is a pointer to the DEVICE_OBJECT</code> and the second is a pointer to the IRP</code> struct, where IRP is short for I/O request packet</code></a>. When we open or close the device file from userspace, the I/O request will generally pass through a number of operating system components and drivers, each them calling IoCallDriver</code> to pass the I/O request to the next. Therefore, as the I/O request goes through the device stack, the IRP</code> ends up being a stack of I/O requests. We will first create an IoRequest</code> type to wrap the IRP</code> pointer: pub struct IoRequest { raw: *mut IRP, } impl IoRequest { pub unsafe fn from_raw(irp: *mut IRP) -> Self { Self { raw: irp } } pub fn as_raw(&self) -> &IRP { unsafe { &*self.raw } } pub fn as_raw_mut(&self) -> &mut IRP { unsafe { &mut *self.raw } } pub fn stack_location(&self) -> &IO_STACK_LOCATION { unsafe { &*IoGetCurrentIrpStackLocation(self.irp_mut()) } } pub fn major(&self) -> u8 { self.stack_location().MajorFunction } } </code></pre> Our dispatch_device</code> function will first use IoGetCurrentIrpStackLocation</code></a> to get the data associated with the I/O request, more specifically the major function. Then it can dispatch the request to handler of the corresponding device. then handle the request and to indicate that the request has been completed it will invoke IoCompleteRequest</code></a>. There will be three types of I/O request we will currently be handling, which are: IRP_MJ_CREATE</code>: userspace opened a handle to the device file.</li> IRP_MJ_CLOSE</code>: userspace closed the handle to the device file.</li> IRP_MJ_CLEANUP</code>: the reference count reached zero, i.e. all handles to the device file have been closed.</li> </ul> Let's first extend the DeviceOperations</code> trait to handle these: pub trait DeviceOperations: Sync + Sized { fn create(&mut self, _device: &Device: request: &IoRequest) -> Result<(), Error> { request.complete(Ok(0)); Ok(()) } fn close(&mut self, _device: &Device, request: &IoRequest) -> Result<(), Error> { request.complete(Ok(0)); Ok(()) } fn cleanup(&mut self, _device: &Device, request: &IoRequest) -> Result<(), Error> { request.complete(Ok(0)); Ok(()) } } </code></pre> As you can see in the functions above, we simply mark the I/O requests as completed. The complete</code> functions simply sets the Information</code> and Status</code> fields of IoStatus</code> in the IRP</code> struct, and then invokes IoCompleteRequest</code></a>. The Information</code> field contains the number of bytes returned, whereas the Status</code> field contains the success or failure status of the I/O request. The complete</code> function is implemented as follows: pub fn complete(&self, value: Result<u32, Error>) { let irp = self.irp_mut(); match value { Ok(value) => { irp.IoStatus.Information = value as _; irp.IoStatus.__bindgen_anon_1.Status = STATUS_SUCCESS; } Err(error) => { irp.IoStatus.Information = 0; irp.IoStatus.__bindgen_anon_1.Status = error.to_kernel_errno(); } } unsafe { IoCompleteRequest(irp, IO_NO_INCREMENT as _); } } </code></pre> Now we can also extend our virtual table device_operations</code> by adding a dispatch</code> callback that takes a pointer to the DEVICE_OBJECT</code>, a pointer to the IRP</code> and the major function: #[repr(C)] pub struct device_operations { dispatch: Option<extern "C" fn (*mut DEVICE_OBJECT, *mut IRP, u8) -> NTSTATUS>, release: Option<extern "C" fn (*mut DEVICE_OBJECT)>, } </code></pre> In this callback we will basically wrap the pointers into Device</code> and IoRequest</code> as we don't want the user to have direct access to these pointers. Using Device</code> we can access the user data which implements the DeviceOperations</code> trait, and therefore we can map the major function code to the corresponding function in that trait and call that function. To prevent the Drop</code> implementation of Device</code> from dropping the DEVICE_OBJECT</code>, we use into_raw()</code> to take the pointer and swap it with a NULL</code> pointer. The code looks like this: extern "C" fn dispatch_callback<T: DeviceOperations>( device: *mut DEVICE_OBJECT, irp: *mut IRP, major: u8, ) -> NTSTATUS { let device = unsafe { Device::from_raw(device) }; let data: &mut T = device.data_mut(); let request = unsafe { IoRequest::from_raw(irp) }; let status = match major as _ { IRP_MJ_CREATE => data.create(&device, &request), IRP_MJ_CLOSE => data.close(&device, &request), IRP_MJ_CLEANUP => data.cleanup(&device, &request), _ => { request.complete(Err(Error::INVALID_PARAMETER)); Err(Error::INVALID_PARAMETER) } }; device.into_raw(); match status { Ok(()) => STATUS_SUCCESS, Err(e) => { let status = e.to_kernel_errno(); request.complete(Err(e)); status } } } </code></pre> With the dispatch callback implemented, we can then extend the VTABLE</code> constant: impl<T: DeviceOperations> DeviceOperationsVtable<T> { pub(crate) const VTABLE: device_operations = device_operations { dispatch: Some(dispatch_callback::<T>), release: Some(release_callback::<T>), }; } </code></pre> Now all that is left is the actual implementation of the dispatch_device</code> function. First, we will use IoGetCurrentIrpStackLocation</code></a> to get the associated data of the I/O request. In addition, we wrap the pointer to the DEVICE_OBJECT</code> into a Device</code>, so we can easily access the vtable. We then simply dispatch the request to the dispatch</code> callback. pub extern "C" fn dispatch_device( device: *mut DEVICE_OBJECT, irp: *mut IRP, ) -> NTSTATUS { let stack_location = unsafe { &*IoGetCurrentIrpStackLocation(irp) }; let device = unsafe { Device::from_raw(device) }; let vtable = device.vtable(); match vtable.dispatch { Some(dispatch) => dispatch(device.into_raw(), irp, stack_location.MajorFunction), _ => { device.into_raw(), STATUS_SUCCESS } } } </code></pre> That concludes our dispatch_device</code> function. Symbolic Links</h1> There is just one more issue and that is when we create our device, it will have an internal Windows NT name like \Device\Example</code>, but that won't be accessible to userspace at all. In order for our device to be accessible to userspace, it needs a path that lives in the global namespace. Such a path looks something like the following: \??\Example</code> and we simply can create a symbolic link to \Device\Example</code>. Fortunately, compared to the rest we have implemented thus far, implementing the code to create and delete symbolic links is rather trivial. We just need to wrap the IoCreateSymbolicLink</code></a> and IoDeleteSymbolicLink</code></a> functions: pub struct SymbolicLink { name: U16CString, } impl SymbolicLink { pub fn new(name: &str, target: &str) -> Result<Self, Error> { // Convert the name to UTF-16 and then create a UNICODE_STRING. let name = U16CString::from_str(name).unwrap(); let mut name_ptr = create_unicode_string(name.as_slice()); // Convert the target to UTF-16 and then create a UNICODE_STRING. let target = U16CString::from_str(target).unwrap(); let mut target_ptr = create_unicode_string(target.as_slice()); let status = unsafe { IoCreateSymbolicLink(&mut name_ptr, &mut target_ptr) }; if status != STATUS_SUCCESS { return Err(Error::from_kernel_errno(status)); } Ok(Self { name, }) } } impl Drop for SymbolicLink { fn drop(&mut self) { let mut name_ptr = create_unicode_string(self.name.as_slice()); unsafe { IoDeleteSymbolicLink(&mut name_ptr); } } } </code></pre> The code above mostly has to deal with the usual Unicode conversions as we have seen before and simply stores the path of the symbolic link, such that we can call IoDeleteSymbolicLink</code></a> in our Drop</code> implementation. Extending our Example</h1> Wow, that was a lot we had to go through to, but now we have a nice and clean API to use to create and delete devices. Let's extend our driver example and set up our own device that is visible to userspace. We won't do much interesting other than print whenever an application opens or closes a handle to our device file and mark the I/O request as completed: struct MyDevice; impl DeviceOperations for MyDevice { fn create(&mut self, _device: &Device, request: &IoRequest) -> Result<(), Error> { println!("userspace opened the device"); request.complete(Ok(0)); Ok(()) } fn close(&mut self, _device: &Device, request: &IoRequest) -> Result<(), Error> { println!("userspace closed the device"); request.complete(Ok(0)); Ok(()) } fn cleanup(&mut self, _device: &Device, request: &IoRequest) -> Result<(), Error> { println!("device is no longer in use by userspace"); request.complete(Ok(0)); Ok(()) } } </code></pre> First we declared MyDevice</code> which is the user associated data for our Device</code> and implemented the create</code>, close</code> and cleanup</code> functions to print messages when an application interacts with our device file. We then create the device file and the symbolic link in our Module</code>: struct Module { _device: Device, _symbolic_link: SymbolicLink, } impl KernelModule for Module { fn init(mut driver: Driver, _: &str) -> Result<Self, Error> { let device = driver.create_device( "\\Device\\Example", DeviceType::Unknown, DeviceFlags::SECURE_OPEN, Access::NonExclusive, MyDevice, )?; let symbolic_link = SymbolicLink::new("\\??\\Example", "\\Device\\Example")?; Ok(Module { _device: device, _symbolic_link: symbolic_link, }) } } </code></pre> As usual, the code can also be found at Github</a>. Testing</h1> Run both WinObj.exe</code> and DebugView.exe</code> in the background. In WinObj.exe</code>, click on Driver on the left. Then create and start the driver service: sc create example binPath=$(realpath target/x86_64-pc-windows-msvc/debug/driver.sys) type=kernel sc start example </code></pre> This should show the Example device in WinObj.exe</code> as shown in Figure 1. Figure 1: WinObj.exe</code> should show the "Example" device. </figcaption> </figure> </center> Right-click on the Example device in WinObj.exe and then click on Properties... Then a dialog window should appear with the properties of the device. Simply click on OK to close it. This interaction should show the messages from our driver as shown in Figure 2. Figure 2: Output in DebugView.exe</code> after interacting with the "Example" device. </figcaption> </figure> </center> If we go to WinObj.exe</code> and click on GLOBAL?? on the left, it should also show the Example device with a symbolic link to \Device\Example</code>. Userspace</h1> Instead of relying on WinObj.exe</code>, we can also write our own userspace program in Rust to open and close the device file. It's actually very simple to do that: use std::fs::File; fn main() -> Result<(), std::io::Error> { let _file = File::open("\\??\\Example")?; Ok(()) } </code></pre> While the above program does not much other than opening and closing the device file, it will become very useful in later tutorials as we will be extending our device file implementation. Running the above program will produce similar messages in DebugView.exe</code> as interacting with our driver using WinObj.exe</code>. The above user code can also be found on Github</a>. What's Next?</h1> In the next article</a> we will look at how to extend our device with read and write callbacks such that we can handle reads from and writes to our device files from userspace. References</h1> IoCreateDevice</code> - MSDN</a></li> IoDeleteDevice</code> - MSDN</a></li> "Specifying Device Types" - MSDN</a></li> IoGetCurrentIrpStackLocation</code> - MSDN</a></li> IoCompleteRequest</code> - MSDN</a></li> "I/O Request Packets" - MSDN</a></li> IoCreateSymbolicLink</code> - MSDN</a></li> IoDeleteSymbolicLink</code> - MSDN</a></li> </ol> Windows Drivers in Rust: Safe Framework 2021-09-25T00:00:00+00:00 Introduction</h1> In the previous article</a> we looked at using bindgen</a> to generate more complete bindings for the Windows kernel API. These bindings can also be found at Github</a>. However, the problem is that we still have to deal with writing unsafe code. In this article we will be looking at creating a Microsoft Windows kernel driver framework with safe Rust abstractions in a new crate called windows-kernel-rs</a>. Memory Allocations</h1> As mentioned before we can currently not rely on the standard Rust library which provides types and data structures such as String</code>, Vec</code> and HashMap</code>, and it would be very convenient if we could actually use those. It turns out that these will help us implement the print!</code> and println!</code> macros that we are used to in Rust. Fortunately, these types are available in the alloc</code> crate, which we can use once we implemented our very own allocator. In fact, the Windows kernel API provides us with functions to allocate and free our own memory called ExAllocatePool</code> and ExFreePool</code> which we can use to implement our own allocator in Rust. We just have to define our own type and then implement the core::alloc::GlobalAlloc</code> trait. So we set up our new crate as before: cargo new --lib windows-kernel-rs </code></pre> Similar to our example driver we will be setting the panic handling to abort and to use our windows-kernel-sys</a> crate, which provides the unsafe bindings. [profile.dev] panic = "abort" [profile.release] panic = "abort" [dependencies] windows-kernel-sys = { path = "../windows-kernel-sys" } </code></pre> We then change src/lib.rs</code> to state that we are not using the standard library and import our allocator</code> module that we will be writing: #![no_std] pub mod allocator; </code></pre> We then implement our own allocator called KernelAllocator</code> in src/allocator.rs</code>: use core::alloc::{GlobalAlloc, Layout}; use windows_kernel_sys::base::_POOL_TYPE as POOL_TYPE; use windows_kernel_sys::ntoskrnl::{ExAllocatePool, ExFreePool}; pub struct KernelAllocator; unsafe impl GlobalAllocator for KernelAllocator { unsafe fn alloc(&self, layout: Layout) -> *mut u8 { let ptr = ExAllocatePool(POOL_TYPE::NonPagedPool, layout.size() as u64); if ptr.is_null() { panic!("[kernel-alloc] failed to allocate pool."); } ptr as _ } unsafe fn dealloc(&self, ptr: *mut u8, _layout: Layout) { ExFreePool(ptr as _) } } </code></pre> Then we have to set up the allocator by using the global_allocator</code> attribute as follows in src/lib.rs</code>: #[global_allocator] static ALLOCATOR: allocator::KernelAllocator = allocator::KernelAllocator; </code></pre> However, if we try building this we will run into an error that there is no alloc_error_handler</code>. So let's define one. We will tell Rust that we want to use the experimental alloc_error_handler</code> feature as follows in src/lib.rs</code>: #![feature(alloc_error_handler)] </code></pre> Then we define the actual allocation error handler in src/allocator.rs</code>: #[alloc_error_handler] fn alloc_error(_: Layout) -> ! { loop {} } </code></pre> While we are at it, we can also move our panic handler into this crate: #[panic_handler] fn panic(_info: &core::panic::PanicInfo) -> ! { loop {} } </code></pre> Then there are currently two more issues with _fltused</code></a> and __CxxFrameHandler3</code> caused by the fact that we actually invoke the panic!</code> macro from our allocator. These are actually bugs that we can temporarily resolve as follows: #[used] #[no_mangle] pub static _fltused: i32 = 0; #[no_mangle] pub extern "system" fn __CxxFrameHandler3() -> i32 { 0 } </code></pre> As of writing I cannot find much about the current state of __CxxFrameHandler3</code>. The issue</a> was opened before, and this pull request</a> should address the issue somewhat, but that does not seem to be entirely true. </blockquote> With all this our kernel allocator is in place and we can actually start using the alloc</code> crate. It turns out that ExAllocatePool</code> and ExFreePool</code> have been deprecated</a> since Microsoft Windows 10, version 2004 and later. Since we used bingen the replacements are actually available to us, but I still have to look into actually using them. I will update this part of the article once I get to that point. </blockquote> Printing Macros</h1> Now that our allocator is set up, we will actually have access to the format!</code> macro which will turn out to be very useful when implementing the print!</code> and println!</code> macros. The details have actually been covered by both Matthias</a> and Philipp</a> who both provide excellent articles on implementing print!</code> and println!</code> macros. Remember that until now we have been using the DbgPrint</code> function which expects a null-terminated or C-style string. In our case we will implement a function called _print</code> that accepts core::fmt::Arguments</code> and then uses alloc::format!</code> to actually format the string, but at the same time ensure that the string is null-terminated. Thus we first import the DbgPrint</code> function in src/io.rs</code>: use windows_kernel_sys::ntoskrnl::DbgPrint; </code></pre> Then we will implement our _print</code> function as follows: #[doc(hidden)] pub fn _print(args: core::fmt::Arguments) { let s = alloc::format!("{}\0", args); // Print the string. unsafe { DbgPrint(s.as_ptr() as _) }; } </code></pre> Note that we are using #[doc(hidden)]</code> as this is more of an internal function that we are not going to use directly, but to implement the print!</code> and println!</code> macros. For the print!</code> macro, we will use format_args!</code> and then simply call this function: #[macro_export] macro_rules! print { ($($arg:tt)*) => ($crate::io::_print(format_args!($($arg)*))); } </code></pre> As you may have noticed, our macro refers to the _print</code> function using $crate::io::_print</code>, this way when we actually import and use the macro from our driver, we don't have to also import the _print</code> function itself. Now that we have the print!</code> macro, we can use that to implement the println!</code> macro which simply prints a newline if no arguments are given, or formats the arguments and prints a newline: #[macro_export] macro_rules! println { () => ($crate::print!("\n")); ($($arg:tt)*) => ($crate::print!("{}\n", format_args!($($arg)*))); } </code></pre> This completes our implementation of the print!</code> and println!</code> macros. Handling Errors</h1> The functions in the Windows kernel API uses the NTSTATUS</code> type to indicate success or return an error. Therefore, we will simply wrap NTSTATUS</code> using our own Error</code> type in src/error.rs</code>: pub struct Error(NTSTATUS) </code></pre> We can then add existing status codes as error by defining constants and add methods to wrap NTSTATUS</code> into an Error</code> and to extract the NTSTATUS</code> from the Error</code>: impl Error { pub const UNSUCCESSFUL: Error = Error(STATUS_UNSUCCESSFUL); pub const NOT_IMPLEMENTED: Error = Error(STATUS_NOT_IMPLEMENTED); pub fn from_kernel_errno(status: NTSTATUS) -> Error { Error(status) } pub fn to_kernel_errno(&self) -> NTSTATUS { self.0 } } </code></pre> With the Error</code> type defined, we can now simply use Result<T, Error></code> where Ok</code> replaces STATUS_SUCCESS</code>. Boilerplate Code</h1> The last part of our framework involves providing a macro that essentially creates the driver_entry</code> and driver_exit</code> functions for us. We first start by providing a trait that the user of our crate can implement that simply has an init</code> and cleanup</code> function. pub trait KernelModule: Sized + Sync { fn init(driver: Driver, registry_path: &str) -> Result<Self, Error>; fn cleanup(&mut self, driver: Driver); } </code></pre> Since we can't couple the KernelModule</code> to the DRIVER_OBJECT</code> we have to globally declare it. We will be using a trick in our macro where we declare it as follows: static mut __MOD: Option<$module> = None; </code></pre> However, to be able to declare it as such, we have to make sure the KernelModule</code> is both Sized</code> and Sync</code>. We also want to provide safe abstractions for the parameters passed to the driver_entry</code> and the driver_exit</code> functions. For the first parameter, we will start with a basic abstraction for the DRIVER_OBJECT</code> in src/driver.rs</code>: pub struct Driver { pub(crate) raw: *mut DRIVER_OBJECT, } impl Driver { pub unsafe fn from_raw(raw: *mut DRIVER_OBJECT) -> Self { Self { raw, } } pub unsafe fn as_raw(&self) -> *const DRIVER_OBJECT { self.raw as _ } pub unsafe fn as_raw_mut(&mut self) -> *mut DRIVER_OBJECT { self.raw as _ } } </code></pre> For the second parameter, we will be relying on the widestring</a> crate to convert the UTF-16 string into &str</code>. We can then use the following code to safely wrap these parameters: let driver = unsafe { Driver::from_raw(driver) }; let registry_path = unsafe { U16CString::from_ptr_str(registry_path.Buffer) }; let registry_path = registry_path.to_string_lossy(); </code></pre> Finally, we can implement the full macro kernel_module</code> as follows: pub use crate::driver::Driver; pub use crate::error::Error; pub use widestring::U16CString; pub use windows_kernel_sys::base::{DRIVER_OBJECT, NTSTATUS, STATUS_SUCCESS, UNICODE_STRING}; #[macro_export] macro_rules! kernel_module { ($module:ty) => { static mut __MOD: Option<$module> = None; #[no_mangle] pub extern "system" fn driver_entry( driver: &mut $crate::DRIVER_OBJECT, registry_path: &$crate::UNICODE_STRING, ) -> $crate::NTSTATUS { unsafe { driver.DriverUnload = Some(driver_exit); } let driver = unsafe { Driver::from_raw(driver) }; let registry_path = unsafe { $crate::U16CString::from_ptr_str(registry_path.Buffer) }; let registry_path = registry_path.to_string_lossy(); match <$module as $crate::KernelModule>::init(driver, registry_path.as_str()) { Ok(m) => { unsafe { __MOD = Some(m); } $crate::STATUS_SUCCESS } Err(e) => { e.to_kernel_errno() } } } pub unsafe extern "C" fn driver_exit( driver: *mut $crate::DRIVER_OBJECT, ) { let driver = unsafe { Driver::from_raw(driver) }; match __MOD.take() { Some(mut m) => m.cleanup(driver), _ => (), } } }; } </code></pre> The macro above generates the boilerplate for our kernel driver implementing both the driver_entry</code> and driver_exit</code> functions. The driver_entry</code> function converts the parameters into safe types, calls the init</code> function of the KernelModule</code> and stores the constructed module into __MOD</code>. The driver_exit</code> function takes the module from __MOD</code> and then calls the cleanup</code> function and implicitly drops the module. Using the Framework</h1> With our new framework in place, we can rewrite our driver into a much cleaner version. The code can also be found at Github</a>: #![no_std] use windows_kernel_rs::{Driver, Error, kernel_module, KernelModule, println}; struct Module; impl KernelModule for Module { fn init(_: Driver, _: &str) -> Result<Self, Error> { println!("Hello, world!"); Ok(Module) } fn cleanup(&mut self, _: Driver) { println!("Bye bye!"); } } kernel_module!(Module); </code></pre> What's Next?</h1> We now have a nice safe framework in Rust that we can use to write Windows kernel drivers, but we haven't been doing much interesting as our driver only prints "Hello, world" and "Bye bye" whenever we load and unload it. In the next article</a> we will look at how to create device files, such that userspace can actually interact with our driver. Acknowledgements</h1> Thanks to Matthias</a> for taking the time to explore how to write kernel drivers for Microsoft Windows and especially taking the time to write up the details of the process. Especially the blog post about kernel printing with Rust</a> has been valuable. Similarly, I would like to thank Philipp</a> for explaining how to implement the print!</code> and println!</code> macros in an operating system or no_std</code> environment in general. Finally, I would like to thank Alex and Geoffrey (among others) for the Linux Kernel Modules in Rust project</a> which explores safe abstractions in Rust for Linux kernel modules. References</h1> https://github.com/Trantect/win_driver_example/issues/4</a></li> 'Updating Deprecated ExAllocatePool Calls" - MSDN</a></li> "Kernel Printing with Rust" - Not Matthias</a></li> "Writing an OS in Rust - VGA Text Mode" - Philipp Oppermann</a></li> </ol> Windows Drivers in Rust: Hello World 2021-09-22T00:00:00+00:00 Introduction</h1> In the previous article</a> we went through a number of prerequisites that we need installed to start writing our kernel driver for Microsoft Windows. Now that we have all of those installed, we can start writing our first kernel driver. Getting Started</h1> To get started, we will create a new Rust project for our driver using the following command: cargo new --lib driver </code></pre> As you may have noticed, we asked cargo</code> to create a library. Unfortunately, Rust does not know how to link a kernel driver by default, it just knows how to link executables, static libraries and dynamic libraries. Fortunately for us, a kernel driver is technically a dynamic link library or .dll</code> file, so we can start out by creating a library. Next, we add the following to the Cargo.toml</code> file to tell Rust to actually build a dynamic library: [lib] crate-type = ["cdylib"] </code></pre> In addition, we will tell Cargo that our project depends on the Rust Nightly toolchain. To do that we will run the following in our project's directory: echo "nightly" > rust-toolchain </code></pre> Passing Linker Options</h1> Now we have set up an empty project that builds an actual .dll</code> file. One problem with this is that Rust does not know that we are trying to create a kernel driver, so it will call the MSVC linker with the default options to link a .dll</code> to be used by normal Windows applications. So we have to specify the options to actually pass to the MSVC linker. We can do this by writing the following to .cargo/config</code> in your project: [build] target = "x86_64-pc-windows-msvc" rustflags = [ # Pre Link Args "-Z", "pre-link-arg=/NOLOGO", "-Z", "pre-link-arg=/NXCOMPAT", "-Z", "pre-link-arg=/NODEFAULTLIB", "-Z", "pre-link-arg=/SUBSYSTEM:NATIVE", "-Z", "pre-link-arg=/DRIVER", "-Z", "pre-link-arg=/DYNAMICBASE", "-Z", "pre-link-arg=/MANIFEST:NO", # Post Link Args "-C", "link-arg=/OPT:REF,ICF", "-C", "link-arg=/ENTRY:driver_entry", "-C", "link-arg=/MERGE:.edata=.rdata", "-C", "link-arg=/MERGE:.rustc=.data", "-C", "link-arg=/INTEGRITYCHECK", ] </code></pre> Here is an explanation of the MSVC linker options</a> that we are passing to the linker: The /NOLOGO</code> option simply supresses the startup banner containing the copyright message and the version number.</li> The /NXCOMPAT</code> option marks the executable as compatible with Data Execution Prevention (DEP). DEP is a feature in Microsoft Windows that marks data pages as non-executable. This prevents an attacker from injecting malicious code into those data pages, that can then get executed.</li> The /NODEFAULTLIB</code> option tells the linker to not link in any default libraries, since those won't be available when writing a kernel driver.</li> The /SUBSYSTEM:NATIVE</code> option specifies the subsystem or the environment to use for the executable. NATIVE</code> provides us with an environment suitable for building a kernel mode driver for Windows NT and derivates.</li> The /DRIVER</code> option tells the linker that we are building a Windows NT (and derivates) kernel mode driver.</li> The /DYNAMICBASE</code> option generates an executable image that can be randomly rebased at load time by using the Address Space Layout Randomization (ASLR) feature. ASLR is a feature that randomizes the virtual memory allocations, such that the locations of heap, stacks and the code are randomized, to effectively make buffer overflow attacks less trivial to exploit.</li> The MANIFEST:NO</code> option tells the linker to not generate a Manifest file</a>.</li> The /OPT:REF,ICF</code> option specifies the linker optimizations to apply. /OPT:REF</code> eliminates functions and data that are never referenced. /OPT:ICF</code> performs identiical COMDAT (common data) folding</a>.</li> The /ENTRY:driver_entry</code> option tells the linker that the entry point of our driver will be the driver_entry</code> function.</li> The /MERGE:.edata=.rdata</code> option merges the .edata</code> section into the .rdata</code> section, while the /MERGE:.rustc=.data' option merges the </code>.rustcsection into the</code>.datasection. Executables consist of a number of sections such as</code>.text(code),</code>.rdata(read-only or constant data),</code>.data(initialized data) and</code>.bss` (uninitialized data).</li> The /INTEGRITYCHECK</code> option specifies that the digital signature of the binary image must be checked at load time.</li> </ul> The Driver Entry Point</h1> Note that we set the entry point of our driver to the driver_entry</code> function by specifying the /ENTRY:driver_entry</code> option. Remove any code in src/lib.rs</code> and let's get started with that function in src/lib.rs</code>: #[no_mangle] pub extern "system" fn driver_entry() -> u32 { 0 /* NT_STATUS_SUCCESS */ } </code></pre> In the code above, #[no_mangle]</code> attribute tells the Rust compiler to not perform name mangling on the function name, such that it is simply visible as driver_entry</code> during the linking process. "system"</code> tells the Rust compiler that the driver_entry</code> function should be using the "system ABI"</a>, which defaults to the "stdcall" ABI on Microsoft Windows. Since we are writing a kernel driver, we won't have access to the Windows API that is normally available to us when writing Windows applications or libraries. Therefore, as the Rust standard library depends on the Windows API, we cannot use it from our kernel driver. Add the following line to the top of src/lib.rs</code> to tell the Rust compiler that we don't want to use the standard library: #![no_std] </code></pre> By disabling the standard library, we also disable the panic handler that comes with it by default. Thus we have to define our own in src/lib.rs</code>: use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} } </code></pre> By default Rust will unwind the stack whenever a panic occurs</a>, which in our case will produce an error about Rust not being able to find eh_personality</code>. Fortunately, we can simply change the panic behaviour to abort immediately by adding the following to Cargo.toml</code>. [profile.dev] panic = "abort" [profile.release] panic = "abort" </code></pre> Signing the Driver</h1> Running cargo build</code> has a number of problems: First, it produces a .dll</code> file, but the file extension for a Windows kernel driver should be .sys</code>, so we have to rename it. Second, in order to be able to load our driver, the binary file has to be signed. We will be using cargo-make</a> to automate these processes. Install it by running the following command (if you don't have it yet): cargo install --force cargo-make </code></pre> Then we can start writing our Makefile.toml</code> file for our project. We will first add a task named build-driver</code> to build our project: [tasks.build-driver] script = ''' cargo b %BUILD_FLAGS% ''' </code></pre> In order to rename the binary file, we need to know where it is located based on whether we are building it in debug or in release mode. So let's add some environment variables to the top of our Makefile.toml</code>: [env.development] TARGET_PATH = "target/x86_64-pc-windows-msvc/debug" [env.production] TARGET_PATH = "target/x86_64-pc-windows-msvc/release" BUILD_RELEASE = "--release" </code></pre> Now we can write the task that actually renames the file: [tasks.rename] ignore_errors = true script = ''' cd %TARGET_PATH% rename driver.dll driver.sys ''' </code></pre> Finally, we will need to sign our driver, but to do that we need access to the MSVC build tools. To set up the build environment, we will need to load the vcvars64.bat</code> file. However, where that file is located depends on what version of Microsoft Visual Studio you have installed: Microsoft Visual Studio Build Tools: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvars64.bat</code></li> Microsoft Visual Studio 2019 Community Edition: C:\Program Files(x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat</code></li> </ul> Rather than trying to automatically detect this path, we will just use an environment variable at the top of our Makefile.toml</code>: [env] VC_BUILD_DIR="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build\\vcvars64.bat" </code></pre> Now that the path to the vcvars64.bat</code> file has been defined, we can write our task to sign the driver. The task will simply load up the build environment, create a certificate if we don't have one, and then sign the driver with our certificate: [tasks.sign] dependencies = ["build-driver", "rename"] script = ''' call "%VC_BUILD_DIR%" if not exist DriverCertificate.cer ( makecert -r -pe -ss PrivateCertStore -n CN=DriverCertificate DriverCertificate.cer ) else ( echo Certificate already exi sts. ) signtool sign /a /v /s PrivateCertStore /n DriverCertificate /fd certHash /t http://timestamp.digicert.com %TARGET_PATH%/driver.sys ''' </code></pre> At this point, we should be able to run the following command to build and sign our driver: cargo make sign </code></pre> Accessing the Kernel API</h1> Currently our driver isn't doing anything useful, but we cannot access the kernel libraries yet as we need bindings to the kernel API, i.e. function and type definitions in Rust, and as we have to link our kernel driver against the kernel libraries providing those functions. To solve the bindings issue, we will simply use Trantect's fork of winapi-rs</a> for now. Add the following to Cargo.toml</code>: [dependencies.winapi] git = "https://github.com/Trantect/winapi-rs.git" branch = "feature/km" features = [ "wdm", "ntstatus", ] </code></pre> The next problem is that we have to tell our linker where to find the kernel libraries to link against. These can normally be found in C:\Program Files (x86)\Windows Kit\10\lib\10.\*\km</code>, but instead of defining the path ourselves, we will write a build.rs</code> to find the correct path for us. We will be using the thiserror and winreg crates for this, so we will add the following to Cargo.toml</code>: [build-dependencies] thiserror = "1.0" winreg = "0.10" </code></pre> Our build.rs</code> starts with a number of things we want to import: use std::path::PathBuf; use thiserror::Error; use winreg::RegKey; use winreg::enums::HKEY_LOCAL_MACHINE; </code></pre> Then we use thiserror to define our own Error</code> type: #[derive(Debug, Error)] pub enum Error { #[error(transparent)] IoError(#[from] std::io::Error), #[error("cannot find the directory")] DirectoryNotFound, } </code></pre> We can get the C:\Program Files (x86)\Windows Kits\10</code> path from the Windows registry from HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Kits\Installed Roots\KitsRoot10</code>. We will use the winreg crate to get the value of this key: /// Retrieves the path to the Windows Kits directory. The default should be /// `C:\Program Files (x86)\Windows Kits\10`. pub fn get_windows_kits_dir() -> Result<PathBuf, Error> { let hklm = RegKey::predef(HKEY_LOCAL_MACHINE); let key = r"SOFTWARE\Microsoft\Windows Kits\Installed Roots"; let dir: String = hklm.open_subkey(key)?.get_value("KitsRoot10")?; Ok(dir.into()) } </code></pre> While we are only interested in the location of the libraries for now, we will need the location of header files later. Therefore we define a DirectoryType</code> type to pass to the get_km_dir</code> function: pub enum DirectoryType { Include, Library, } </code></pre> We can then call the get_windows_kit_dir</code> function to get the Windows Kits directory. Based on the DirectoryType</code> that we want, we can then append Lib</code> or Include</code> to this path. At this path we can find one or more directory versions containing the version number, so we will just pick the highest version number. Finally, we need to append km</code> to the path. The following code does all of this: /// Retrieves the path to the kernel mode libraries. The path may look something like: /// `C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\km`. pub fn get_km_dir(dir_type: DirectoryType) -> Result<PathBuf, Error> { // We first append lib to the path and read the directory.. let dir = get_windows_kits_dir()? .join(match dir_type { DirectoryType::Include => "Include", DirectoryType::Library => "Lib", }) .read_dir()?; // In the lib directory we may have one or more directories named after the version of Windows, // we will be looking for the highest version number. let dir = dir .filter_map(|dir| dir.ok()) .map(|dir| dir.path()) .filter(|dir| { dir.components() .last() .and_then(|c| c.as_os_str().to_str()) .map(|c| c.starts_with("10.") && dir.join("km").is_dir()) .unwrap_or(false) }) .max() .ok_or_else(|| Error::DirectoryNotFound)?; // Finally append km to the path to get the path to the kernel mode libraries. Ok(dir.join("km")) } </code></pre> Then finally in our main</code> function, we use the get_km_dir</code> function to get the path to the kernel libraries. Then based on the target we are building, we append the architecture to this path and emit this path as a linker path: fn main() { // Get the path to the kernel libraries. let dir = get_km_dir(DirectoryType::Library).unwrap(); // Append the architecture based on our target. let target = std::env::var("TARGET").unwrap(); let arch = if target.contains("x86_64") { "x64" } else if target.contains("i686") { "x86" } else { panic!("The target {} is currently not supported.", target); }; let dir = dir.join(arch); // Specify the link path. println!("cargo:rustc-link-search=native={}", dir.to_str().unwrap()); } </code></pre> Hello, world!</h1> With access to the kernel API, we can now extend our driver to print "Hello, world!"</code>: use winapi::km::wdm::DbgPrint; use winapi::shared::ntdef::NTSTATUS; use winapi::shared::ntstatus::STATUS_SUCCESS; #[no_mangle] pub extern "system" fn driver_entry() -> NTSTATUS { unsafe { DbgPrint("Hello, world!\0".as_ptr()); } STATUS_SUCCESS } </code></pre> When calling the DbgPrint</code> function, make sure to add the \0</code> add the end of the string, as the functions expects null-terminated or C-style strings. Later on we will implement the print!</code> and println!</code> macros that we normally expect in Rust. </blockquote> Finally, our driver is still a bit inconvenient as we cannot unload the driver without rebooting Microsoft Windows. The driver_entry</code> function actually gets a number of arguments including a DRIVER_OBJECT</code> struct which has a function pointer called DriverUnload</code>. We will define a function driver_exit</code> that simply prints "Bye bye!"</code> and point DriverUnload</code> to this function: use winapi::km::wdm::{DbgPrint, DRIVER_OBJECT}; use winapi::shared::ntdef::{NT_SUCCESS, UNICODE_STRING}; use winapi::shared::ntstatus::STATUS_SUCCESS; #[no_mangle] pub extern "system" fn driver_entry(driver: &mut DRIVER_OBJECT, _: *const UNICODE_STRING) -> NTSTATUS { unsafe { DbgPrint("Hello, world!\0".as_ptr()); } driver.DriverUnload = Some(driver_exit); STATUS_SUCCESS } pub extern "system" fn driver_exit(driver: &mut DRIVER_OBJECT) { unsafe { DbgPrint("Bye bye!\0".as_ptr()); } } </code></pre> Loading the Driver</h1> Now that we have our driver code set up, we can simply build it as follows: cargo make sign </code></pre> The driver can be found at target/x86_64-pc-windows-msvc/debug/driver.sys</code>, but how do we actually load our driver and see the debug messages? Well, let's first run the DbgView.exe</a> utility as an administrator so we can actually capture our debug messages. In the Capture submenu, make sure that Capture Kernel and Enable Verbose Kernel Output are enabled. Now that we are capturing the debug messages, we can actually load our driver and to do that we will be using the service controller or sc.exe</code>. Run the following command to create the service for our driver: sc create example binPath=$(realpath target/x86_64-pc-windows-msvc/debug/driver.sys) type=kernel </code></pre> We can then start the service as follows, which loads our driver: sc start example </code></pre> We can then stop the service as follows, which unloads our driver: sc stop example </code></pre> Then after stopping it, we can delete the service: sc delete example </code></pre> If you closely watch DebugView, you should see the "Hello, world!"</code> and "Bye bye!"</code> messages from our kernel driver. The code can be found on Github</a>. What's Next?</h1> We have now programmed our very first kernel driver for Microsoft Windows that is capable of printing some debug messages. In the next article</a> we will look at how to actually generate more complete bindings for the kernel API, so we can drop the winapi-rs crate that we are currently using and don't have to define the functions and types ourselves. Acknowledgements</h1> Thanks to Matthias</a> for taking the time to explore how to write kernel drivers for Microsoft Windows and especially taking the time to write up the details of the process. References</h1> https://docs.microsoft.com/en-us/cpp/build/reference/linker-options?view=msvc-160</a></li> https://docs.microsoft.com/en-us/windows/win32/SbsCs/manifest-files-reference</a></li> https://devblogs.microsoft.com/oldnewthing/20161024-00/?p=94575</a></li> https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-with-panic.html</a></li> </ol> Windows Drivers in Rust: Prerequisites 2021-09-22T00:00:00+00:00 Introduction</h1> While I am definitely not the first to write about writing kernel drivers for Microsoft Windows in Rust</a>, I feel there are still a number of gaps that have to be addressed to make this more approachable. That is why I will be covering how to write kernel drivers in Rust for Microsoft Windows in a series of articles that together hopefully end up being a more complete guide. Before we can even get started developing Windows drivers in Rust, we first have to install a number of prerequisites, namely: Microsoft Visual Studio</li> Windows SDK</li> Windows Driver Kit</li> LLVM and Clang (for bindgen)</li> Microsoft Windows running in Testing Mode (to load self-signed Windows drivers)</li> Sysinternals DebugView</li> </ul> Setting up Visual Studio</h1> Download Build Tools for Visual Studio</a> on the Windows box, and run the installer. When the installer presents you with workloads to select, you can select "Desktop development with C++" as that will pretty much install everything you will need. Then click the button on the bottom-right of the installer to download and install Visual Studio Build Tools. This may take a while to install, so this is probably a good moment to brew yourself a cup of coffee. If for some reason, you want to modify the workloads or individual components later on, you can: Select the Start button in the bottom-left, and then Settings to open your settings </li> Type Apps & Features</code> in the search bar to find the Apps & Features setting. Scroll down to the Microsoft Visual Studio Installer.</li> Select Microsoft Visual Studio Installer and then click on the Modify button.</li> This should open the same installer as before, allowing you to change your workloads/individual components.</li> </ol> If you prefer a more complete installation of Visual Studio instead, you can download and install Visual Studo 2019 Community Edition</a> rather than the Build Tools for Visual Studio. </blockquote> Windows Driver Kit</h1> To install the Windows Driver Kit, we need a matching version of the Windows SDK first. Unfortunately, Visual Studio does not ship the most recent version of the Windows SDK yet, so we have to install it separately. Download the Windows SDK ISO</a> and right-click the file and select Mount. Then once the ISO has been mounted, you can run WinSDKSetup.exe</code> to install the Windows SDK. Now that we have the right version of the Windows SDK installed, we can proceed to download and install the Windows WDK</a>. You should now have version 10.22000.1 of the Windows SDK and the WDK. LLVM</h1> While we won't need LLVM until we look at generating bindings for the Windows Driver API in Windows Drivers in Rust: Generating Bindings</a>, it would be nice to have it already installed for when we get to using bindgen</a>, as it depends on libclang to generate Rust bindings from the C/C++ headers. The available releases for LLVM can be found on their download page</a>. At the time of writing the most recent release of LLVM is 12.0.1, which is what we will be using. We will be downloading the Windows installer</a> to install LLVM, which is named LLVM-12.0.1-win64.exe</code>. Download and run the installer to install LLVM and Clang. While MSYS2, WSL and Microsoft Visual Studio also provide their own versions of LLVM and Clang, these are not necessarily suitable for our needs. For instance, bindgen will not be able to parse the Windows Driver headers using the MSYS2 version at all due to compatibility reasons. Similarly, Microsoft Visual Studio seems to provide a 32-bit version of libclang.dll, while we need a 64-bit version. </blockquote> Testing Mode</h1> Open PowerShell in Administrator mode and run the following to allow Microsoft Windows to self-signed Windows driver: Bcdedit.exe -set TESTSIGNING ON </code></pre> After running the command above, you have to reboot your system and Microsoft Windows should boot up in testing mode. If the changes are successful, the bottom-right corner should indicate that Microsoft Windows is now running in Test Mode. In case you want to revert the change, you can run the following command in PowerShell running as an administrator: Bcdedit.exe -set TESTSIGNING OFF </code></pre> Sysinternals</h1> In order to see debug messages from our drivers, we will need a tool called DebugView from Sysinternals</a>. Download DebugView</a> to your system. It can simply be run in administrator mode to start capturing the debug messages. In addition, we will need a tool called WinObj from Sysinternals</a>, which will help us investigate the object manager namespace later on. Download WinObj</a> to your system. What's Next?</h1> In the next article</a> we will be looking at actually setting up our driver, such that we end up with a minimal driver that can print debug messages. References</h1> https://not-matthias.github.io/kernel-driver-with-rust/</a></li> https://docs.microsoft.com/en-us/windows-hardware/drivers/install/the-testsigning-boot-configuration-option</a></li> </ol> Windows Drivers in Rust: Generating Bindings 2021-09-21T00:00:00+00:00 Introduction</h1> In the previous article</a> we got our first driver to work and print some debug messages. However, at the time of writing there aren't any crates that really cover the Windows kernel API too well. Of course, we could write the bindings ourselves whenever we need them, but that is a lot of work. Wouldn't it be nice if we could just generate the Rust bindings from the C header files? Well, that is why we are going to be using bindgen</a>, which does exactly that! If you just would like to use the bindings, they can be found on Github</a>. This serves mostly as a documentation of the process, which may be useful in case you have to generate Rust bindings using bindgen yourself. </blockquote> Getting Started</h1> We will create a new crate for our unsafe bindings called windows-kernel-sys</code>: cargo new --lib windows-kernel-sys </code></pre> For this project, we will also be using Rust Nightly: echo nightly > rust-toolchain </code></pre> In addition, we copy over the build.rs</code> from our previous driver, but make sure that the main</code> function is empty: fn main() { } </code></pre> Finally, we will add the same build dependencies as before to Cargo.toml</code>: [build-dependencies] thiserror = "1.0" winreg = "0.10" </code></pre> Generating Bindings</h1> As mentioned before, we will be using bindgen, so let's add that to our build dependencies in Cargo.toml</code>: [build-dependencies] bindgen = "0.59" </code></pre> Then we create a file named src/wrapper.h</code> which will include the headers for which we want to generate bindings: #define _AMD64_ #include "ntdef.h" #include "ntifs.h" </code></pre> We will then use bindgen to generate the type definitions from this header file. We will do this in the generate_base</code> function in build.rs</code>: fn generate_base() { // Tell Cargo to re-run this if src/wrapper.h gets changed. println!("cargo:rerun-if-changed=src/wrapper.h"); // Find the include directory containing the kernel headers. let include_dir = get_km_dir(DirectoryType::Include).unwrap(); // Get the build directory. let out_path = PathBuf::from( std::env::var_os("OUT_DIR") .expect("the environment variable OUT_DIR is undefined") ); // Generate the bindings bindgen::Builder::default() .header("src/wrapper.h") .use_core() .derive_debug(false) .layout_tests(false) .ctypes_prefix("cty") .default_enum_style(bindgen::EnumVariation::ModuleConsts) .clang_arg(format!("-I{}", include_dir.to_str().unwrap())) .parse_callbacks(Box::new(bindgen::CargoCallbacks)) .ignore_functions() .generate() .unwrap() .write_to_file(out_path.join("base.rs")) .unwrap(); } </code></pre> The function above tells Cargo to re-run build.rs</code> whenever src/wrapper.h</code> changes. Then we look up the include directory that contains the kernel headers as well as our build directory. Finally, we pass a lot of options to bindgen::Builder</code>: .header("src/wrapper.h")</code>: we want to generate the bindings from this header.</li> .use_core()</code>: use core</code> instead of std</code>.</li> .derive_debug(false)</code>: don't derive Debug</code> for our types.</li> .layout_tests(false)</code>: don't generate layout tests.</li> .ctypes_prefix("cty")</code>: we will be importing the C-types from the cty crate.</li> .default_enum_style(bindgen::EnumVariation::ModuleConsts)</code>: generate enums as a module containing the constants.</li> .clang_arg(format!("-I{}", include_dir.to_str().unwrap())</code>: tell Clang where to find the kernel headers.</li> .parse_callbacks(Box::new(bindgen::CargoCallbacks))</code>: outputs re-run if changed for the files included by our header, such that when any of the headers change, build.rs</code> gets executed to regenerate the bindings.</li> .ignore_functions()</code>: ignore any function declarations, we will just generate the types for now.</li> </ul> Then in our main</code> function, we can simply call this function: fn main() { generate_base(); } </code></pre> Unfortunately, if we run cargo build</code>, then bindgen will panic with "Unable to get layout information?"</code>. This is caused by union _KGDTENTRY64 *GdtBase;</code> and union _KIDTENTRY64 *IdtBase;</code> in ntddk.h</code>, which are forward declarations of these unions without any declaration of the actual union. Fortunately, the definitions of these unions can be found online for _KGDTENTRY64</a> and _KIDTENTRY64</a>, so we can just copy them over into src/wrapper.h</code>: typedef union _KGDTENTRY64 { struct { unsigned short LimitLow; unsigned short BaseLow; union { struct { unsigned char BaseMiddle; unsigned char Flags1; unsigned char Flags2; unsigned char BaseHigh; } Bytes; struct { unsigned long BaseMiddle : 8; unsigned long Type : 5; unsigned long Dpl : 2; unsigned long Present : 1; unsigned long LimitHigh : 4; unsigned long System : 1; unsigned long LongMode : 1; unsigned long DefaultBig : 1; unsigned long Granularity : 1; unsigned long BaseHigh : 8; } Bits; }; unsigned long BaseUpper; unsigned long MustBeZero; }; unsigned __int64 Alignment; } KGDTENTRY64, *PKGDTENTRY64; typedef union _KIDTENTRY64 { struct { unsigned short OffsetLow; unsigned short Selector; unsigned short IstIndex : 3; unsigned short Reserved0 : 5; unsigned short Type : 5; unsigned short Dpl : 2; unsigned short Present : 1; unsigned short OffsetMiddle; unsigned long OffsetHigh; unsigned long Reserved1; }; unsigned __int64 Alignment; } KIDTENTRY64, *PKIDTENTRY64; </code></pre> Now if we call cargo build</code>, the generation actually succeeds. Importing Bindings</h1> First, we will add the cty crate as a dependency for our crate: [dependencies] cty = "0.2" </code></pre> Then in src/base.rs</code>, we can include the generated bindings: #![allow(non_upper_case_globals)] #![allow(non_camel_case_types)] #![allow(non_snake_case)] pub use cty::*; include!(concat!(env!("OUT_DIR"), "/base.rs")); pub const STATUS_SUCCESS: NTSTATUS = 0x00000000; </code></pre> We can then export the module in src/lib.rs</code>: #![no_std] #![feature(untagged_unions)] pub mod base; pub use cty::*; </code></pre> Generating Functions</h1> Now that we have the base</code> modules with all the types, we can proceed with generating the function declarations for ntoskrnl.lib</code>. As generating bindings is rather slow, we are going to feature gate the different kernel libraries that can be used. Let's define the features in Cargo.toml</code>: [features] default = ["ntoskrnl"] ntoskrnl = [] </code></pre> We will now implement the generate_ntoskrnl</code> function in our build.rs</code> to generate the function bindings: fn generate_ntoskrnl() { // Tell Cargo to re-run this if src/wrapper.h gets changed. println!("cargo:rerun-if-changed=src/wrapper.h"); // Link against ntoskrnl. println!("cargo:rustc-link-lib=ntoskrnl"); // Find the include directory containing the kernel headers. let include_dir = get_km_dir(DirectoryType::Include).unwrap(); // Get the build directory. let out_path = PathBuf::from( std::env::var_os("OUT_DIR") .expect("the environment variable OUT_DIR is undefined") ); // Generate the bindings bindgen::Builder::default() .header("src/wrapper.h") .use_core() .derive_debug(false) .layout_tests(false) .ctypes_prefix("cty") .default_enum_style(bindgen::EnumVariation::ModuleConsts) .clang_arg(format!("-I{}", include_dir.to_str().unwrap())) .parse_callbacks(Box::new(bindgen::CargoCallbacks)) .blocklist_type(".*") .allowlist_function(".*") .allowlist_recursively(false) .generate() .unwrap() .write_to_file(out_path.join("base.rs")) .unwrap(); } </code></pre> The code above is pretty much the same as the generate_base</code> function, except that we tell Cargo to link against ntoskrnl</code> and that we now generate the function declarations rather than the type declarations. Then we can change the main</code> function in build.rs</code> to generate the bindings: fn main() { generate_base(); #[cfg(feature = "ntoskrnl")] generate_ntoskrnl(); } </code></pre> We can then include the ntoskrnl bindings in our crate from src/ntoskrnl.rs</code>: #![allow(non_upper_case_globals)] #![allow(non_camel_case_types)] #![allow(non_snake_case)] use crate::base::*; include!(concat!(env!("OUT_DIR"), "/ntoskrnl.rs")); </code></pre> Then in src/lib.rs</code>, we can export the ntoskrnl</code> module: #[cfg(feature = "ntoskrnl")] pub mod ntoskrnl; </code></pre> Dealing with Inline Functions</h1> One limitation of bindgen is that it does not generate bindings to inline functions by default. Fortunately, this is easy to solve by creating a C file that contains non-inline functions that simply call the inline functions. For instance, to wrap IoGetCurrentIrpStackLocation</code>, we can write the following src/wrapper.c</code>: #include "wrapper.h" PIO_STACK_LOCATION _IoGetCurrentIrpStackLocation(irp: PIRP) { return IoGetCurrentIrpStackLocation(); } </code></pre> Then we can use the cc crate to compile the C source file. Let's add the build dependency to our Cargo.toml</code>: [build-dependencies] cc = "1.0" </code></pre> Then we can use the cc crate from the generate_ntoskrnl</code> function in build.rs</code> to build the wrapper: cc::Build::new() .flag("/kernel") .include(include_dir) .file("src/wrapper.c") .compile("wrapper_ntoskrnl"); </code></pre> Then we can define the function in src/ntoskrnl.rs</code>: #[link(name = "wrapper_ntoskrnl")] extern "C" { pub fn _IoGetCurrentIrpStackLocation(irp: PIRP) -> PIO_STACK_LOCATION; } pub use self::_IoGetCurrentIrpStackLocation as IoGetCurrentIrpStackLocation; </code></pre> Using the Bindings</h1> If we go back to our driver, we can now drop the winapi-rs crate and use our own bindings as a dependency. Let's add them to the Cargo.toml</code>: [dependencies] windows-kernel-sys = { path = "../windows-kernel-sys" } </code></pre> Then we have to change our src/lib.rs</code> as follows: use windows_kernel_sys::base::{DRIVER_OBJECT, NTSTATUS, UNICODE_STRING, STATUS_SUCCESS}; use windows_kernel_sys::ntoskrnl::DbgPrint; #[no_mangle] pub extern "system" fn driver_entry(driver: &mut DRIVER_OBJECT, _: *const UNICODE_STRING) -> NTSTATUS { unsafe { DbgPrint("Hello, world!\0".as_ptr() as _); } driver.DriverUnload = Some(driver_exit); STATUS_SUCCESS } pub unsafe extern "C" fn driver_exit(driver: *mut DRIVER_OBJECT) { DbgPrint("Bye bye!\0".as_ptr() as _); } </code></pre> If everything went well, the above will build against our own bindings: cargo make sign </code></pre> The code can be found on Github</a>. More complete bindings can also be found on Github</a>. What's Next?</h1> Now that we have our own bindings for the Windows kernel API, we can start implementing safe abstractions in Rust on top of our unsafe bindings. In the next article</a> we will be looking at how to implement a safe framework for writing Windows kernel drivers in Rust. More specifically, we will implement a global allocator such that we can use the heap-based types from the alloc</code> crate such as strings and Vec</code>, we will implement the print!</code> and println!</code> macros and a macro to produce the unsafe boilerplate code for the Windows kernel driver. Acknowledgements</h1> Thanks to Hussein Aitlahcen</a> for his tremendous work on figuring out how to get bindgen to work with the Windows kernel headers. References</h1> https://github.com/hussein-aitlahcen/windows-kernel-rs</a></li> https://rust-lang.github.io/rust-bindgen/</a></li> http://librecrops.github.io/lost-sdk/files/KGDTENTRY64.h.html</a></li> http://librecrops.github.io/lost-sdk/files/KIDTENTRY64.h.html</a></li> </ol> Setting up Windows for Remote Rust Development 2021-09-18T00:00:00+00:00 Introduction</h1> Having used Linux Gentoo with i3 as a tiling window manager, Vim as a text editor and terminals for too long, Microsoft Windows' GUI feels rather clunky to use when developing anything on Microsoft Windows. It would be ideal if we could just SSH into our Windows box and have an experience similar to the one we have on our Linux installation with access to tools such as Git, Vim and coreutils from our terminal. Yet at the same time, we want to be able to use Visual Studio to build native Windows binaries for our Rust applications. In this guide we will be setting up Visual Studio, Rust, MSYS 2 and OpenSSH server on our Windows box, such that we have a nice remote environment to use. Remote Desktop Protocol</h1> While we will be setting up a Linux-like environment on Microsoft Windows for our Rust development, we sometimes really have to resort to the GUI. For instance, to set up our Windows box in the first place. Therefore, we will first enable Remote Desktop on our Windows box: Unfortunately, the Remote Desktop feature does not seem to be available on Microsoft Windows 10 Home Edition out of the box. While it is possible to use something like the RDP Wrapper Library</a> to get Remote Desktop to work, I am assuming you have either Microsoft Windows 10 Pro Edition or Enterprise Edition installed. </blockquote> On the Windows box, select the Start button (Windows logo) in the bottom-right and then click the Settings (cog wheel icon) on the left.</li> Select the System (computer icon) button and then scroll down to Remote Desktop.</li> Click on the slider to enable Remote Desktop.</li> Select Select users that can remotely access this PC and add your account. If your account is an administrator, it will have access to Remote Desktop by default.</li> Select Power & sleep on the left, and set Sleep to never, such that the PC never goes to sleep.</li> </ol> Now that Remote Desktop has been set up, we need to note down the IP address that has been assigned to our Windows box: On the Windows box, type PowerShell.exe</code> in the search bar at the bottom-left.</li> Select Windows PowerShell, this will open PowerShell and present you with a blue terminal.</li> Type ipconfig</code> in PowerShell and note down the IPv4 address. We will be using this to connect to the Windows box from Linux.</li> </ol> On Linux, you can either use rdesktop</a> or freerdp</a> to connect to the Windows box. My recommendation is to install freerdp, as rdesktop will likely end up giving you the error message: "Failed to connect, CredSSP required by server." Install freerdp using your package manager: emerge freerdp </code></pre> Assuming your username is USER and the IP address of your Windows box is 192.168.1.42 (feel free to replace these arguments), you can then run the following command to connect to your Windows box over RDP: xfreerdp /u:"USER" /v:192.168.1.42:3389 /size:1920x1080 -grab-keyboard </code></pre> Setting up Visual Studio</h1> Download Build Tools for Visual Studio</a> on the Windows box, and run the installer. When the installer presents you with workloads to select, you can select "Desktop development with C++" as that will pretty much install everything you will need. Then click the button on the bottom-right of the installer to download and install Visual Studio Build Tools. This may take a while to install, so this is probably a good moment to brew yourself a cup of coffee. If for some reason, you want to modify the workloads or individual components later on, you can: Select the Start button in the bottom-left, and then Settings to open your settings </li> Type Apps & Features</code> in the search bar to find the Apps & Features setting. Scroll down to the Microsoft Visual Studio Installer.</li> Select Microsoft Visual Studio Installer and then click on the Modify button.</li> This should open the same installer as before, allowing you to change your workloads/individual components.</li> </ol> If you prefer a more complete installation of Visual Studio instead, you can download and install Visual Studo 2019 Community Edition</a> rather than the Build Tools for Visual Studio. </blockquote> Setting up MSYS2</h1> Download and install MSYS2</a>. Open the MSYS2 shell and type the following command to make sure it is up-to-date: pacman -Syu </code></pre> Then install any tools you want such as Git and Vim: pacman -Sy vim git </code></pre> Then we will be setting up MSYS2 to use the conventional location of our user directory at C:\Users\USER</code>, as this is also where our .ssh</code> directory will live. In the MSYS2 shell, change db_home: cygwin desc</code> in /etc/nsswitch.conf</code> to the following: db_home: windows </code></pre> Then we will be adding a HOME</code> environment variable for our account (this will be used by MSYS2 over OpenSSH): Select Start and then Settings.</li> Then search for Edit environment variables for your account</code>.</li> Click the New... button to add a new user variable.</li> The variable name should be HOME</code> and the value should be your user directory on Microsoft Windows (e.g. C:\Users\USER</code>).</li> </ol> Now if you start up a MSYS2 shell, your home directory should be C:\Users\USER</code>. Copy over the default .bashrc</code>, .bash_profile</code> and .profile</code> files from the old home directory: cp /home/USER/{.bash_profile, .bashrc, .profile} ~/ </code></pre> Setting up OpenSSH</h1> Run PowerShell as an administrator. Then run the following command to make sure that OpenSSH is available: Get-WindowsCapability -Online | ? Name -like 'OpenSSH*' </code></pre> This should return the following output if neither the client or the server are installed: Name : OpenSSH.Client~~~~0.0.1.0 State : NotPresent Name : OpenSSH.Server~~~~0.0.1.0 State : NotPresent </code></pre> Then install the server component as follows: Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0 </code></pre> While we don't necessarily need the client component, you can install it as follows: Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0 </code></pre> These should return the following output: Path : Online : True RestartNeeded : False </code></pre> Start the sshd service: Start-Service sshd </code></pre> Set up the sshd service to automatically start up whenever the Windows box boots: Set-Service -Name sshd -StartupType 'Automatic' </code></pre> Confirm that the firewall has been configured: Get-NetFirewallRule -Name *ssh* </code></pre> If there is no firewall rule named "OpenSSH-Server-In-TCP", then you can create it as follows: New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 </code></pre> By default, OpenSSH server will use the Windows command line, but we want to present the Bash shell from MSYS2 instead. Where MSYS2 is installed (e.g. C:\msys64\</code>) we will create a batch script named msys2.bat</code>: @echo off IF DEFINED SSH_TTY (C:\msys64\usr\bin\bash.exe -li %*) ELSE (C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe %*) </code></pre> Let's dissect the above batch script to get some understanding of what it does: @echo off</code> disables the commands from being echoed back to stdout.</li> IF DEFINED SSH_TTY () ELSE ()</code> checks if the SSH_TTY</code> environment variable is set. If it is set, it means that we have a tty and should start an interactive Bash shell. Otherwise, the user may be running something like scp</code>, in which case we default to PowerShell. Without this check we would end up breaking commands like scp</code>.</li> </ul> Point the default shell to use for OpenSSH connections to our batch script: New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\msys64\msys2.bat" -PropertyType String -Force </code></pre> Now when you ssh into the Windows box, it should present you with Bash from MSYS2. Using Public Keys</h1> The default sshd configuration on Microsoft Windows is a bit odd, as it will not look at $HOME/.ssh/authorized_keys</code> for administrator accounts, but instead at C:\ProgramData\ssh\administrators_authorized_keys</code>. To get consistent behaviour with what we expect coming from Linux, we will edit the configuration to always use $HOME/.ssh/authorized_keys</code> instead. On Microsoft Windows, the sshd_config</code> lives at C:\ProgramData\ssh\sshd_config</code>. Since the editing that file requires administrator rights, the easiest way to edit it is to run PowerShell as an administrator, point to the directory C:ProgramData\ssh\sshd_config</code> and run notepad</code> to open it: notepad sshd_config </code></pre> Open sshd_config</code> and remove the following lines: Match Group administrators AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys </code></pre> In addition, you should also disable password authentication: PasswordAuthentication no </code></pre> Run PowerShell as an administrator, and run the following to restart OpenSSH: Restart-Service ssh </code></pre> Then put your public key(s) in C:\Users\USER\.ssh\authorized_keys</code> (note the dot before ssh). If everything is set up correctly, you should be able to connect over SSH without needing to enter a password, and it should never ask for a password when the account does not exist or when the wrong private key is being used to connect. Setting up Rust</h1> Now connect to your Windows box over SSH and install rustup</a>: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh </code></pre> Just press enter to select the default option. rustup will simply use the stable-x86_64-pc-windows-msvc</code> toolchain by default. Add the following line to ~/.bash_profile</code>: export PATH="$HOME/.cargo/bin":$PATH </code></pre> Then you can type the following to reload your bash profile: source ~/.bash_profile </code></pre> Commands like cargo</code>, rustc</code> and rustup</code> should work now. You can make sure that stable-x86_64-pc-windows-msvc</code> is the default toolchain as follows: rustup toolchain list </code></pre> Finally, you should be able to use Rust: cargo new hello cd hello cargo run </code></pre> Why not WSL?</h1> Personally, I prefer MSYS2 due to my familiarity with MSYS2, as I have been using MinGW and MSYS2 on Microsoft Windows before WSL even became an option. Even though WSL feels more like an environment for Microsoft Windows users to use and develop Linux applications without having to use a VM or install a Linux distribution, WSL would also work for our purpose as the goal is to get an environment similar to Linux that we can ssh into and use to build native Windows applications. Therefore, I will also describe the changes needed to use something like WSL instead of MSYS2 for completeness. While it is possible to use OpenSSH from inside your WSL installation using something like subsystemctl</a> and genie</a> in combination with Windows Task Scheduler, my recommendation is to use the OpenSSH Server that comes with Microsoft Windows 10 instead. To set up OpenSSH on Microsoft Windows 10, you can simply follow the instructions outlined before. You can then use the following batch script as a default shell for OpenSSH: @echo off IF DEFINED SSH_TTY (C:\Windows\System32\wsl.exe ~ -d Alpine %*) ELSE (C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe %*) </code></pre> Replace Alpine</code> in the above batch script with the name of your WSL distribution to run the shell of the WSL environment you want. If you now use ssh to connect to your Windows box, you should end up in your WSL environment. Similarly, because we check whether the SSH_TTY</code> environment variable is set or not, scp</code> will work, but copy to or from C:\Users\USER by default. With some more work, it is probably to make scp use the WSL root filesystem instead. We can do this by using the first instance of PowerShell to first change the directory and then run a second instance of PowerShell to execute the SSH command. @echo off IF DEFINED SSH_TTY (C:\Windows\System32\wsl.exe ~ -d Arch %*) ELSE (C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe "cd C:\Users\USER\Desktop\rootfs\root; PowerShell %*") </code></pre> If we run scp now, it will copy files from/to /root in our WSL environment by default. Note: from the WSL environment's perspective the authorized_keys</code> are in /mnt/c/Users/USER/.ssh/authorized_keys</code>. To set up rustup to use MSVC in WSL, you can refer to https://github.com/strickczq/msvc-wsl-rust</a>. References</h1> https://docs.microsoft.com/en-us/windows-server/remote/remote-desktop-services/clients/remote-desktop-allow-access</a></li> https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse</a></li> https://docs.microsoft.com/en-us/windows/wsl/install-win10</a></li> </ol> Setting up mail autoconfiguration 2020-12-29T00:00:00+00:00 Installation</h1> Install the build dependencies: apk add build-base git openssl-dev </code></pre> Install Rustup: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env </code></pre> Clone the repository: git clone https://github.com/StephanvanSchaik/automail cd automail </code></pre> Build automail: RUSTFLAGS="-C target-feature=-crt-static" cargo build --release </code></pre> Install the binary: cp target/release/automail /usr/bin </code></pre> Create the automail user: useradd -MU automail </code></pre> Edit /etc/init.d/automail</code>: #!/sbin/openrc-run name=$RC_SVCNAME cfgfile="/etc/$RC_SVCNAME/$RC_SVCNAME.conf" command="/usr/bin/automail" command_args="" command_user="automail" pidfile="/run/$RC_SVCNAME/$RC_SVCNAME.pid" start_stop_daemon_args="" command_background="yes" depend() { need net } start_pre() { checkpath --directory --owner $command_user:$command_user --mode 0775 \ /run/$RC_SVCNAME /var/log/$RC_SVCNAME } </code></pre> Mark /etc/init.d/automail</code> as executable: chmod +x /etc/init.d/automail </code></pre> Configuration</h1> Create the /etc/automail</code> directory: mkdir /etc/automail </code></pre> Edit /etc/automail/Rocket.toml</code>: [default] address = "0.0.0.0" port = 3721 </code></pre> Edit /etc/automail/config.toml</code>: [ssl] key = "/etc/letsencrypt/live/example.com/privkey.pem" cert = "/etc/letsencrypt/live/example.com/cert.pem" chain = "/etc/letsencrypt/live/example.com/chain.pem" [[domain]] domain = "example.com" name = "Example Mail Configuratioon" short_name = "Example" [[domain.server]] protocol = "imap" hostname = "imap.example.com" port = 993 auth = "plain" encrypt = "ssl" [[domain.server]] protocol = "imap" hostname = "imap.example.com" port = 143 auth = "plain" encrypt = "starttls" [[domain.server]] protocol = "pop3" hostname = "pop3.example.com" port = 995 auth = "plain" encrypt = "ssl" [[domain.server]] protocol = "pop3" hostname = "pop3.example.com" port = 110 auth = "plain" encrypt = "starttls" [[domain.server]] protocol = "smtp" hostname = "smtp.example.com" port = 587 auth = "plain" encrypt = "starttls" [[domain.server]] protocol = "smtp" hostname = "smtp.example.com" port = 465 auth = "plain" encrypt = "ssl" </code></pre> Start the service: /etc/init.d/automail start rc-update add automail </code></pre> Edit /etc/nginx/sites-available/01_mail.example.com</code>: index index.html; server { listen 80; listen [::]:80; server_name autoconfig.example.com autodiscover.example.com mail.example.com; return 301 https://$host$request_uri; } server { listen 443 ssl; listen [::]:443 ssl; server_name autoconfig.example.com autodiscover.example.com mail.example.com; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot location / { proxy_pass http://localhost:3721; } location /.well-known { alias /var/www/example.com/htdocs/.well-known; } } </code></pre> Create the symlink: ln -s /etc/nginx/sites-available/01_mail.example.com /etc/nginx/sites-enabled/01_mail.example.com </code></pre> Reload the Nginx configuration: /etc/init.d/nginx reload </code></pre> Add the following DNS-record: _autodiscover._tcp SRV 0 1 443 autodiscover.example.com. </code></pre> Add the subdomains to the Let's Encrypt certificate. Testing</h1> Use an e-mail client with an account wizard like Thunderbird to test the setup. It should now be able to automatically detect the right settings for e-mail. You should also be able to download a MobileConfig file for Apple devices via [email protected]">https://mail.example.com/[email protected]</a>. Setting up ClamAV 2020-12-29T00:00:00+00:00 Installation</h1> Install ClamAV: apk add acf-clamav clamsmtp </code></pre> Add the postfix user to the clamav group: usermod -aG clamav postfix </code></pre> Configuration</h1> Start the services: /etc/init.d/clamd start /etc/init.d/clamsmtpd start rc-update add clamd rc-update add clamsmtpd </code></pre> Check if the ClamAV service is listening: netstat -anp | grep clamsmtpd </code></pre> Edit /etc/postfix/main.cf</code>: content_filter = scan:[127.0.0.1]:10025 </code></pre> Add the following to /etc/postfix/master.cf</code>: # AV scan filter (used by content_filter) scan unix - - n - 16 smtp -o smtp_send_xforward_command=yes -o smtp_enforce_tls=no # For injecting mail back into postfix from the filter 127.0.0.1:10026 inet n - n - 16 smtpd -o content_filter= -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks -o smtpd_helo_restrictions= -o smtpd_client_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks_style=host -o smtpd_authorized_xforward_hosts=127.0.0.0/8 </code></pre> Reload the Postfix configuration: /etc/init.d/postfix reload </code></pre> Testing</h1> To test your ClamAV installation, you can send an e-mail from another domain to yourself and check the e-mail headers. They should contain a X-Virus-Scanned</code> field if ClamAV is operational. X-Virus-Scanned: ClamAV using ClamSMTP </code></pre> Setting up Dovecot 2020-12-29T00:00:00+00:00 Installation</h1> Install dovecot</code>: apk add dovecot dovecot-lmtpd dovecot-pgsql dovecot-pigeonhole-plugin dovecot-pop3d </code></pre> Configuration</h1> Edit /etc/dovecot/dovecot.conf</code>: protocols = imap pop3 lmtp </code></pre> Edit /etc/dovecot/conf.d/10-mail.conf</code>: first_valid_uid = 105 last_valid_uid = 105 first_valid_gid = 107 last_valid_gid = 107 mail_location = maildir:/var/mail/domains/%d/%n </code></pre> Edit the imap service in /etc/dovecot/conf.d/10-master.conf</code>: service imap { vsz_limit = 1024M } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0666 user = postfix group = postfix } user = vmail group = postdrop } service auth { unix_listener auth-userdb { } unix_listener /var/spool/postfix/private/auth { mode = 0666 user = postfix group = postfix } unix_listener /var/spool/postfix/auth-master { mode = 0660 user = vmail group = postfix } user root } </code></pre> Edit the mailboxes in /etc/dovecot/conf.d/15-mailboxes.conf</code>: namespace inbox { inbox = yes mailbox Trash { auto = create special_use = \Trash } mailbox Spam { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } } </code></pre> Edit /etc/dovecot/conf.d/10-auth.conf</code>: auth_mechanisms = plain login auth_username_format = %Lu disable_plaintext_auth = no </code></pre> Edit /etc/dovecot/conf.d/20-lmtp.conf</code>: protocol lmtp { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins sieve } </code></pre> Edit /etc/dovecot/conf.d/20-managesieve.conf</code>: protocols = $protocols sieve </code></pre> Make sure to include auth-sql.conf.ext</code> from /etc/dovecot/conf.d/10-auth.conf</code>: !include auth-sql.conf.ext </code></pre> Edit /etc/dovecot/conf.d/auth-sql.conf.ext</code>: passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { args = uid=105 gid=107 home=/var/mail/domains/%d/%n driver = static } </code></pre> Edit /etc/dovecot/dovecot-sql.conf.ext</code>: connect = host=localhost dbname=postfix user=postfix password=password default_pass_scheme = SHA512-CRYPT password_query = \ SELECT username,password \ FROM mailbox \ WHERE local_part = '%n' AND domain = '%d' </code></pre> Allow TCP traffic on the IMAP and POP3 port (143 and 110) and port 4190: ufw allow imap ufw allow pop3 ufw allow 4190/tcp </code></pre> Check whether the Dovecot configuration is valid: dovecot -n </code></pre> Start the service: /etc/init.d/dovecot init rc-update add dovecot </code></pre> Check if Dovecot is listening on the IMAP and the POP3 port (143 and 110), as well as the /var/spool/postfix/private/dovecot-lmtp</code> and /var/spool/postfix/private/auth</code> UNIX sockets: netstat -anp | grep dovecot </code></pre> Configuring DNS</h1> Add the following DNS records: _imap._tcp SRV 143 imap.example.com. _pop3._tcp SRV 110 pop3.example.com. </code></pre> Thunderbird Add-on</h1> Download and install the Sieve Add-on for Thunderbird</a>. If you go to Tools > Sieve Message Filters, Thunderbird should open the page that lists your server connections. You should be able to connect to the Sieve server and manage the sieve scripts for your accounts. Setting up Alpine Linux 2020-12-29T00:00:00+00:00 Installation</h1> Use any live environment (LiveISO or rescue image) to boot up the system you want to set up. The image doesn't necessarily have to be Alpine Linux, but if you can boot up Alpine linux, the installation may be slightly easier. Use fdisk</code>, gdisk</code> or gparted</code> to partition the disk. Mount the partition that you want to use as the root filesystem: mkdir /mnt/chroot mount /dev/vda1 /mnt/chroot </code></pre> Download and extract the Alpine Linux rootfs: wget https://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-minirootfs-3.12.3-x86_64.tar.gz tar -xvpzf alpine-minirootfs-3.12.3-x86_64.tar.gz -C /mnt/chroot/ </code></pre> Check https://alpinelinux.org/downloads/</a> for the latest version. Edit /mnt/chroot/chroot.sh</code>: mount --rbind /dev /mnt/chroot/dev mount --make-rslave /mnt/chroot/dev mount -t proc /proc /mnt/chroot/proc mount --rbind /sys /mnt/chroot/sys mount --make-rslave /mnt/chroot/sys mount --rbind /tmp /mnt/chroot/tmp cp /etc/resolv.conf /mnt/chroot/etc/resolv.conf </code></pre> Using this script you can easily chroot</code> into the installation: sh chroot.sh chroot /mnt/chroot /bin/bash </code></pre> Add the following lines to /etc/apk/repositories</code>: @edge http://dl-cdn.alpinelinux.org/alpine/edge/main @community http://dl-cdn.alpinelinux.org/alpine/edge/community @testing http://dl-cdn.alpinelinux.org/alpine/edge/testing </code></pre> Installing basic software: apk add bash shadow openrc vim </code></pre> Remove the #</code> from the following line in /etc/inittab</code> to enable the serial console at boot: ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100 </code></pre> If you installed Alpine to a VPS or a container, then you may not need a boot loader, a kernel and the firmware to boot into Alpine. Otherwise, you can set up a Linux kernel: apk add linux-firmware linux-lts grub grub-efi </code></pre> Install GRUB: grub-install --target=x86_64-efi --efi-directory=/boot /dev/vda </code></pre> Generate the GRUB config: grub-mkconfig -o /boot/grub/grub.cfg </code></pre> Create user</h1> Create a user</code> that you can use to log onto the server: useradd -mUG wheel user </code></pre> Configure a password for the user</code>: passwd user </code></pre> Setting the timezone</h1> Install the tzdata</code> package: apk add tzdata </code></pre> You can list the available timezones as follows: ls /usr/share/zoneinfo </code></pre> Assuming you want to use the Europe/Amsterdam timezone, you can copy it as follow to the localtime file: cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime </code></pre> Do also specify your timezone: echo "Europe/Amsterdam" > /etc/timezone </code></pre> You can check the current time and date: date </code></pre> After configuring the timezone, you can remove the tzdata</code> package: apk del tzdata </code></pre> Setting the hostname</h1> Edit /etc/hostname</code>: hostname </code></pre> Networking</h1> Edit /etc/network/interfaces</code> and adjust the settings for IPv6 accordingly: auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp iface eth0 inet6 static address 2001:bc8:600:1236::1 netmask 64 gateway 2001:bc8:600:1236:: pre-up echo 1 > /proc/sys/net/ipv6/conf/eth0/accept_ra </code></pre> Restart the service: /etc/init.d/networking restart </code></pre> Firewall</h1> Install ufw: apk add ip6tables ufw@testing </code></pre> Block incoming traffic by default: ufw default deny incoming ufw default allow outgoing </code></pre> Start the service: ufw enable rc-update add ufw </code></pre> Check the status of ufw: ufw status </code></pre> SSH</h1> Install OpenSSH: apk add openssh-server </code></pre> Edit /etc/ssh/sshd_config</code>: PasswordAuthentication no AllowTcpForwarding yes </code></pre> Open the SSH port, but limit the amount of possible traffic: ufw limit ssh </code></pre> Start the service: /etc/init.d/sshd start rc-update add sshd </code></pre> Log in as user</code>: su user mkdir .ssh chmod 0700 </code></pre> Upload your public key as ~/.ssh/authorized_keys</code>. DNS</h1> Add an A-record and AAAA-record with your IPv4 and IPv6 addresses to your DNS records: @ IN A ( "163.172.149.186" ); @ IN AAAA ( "2001:bc8:600:1236::1" ); </code></pre> Add a CNAME-record to your DNS record: * IN CNAME ( "@" ); </code></pre> Finally, you also want to set up reverse DNS to map your IP addresses back to the hostname. This can typically be done through the provider of your server instance. Setting up Gitea 2020-12-29T00:00:00+00:00 Installation</h1> Install Gitea: apk add gitea </code></pre> Configuration</h1> Create the user and database: createuser -U postgres --pwprompt gitea createdb -U postgres --owner=gitea gitea </code></pre> Start the service: /etc/init.d/gitea start rc-update add gitea </code></pre> Forward the port using SSH: ssh -L 3000:localhost:3000 example.com </code></pre> Point your browser to http://localhost:3000/</a> and configure Gitea. Edit /etc/nginx/sites-available/01_git.example.com</code>: server { listen 80; listen [::]:80; server_name git.codentium.com; # Redirect HTTP traffic to HTTPS. return 301 https://$host$request_uri; } server { listen 443 ssl; listen [::]:443 ssl; server_name git.codentium.com; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem # managed by Certbot index index.html; # HTTP Strict Transport Security. add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; location / { proxy_pass http://localhost:3000; } location /.well-known { alias /var/www/example.com/htdocs/.well-known; } } </code></pre> Create a symlink to enable the site: ln -s /etc/nginx/sites-available/01_git.example.com /etc/nginx/sites-enabled/ </code></pre> Reload the Nginx configuration: /etc/init.d/nginx reload </code></pre> Setting up Let's Encrypt SSL 2020-12-29T00:00:00+00:00 Installation</h1> Install certbot</code>: apk add certbot certbot-nginx </code></pre> certbot --nginx -d example.com -d www.example.com </code></pre> Renewing the Certificates</h1> To test if the certificate renewal works, you can perform a dry run: certbot renew --dry-run </code></pre> To renew the certificates, run the following: certbot renew </code></pre> Add a DNS CAA record for your domain: @ CAA 0 issue "letsencrypt.org" </code></pre> Setting up Nginx</h1> Generate the Diffie-Hellman parameters: openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096 </code></pre> Edit /etc/nginx/nginx.conf</code>: http { include /etc/nginx/sites-enabled/*; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers !ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384; ssl_prefer_server_ciphers on; ssl_session_cache shared:SSL:10m; ssl_dhparam /etc/ssl/certs/dhparam.pem; ssl_ecdh_curve secp384r1; } </code></pre> Edit your per site-configurations in /etc/nginx/sites-available</code> to enable HTTPS: server { listen 80; listen [::]:80; listen 443 ssl; listen [::]:443 ssl; server_name .example.com; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot </code></pre> Allow TCP traffic on the HTTPS port (443): ufw allow https </code></pre> Reload the Nginx configuration: /etc/init.d/nginx reload </code></pre> Setting up Postfix</h1> Edit /etc/postfix/main.cf</code>: smtpd_use_tls = yes smtpd_tls_key_file = /etc/letsencrypt/live/example.com/privkey.pem smtpd_tls_cert_file = /etc/letsencrypt/live/example.com/fullchain.pem smtpd_tls_security_level = may smtpd_tls_auth_only = no smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_timeout = 3600s smtpd_tls_eecdh_grade = strong smtpd_tls_protocols= !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_mandatory_protocols= !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_mandatory_ciphers = high smtpd_tls_security_level=may smtpd_tls_ciphers = high tls_preempt_cipherlist = yes smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, SRP, 3DES, eNULL smtpd_tls_exclude_ciphers = aNULL, MD5 , DES, ADH, RC4, PSD, SRP, 3DES, eNULL smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 tls_random_source = dev:/dev/urandom </code></pre> Edit /etc/postfix/master.cf</code>: submission inet n - n - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sals_auth_enable=yes smtps inet n - n - - smtpd -o syslog_name=postfix/smtps -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes </code></pre> Allow TCP traffic on the SMTPS and Submission ports (port 465 and 587): ufw allow smtps ufw allow 587/tcp </code></pre> Reload the Postfix configuration: /etc/init.d/postfix reload </code></pre> Add the following DNS records: _submission._tcp SRV 587 smtp.example.com. </code></pre> Setting up Dovecot</h1> Generate the Diffie-Hellman parameters: openssl dhparam -out /etc/dovecot/dh.pem 4096 </code></pre> Edit /etc/dovecot/conf.d/10-ssl.conf</code>: ssl = required ssl_cert = </etc/letsencrypt/live/example.com/fullchain.pem ssl_key = </etc/letsencrypt/live/example.com/privkey.pem ssl_dh = </etc/dovecot/dh.pem ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes </code></pre> Allow TCP traffic on the IMAPS and POP3S ports (port 993 and 995): ufw allow imaps ufw allow pop3s </code></pre> Reload the Dovecot configuration: /etc/init.d/dovecot reload </code></pre> Add the following DNS records: _imaps._tcp SRV 993 imap.example.com. _pop3s._tcp SRV 995 pop3.example.com. </code></pre> Testing</h1> Use OpenSSL to test the connection: openssl s_client -connect imap -connect example.com:443 openssl s_client -starttls smtp -connect mail.example.com:587 openssl s_client -starttls imap -connect mail.example.com:143 openssl s_client -connect mail.example.com:993 </code></pre> Setting up Murmur 2020-12-29T00:00:00+00:00 Installation</h1> Install Murmur: apk add murmur </code></pre> Edit /etc/murmur.ini</code>: sslCert=/etc/letsencrypt/live/example.com/fullchain.pem sslKey=/etc/letsencrypt/live/example.com/privkey.pem </code></pre> Allow TCP traffic on port 64738: ufw allow 64738 </code></pre> Start the service: /etc/init.d/murmur start rc-update add murmur </code></pre> Add the following DNS record: _mumble._tcp SRV 64738 mumble.example.com. </code></pre> Setting up HTTP Strict Transport Security 2020-12-29T00:00:00+00:00 Configuration</h1> Set up Let's Encrypt</a> first. Edit /etc/nginx/sites-available/02_example.com</code>: server { listen 80; listen [::]:80; server_name .example.com; # Redirect HTTP traffic to HTTPS. return 301 https://$host$request_uri; } server { listen 443 ssl; listen [::]:443 ssl; server_name .example.com; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot add_header Strict-Transport-Security "max-age=31536000; includeSubDomains"; } </code></pre> References</h1> https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/</a></li> </ol> Setting up PHP 7 with Nginx 2020-12-29T00:00:00+00:00 Installation</h1> Install PHP 7: apk add php7-fpm php7-mcrypt php7-soap php7-openssl php7-gmp php7-pdo_odbc php7-json php7-dom php7-pdo php7-zip php7-mysqli php7-sqlite3 php7-apcu php7-pdo_pgsql php7-bcmath php7-gd php7-odbc php7-pdo_mysql php7-pdo_sqlite php7-gettext php7-xmlreader php7-xmlrpc php7-bz2 php7-iconv php7-pdo_dblib php7-curl php7-ctype </code></pre> Configuration</h1> Edit /etc/php7/php.ini</code>: [Date] date.timezone = Europe/Amsterdam </code></pre> Start the service: /etc/init.d/php-fpm7 start rc-update add php-fpm7 </code></pre> Edit /etc/nginx/sites-available/02_localhost</code> to add in the following within server</code>: server { location ~\.php$ { fastcgi_split_path_info ^(.+\.php)(.*)$; include fastcgi.conf; fastcgi_pass 127.0.0.1:9000; } } </code></pre> Edit /var/www/localhost/htdocs/version.php</code>: <?php phpinfo(); ?> </code></pre> Reload the Nginx configuration: /etc/init.d/nginx reload </code></pre> Setting up uWSGI 2020-12-29T00:00:00+00:00 Installation</h1> Install uWSGI: apk add python3 py3-pip py3-virtualenv uwsgi uwsgi-python3 </code></pre> Configuration</h1> Start the service: /etc/init.d/uwsgi start rc-update add uwsgi </code></pre> Per-Application Setup</h1> Add a user for the Python application: useradd -m flask </code></pre> Log in as the user: su flask cd ~ </code></pre> Create the virtual environment for the application: virtualenv env </code></pre> Activate the virtual environment: source env/bin/activate </code></pre> Install Flask; pip install flask </code></pre> Edit ~/app.py</code>: from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!' </code></pre> Deactivate the virtual environment: deactivate </code></pre> Press Ctrl+D or type exit</code> to log out of the flask user. To test our uWSGI installation, we can first run uwsgi</code> from the command line: uwsgi --socket 127.0.0.1:5000 --protocol=http --plugins python --uid flask --chdir /home/flask --virtualenv /home/flask/env -w app:app </code></pre> Forward the port using SSH: ssh -L 5000:localhost:5000 example.com </code></pre> Point your browser to http://localhost:5000</a> to test the Python application. uWSGI Emperor</h1> Create a directory for the uWSGI application sockets: mkdir /var/run/uwsgi-apps chown -hR uwsgi: /var/run/uwsgi-apps </code></pre> Edit /etc/uwsgi/conf.d/flask</code>: [uwsgi] uid = flask gid = flask chown-socket = nginx socket = /var/run/uwsgi-apps/flask.sock chdir = /home/flask/app virtualenv = /home/flask/app module = app:app plugin = python </code></pre> Correct the permissions for /etc/uwsgi: chown -R uwsgi: /etc/uwsgi </code></pre> Reload uWSGI emperor: /etc/init.d/uwsgi reload </code></pre> Nginx</h1> If you want to add the application to a subdirectory, edit your configuration as follows: location /flask { try_files $uri @flask; } location @flask { rewrite /foo/(.*)$ /$1 break; include uwsgi_params; uwsgi_pass unix:/var/run/uwsgi-apps/flask.sock; } </code></pre> Otherwise, you can also host your application as part of a (sub)domain: location / { try_files $uri @flask; } location @flask { include uwsgi_params; uwsgi_pass unix:/var/run/uwsgi-apps/flask.sock; } </code></pre> Setting up nginx 2020-12-29T00:00:00+00:00 Installation</h1> Install nginx</code>: apk add nginx </code></pre> Configuration</h1> Edit /etc/nginx/nginx.conf</code>: user nginx nginx; worker_processes 1; error_log /var/log/nginx/error_log info; events { worker_connections 1024; use epoll; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] ' '"$request" $status $bytes_sent ' '"$http_referer" "$http_user_agent" ' '"$gzip_ratio"'; client_header_timeout 10m; client_body_timeout 10m; client_max_body_size 50m; send_timeout 10m; connection_pool_size 256; client_header_buffer_size 1k; large_client_header_buffers 4 2k; request_pool_size 4k; gzip on; gzip_min_length 1100; gzip_buffers 4 8k; gzip_types text/plain; output_buffers 1 32k; postpone_output 1460; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 75 20; ignore_invalid_headers on; include /etc/nginx/sites-enabled/*; } </code></pre> Create the directories for the per-site configurations: mkdir -p /etc/nginx/sites-available mkdir -p /etc/nginx/sites-enabled </code></pre> Edit /etc/nginx/sites-available/02_localhost</code>: server { listen 8000; listen [::]:8000; server_name localhost; root /var/www/localhost/htdocs; index index.html index.htm; error_page 404 /404.htm; location / { } location ~* ".(htm)$" { try_files $uri "${uri}l" =404; } } </code></pre> Edit /etc/nginx/sites-available/02_example.com</code>: server { listen 80; listen [::]:80; server_name .example.com; root /var/www/example.com/htdocs; index index.html index.htm; error_page 404 /404.htm; location / { } location ~* ".(htm)$" { try_files "${uri}l" =404; } } </code></pre> Create the symlinks to enable the per-site configurations: ln -s /etc/nginx/sites-available/02_localhost /etc/nginx/sites-enabled/ ln -s /etc/nginx/sites-available/02_example.com /etc/nginx/sites-enabled/ </code></pre> Create the directories: mkdir -p /var/www/localhost/htdocs mkdir -p /var/www/example.com/htdocs </code></pre> Edit /var/www/localhost/htdocs/index.htm</code> and /var/www/example.com/htdocs/index.htm</code>: <html> <head> <title>Test</title> </head> <body> Hello, world! </body> </html> </code></pre> Allow TCP traffic on the HTTP port (80): ufw allow http </code></pre> Check whether the Nginx configuration is valid: nginx -t </code></pre> Start the service: /etc/init.d/nginx start rc-update add nginx </code></pre> Check if Nginx is listening on the HTTP port (80): netstat -anp | grep nginx </code></pre> You should now be able to access your website from http://example.com</a>. In addition, you can forward the port 8000 over SSH: ssh -L 8000:localhost:8000 example.com </code></pre> Then you can point your browser to http://localhost:8000</a>. Setting up OpenVPN 2020-12-29T00:00:00+00:00 Installation</h1> Install OpenVPN: apk add openvpn </code></pre> Public Key Infrastructure</h1> Install EasyRSA: apk add easy-rsa </code></pre> Copy the template at /usr/share/easy-rsa</code> to its own directory: cp -a /usr/share/easy-rsa ~/certs cd ~/certs mv vars.example vars </code></pre> Edit ~/certs/vars</code>: set_var EASYRSA_DN "org" set_var EASYRSA_REQ_COUNTRY "US" set_var EASYRSA_REQ_PROVINCE "CA" set_var EASYRSA_REQ_CITY "SanFrancisco" set_var EASYRSA_REQ_ORG "YourOrganization" set_var EASYRSA_REQ_EMAIL "[email protected]" </code></pre> Set up the Public Key Infrastructure: ./easyrsa init-pki </code></pre> Create the CA certificate: ./easyrsa build-ca </code></pre> Generate the server certificate request and key: ./easyrsa gen-req games nopass </code></pre> Sign the server certificate request: ./easyrsa sign-req server games </code></pre> Generate the Diffie-Hellman parameters: ./easyrsa gen-dh </code></pre> Generate the scret Hash-based message Authentication Code (HMAC): ./easyrsa --genkey --secret pki/ta.key </code></pre> Upload the following files: ~/certs/pki/ca.crt</code> => /etc/openvpn/games/ca.crt</code></li> ~/certs/pki/dh.pem</code> => /etc/openvpn/games/dh.crt</code></li> ~/certs/pki/ta.key</code> => /etc/openvpn/games/ta.key</code></li> ~/certs/pki/issued/games.crt</code> => /etc/openvpn/games/games.crt</code></li> ~/certs/pki/private/games.key</code> => /etc/openvpn/games/games.key</code></li> </ul> Configuration</h1> Edit /etc/openvpn/games.conf</code>: port 1194 proto udp proto udp6 dev tap user nobody group nobody ca /etc/openvpn/games/ca.crt cert /etc/openvpn/games/games.crt key /etc/openvpn/games/games.key dh /etc/openvpn/games/dh.pem tls-auth /etc/openvpn/games/ta.key 0 server 10.42.42.0 255.255.255.0 client-to-client push "route 10.42.42.0 255.255.255.0" push "route-metric 512" push "route 0.0.0.0 0.0.0.0" topology subnet persist-key ifconfig-pool-persist games-ips.txt keepalive 10 120 comp-lzo status /var/log/openvpn/games-status.log log /var/log/openvpn/games.log verb 4 </code></pre> mkdir -p /var/log/openvpn chown -hR openvpn: /var/log/openvpn </code></pre> Create the symlink for a game-specific service: ln -s /etc/init.d/openvpn /etc/init.d/openvpn.games </code></pre> Allow TCP traffic on port 1194: ufw allow 1194 </code></pre> Start the service: /etc/init.d/openvpn.games start rc-update add openvpn.games </code></pre> Issuing Client Certificates</h1> Create a private key, create a certificate request and sign it for the client (replace client with a unique identifier for the client): ./easyrsa build-client-full client nopass </code></pre> Edit games.ovpn</code>: client dev tap proto udp remote example.com 1194 resolv-retry 30 nobind persist-key ca ca.crt cert client.crt key client.key tls-auth ta.key 1 comp-lzo status /var/log/openvpn/games-status.log log /var/log/openvpn/games.log verb 3 </code></pre> Provide the user with the following files: ~/certs/psk/ca.crt</code></li> ~/certs/psk/ta.key</code></li> ~/certs/psk/private/client.key</code></li> ~/certs/psk/issued/client.crt</code></li> </ul> You can then use OpenVPN to open the games.ovpn</code> directly. On Linux, it should also be possible to use the games.ovpn</code> as /etc/openvpn/games.conf</code>. Adjust the paths to the keys accordingly (e.g. by storing them in /etc/openvpn/games/</code>). Then create symlink and run the OpenVPN service: ln -s /etc/init.d/openvpn /etc/init.d/openvpn.games /etc/init.d/openvpn.games start rc-update add openvpn.games </code></pre> Game Discovery</h1> One of the problems that many games have is that when you try to find LAN games, that they will end up using the wrong network interface by default. To fix this, the VPN network interface has to be priortised over the default one. Go to Network Connections from Networking and Sharing Center. There you should see a VPN Network TAN-Windows Adapter V9 interface. Right-click it and select "Properties", you should be presented with the following window: Figure 1: The VPN Properties window for the VPN Network TAN-Windows Adapter V9 interface. </figcaption> </figure> </center> Select "Internet Protocol Version 4 (TCP/IPv4)" and click on "Properties": Figure 2: The Internet Protocol Version 4 Properties window. </figcaption> </figure> </center> Click on "Advanced": Figure 3: The Advanced TCP/IP Settings window. </figcaption> </figure> </center> Set the interface metric from automatic to manual and input 5 as its value and close the windows. Now you should be able to find your friend's hosted game using LAN discovery. Setting up Postfix 2020-12-29T00:00:00+00:00 Installation</h1> Install postfix</code>: apk add postfix postfix-pgsql postfix-pcre </code></pre> Configuration</h1> Create the mail directory and assign vmail as the owner: mkdir -p /var/mail/domains chown -R vmail:postdrop /var/mail/domains </code></pre> Get the uid and gid of the vmail</code> user (in our case the uid is 105 and the gid is 107): grep vmail /etc/passwd </code></pre> Edit /etc/postfix/main.cf</code>: inet_protocols = ipv4 ipv6 myhostname = example.com mydomain = example.com relayhost = mynetworks = 127.0.0.0/8 mydestination = mydestination = localhost.$mydomain, localhost mynetworks_style = subnet mynetworks = 127.0.0.0/8 virtual_mailbox_domains = proxy:pgsql:/etc/postfix/sql/pgsql_virtual_domains_maps.cf virtual_alias_maps = proxy:pgsql:/etc/postfix/sql/pgsql_virtual_alias_maps.cf, proxy:pgsql:/etc/postfix/sql/pgsql_virtual_alias_domain_maps.cf, proxy:pgsql:/etc/postfix/sql/pgsql_virtual_alias_domain_catchall_maps.cf virtual_mailbox_maps = proxy:pgsql:/etc/postfix/sql/pgsql_virtual_mailbox_maps.cf, proxy:pgsql:/etc/postfix/sql/pgsql_virtual_alias_domain_mailbox_maps.cf virtual_mailbox_base = /var/mail/domains/ virtual_gid_maps = static:107 virtual_uid_maps = static:105 virtual_minimum_uid = 100 virtual_transport = lmtp:unix:private/dovecot-lmtp mailbox_transport = virtual local_transport = virtual local_transport_maps = $virtual_mailbox_maps smtpd_helo_required = yes disable_vrfy_command = yes message_size_limit = 104857600 virtual_mailbox_limit = 104857600 queue_minfree = 51200000 smtpd_sender_restrictions = permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain smtpd_recipient_restrictions = reject_non_fqdn_recipient, reject_unknown_recipient_domain, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_rbl_client dnsbl.sorbs.net, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net smtpd_data_restrictions = reject_unauth_pipelining smtputf8_enable = no broken_sasl_auth_clients = no smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_sasl_security_options = noanonymous smtpd_sasl_local_domain = smtpd_sasl_authenticated_header = no </code></pre> Adjust the password accordingly and run the following sequence of commands to create the PostgreSQL scripts for Postfix to access the Postfix database: cd /etc/postfix mkdir sql PASSWORD="ChangeMe" cat - <<EOF >sql/pgsql_virtual_alias_domain_catchall_maps.cf user = postfix password = $PASSWORD hosts = localhost dbname = postfix query = SELECT goto FROM alias,alias_domain WHERE alias_domain.alias_domain = '%d' AND alias.address = '@' || alias_domain.target_domain AND alias.active = true AND alias_domain.active = true EOF cat - <<EOF >sql/pgsql_virtual_alias_domain_mailbox_maps.cf user = postfix password = $PASSWORD hosts = localhost dbname = postfix query = SELECT maildir FROM mailbox,alias_domain WHERE alias_domain.alias_domain = '%d' AND mailbox.username = '%u' || '@' || alias_domain.target_domain AND mailbox.active = true AND alias_domain.active EOF cat - <<EOF >sql/pgsql_virtual_alias_domain_maps.cf user = postfix password = $PASSWORD hosts = localhost dbname = postfix query = SELECT goto FROM alias,alias_domain WHERE alias_domain.alias_domain = '%d' AND alias.address = '%u' || '@' || alias_domain.target_domain AND alias.active = true AND alias_domain.active = true EOF cat - <<EOF >sql/pgsql_virtual_alias_maps.cf user = postfix password = $PASSWORD hosts = localhost dbname = postfix query = SELECT goto FROM alias WHERE address = '%s' AND active = '1' EOF cat - <<EOF >sql/pgsql_virtual_domains_maps.cf user = postfix password = $PASSWORD hosts = localhost dbname = postfix query = SELECT domain FROM domain WHERE domain = '%s' AND active='1' EOF cat - <<EOF >sql/pgsql_virtual_mailbox_maps.cf user = postfix password = $PASSWORD hosts = localhost dbname = postfix query = SELECT maildir FROM mailbox WHERE username = '%s' AND active = true EOF chown -R postfix:postfix sql chmod 640 sql/* </code></pre> Allow TCP traffic on the SMTP port (port 25): ufw allow smtp </code></pre> Check whether the Postfix configuration is valid: postconf -n </code></pre> Start the Postfix service: newaliases /etc/init.d/postfix start rc-update add postfix </code></pre> Configuring DNS</h1> Add the following DNS records: @ MX 10 mail.example.com. </code></pre> Setting up Postfixadmin 2020-12-29T00:00:00+00:00 Installation</h1> Install postfixadmin</code>, dovecot</code> (necessary for the SHA512-CRYPT) and the necessary PHP extensions: apk add postfixadmin dovecot php-pgsql php-imap php-mbstring php-session </code></pre> Configuration</h1> Create the /usr/share/webapps/postfixadmin/templates_c</code> directory: mkdir -p /usr/share/webapps/postfixadmin/templates_c chown www: /usr/share/webapps/postfixadmin/templates_c </code></pre> Copy /usr/share/webapps/postfixadmin/config.inc.php</code>: cp /etc/postfixadmin/config.inc.php /usr/share/webapps/postfixadmin/config.local.php </code></pre> Edit /usr/share/webapps/postfixadmin/config.local.php</code>: $CONF['configured'] = true; $CONF['setup_password'] = ""; // Don't change this yet. $CONF['database_type'] = 'pgsql'; $CONF['database_host'] = 'localhost'; $CONF['database_user'] = 'postfixadmin'; $CONF['database_password'] = 'password'; // Change this to your password. $CONF['database_name'] = 'postfix'; $CONF['database_prefix'] = ""; $CONF['admin_email'] = '[email protected]'; // Replace with your e-mail address. $CONF['encrypt'] = 'dovecot:SHA512-CRYPT'; $CONF['authlib_default_flavor'] = 'SHA'; $CONF['dovecotpw'] = "/usr/bin/doveadm pw"; $CONF['domain_path'] = 'YES'; $CONF['domain_in_mailbox'] = 'NO'; $CONF['aliases'] = '10'; $CONF['mailboxes'] = '10'; $CONF['maxquota'] = '10'; $CONF['quota'] = 'YES'; $CONF['quota_multiplier'] = '1024000'; $CONF['vacation'] = 'NO'; $CONF['vacation_control'] ='NO'; $CONF['vacation_control_admin'] = 'NO'; $CONF['alias_control'] = 'YES'; $CONF['alias_control_admin'] = 'YES'; $CONF['special_alias_control'] = 'YES'; $CONF['fetchmail'] = 'NO'; $CONF['user_footer_link'] = "http://localhost:8000/postfixadmin"; $CONF['footer_link'] = 'http://localhost:8000/postfixadmin/main.php'; $CONF['create_mailbox_subdirs_prefix']=""; $CONF['used_quotas'] = 'YES'; $CONF['new_quota_table'] = 'YES'; </code></pre> Replace change-this-to-your-domain.tld</code> to your domain name: sed -i -e 's/change-this-to-your.domain.tld/example.com/g' /usr/share/webapps/postfixadmin/config.local.php </code></pre> Add the following to /etc/nginx/sites-available/02_localhost</code>: location /postfixadmin { alias /usr/share/webapps/postfixadmin/public; } location ~ \.php$ { location ~ ^/postfixadmin/ { alias /usr/share/webapps/postfixadmin/public; fastcgi_split_path_info ^/postfixadmin(.+?\.php)(.*)$; include fastcgi.conf; fastcgi_pass 127.0.0.1:9000; } fastcgi_split_path_info ^(.+?\.php)(.*)$; include fastcgi.conf; fastcgi_pass 127.0.0.1:9000; } </code></pre> Forward the port using SSH: ssh -L 8000:localhost:8000 example.com </code></pre> Point your browser to http://localhost:8000/postfixadmin/setup.php</a> and create the password hash. Then add it to /usr/share/webapps/postfixadmin/config.local.php</code>. Go back to http://localhost:8000/postfixadmin/setup.php</a> and create the superadmin account. Setting up PostgreSQL 2020-12-29T00:00:00+00:00 Installation</h1> Install PostgreSQL: apk add postgresql postgresql-client </code></pre> Configuration</h1> Start the service: /etc/init.d/postgresql setup /etc/init.d/postgresql start rc-update add postgresql </code></pre> Create the PostgreSQL users and database for the Postfix database: createuser -U postgres --pwprompt postfixadmin createuser -U postgres --pwprompt postfix createdb -U postgres --owner=postfixadmin postfix </code></pre> Grant the postfix</code> user access to the database: psql -U postfixadmin postfix GRANT SELECT ON alias TO postfix; GRANT SELECT ON domain TO postfix; GRANT SELECT ON mailbox TO postfix; GRANT SELECT on alias_domain TO postfix; \q </code></pre> Setting up rspamd 2020-12-29T00:00:00+00:00 Installation</h1> Install redis</code> and rspamd</code>: apk add redis rspamd rspamd-client rspamd-controller rspamd-fuzzy rspamd-proxy </code></pre> Configuration</h1> Run the rspamd configuration wizard: rspamadm configwizard </code></pre> Set the WebUI controller password, set up Redis with the defaults and set up the DKIM signing feature. In terms of DKIM, select option 1. Then press enter until it asks for a domain to sign. Now you can specify a domain, use the default selector, create a private key for each domain you want to add. Start the services: /etc/init.d/redis start /etc/init.d/rspamd start rc-update add redis rc-update add rspamd </code></pre> Check if the rspamd services are listening: netstat -anp | grep rspamd </code></pre> Add the milter to /etc/postfix/main.cf</code>: milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:127.0.0.1:11332 non_smtpd_milters = $smtpd_milters </code></pre> Add -o smtpd_milters=</code> to /etc/postfix/master.cf</code> to prevent mail from being routed twice to rspamd: # For injecting mail back into postfix from the filter 127.0.0.1:10026 inet n - n - 16 smtpd -o content_filter= -o smtpd_milters= -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks -o smtpd_helo_restrictions= -o smtpd_client_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks_style=host -o smtpd_authorized_xforward_hosts=127.0.0.0/8 </code></pre> Reload the Postfix configuration to enable the milters: /etc/init.d/postfix reload </code></pre> Configure the DNS records: @ TXT v=spf1 a mx mx:example.com -all _adsp._domainkey TXT dkim=all _dmarc TXT v=DMARC1; p=reject; sp=reject; rua=mailto:[email protected]; aspf=s; adkim=s; </code></pre> Also add the DKIM records that can be found in /var/lib/rspamd/dkim/example.com.dkim.key.pub</code>: dkim._domainkey TXT v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDLaiAmj5xUQ6s4AlEhwHwnW3JqNc0LZH2SEMZ9y7qIk+C7EvplDYkLf8tG6iVFSb1+ouPCESgRza6/sM4BZYdIYB5SUkM5bn+CqpTtBEWUPvaGawzqer+on1/+Y9pXFKgV3O8WaG223w+THrvfCj9g0FsRKff6lfWekQr2+8G70wIDAQAB </code></pre> Dovecot Sieve</h1> Now that rspamd is running, we will use Dovecot's sieving functionality to automatically move any mail marked as spam by rspamd to the spam folder. In addition, we will add two sieve scripts that monitor any transactions moving mail from or to the spam folder to use that mail to train rspamd Enable the imap_sieve plugin for the imap protocol in /etc/dovecot/conf.d/20-imap.conf</code>: protocol imap { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins imap_sieve ... } </code></pre> Add or edit the following options in /etc/dovecot/conf.d/90-sieve.conf</code>: sieve_before = /var/mail/sieve/global/spam-global.sieve sieve_pipe_bin_dir = /usr/bin sieve_global_extensions = +vnd.dovecot.pipe sieve_plugins = sieve_imapsieve sieve_extprograms </code></pre> Reload the Dovecot configuration to enable the Sieve extensions: /etc/init.d/dovecot reload </code></pre> Create the directory for the sieve scripts: mkdir -p /var/mail/sieve/global </code></pre> Edit /var/mail/sieve/global/spam-global.sieve</code>: require ["fileinto", "mailbox"]; if anyof( header :contains ["X-Spam-Flag"] "YES", header :contains ["X-Spam"] "Yes", header :contains ["Subject"] "*** SPAM ***" ) { fileinto :create "Spam"; stop; } </code></pre> Edit /var/mail/sieve/global/learn-spam.sieve</code>: require ["vnd.dovecot.pipe", "copy", "imapsieve"]; pipe :copy "rspamc" ["learn_spam"]; </code></pre> Edit /var/mail/sieve/global/learn-ham.sieve</code>: require ["vnd.dovecot.pipe", "copy", "imapsieve"]; pipe :copy "rspamc" ["learn_ham"]; </code></pre> Compile the scripts: sievec /var/mail/sieve/global/spam-global.sieve sievec /var/mail/sieve/global/learn-spam.sieve sievec /var/mail/sieve/global/learn-ham.sieve </code></pre> Correct the permissions: chown -hR vmail: /var/mail/sieve </code></pre> Add the following to /etc/dovecot/conf.d/90-sieve.conf</code>: # Learn about spam when mail is moved from any mailbox into spam. imapsieve_mailbox1_name = Spam imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_before = file:/var/mail/sieve/global/learn-spam.sieve # Learn aboout ham when mail is moved from spam into any mailbox. imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/var/mail/sieve/global/learn-ham.sieve </code></pre> Reload the Dovecot configuration to enable the sieve filters: /etc/init.d/dovecot reload </code></pre> Web Interface</h1> Add the following to /etc/nginx/sites-available/02_localhost</code>: location /rspamd { alias /usr/share/rspamd/www; try_files $uri @rspamd; } location @rspamd { rewrite /rspamd/(.*) /$1 break; proxy_pass http://127.0.0.1:11334; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } </code></pre> Forward the port using SSH: ssh -L 8000:localhost:8000 example.com </code></pre> Point your browser to http://localhost:8000/rspamd/</a>. Testing</h1> To test your rspamd installation, you can send an e-mail from another domain to yourself and check the e-mail headers. They should contain an Authentication-Results</code> field if SPF, DKIM and DMARC are operational. Authentication-Results: example.com; dkim=pass header.d=gmail.com header.s=smtpapi header.b=XXXXXXXX; dmarc=none; spf=pass </code></pre> Furthermore, if the spam filter is working, then the header should also contain a X-Spam-Score</code> field. Vice versa, if you send an e-mail, rspamd should now sign the e-mail with the DKIM private key belonging to your domain name. You can use https://dkimvalidator.com</a> to fully verify your SPF, DKIM and DMARC setup when sending e-mails.

Setting up Intel SGX

References</h1> https://www.phoronix.com/news/Intel-SGX-Linux-5.11</a></li> </ol>

Building the Linux Asahi kernel

Building m1n1 and the Linux Asahi kernel from source</h1> First, we clone the PKGBUILDs repository:</p>

Setting up Prosody

Installation</h1> Install Prosody:</p>

Configuration</h1> The configuration file for Prosody can be found in /etc/prosody/prosody.cnf.lua</code>. Getting started with Prosody is relatively straight-forward. Just comment out the VirtualHost</code> directive for "localhost", and add one for your domain instead:</p>

Setting up file sharing</h1> Setting up file sharing is as simple as enabling the http_file_share</code> component:</p>

Building a Linux kernel with Rust support on Gentoo

Building the kernel with Clang</h1> We can try to set CONFIG_GCC_PLUGINS=n</code> by deselecting the following option in make menuconfig</code> to set CONFIG_RUST=y</code>:</p>

Testing our kernel</h1> After booting into our new Linux kernel with Rust support, we can run zgrep RUST /proc/config.gz</code> to confirm that Rust support is enabled as shown in Figure 2.</p>

Accessing physical memory from userspace on Linux

Upgrading and downgrading CPU microcode

Linux</h1>

Disabling Mitigations</h2> In /etc/default/grub</code>, add mitigations=off</code> to GRUB_CMDLINE_LINUX</code>:</p>

Updating Alpine Linux

Building the Linux Kernel (Ubuntu)

Installing build dependencies</h1> There are number of dependencies that need to be installed before we can build the Linux kernel. They can be installed by running the following command on Ubuntu:</p>

Building the Linux Kernel (Ubuntu)

Steamcmd

Setting up a Valheim Server

Setting up a VoidLinux chroot

Installation</h1> We will first create a directory for our VoidLinux installation:</p>

Windows Drivers in Rust: I/O Controls

Windows Drivers in Rust: Reading and Writing

Windows Drivers in Rust: Creating Devices

Windows Drivers in Rust: Safe Framework

Windows Drivers in Rust: Hello World

The Driver Entry Point</h1> Note that we set the entry point of our driver to the driver_entry</code> function by specifying the /ENTRY:driver_entry</code> option. Remove any code in src/lib.rs</code> and let's get started with that function in src/lib.rs</code>:</p>

Windows Drivers in Rust: Prerequisites

Testing Mode</h1> Open PowerShell in Administrator mode and run the following to allow Microsoft Windows to self-signed Windows driver:</p>

Windows Drivers in Rust: Generating Bindings

Generating Bindings</h1> As mentioned before, we will be using bindgen, so let's add that to our build dependencies in Cargo.toml</code>:</p>

Setting up Windows for Remote Rust Development

Setting up OpenSSH</h1> Run PowerShell as an administrator. Then run the following command to make sure that OpenSSH is available:</p>

Setting up mail autoconfiguration

Installation</h1> Install the build dependencies:</p>

`References</h1> https://www.phoronix.com/news/Intel-SGX-Linux-5.11</a></li> </ol>`