tpwo.github.io

Useful Git settings

2025-05-31T00:31:00+02:00

In my dotfiles I always have a .gitconfig file with some of the settings preconfigured. Without them using Git for me would be much more cumbersome. Here’s a list of them along with some comments and sources.

Core settings

`core.autocrlf`

Line ending characters are one big mess due to backwards compatibility. Windows systems use CRLF, Unix and consequently Linux uses LF. More than that, older Macs used CR, so the only missing option was LFCR 🤠. Thankfully Apple switched to LF with Mac OS X 10.0, so we’re left with two variants.[1]

Git was created by Linus Torvalds so it’s no surprise that it prefers LF by default, and warns you if you try committing files with other line ending characters.

Theoretically you can stick to e.g. CRLF if you’re doing a strictly Windows project, but I find it much easier to always commit LF, and configure Git (and my CI[2]) to guarantee that.

In practice, you might notice that a lot of files in a lot of repositories in your project were already committed with CRLF. Such is life. Because of that I don’t like setting autocrlf to true, as it tells Git to always convert from CRLF to LF when doing a commit.

This is a bit annoying, as then it shows that the whole file was modified (which is in fact true) in the commit view, and it makes much harder to see the actual changes. This depends on the tool you’re using as some of them can hide line ending changes pretty well. Still, I would advocate to isolate line ending changes from other types of changes.

E.g. you might want to do a single commit in a repo in which you clean it up for all files[3], and then add this commit to .git-blame-ignore-revs, so it doesn’t makes mess with the history regardless of the tool and/or settings you are using.

TLDR

My rule of thumb:

git config --global core.autocrlf true  1
git config --global core.autocrlf input 2

Use that setting on Windows
Use that setting otherwise (Linux, Mac, WSL)

`core.excludesFile`

Have you noticed that due to your setup, i.e. editor or IDE you’re using, you are the person who always adds particular entries to .gitignore in all repos? Are you tired of that? excludesFile is the solution for you. It’s a globally defined .gitignore which Git uses even if a .gitignore is missing in a repo.

I like to keep it in my home folder along with .gitconfig, and I use the name .gitignore_global:

git config --global core.excludesFile '~/.gitignore_global'

`core.sshcommand`

I’m a Windows user, but I prefer bash. As terminal is very important in my workflow, I use WSL both at work and in my private projects. There are a lot of quirks about WSL and one of them is having Git remember you SSH passwords. There are Linux solutions for that, and I experimented with them. Unfortunately, it wasn’t working out great for me (but maybe something has changed in recent years). Another solution is to use password-less keys, but you don’t want to do that, right?

Thankfully we can use Windows built-in SSH agent to remember passwords for us even in WSL.[4]

To do that, we have to tell WSL Git to use SSH binary from Windows:

git config --global core.sshcommand '/mnt/c/Windows/System32/OpenSSH/ssh.exe'

Credential settings

`credential.helper`

This is quite similar to core.sshcommand I mentioned above when using Git in WSL. You can choose to use HTTPS over SSH when talking with Git remotes. But WSL doesn’t have a built-in way to store these credentials, and using plaintext file .git-credentials is not something you want to be doing.

Again, Windows comes to the rescue. Here, you have to install Git on Windows to be able to use its credential manager from WSL Git. Then, you can authenticate once, usually via a browser window, and Git will remember this.

git config --global credential.helper '/mnt/c/Program\ Files/Git/mingw64/bin/git-credential-manager.exe'

Make sure to quote the path as space in Program Files would cause some issues otherwise. And use the actual path of where you have Git for Windows installed, other default alternative is AppData of your Windows user.

Git features

`feature.manyFiles`

Git introduced this feature a few years ago to improve support for bigger repos. I naively enabled it after reading the change log and it instantly broke some tools for me, particularly Matlab Git integration was completely broken. After these experiences I disabled this feature, so my setting repeats the current default.[5]

git config --global feature.manyFiles false

If you’re struggling with Git performance you might consider to enable it. Hopefully after a few years tools already support it.

`git commit`

`commit.verbose`

git commit has a --verbose flag which includes committed diff in a commit window. The commit windows is nothing more but an automatically opened file which you edit, save, and quit, which makes git do the commit.

This option always adds this flag, so you can see what are you committing every time.

git config --global commit.verbose true

`git fetch`

`fetch.prune`

You can remove any remote-tracking references (i.e. origin/<branch-name>) which were removed on the remote with git fetch --prune. I find it very convenient to be the default, as branches are usually removed after merging pull request, so you can quickly have a lot of dangling references.

git config --global fetch.prune true

If you have multiple remotes, and would like to prune references from only some of them, it’s also an option. For that, there is another command:

git config --global remote.<name>.prune true

<name> can be anything, and by convention origin is used as the default remote name.

You might be wondering if setting something like git config --global remote.prune true is allowed. Yes, it is, and it does pretty much the same thing as setting fetch.prune to true.[6]

`git pull`

`pull.rebase`

I like explanation from this video why default behavior of git pull is a bit annoying. Its title is a little bit clickbaitish, but I agree with the arguments.

I was also a bit surprised that author didn’t mentioned option we’re talking about here. To do git pull --rebase automatically with each pull set:

git config --global pull.rebase true

`pull.ff only`

There is also another option which enforces fast-forward merge in git pull. The difference is that --ff-only merge works only if there are no newer commits coming from the remote. Otherwise it fails, and you have to do something with it manually. That means you are forced to do the rebase on your own.

This adds you more work but one can advocate that it’s safer, as you have to explicitly rebase to the new version.

This is in line with the approach never use git pull (i.e. use git fetch and git rebase instead). If you’re into that, consider using this option.

`git push`

The biggest problem with this command is people overusing --force. There is a safer replacement for it.

You can use --force-with-lease instead of --force to only overwrite changes you’re expecting. Either the ones you have locally, or ones you fetched before. If there will be any new commit on a remote, force push with lease fails.

`git merge`

`merge.conflictstyle`

I think I don’t know a person who likes to solve merge conflicts. But I noticed that many people are more scared about them then they probably should be.

Setting a better conflict style in Git is one of the steps to make them less scary.

zdiff3 is a relatively new conflict style in Git which was introduced in 2022 with version 2.35. That means that you still can work with systems which don’t have this feature, so make sure to see if your Git version is new enough.

But what is zdiff3? It’s an updated version of diff3 which is much older algorithm[7] and which has one big benefit: in classic conflict view you see two versions of the code, ours and theirs, as Git labels it. diff3 also shows what was before that, which we can name a common ancestor of both commits.[8]

This simple addition often adds much needed context, to solve merge conflict faster, and do it correctly.

Unfortunately, diff3 isn’t ideal, and sometimes creates more noisy conflicts, marking more lines as conflicting.[9]

zdiff3 addresses these issues. Interestingly enough, z in the name comes from the word zealous.

git config --global merge.conflictstyle zdiff3

`git init`

`init.defaultBranch`

Like it or not, but main is the new master in Git, and it doesn’t seem to be changing any time soon.

Git though still uses default master, so this option has to be set.

git config --global init.defaultBranch main

Git aliases

`git git`

I borrowed that idea from Anthony Sottile[10]. It allows you to type git git <command> and make the command still work. In fact, you might type as many git commands as you like, as it’s recursive.

Why? Same as Anthony, I sometimes type git in the terminal, then start thinking about something, then and go back to typing, often typing git again due to muscle memory. Adding an alias is simpler than fighting with it, apparently 😛

git config --global alias.git '!git'

`git l`

Quickly output oneline log for all branches. Very handy to quickly orient around in the repo. This is one of my most-used git commands.

git config --global alias.l 'log --graph --all --oneline'

`git lg`

git l with a bit more details. It uses more formatting and I copied it from someone’s dotfiles years ago.

git config --global alias.lg "log --graph --all --format=format:'%C(bold blue)%h%C(reset) - %C(bold green)(%cr)%C(reset) %C(white)%s%C(reset) %C(bold yellow)- %cn%C(reset)%C(bold red)%d%C(reset)' --abbrev-commit --date=relative"

`git llg`

Even more detailed log. Also copied from that same person.

git config --global alias.llg "log --graph --all --format=format:'%C(bold blue)%h%C(reset) - %C(bold cyan)%cD%C(reset) %C(bold green)(%cr)%C(reset)%C(bold yellow)%d%C(reset)%n'' %C(white)%s%C(reset) %C(bold yellow)- %cn%C(reset)' --abbrev-commit"

`git k`

You migth not know this, but Git comes with a GUI called gitk. It isn’t always installed (e.g. MacOS Git via homebrew), but when you have it, it might be handy in some cases.

I’m used to Git commits starting with git <cmd>, so I added an alias which reflects it.

git config --global alias.k '!gitk &'

`git ka`

Another gitk alias to run the program with --all flag which then shows all branches.

git config --global alias.ka '!gitk --all &'

I recommend a video from Scott Hanselman if you want to learn more about this topic ↩
pre-commit with its mixed-line-ending hook set to lf is a great tool for that ↩
pre-commit mentioned above is a great way to do this. Also make sure to set up it to check it with each new commit or these CRLF files might start reappearing, as we live in a society ↩
Refer to MS docs on how to enable SSH agent ↩
When writing this post, I did some research, and it seems that this might be due to libgit2 incomparability with manyFiles. More details on Stack Overflow ↩
There is quite a good summary of the topic again on SO ↩
Introduced in 2007 with Git 1.5.0 ↩
A pretty nice explanation of that is in this blog post ↩
Again, SO for the rescue ↩
Here’s his video about it: https://youtu.be/BkUW2NgfZao ↩

Convert dvdsub into srt subtitles

2025-05-30T22:56:00+02:00

I wanted to extract subtitles from a bunch of .mp4 files, as they were containing song names playing in the background of the movie.

This was more complicated than I thought.

It turns out video subtitles can come in very different formats. One of them is dvdsub, and it’s a bitmap subtitle format.

To convert it into something you can grep, you have to go through a lot of steps including OCR.

Fortunately, good people created tools to do just that.

ffmpeg for the win

I loved ffmpeg once I discovered it many years ago. I really don’t like bloated constrained GUI-based apps to manipulate video and audio files.

It can help with subtitles as well.

To install on MacOS you can use homebrew:

brew install ffmpeg

On Linux it should be available in your package manager. On Windows probably the best solution is to use WSL2.

Check your subs

ffmpeg can help you verifying that your subs are actually in dvdsub format. Use -i and point to the video file:

ffmpeg -i <video-file>

e.g. ffmpeg -i video.mp4

Output should look like this:

<...snip...>
Stream #0:0[0x1](eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 448x336 [SAR 1:1 DAR 4:3], 825 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
      Metadata:
        creation_time   : 2007-09-27T17:47:49.000000Z
        handler_name    : Video Media Handler
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 74 kb/s (default)
      Metadata:
        creation_time   : 2007-09-27T17:47:49.000000Z
        handler_name    : Sound Media Handler
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](eng): Subtitle: dvd_subtitle (dvdsub) (mp4s / 0x7334706D), 720x480, 0 kb/s (default)
      Metadata:
        creation_time   : 2007-09-27T17:47:57.000000Z
        handler_name    : Unspecified
At least one output file must be specified

The part we’re interested in is Subtitle: dvd_subtitle (dvdsub). This confirms that our subtitles are bitmap-based, and are one of the three streams in the file.

Indexing is 0-based with a number after Stream #, i.e. 0:2 in case of subtitle stream.

Stream index is required in the next step.

Extract subs into separate container

ffmpeg comes in handy again:

ffmpeg -i <video-file> -map <stream-index> -c:s copy <output-file>.mkv

e.g.

ffmpeg -i video.mp4 -map 0:2 -c:s copy output.mkv

Convert `mkv` into `sub` and `idx` files

This step requires mkvextract. Again, you can install it with homebrew:

brew install mkvtoolnix

Then mkvextract should be available. You can use it to create sub and idx file from mkv:

mkvextract tracks <output-file>.mkv 0:<output-file>.sub

e.g. mkvextract tracks output.mkv 0:output.sub

We do it because OCR tools support this format instead of mkv.

Use OCR to get `srt` file

This step took me the most, as the tool we’re using here is pretty old, and I had to manually clone repo and tweak it to be able to compile it.

Install `VobSub2SRT`

You can use my fork, so you don’t have to apply the fix by yourself. The "fix" was bumping the minimum required version of cmake which I did in this one-liner.

Use my homebrew tap

I was able to set up a private homebrew tap, so it’s possible to install my fixed version with these two commands. Homebrew will take care of build dependencies (or at least for cmake and tesseract):

brew tap tpwo/vobsub2srt
brew install --HEAD tpwo/vobsub2srt/vobsub2srt

Setting this up was new for me. It turns out that Homebrew requires <user>/homebrew-<name> repo to be present, so I created it. This repo contains the formula to install VobSub2SRT from my fork.

Manual installation

If you prefer manual installation, it’s also possible. Before compilation, you have to install build dependencies:

brew install cmake
brew install tesseract

Then you can clone my fork and compile the tool:

git clone https://github.com/tpwo/VobSub2SRT
cd VobSub2SRT

# Then you can follow original README
./configure
make

# `vobsub2srt` should be present in `build/bin`

Original README suggests sudo make install, but I just grabbed the compiled binary and moved it to my PATH in ~/.local/bin.

Then you can finally convert sub and idx file into srt file:

# Note we don't pass file extension here
vobsub2srt <output-file>

e.g. vobsub2srt output

Script to run it all with one command

I had a bunch videos with embedded subtitles, so I created a Bash script to quickly run it with for f in *:

title="$1"
title_wo_suffix="${title%.*}"

outdir=music

stream=$(ffmpeg -i "$title" 2>&1 | grep 'dvdsub' | awk -F '[#\[]' '{print $2}')

if [[ -z "$stream" ]]; then
    echo 'No subtitles track in the video!'
    exit 1
fi

mkdir -p "$outdir"

ffmpeg -i "$title" -map "$stream" -c:s copy "$outdir"/"$title_wo_suffix".mkv

mkvextract tracks "$outdir"/"$title_wo_suffix".mkv 0:"$outdir"/"$title_wo_suffix".sub

vobsub2srt "$outdir"/"$title_wo_suffix"

To run it, save it under <name>, do chmod +x <name>, and run:

./<name> <video-file>.<ext>

e.g. ./extract-subs video.mp4.

To run it for e.g. all .mp4 files in the current folder do:

for f in *.mp4; do ./extract-subs.sh "$f"; done

Sources and further reading

Original VobSub2SRT repo (I forked a fork!):
https://github.com/ruediger/VobSub2SRT
Tutorial describing alternative flow with ffmes. I figured VobSub2SRT is easier to run on Apple Sillcon Macbook:
https://subarashii-no-fansub.github.io/Subbing-Tutorial/Convert-vobsub-file-into-srt/

Use SSH key to sign your Git commits

2025-05-30T21:33:00+02:00

But why?

You might wonder why should you sign your commits. This is a valid question, and some people just don’t care about it. If you don’t have a ready opinion[1], I’ll try to convince you.

Choose your warrior committer

In Git you can assign any author to any commit. This is a basic functionality of Git, you can put any name and email in .gitconfig or pass one-time name/email with -C when committing (or set ENV variables etc. – there are many ways).

Make GitHub to link their profile

Then, you can push this commit to GitHub. If the commit email address is already used by someone on GitHub, GitHub links it with their profile. Even if this is obviously not their commit. GitHub cannot know this.

Fork their repo and display fabricated commit from the base URL

You might think that no one cares about you doing some fake commits in your repos. But you can fork any repo and fake commits in the fork.

Then, you can display a commit from fork even when looking at it from the original repo. Actually, this is a big problem with GitHub UI, as it treats all forks to be connected with the base repo.

I once found a fabricated Linus Torvalds commit in which he deleted the whole Linux codebase with a funny commit message.

Unfortunately, I haven’t found it now but here’s something similar (also from Torvalds!).

TLDR: fabrication process is simple:

Fork a repo
Create a fake commit.
Display it in context of the original repo. GitHub allows for that!

What can I do?

You cannot prevent people from using your (public) email and fabricate commits on your account. But you can turn on GitHub vigilant mode which is described in their docs.

Then all of your unsigned commits will be marked explicitly as unverified. Unfortunately, also all of your own commits, prior you started to sign them, will be marked as well.

Old times

Nevermind, let’s go back to the original topic. When you wanted to sign your commits in Git a few years ago, you were forced to use GPG. Unless you already have and use GPG, it was a lot of pain. GPG is quite old, and it doesn’t offer the best user experience. Using it, meant maintaining another key, which you have to protect with another strong password and back it up. This was enough to stop many people from signing their commits.

Alternative to GPG

By the end of 2021 with v2.34.0 Git added a way to sign your commits using SSH keys. I tried switching to SSH a few years ago but it didn’t work, and quick research showed that I’m not the only one.

Out of curiosity I checked it now, and I was able to sign my commits after a few minutes of configuration, and this post is the result of that. Time heals wounds!

Configure Git

I assume that you already have and use a password-protected SSH key[2]. Then you can tell Git to use it:

git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/<your-public-key>

And that’s all! To sign a single commit:

git commit -S -m 'Your commit message'

To always sign your commits:

git config --global commit.gpgsign true

To also always sign your tags:

git config --global tag.gpgsign true

Configure GitHub

To have commits marked as signed in GitHub, you have to add SSH key one more time, and choose it to be your Signing Key:

After doing that, your signed commits will be marked as Verified:

Local signers file

You can also create an allowed signers file for quick local verification of signatures

Create a file (e.g., ~/.config/git/allowed_signers) listing trusted email and public key pairs in this format:

[email protected] ssh-ed25519 AAAAC3NzaC1...

Then tell Git to use it:

git config --global gpg.ssh.allowedSignersFile '~/.config/git/allowed_signers'

You can now verify if your commit was signed properly:

$ git verify-commit <hash>
Good "git" signature for [email protected] with ED25519 key SHA256:<...snip...>

You can do it for git log as well:

$ git log --show-signature
commit <hash> (HEAD -> main, origin/main)
Good "git" signature for [email protected] with ED25519 key SHA256:<...snip...>
Author: John Doe <[email protected]>
Date:   Fri May 30 21:22:36 2025 +0200

    Hello there

Clipboard sync between WSL Neovim and Windows

2024-09-17T21:52:00+02:00

I recently started migration from Vim to Neovim. One of the first issues was proper configuration of clipboard. By proper, I mean a situation where I have a working 2-way-sync clipboard between Neovim running in tmux in WSL in Windows Terminal and Windows, so I can easily copy and paste from my browser or any other program to and from Neovim.

With Vim I intially achieved that with GWSL and configured $DISPLAY env variable which was a quite complex and unstable solution (GWSL sometimes has weird issues). Thankfully, it was needed only on older versions of Windows 10 (my $CURRENT_COMPANY still uses the old Win10 version though…). Up-to-date Windows 10 and Windows 11 support clipboard sync for Vim out of the box. They have something like GWSL built-in, and surprisingly it’s quite stable and just works. Bravo Microsoft 😛

The only caveat is to install Vim version which has clipboard support – usually vim-gtk is recommended on Debian/Ubuntu.

The unexpected

I was a bit surprised to find out that this doesn’t work with Neovim.

Some googling brought the solution: win32yank. This is a small Rust program which integrates with Windows cliboard, providing an interface which can be easily used by Neovim.

Actually, Neovim has built-in support for it. You just need to make sure that win32yank.exe is on your $PATH.

How to get `win32yank.exe`?

I use WSL to do most of my text editting, but I still find it useful to have the same editor on Windows itself.

You can install it most easily with winget, i.e. winget install Neovim.Neovim.

Why this it important? It turns out that Neovim on Windows is shipped with win32yank!

Creating symlink

winget installs Neovim in Program Files, and win32yank.exe is located in the bin subfolder. Use this command to create a symlink in your home folder, so Neovim can see it.

ln --symbolic /mnt/c/Program\ Files/Neovim/bin/win32yank.exe ~/.local/bin/win32yank.exe

Of course, ~/.local/bin has to exist and be in your $PATH for this to work.

Playing Super Mario World in 2024

2024-08-12T23:12:00+02:00

Thank you Mario! But our princess is in another castle!
— Toad

Super Mario World is one of my childhood games I remember really well. It was released in 1990 but recently I found out that we can still play it today on modern platforms with widescreen support enabled!

Cover of the US version of Super Mario World

The base to do it is bsnes, a multi-platform SNES emulator. There is a fork of it which supports HD video features called bsnes-hd, and we’ll be using it.

Super Mario World Widescreen project is a fan-made patch which can be found in wide-snes GitHub repo.

We also need a patcher. According to sneslab.net manual we can use either FLIPS or beam. They host FLIPS directly on their site: https://sneslab.net/tools/floating.zip.

Now, the only missing piece is ROM with the original game. I found a copy of it on archive.org which seems to be a pretty reasonable source. If it’s there, Nintendo probably doesn’t have anything against it, or at least no longer cares.

Patching the ROM

After downloading the files, you should have something like this (we’re skipping emulator for now):

├── smw-widescreen.bps           1
├── Super Mario World (USA).sfc  2
└── floating
    ├── boring.zip
    ├── flips-linux
    ├── flips.exe                3
    ├── license.txt
    └── src.zip

Patch file
ROM file
FLIPS patcher

Run FLIPS <3> and click Apply Patch. At first it asks you for the patch file, then for the ROM file, and then for the filename to save the patched ROM.

You can notice that the patched file is twice the size of the original:

$ du -sh *
1.0M    Super Mario World (USA) Widescreen Project.sfc
512K    Super Mario World (USA).sfc

Creating widescreen configuration file

bsnes-hd requires a bso configuration file in order to correctly run the game in a widescreen mode. Thankfully, you don’t have to download it, as the file is very short and simple, and you can create it with this command:

echo 'w1s1W48S2i0o1p1b1B1c1' > 'Super Mario World (USA) Widescreen Project.bso'

The content of the file is taken from wide-snes GitHub repo. I guess it uses special quirky syntax to tell the emulator how it should behave when dealing with the patched ROM.

Running the game

Now, you can unzip bsnes-hd and run it. To make the game look better, make sure to select Settings > Shader > None. By default it’s set to Blur which makes the game very… well, blurry. You can later switch these options on the fly, maybe you’ll like it.

To start the game, either select System > Load Game or drag and drop patched sfc file into the window. Super Mario World should start in widescreen mode!

Configuring controller

I found the game to be much more enjoyable while playing it with a game pad. Fortunately, bsnes allows for that. Select Settings > Input to configure mappings for your device.

I’m using Xbox 360 controller, and it works like a charm. Here are my settings:

These are pretty much defaults, as much as you can tell that about SNES → Xbox 360 conversion, but I added another mapping for X which is used to run faster in the game.

Python integration tests with pytest and docker compose

2024-07-15T22:48:00+02:00

I really like pytest. It’s a great tool for writing unit tests in Python, as it provides a lot of features out of the box. Some of them are quite magical, but they can make your life much easier, so at the end you need much less code.

Recently, I experimented with using pytest fixture to create a simple integration test skeleton which integrates docker.

Fixtures

Fixtures in pytest are a way to request for something in the test code. There are two basic types of fixtures.[1]

return fixtures which provide a certain value or object
- This means we execute some code before requesting a fixture
yield fixtures which provide a certain state
- This means we execute some code before and after requesting a fixture

In this case the latter is be needed, as we want to perform the test in a state when docker container is running. This way we can startup a docker container and clean it up regardless of the test results.

I’m using docker compose, as it provides more flexibility and it’s cleaner than CLI flags for pure docker. I’d use it even when you have only a single container, as overhead is minimal.

Implementation

You can see the test file skeleton here.

import shutil
import subprocess
import time

import pytest


def tested_function():
    ...


@pytest.fixture
def docker_container():
    out = subprocess.run(
        ('docker', 'compose', '--file', 'testing/docker-compose.yml', 'up', '--detach'),
        check=True,
    )
    time.sleep(10)
    yield out
    subprocess.run(
        ('docker', 'compose', '--file', 'testing/docker-compose.yml', 'down'),
        check=True,
    )


def test_integration(docker_container):
    actual = tested_function()
    expected = '<expected output>'
    assert actual == expected

The flow is quite simple here:

test_integration function starts
It requests docker_container, so the fixture function starts
We reach yield in docker_container and go back to the test function
docker_container value in test_integration is a return value of subprocess.run from the fixture
- I’m not using it in this example
Test code is run up to the end
Either if any exception is thrown or if the test code finished, we go back to the fixture and execute rest of the code
- In this case it’s docker compose down

The biggest drawback of this approach is time.sleep, as out is returned when docker compose up starts running, and it doesn’t mean that the container is ready for the test. Depending on your use case you would need less or even more time here.

A smarter approach would be getting the container notify the code that it’s ready in an event-driven fashion. The presented solution is currently enough for my use case, even though it might be quite limiting if we want to scale it and run multiple test cases.

But for my use case this was good enough, so I stopped here, and maybe I’ll expand it in the future.

Integration

By default pytest picks up all files which are named test_*.py or *_test.py. As integration tests are quite longer than unit tests, we don’t want them to be run each time along with unit tests.

I chose to rename the file with integration tests to integration.py which I put in tests folder along with other test files. This works fine on a small-scale project. In a bigger project, I’d probably create a separate folder in tests or maybe create a new top level folder like integration_tests.

With the simple approach running these tests is a simple pytest tests/integration.py, which I put under Makefile target integration-tests. This way I can type make i and let tab-completion do the rest for me.

In CI, I’m running them in the same job as unit tests, directly after them, as I assume that it doesn’t make sense to run costly integration tests if any of my unit tests is failing (even if GitHub pays for my CI 😛).

You can see the actual implementation in my event-scrapper-srt project.

To understand how they work here’s a great video from Anthony Sottile about it ↩

Python debugging 101: pdb

2024-06-19T20:12:00+02:00

pdb is a simple debugger that comes by default with Python. There are many more advanced alternatives like pudb or ipdb, but pdb is always available out of the box which is a big plus.

breakpoint() was introduced in Python 3.7. Before that, import of pdb and calling a function that creates a breakpoint was required.

`pdb` commands

h / help – display available commands
c / continue – continue running the code until the end
q / quit – quit program by raising BdbQuit exception

`l` / `list`

Prints lines of code around the line that is about to be run (marked with an arrow). Pressing enter will list even more file. list . will reset the view to the original (i.e. some lines before and after a line with arrow).

`ll`

Long list, prints more lines of code. Pressing enter has no effect. There is no support for dot from this reason.

`w` / `where`

Displays a call stack that led to the execution of the given code.

`u` / `up` & `d` / `down`

Goes up or down one stack frame. An arrow showed with where is updated to account for the new position. Also, list will print the code around the current stack frame.

up and down accept step parameter, e.g. up 3 will go up by three stack frames.

`p` & `pp`

For printing and pretty printing. The latter is especially helpful when printing more complex data structures.

`n` / `next`

Goes to the next line.

Whenever, a debugger returns it prints --Return-- to the screen, and pauses right after the return value. Pressing n again will go to the next function.

`s` / `step`

Goes inside a function call.

`r` / `return`

Goes directly to the return statement of the current function. Useful to quickly exit functions that we are not interested in debugging.

Summary

There are a lot of more features that can be used, e.g. creating breakpoints from pdb, but above I described the basics that should be useful most of the time.

For conditional breakpoints you can use Python if and hide breakpoint under a certain condition.

I really recommend this video from Anthony Sottile if you want to learn more and see how to use pdb in practice.

Xtract Ze Files!

2024-06-19T19:45:00+02:00

It’s a neat way to remember tar flags. Xtract Ze Files!

tar -xzf <myarchive>.tar.gz …

Hello, World!

2024-06-05T23:53:00+02:00

We’re live! I had an idea to create a personal blog for quite some time, and here it is.

It’s still in a very WIP state, as I connected all the wires to run it, and didn’t have much time to focus on personalization, themes and other nice-to-eyes features. But at first, I’d like to add some content, so I can see the impact of style changes on real posts.

tpwo.github.io

Useful Git settings

Core settings

core.autocrlf

TLDR

core.excludesFile

core.sshcommand

Credential settings

credential.helper

Git features

feature.manyFiles

git commit

commit.verbose

git fetch

fetch.prune

git pull

pull.rebase

pull.ff only

git push

git merge

merge.conflictstyle

git init

init.defaultBranch

Git aliases

git git

git l

git lg

git llg

git k

git ka

Convert dvdsub into srt subtitles

ffmpeg for the win

Check your subs

Extract subs into separate container

Convert mkv into sub and idx files

Use OCR to get srt file

Install VobSub2SRT

Use my homebrew tap

Manual installation

Script to run it all with one command

Sources and further reading

Use SSH key to sign your Git commits

But why?

Choose your warrior committer

Make GitHub to link their profile

Fork their repo and display fabricated commit from the base URL

What can I do?

Old times

Alternative to GPG

Configure Git

Configure GitHub

Local signers file

Further reading

Clipboard sync between WSL Neovim and Windows

The unexpected

How to get win32yank.exe?

Creating symlink

Playing Super Mario World in 2024

Patching the ROM

Creating widescreen configuration file

Running the game

Configuring controller

Python integration tests with pytest and docker compose

Fixtures

Implementation

Integration

Python debugging 101: pdb

pdb commands

l / list

ll

w / where

u / up & d / down

p & pp

n / next

s / step

r / return

Summary

Xtract Ze Files!

Hello, World!

`core.autocrlf`

`core.excludesFile`

`core.sshcommand`

`credential.helper`

`feature.manyFiles`

`git commit`

`commit.verbose`

`git fetch`

`fetch.prune`

`git pull`

`pull.rebase`

`pull.ff only`

`git push`

`git merge`

`merge.conflictstyle`

`git init`

`init.defaultBranch`

`git git`

`git l`

`git lg`

`git llg`

`git k`

`git ka`

Convert `mkv` into `sub` and `idx` files

Use OCR to get `srt` file

Install `VobSub2SRT`

How to get `win32yank.exe`?

`pdb` commands

`l` / `list`

`ll`

`w` / `where`

`u` / `up` & `d` / `down`

`p` & `pp`

`n` / `next`

`s` / `step`

`r` / `return`