Pijul - A Distributed Version Control System

Elpe. Yaks big. Razors bigger

Sun, 08 Jun 2025 00:00:00 +0200

Elpe is a library to describe build environments with packages from existing Linux distributions, and run scripts in these environments inside containers.

Pijul (and Sanakirja) on the mainframe

Tue, 13 Feb 2024 15:50:00 +0100

An underappreciated aspect of Pijul is its backend, Sanakirja, which is easy to confuse with a mere key-value store. Sanakirja does include a key-value store, but is primarily a transactional block allocator in a file.

Update in Thrussh: the SSH Terrapin attack

Mon, 18 Dec 2023 16:00:00 +0100

A lot of things are currently happening in Pijul, but this short post is about a satellite project, Thrussh, which we use in Pijul and in the Nest.

A new direction for Pijul's hosting service

Wed, 24 May 2023 09:20:27 +0200

I gave a talk at GOTO Aarhus yesterday, where I announced a change of direction for the Nest, the hosting service we’ve been running for a few years already. Its new incarnation will be open source and serverless.

Some context

The first version of the Nest, using old versions of the formats and algorithms, started operating in 2016. The goal at the time was to make it easy to test Pijul at scale, in particular the apply and unrecord algorithms, as well as the protocol. This goal has been achieved, as we have seen less and less failures over time.

Announcing Pijul 1.0 beta

Tue, 18 Jan 2022 00:00:00 +0000

I’m proud of finally announcing the beta release of Pijul, after a bit more than a year of alpha. Sorry for the long post, and Happy New Year!

Modular monorepos

Fri, 07 Jan 2022 01:00:00 +0000

A recent change introduced a new structure in Pijul repositories, allowing one to merge unrelated repositories while preserving commutation across the different source repos.

Towards a hybrid snapshot/patch version control system

Fri, 07 Jan 2022 00:00:00 +0000

Where I outline the design of tags in Pijul, which are a first step towards a more general hybrid snapshot/patch version control system. The Sanakirja tricks presented here can also be applied outside the context of Pijul, to create efficient compressed databases.

Changes on binary files

Sun, 11 Jul 2021 00:00:00 +0000

We just released Pijul 1.0.0-alpha.52, which includes a pretty cool change: short patches on binary files.

Two changes to changes

Mon, 28 Jun 2021 00:00:00 +0000

In this post, I announce the final changes to the repository and changes formats before we declare Pijul “1.0 beta”.

On fires

Thu, 03 Jun 2021 00:00:00 +0000

Almost three months ago, a fire started in the datacenter that hosted this website, as well as nest.pijul.com. In this post, I try to reflect on what that meant for us, and the consequences it had.

Rethinking Sanakirja

Fri, 05 Feb 2021 00:00:00 +0000

Sanakirja is a key-value store based on B trees, written in Rust. In this post, I reflect on some design and programming methodology mistakes I’ve made. Benchmarks show that the latest version beats LMDB, the fastest C equivalent, by 20%-50% on a use case similar to my most common load (storing graphs in Pijul).

Implementing cross-process Sanakirja locks

Fri, 15 Jan 2021 00:00:00 +0000

I just finished implementing a non-trivial bit of Pijul: cross-process locks for Sanakirja databases. In this post, I explain how it works.

How to survive?

Tue, 05 Jan 2021 00:00:00 +0000

Since we released the alpha version of Pijul 1.0 two months ago, a lot of things have happened. In this post, I want to share some of them, and give a roadmap for the next few weeks or months.

Commutation and scalability

Sun, 20 Dec 2020 00:00:00 +0000

I just finished the implementation of an important feature of Pijul: clones, pushes and pulls on partial repositories. In this post, I explain why this matters.

Announcing Minus, a cross-platform pager

Wed, 09 Dec 2020 00:00:00 +0000

This is a guest post by Arijit Dey, introducing a cross-platform paging crate he wrote, which will allow Pijul to work equally on all platforms.

Towards 1.0

Sat, 07 Nov 2020 00:00:00 +0000

After fixing the performance and scalability problems, we’re on our way to getting a stable Pijul. In this post, I explain what I’ve been up to in the recent months.

Pijul 0.12

Tue, 23 Apr 2019 10:52:09 +0100

We released Pijul 0.12 yesterday. This is a really exciting time for the project, as we have reasons to believe that all the algorithms are now in a pretty good shape.

Sanakirja gets its full concurrency model

Wed, 20 Mar 2019 15:20:00 +0000

I just fixed a few remaining bugs in Sanakirja, the database backend behind Pijul, and took the opportunity to update its concurrency model.

Pijul 0.11

Tue, 20 Nov 2018 22:26:49 +0000

I’d like to announce a new release of Pijul, version 0.11. This release fixes a large number of bugs, follows evolutions in the Rust ecosystem (Tokio), integrates better with existing software (SSH proxys and config files), and introduces an important feature of Pijul: /partial clones/.

Pijul 0.10

Sat, 21 Apr 2018 15:13:57 +0200

I’m pleased to announce the new release of Pijul, version 0.10. This release has been a long time coming, but brings in an important number of new features and stability enhancements.

buildRustCrate just got integrated into nixpkgs

Tue, 12 Dec 2017 00:00:00 +0000

buildRustCrate is a tool I wrote in the Nix programming language to share build products across crates and across versions of a single crate when compiling Rust code. The initial motivation was to speed up deployment times when working on large projects with lots of dependencies, such as the Pijul Nest.

New release of Thrussh, including support for agents

Wed, 06 Dec 2017 00:00:00 +0000

I’d like to announce a new version of Thrussh, which will hopefully solve a number of problems people have been having with SSH support in Pijul.

Downloads

Tue, 19 Sep 2017 21:57:58 +0200

From source

$ cargo install pijul --version "~1.0.0-beta"

On Windows

See Windows binaries.

Using Pijul itself

If you are lucky enough to already have one version of Pijul, here is how to check the sources out:

$ pijul clone https://nest.pijul.com/pijul/pijul

Or, if you have an account on the Nest, via SSH:

$ pijul clone me@nest.pijul.com:pijul/pijul

Other

See the manual

Reporting bugs

https://nest.pijul.com/pijul/pijul/discussions

FAQ

Tue, 19 Sep 2017 21:56:45 +0200

What is a version control system?

A version control system is a piece of software that allows different authors to work collaboratively and asynchronously on a file, keeps track of their changes and alerts them when their edits are conflicting.

It can also be used by single authors to review and revert their changes.

It is distinct from a parallel editor, in which all authors edit the same file concurrently. In a parallel editor, the authors are forced to share all their edits with others, often restricting their creativity and causing data loss.

Community

Tue, 19 Sep 2017 21:47:36 +0200

The vision for our community borrows a lot from the Rust community.

Getting Started

We currently try to maintain two communication channels:

pijul.zulipchat.com, a Zulip channel to ask any question about Pijul.
discourse.pijul.org, a Discourse forum for lengthier discussions.

Code of Conduct

This code of conduct is largely stolen from the Rust Code of Conduct.

We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.

New release - Pijul 0.8

Thu, 14 Sep 2017 00:00:00 +0000

We’re proud to announce a new release of Pijul: version 0.8.

Updates on nest.pijul.com

Sun, 02 Jul 2017 00:00:00 +0000

As nest.pijul.com is slowly stabilising, I’d like to blog about a few lessons learnt during its development so far.

Pijul 0.4, Improvements and breaking changes

Sun, 02 Apr 2017 00:00:00 +0000

Today, we released a new version of Pijul. Still alpha, but lots of updates. Also, still GPL2+ (as in version 0.3, but unlike before).

News of Sanakirja

Thu, 16 Mar 2017 00:00:00 +0000

We’ll release a usable version of Pijul later today. Pijul itself is in an alpha stage, but is based on a number of other things we’ve needed to write, among which a key-value store known as Sanakirja.

Pijul 0.3, our first bootstrapped release! Featuring nest.pijul.com

Thu, 16 Mar 2017 00:00:00 +0000

The pijul team is proud and happy to announce the first bootstrapped release of pijul, and of the Nest, a hosting service for pijul repositories.

First usable Pijul

Tue, 10 Jan 2017 00:00:00 +0000

After several months of silence, I’m pleased to announce that we are starting to test the first usable version of Pijul.

The design of Thrussh

Wed, 20 Jul 2016 00:00:00 +0000

I’ve recently written an SSH library called Thrussh, and I’ve also started to use it for actual tasks like replacing the SCP client in Pijul.

Sprint started!

Fri, 06 May 2016 00:00:00 +0000

I’m blogging live from the first Pijul sprint, at the Aalto Design Factory in Helsinki. Feel free to stop by and say hello if you’re around!

Reorganizing libpijul

Sat, 09 Apr 2016 00:00:00 +0000

Now that Sanakirja is in a usable state, the main operation before we can start to use Pijul for other purposes than tests is to plug Sanakirja in. This post describes our strategy for doing this, and the difficulties we’ve had.

Sprint+Meetup

Sat, 02 Apr 2016 00:00:00 +0000

We’re happy to announce the first joint sprint with the darcs team, which will take place on May 6th, 7th and 8th in Helsinki, Finland. If you’re interested in using and/or learning Rust and Haskell, come hack with us!

Sanakirja 0.2

Thu, 24 Mar 2016 00:00:00 +0000

I just finished Sanakirja 0.2, available from our darcs repository, and soon on crates.io.

Functional semantics in Sanakirja

Thu, 03 Mar 2016 00:00:00 +0000

I announced Sanakirja, our new database backend, on reddit and twitter, but so far I had not had the time to blog about it. This post explains some of the ideas behind this new crate.

Thoughts on branches

Mon, 15 Feb 2016 00:00:00 +0000

Since our release of Pijul 0.2, I’ve been working mostly on the biggest feature of Pijul 0.3: branches. In this post, I try to explain the current plan.

Pijul 0.2

Tue, 02 Feb 2016 00:00:00 +0000

We’re proud to announce the second public release of Pijul, a new version control system aiming to be easy to learn and use, distributed, and fast.

XKCD and Git

Sat, 31 Oct 2015 00:00:00 +0000

Seems like we’re not the only ones! Here is the original picture.

Compressed Sanakirja databases

Mon, 01 Jan 0001 00:00:00 +0000

Contributor license agreement

Mon, 01 Jan 0001 00:00:00 +0000

Practical instructions

If you contribute to Libpijul, we are asking you to allow us to reuse your work, including under a different license than the current GPL-2.0-or-later.

If you agree and have the right to do so, please add IUH7IMWES3KQTHVWA5UNHAO7QWVCC5PQJ6VLK3RC3T4F2MS74P3AC (the patch introducing the CLA) to the dependencies of your patches. Three ways to do it:

Add the following line to .pijul/config, before recording:

extra_dependencies = [ "IUH7IMWES3KQTHVWA5UNHAO7QWVCC5PQJ6VLK3RC3T4F2MS74P3AC" ]

Add the following line in the Dependencies section of your changes:

Documentation

Mon, 01 Jan 0001 00:00:00 +0000

Coming soon. If you are interested in contributing, you can come on our discourse and ask anything, and contribute to our repository.

Model

Mon, 01 Jan 0001 00:00:00 +0000

On this page, we hope to clarify the main ideas behind Pijul.

Why care about patch theory?

Good question. The answer is, because having a simple mental model of a tool allows to predict what it does. Pijul uses only patches, which are atomic components of team work. In Pijul, a branch is exactly a set of patches.

Many users of other distributed version control systems expect their favorite tool to work in a way that is somewhat different from how these systems really work.

Thrussh

Mon, 01 Jan 0001 00:00:00 +0000

Portable SSH client and server library

Thrussh is the first Rust SSH library. The only non-Rust part is the crypto backend (handled by https://briansmith.org/rustdoc/ring/). This means that this library will never be subject to buffer overruns, double frees, etc. It is designed to work on any platform, and to work with asynchronous IO.

This also allows it to be as fast as others, since less time is spent on checking memory allocations (compilers do that better than humans), and more time is spent on performance. Also, contrarily to others, Thrussh uses no Unix-specific features, which means it’s Windows-compatible.