C++ Stories

Understanding std::shared_mutex from C++17

Sat, 14 Feb 2026 00:00:00 +0000

In this article, we’ll start with a basic example using std::mutex, look at its limitations, and then introduce std::shared_mutex, a reader-writer mutex added in C++17. Even in 2026, with many new concurrency features available, std::shared_mutex is still a valuable and practical tool.

Let’s jump in.

A Simple Thread-Safe Counter with `std::mutex`

We’ll begin with a small example (a standard “hello world” for this type of mutexes): a counter object that multiple threads can access:

#include <mutex>

class Counter {
public:
    int get() const {
        std::lock_guard<std::mutex> lock(mutex_);
        return value_;
    }

    void increment() {
        std::lock_guard<std::mutex> lock(mutex_);
        ++value_;
    }
private:
    mutable std::mutex mutex_;
    int value_{0};
};

Nothing scary so far, and this implementation is correct, thread-safe, and easy to understand.

However, there’s a significant limitation: All access is exclusive.

Only one thread can call get() or increment() at any given time, even though get() does not modify the state.

When `std::mutex` Becomes a Bottleneck

In many real-world programs, shared data is read frequently and might be rarely updated/overriden.

For example:

configuration data
caches
lookup tables
statistics and metrics

With std::mutex, even multiple read-only operations block each other. This limits scalability and wastes available parallelism.

Have a look at the following example:

int main() {
    Counter counter;
    std::vector<std::jthread> threads;

    threads.emplace_back([&counter] {
        for (int i = 0; i < 10; ++i) {
            counter.increment();
            std::this_thread::sleep_for(15ms);
        }
    });

    for (int i = 0; i < 4; ++i) {
        threads.emplace_back([&counter, i] {
            for (int j = 0; j < 10; ++j) {
                std::println("reader {} sees {}", i, counter.get());
                std::this_thread::sleep_for(10ms);
            }
        });
    }
}

On Compiler Explorer - see here https://godbolt.org/z/cdjrj8hGd - I’m getting the following output:

Program returned: 0
reader 0 sees 0
reader 1 sees 1
reader 2 sees 1
reader 3 sees 1
reader 0 sees 1
reader 1 sees 1
reader 2 sees 1
reader 3 sees 1
reader 2 sees 2
reader 3 sees 2
reader 0 sees 2
reader 1 sees 2
reader 0 sees 3
reader 1 sees 3
reader 2 sees 3
reader 3 sees 3
reader 3 sees 3
reader 0 sees 3
reader 1 sees 3
reader 2 sees 3
reader 3 sees 4
reader 1 sees 4
reader 2 sees 4
reader 0 sees 4
reader 3 sees 5
reader 1 sees 5
reader 2 sees 5
reader 0 sees 5
reader 1 sees 5
reader 2 sees 5
reader 3 sees 5
reader 0 sees 5
reader 2 sees 6
reader 3 sees 6
reader 0 sees 6
reader 1 sees 6
reader 3 sees 7
reader 0 sees 7
reader 2 sees 7
reader 1 sees 7

Notice that some lines repeat the same value. That’s expected. The writer increments the counter every ~15 ms, but readers sample it every ~10 ms, so they can observe the same value more than once. Also, thread scheduling is not deterministic, so each reader may see a slightly different sequence.

In the demo code, there are many more reads than writes, so is the mutex the best solution here?

Introducing `std::shared_mutex`

std::shared_mutex is a reader-writer mutex. It supports two locking modes:

shared ownership - many threads can hold the lock simultaneously
exclusive ownership - only one thread can hold the lock

This makes std::shared_mutex a good fit for read-mostly data structures.

Refactoring the Counter with `std::shared_mutex`

Let’s update the counter example:

#include <shared_mutex>
#include <mutex> // for locks

class Counter {
public:
    int get() const {
        std::shared_lock lock(mutex_);
        return value_;
    }

    void increment() {
        std::unique_lock lock(mutex_);
        ++value_;
    }

private:
    mutable std::shared_mutex mutex_;
    int value_{0};
};

What changed?

get() now uses std::shared_lock
increment() still uses exclusive access via std::unique_lock
the mutex type is std::shared_mutex

With this change, many threads can call get() concurrently, writes remain fully protected, and read scalability improves with almost no extra complexity/

On Compiler Explorer - https://godbolt.org/z/W1es4qM8x - I’m getting:

reader 0 sees 0
reader 1 sees 0
reader 2 sees 0
reader 3 sees 1
reader 1 sees 1
reader 0 sees 1
reader 2 sees 1
reader 3 sees 1
reader 1 sees 2
reader 0 sees 2
reader 2 sees 2
reader 3 sees 2
reader 3 sees 2
reader 1 sees 2
reader 0 sees 2
reader 2 sees 2
reader 1 sees 3
reader 2 sees 3
reader 3 sees 3
reader 0 sees 3
reader 2 sees 4
reader 1 sees 4
reader 3 sees 4
reader 0 sees 4
reader 2 sees 4
reader 1 sees 5
reader 3 sees 5
reader 0 sees 5
reader 3 sees 5
reader 1 sees 5
reader 0 sees 5
reader 2 sees 5
reader 3 sees 6
reader 0 sees 6
reader 2 sees 6
reader 1 sees 6
reader 3 sees 6
reader 0 sees 7
reader 2 sees 7
reader 1 sees 7

The output is still nondeterministic, and it may look similar between std::mutex and std::shared_mutex. The key difference is not the values printed, but the fact that with std::shared_mutex multiple readers can hold the lock at the same time. This matters when the protected read section is non-trivial (parsing, lookups, copying data, etc.) and when the program is under real contention.

Measuring gains

Ok, but let’s try to measure the gains to see the difference. I’ll add some extra sleep code to simulate a real “workload”, and then we can compare times.

#include <chrono>
#include <mutex>
#include <print>
#include <shared_mutex>
#include <thread>
#include <vector>

using namespace std::chrono_literals;

// Simulated work while holding the lock
constexpr auto ReadWork  = 2ms;
constexpr auto WriteWork = 1ms;
constexpr auto GapWork   = 1ms;

// --- Version 1: std::mutex (exclusive for both read and write) ---
class CounterMutex {
public:
    int get() const {
        std::lock_guard<std::mutex> lock(mutex_);
        std::this_thread::sleep_for(ReadWork);
        return value_;
    }

    void increment() {
        std::lock_guard<std::mutex> lock(mutex_);
        ++value_;
        std::this_thread::sleep_for(WriteWork);
    }

private:
    mutable std::mutex mutex_;
    int value_{0};
};

// --- Version 2: std::shared_mutex (shared reads, exclusive writes) ---
class CounterSharedMutex {
public:
    int get() const {
        std::shared_lock lock(mutex_);
        std::this_thread::sleep_for(ReadWork);
        return value_;
    }

    void increment() {
        std::unique_lock lock(mutex_);
        ++value_;
        std::this_thread::sleep_for(WriteWork);
    }

private:
    mutable std::shared_mutex mutex_;
    int value_{0};
};

void run_test(const char* label, auto& counter) {
    constexpr int kReaders = 4;
    constexpr int kReadsPerReader = 30;
    constexpr int kWrites = 20;

    const auto start = std::chrono::steady_clock::now();

    {
        std::vector<std::jthread> threads;
        threads.reserve(1 + kReaders);

        // Writer
        threads.emplace_back([&] {
            for (int i = 0; i < kWrites; ++i) {
                counter.increment();
                std::this_thread::sleep_for(GapWork);
            }
        });

        // Readers
        for (int id = 0; id < kReaders; ++id) {
            threads.emplace_back([&counter] {
                for (int i = 0; i < kReadsPerReader; ++i) {
                    (void)counter.get();
                    std::this_thread::sleep_for(GapWork);
                }
            });
        }
    }

    const auto end = std::chrono::steady_clock::now();
    const auto ms =
        std::chrono::duration_cast<std::chrono::milliseconds>(end - start);

    std::println("{}: {} ms", label, ms.count());
}

int main() {
    std::println("hardware_concurrency: {}", std::thread::hardware_concurrency());

    CounterMutex counter_mutex;
    CounterSharedMutex counter_shared;

    run_test("std::mutex", counter_mutex);
    run_test("std::shared_mutex", counter_shared);
}

See at Compiler Explorer. The output:

hardware_concurrency: 2
std::mutex: 285 ms
std::shared_mutex: 102 ms

On this machine, std::thread::hardware_concurrency() reports 2 hardware threads. In this setup, the workload is intentionally read-heavy and each read holds the lock for a short but non-trivial amount of time. With std::mutex, all reads and writes use exclusive locking, so reader threads are effectively serialized and must wait for each other. As a result, the total runtime is about 285 ms.

With std::shared_mutex, read operations acquire a shared lock and can therefore run concurrently. On a system with two hardware threads, this allows two readers to make progress at the same time, significantly reducing contention. Under the same workload and timing parameters, the total runtime drops to about 102 ms. The exact numbers will vary between runs and platforms. Still, the difference clearly illustrates the main advantage of std::shared_mutex for read-mostly workloads: improved throughput when multiple readers can proceed in parallel.

A More Realistic Example: Read-Mostly Cache

The “counter” example is small, but the pattern becomes more useful with real data structures. A typical example is a cache.

#include <shared_mutex>
#include <unordered_map>
#include <string>

class Cache {
public:
    std::string get(const std::string& key) const {
        std::shared_lock lock(mutex_);
        if (auto it = data_.find(key); it != data_.end())
            return it->second;
        return {};
    }

    void put(std::string key, std::string value) {
        std::unique_lock lock(mutex_);
        data_[std::move(key)] = std::move(value);
    }

private:
    mutable std::shared_mutex mutex_;
    std::unordered_map<std::string, std::string> data_;
};

This pattern appears often in real systems:

many threads read cached values
updates are relatively rare
correctness is more important than extreme micro-optimizations

Using std::shared_mutex allows reads to scale while still keeping writes safe.

Common Pitfalls

Recursive locking is undefined

A thread must not lock the same std::shared_mutex recursively:

mutex.lock();
mutex.lock(); // undefined behavior

This applies to both shared and exclusive ownership.

You cannot upgrade a shared lock

This pattern will deadlock:

std::shared_lock read_lock(mutex_);
std::unique_lock write_lock(mutex_); // bad idea

If upgrading is required, the locking strategy usually needs to change.

More locks do not always mean more performance

std::shared_mutex has overhead. If:

writes are frequent
contention is low
critical sections are small

then std::mutex may be just as fast or even faster. Measuring is important.

You can read this recent article about some detailed measurements: When std::shared_mutex Outperforms std::mutex: A Google Benchmark Study – Tech For Talk

Newer Concurrency Tools in C++20 and Above

Since C++17, the concurrency library has expanded significantly. We now have:

std::jthread and cooperative cancellation (C++20)
semaphores, latches, and barriers (C++20)
improved atomic operations (C++20)
safe memory reclamation mechanisms (RCU, hazard pointers in C++26)

These tools focus mostly on thread lifetime, coordination, cancellation or lock-free programming.

std::shared_mutex fills a different role.

It is still a mutual-exclusion primitive, explicitly designed to protect a shared state with many readers and few writers. It does not compete with atomics, condition variables, or RCU.

Summary

In this article, we explored std::shared_mutex, a synchronization primitive designed for protecting read-mostly shared data. Unlike std::mutex, it allows multiple threads to read the same resource concurrently, while still ensuring exclusive access for writers.

Starting from a simple counter example, we showed how a plain mutex serializes all access and how switching to std::shared_mutex enables concurrent reads with only small changes to the code. A simple benchmark experiment demonstrated that, under a read-heavy workload, this can significantly reduce contention and improve throughput.

Even with newer concurrency features added in C++20 and later, std::shared_mutex remains a useful and practical tool when reads dominate writes and simplicity is preferred over more complex synchronization techniques.

References

C++ Concurrency in Action 2nd Edition - by Anthony Williams
Programming: Principles and Practice Using C++ 3rd Edition - by Bjarne Stroustrup
Concurrency with Modern C++: What every professional C++ programmer should know about concurrency - by Rainer Grimm
<shared_mutex> | Microsoft Learn
std::shared_mutex - cppreference.com

Back to you

Have you tried shared_mutex? In what situations?
Do you use concurrency tools from C++20 and above?

Share your comments below

IIFE for Complex Initialization

Sat, 24 Jan 2026 00:00:00 +0000

What do you do when the code for a variable initialization is complicated? Do you move it to another method or write inside the current scope?

In this blog post, I’d like to present a trick that allows computing a value for a variable, even a const variable, with a compact notation.

Updated in Jan 2026: improved code, added C++26 section, added [&] section, updated code samples and added links to Compiler Explorer.

Intro

I hope you’re initializing most variables as const (so that the code is more explicit, and also compiler can reason better about the code and optimize).

For example, it’s easy to write:

const int myParam = inputParam * 10 + 5;

or even:

const int myParam = bCondition ? inputParam*2 : inputParam + 10;

But what about complex expressions? When we have to use several lines of code, or when the ? operator is not sufficient.

‘It’s easy’ you say: you can wrap that initialization into a separate function.

While that’s the right answer in most cases, I’ve noticed that in reality a lot of people write code in the current scope. That forces you to stop using const and code is a bit uglier.

You might see something like this:

int myVariable = 0; // this should be const...

if (bFirstCondition)
    myVariable = bSecondCondition ? computeFunc(inputParam) : 0;
else
    myVariable = inputParam * 2;

// more code of the current function...
// and we assume 'myVariable` is const now

The code above computes myVariable which should be const. But since we cannot initialize it in one line, then the const modifier is dropped.

I highly suggest wrapping such code into a separate method, but recently I’ve come across a new option.

I’ve got the idea from a great talk by Jason Turner about “Practical Performance Practices” where among various tips I’ve noticed “IIFE”.

The IIFE acronym stands for “Immediately-invoked function expression”. Thanks to lambda expression, it’s now available in C++. We can use it for complex initialization of variables.

Extra: You might also encounter: IILE, which stands for “Immediately Invoked Lambda Expression”.

How does it look like?

IIFE

The main idea behind IIFE is to write a small lambda that computes the value:

const auto var = [&...] { 
    return /* some complex code here */; 
}(); // call it!

var is const even when you need several lines of code to initialize it!

The critical bit is to call the lambda at the end. Otherwise it’s just a definition.

The imaginary code from the previous section could be rewritten to:

const int myVariable = [&] {
    if (bFirstCondition)
        return bSecondCondition ? computeFunc(inputParam) : 0;
    else
       return inputParam * 2;
}(); // call!

// more code of the current function...

The above example shows that the original code was enclosed in a lambda.

We can also be more expressive and capture only needed things:

const int myVariable = [&inputParam] {
    if (bFirstCondition)
        return bSecondCondition ? computeFunc(inputParam) : 0;
    else
       return inputParam * 2;
}(); // call!

// more code of the current function...

The expression takes no parameters but captures the current scope by reference (or some explicit variables). Also, look at the last line of the declaration, () - we’re invoking the function immediately.

Additionally, since this lambda takes no parameters, we can skip () in the declaration. Only [] is required at the beginning, since it’s the lambda-introducer .

Improving Readability of IIFE

One of the main concerns behind IIFE is readability. Sometimes it’s not easy to see that () at the end.

How can we fix that?

Some people suggest declaring a lambda above the variable declaration and just calling it later:

auto initialiser = [&] { 
    return /* some complex code here */; 
};
const auto var = initialiser(); // call it

The issue here is that you need to find a name for the initializer lambda, but I agree that’s easy to read.

And another technique involves std::invoke() that is expressive and shows that we’re calling something:

const auto var = std::invoke([&] { 
    return /* some complex code here */; 
});

Note: std::invoke() is located in the <functional> header and it’s available since C++17.

In the above example, you can see that we clearly express our intention, so it might be easier to read such code.

Now back to you:

Which method do you prefer?

just calling () at the end of the anonymous lambda?
giving a name to the lambda and calling it later?
using std::invoke()
something else?

Should we capture the whole context with `&` ?

Short answer: capture only what you need.

Using [&] is concise and mirrors the “local scope” feel of the original code. But capture‑all by reference hides dependencies and makes it easy to accidentally use something that later becomes a dangling reference.

That’s why it’s best to write:

const auto value = [&inputParam, &anotherValue] {
    // ...
    return /* computed value */;
}();

[&] is convenient and safe for IIFE in most cases, but it’s also the least explicit. For code that will evolve or be reviewed often, prefer a precise capture list to make dependencies obvious and avoid surprises.

C++26 Updates

P1102R2: Down with ()!

From the start of the article I wrote that you don’t need to write () to indicate the argument list… but until C++26 we had had to write:

const auto value = [inputParam]() noexcept {
    if (/*condition*/)
        return inputParam*2.0;

    return inputParam;
}();

Do you see []() noexcept… ?

Up to C++26 there was a limitation:

Confusingly, the current Standard requires the empty parens when using the mutable (an other) keyword. This rule is unintuitive, causes common syntax errors, and clutters our code.

This also applied to noexcept, attributes and other specifiers.

In summary, since C++26 you can omit the () if there are no parameters to declare/pass to the lambda.

Use Case of IIFE

Ok, the previous examples were all super simple, and maybe even convoluted… is there a better and more practical example?

How about building a simple HTML string?

Our task is to produce an HTML node for a link:

As input, you have two strings: link and text (might be empty).

The output: a new string:

<a href="link">text</a>

<a href="link">link</a> (when text is empty)

We can write a following function:

std::string BuildStringTest(std::string link, std::string text) {
    std::string html;
    html = "<a href=\"" + link + "\">";
    if (!text.empty())
        html += text;
    else
        html += link;
    html += "</a>";

    return html;
}

Alternatively we can also compact the code:

std::string BuildStringTest2(std::string link, std::string text) {
    std::string html;
    const auto& inText = text.empty() ? link : text;
    html = "<a href=\"" + link + "\">" + inText + "</a>";
    return html;
}

Ideally, we’d like to have html as const, so we can rewrite it as:

std::string BuildStringTestIIFE(std::string link, std::string text) {
    const std::string html = [&link, &text] {
        std::string out = "<a href=\"" + link + "\">";
        if (!text.empty())
            out += text;
        else
            out += link;
        out += "</a>"; 
        return out;
    }(); // call ()!

    return html;
}

Or with a more compact code:

std::string BuildStringTestIIFE2(std::string link, std::string text) {
    const std::string html = [&link, &text] {
        const auto& inText = text.empty() ? link : text;
        return "<a href=\"" + link + "\">" + inText + "</a>";
    }(); // call!

    return html;
}

Here’s the code @Compiler Explorer

Do you think that’s acceptable?

Try rewriting the example below, maybe you can write nicer code?

A Benchmark of IIFE

With IIFE, we not only get a clean way to initialize const variables, but since we have more const objects, we might get better performance.

Is that true? Or maybe longer code and creation of lambda makes things slower?

For the HTML example, I wrote a benchmark that tests all four version:

@QuickBench

And it looks like we’re getting 10% with IIFE! (also similar result with recent compilers, as of 2026)

Some notes:

This code shows the rough impact of the IIFE technique, but it was not written to get the super-fast performance. We’re manipulating string here so many factors can affect the final result.
it seems that if you have less temporary variables, the code runs faster (so StringBuild is slightly faster than StringBuild2 and similarly IIFE and IIFE2)
We can also use string::reserve to preallocate memory, so that each new string addition won’t cause reallocation.

You can check other tests here: @QuickBench, and see the recent version @QuickBench Clang 17.0

It looks like the performance is not something you need to be concerned with. The code works sometimes faster, and in most of the cases the compiler should be able to generate similar code as the initial local version

Summary

Would you use such a thing in your code?

In C++ Coding Guideline we have a suggestion that it’s viable to use it for complex init code:

C++ Core Guidelines - ES.28: Use lambdas for complex initialization,

I am a bit sceptical to such expression, but I probably need to get used to it. I wouldn’t use it for a long code. It’s perhaps better to wrap some long code into a separate method and give it a proper name. But if the code is 2 or three lines long… maybe why not.

Also, if you use this technique, make sure it’s readable. Leveraging std::invoke() seems to be a great option.

I want to thank Mariusz Jaskółka from for the review, hints about compacting the code and also perf improvements with reserve().

Back to you

What do you think about such syntax? Have you used it in your projects?
Do you have any guidelines about such thing?
Is such expression better than having lots of small functions?

References and Books

Books:

7 Practical std::chrono Calendar Examples (C++20)

Tue, 30 Dec 2025 00:00:00 +0000

This article collects small, self-contained, and practical examples for working with std::chrono calendar types.

The previous blog post - see Exploring C++20 std::chrono - Calendar Types - C++ Stories - focused on the building blocks: calendar types, operators, and arithmetic rules. In this post, we’ll focus on practical examples like:

What’s the last business day of the month?
When is the third Friday in June?
How many days since the start of the year?
What happens when I add a month to January 31?
And a few more

Let’s start.

1. Which day of the year is that?

Let’s begin with a simple example of counting how many days a specific date is from the start of the year. In the previous article, we covered basic arithmetic, so now we can apply that knowledge to calculate the difference between a given date and the beginning of the year.

See below:

#include <chrono>
#include <print>

using namespace std::chrono;

constexpr std::string_view ordinal_suffix(const days& d) {
    const auto n = d.count();

    const auto last_two = n % 100;
    if (last_two >= 11 && last_two <= 13)
        return "th";

    switch (n % 10) {
        case 1: return "st";
        case 2: return "nd";
        case 3: return "rd";
        default: return "th";
    }
}

void print_day_of_year(year_month_day ymd) {
    const sys_days start_of_year{ ymd.year() / January / 1 };
    const sys_days current{ ymd };

    const days day_index = (current - start_of_year) + days{1};

    std::print("{} is the {}{} day of the year\n",
               ymd,
               day_index.count(),
               ordinal_suffix(day_index));
}

int main() {
    print_day_of_year(2025y / January / 2);
    print_day_of_year(2025y / November / 11);
}

Run here @Compiler Explorer

A few small details worth noticing:

The arithmetic is done entirely in sys_days.
The result is one-based (January 1 is day 1), which is usually what people expect.
Leap years fall out naturally from the sys_days conversion.

2. Nth weekday of a month

If you want to check the date of the third Friday in a given month:

#include <chrono>
#include <print>

int main() {
    using namespace std::chrono;
    using namespace std::literals;
    auto thirdFriday = 2026y / June / Friday[3];
    std::print("Third Friday of {:%B %Y}: {}\n", thirdFriday, 
                sys_days{thirdFriday});
}

Run at Compiler Explorer

The type of thirdFriday is year_month_weekday, and if we want to show it as a full date, we can convert it to sys_days.

3. Last day of the month

End-of-month logic sounds simple, but it can be painful to implement. With year_month_day_last it’s now super simple:

#include <chrono>
#include <print>

int main() {
    using namespace std::chrono;

    year_month_day_last endOfFeb2025 =
        year{2025} / February / last;

    year_month_day_last endOfFeb2024 =
        year{2024} / February / last;

    std::print("{}\n", sys_days{endOfFeb2025});
    std::print("{}\n", sys_days{endOfFeb2024});
}

See at Compiler Explorer

Output:

2025-02-28
2024-02-29

Again, we need to convert to sys_days to print the date; otherwise, you will see something like 2025/Feb/last.

4. Computing “last” dates for the whole year

We previously computed the n-th weekday, but how can we determine the last weekday of a month? This is useful for payrolls, meetings, and similar purposes.

The nice part is that this is exactly what weekday[last] is for:

#include <chrono>
#include <print>

int main() {
    using namespace std::chrono;

    const auto y = 2026y;

    for (unsigned m = 1; m <= 12; ++m) {
        auto lastFriday = y / m / Friday[last]; 
        sys_days date = sys_days{lastFriday}; 

        std::print("{:%B %Y}: last Friday is {:%F}\n", date, date);
    }
}

Example output (shape):

January 2026: last Friday is 2026-01-30
February 2026: last Friday is 2026-02-27
...
December 2026: last Friday is 2026-12-25

Warning: The loop for (month m = January; m <= December; ++m) will not work as intended. It creates an infinite loop because ++m uses modulo arithmetic, which wraps it back to January.

5. Adding months

Month arithmetic in <chrono> is field-based, not duration-based. That distinction matters, especially once you involve sys_days.

#include <chrono>
#include <print>

int main() {
    using namespace std::chrono;

    year_month_day jan31{ year{2025} / January / 31 };
    auto feb = jan31 + months{1};

    std::print("Jan 31 + 1 month = {}, ok: {}\n",
               feb, feb.ok());
}

See @Compiler Explorer

Output:

Jan 31 + 1 month = 2025-02-31 is not a valid date, ok: false

This is intentional. Adding months to a year_month_day preserves the day field, even if the result is not a valid civil date. The library does not clamp or normalize automatically.

How to clamp the result and get the correct date?

If you first perform calendar arithmetic and then convert to sys_days, overflow is normalized into the following month:

auto normalized = sys_days{ jan31 + months{1} };
std::print("Normalized via sys_days: {}\n",
           year_month_day{normalized});

Output:

Normalized via sys_days: 2025-03-03

See @Compiler Explorer

What happens here:

jan31 + months{1} is 2025-02-31 (invalid, but allowed)
Converting to sys_days forces normalization
February 2025 has 28 days
February 31 overflows by 3 days so we get March 3

You can also try a different “order”:

auto durationBased = sys_days{jan31} + months{1};
std::print("Duration-based result: {}\n", durationBased);

Typical output (libstdc++):

2025-03-02 10:29:06

See @Compiler Explorer

Here, months{1} is treated as an average month duration (30 days, 10 hours, 29 minutes, 6 seconds).

6. Next business day

A very common rule in finance is “move to the next business day”. Settlement dates, payment processing, and reporting cutoffs often use this exact logic, at least as a baseline.

For now, we’ll define a business day as Monday to Friday, skipping weekends. Holidays can be layered on later without changing the structure of the code.

#include <chrono>
#include <print>

using namespace std::chrono;

sys_days next_business_day(sys_days d) {
    while (true) {
        d += days{1};

        weekday wd{d};

        if (wd != Saturday && wd != Sunday)
            return d;
    }
}

int main() {
    sys_days tradeDate = sys_days{ year{2025}/November/21 };
    auto settlement = next_business_day(tradeDate);

    std::print("Trade date: {}\n", year_month_day{tradeDate});
    std::print("Next business day: {}\n", year_month_day{settlement});

    sys_days tuesday = sys_days{ year{2025}/November/18 };
    std::print("Next business day after {} is {}\n",
               year_month_day{tuesday},
               year_month_day{next_business_day(tuesday)});
}

Output:

Trade date: 2025-11-21
Next business day: 2025-11-24
Next business day after 2025-11-18 is 2025-11-19

See @Compiler Explorer

7. Count Fridays in a month

As a last example, let’s count how many Fridays do we have a in a given month:

#include <chrono>
#include <print>

using namespace std::chrono;

int count_weekdays_in_month_simple(year_month ym, weekday target) {
    sys_days d = sys_days{ ym / day{1} };

    int count = 0;
    while (year_month_day{d}.month() == ym.month()) {
        if (weekday{d} == target)
            ++count;
        d += days{1};
    }
    return count;
}

unsigned count_weekdays_in_month(year_month ym, weekday wd) {
    unsigned count = 0;

    for (unsigned i = 1; ; ++i) {
        const year_month_weekday ymwd = ym / wd[i];
        if (!ymwd.ok()) 
            break;

        const year_month_day ymd{ ymwd };
        if (ymd.ok())
            ++count;
    }
    return count;
}


int main() {
    auto ym = year{2025} / May;

    int fridays = count_weekdays_in_month(ym, Friday);
    std::print("Fridays in {}: {}\n", ym, fridays);

    int mondays = count_weekdays_in_month(ym, Monday);
    std::print("Mondays in {}: {}\n", ym, mondays);
}

See @Compiler Explorer

The code has two versions:

The simple version iterates day by day and might not be optimal. You can theoretically speed it up by increasing by 7 days…
In the second version, we can iterate using weekdays.

Also, you can make the code even faster with just arthmetic, without any loops. It was suggested in this comment at Reddit:

int count_weekdays_in_month(year_month ym, weekday wd) {
  auto a = sys_days{ym / wd[1]};
  auto b = sys_days{ym / wd[last]};
  return (b - a) / weeks(1) + 1;
}

See the updated example here: https://godbolt.org/z/TPK368W9f

Summary

In this article we covered a set of small, practical examples built on top of the C++20 <chrono> calendar types The examples showed how to count days within a year, resolve n-th and last weekdays of a month, handle end-of-month and month-addition edge cases, distinguish between calendar and duration arithmetic, and express simple business rules such as “next business day” or counting specific weekdays in a month.

In most cases, we have the same pattern: calendar types are used to express structure and intent, while sys_days is used when actual day counting, iteration, or normalization is required.

Documents and Resources

C++ Standard Draft - Time section - N4868 (October 2020 pre-virtual-plenary working draft/C++20 plus editorial changes)
C++20 - The Complete Guide by Nicolai M. Josuttis @Amazon
Modern C++ Programming Cookbook: Master C++ core language and standard library features, with over 100 recipes, updated to C++20 2nd ed. Edition @Amazon
https://github.com/HowardHinnant/date/wiki/Examples-and-Recipes - examples and recipes for Howard Hinnant’s tz.h library, which is a base for chrono.

Back to you

Do you use std::chrono for calendar computations?
What use cases are most important to you for a calendar-type library?

Exploring C++20 std::chrono - Calendar Types

Sat, 27 Dec 2025 00:00:00 +0000

Before C++20, <chrono> gave us a solid foundation for working with clocks, durations, and time points - but it didn’t really know anything about the civil calendar. If you needed to represent “March 15”, check whether a date is valid, compute “the third Monday of November”, or add a few months to a date, you had to rely on custom utilities, std::tm, or external libraries like Howard Hinnant’s excellent date.

C++20 finally fills this gap by standardising a complete set of calendar types: year, month, day, weekday, year_month_day, month_day_last, and many more. These types let you express dates in a clear, strongly typed way, without fragile integer arithmetic or manual conversions.

All of this comes with a very clean API: more than 40 overloads of the / operator let you build date types fluently, almost like writing calendar expressions. In short, C++20 turns most everyday date manipulation tasks into expressive, readable, and fully standard C++.

So let’s review the core types and operations.

Introduction to Calendar Types

In C++20, the library extended durations types to cover days, months, and years:

using hours        = duration<int, ratio<3600>>;

// new in C++20
using days   = duration<int, ratio_multiply<ratio<24>, hours::period>>;
using weeks  = duration<int, ratio_multiply<ratio<7>, days::period>>;
using years  = duration<int, ratio_multiply<ratio<146097, 400>, days::period>>;
using months = duration<int, ratio_divide<years::period, ratio<12>>>;

And in C++20, we have the following new calendrical types:

name	description
`last_(spec)`	The type is used in conjunction with other calendar types to specify the last in a sequence. For example, depending on the context, it can represent the last day of a month, or the last day of the week of a month. See examples below…
`day`	represents a day of a month. It normally holds values in the range 1 to 31, but may hold non-negative values outside this range.
`month`	represents a month of a year. It normally holds values in the range 1 to 12, but may hold non-negative values outside this range.
`year`	represents a year in the civil (Gregorian) calendar. It can represent values in the range `[min(), max()]` (static member functions). It can be constructed with any int value, which will be subsequently truncated to fit into year’s unspecified internal storage
`weekday`	represents a day of the week in the civil calendar. It normally holds values in the range 0 to 6, corresponding to Sunday through Saturday, but it may hold non-negative values outside this range
`weekday_indexed`	represents a `weekday` and a small index in the range 1 to 5. This class is used to represent the first, second, third, fourth, or fifth weekday of a month.
`weekday_last`	represents the last weekday of a month.
`month_day`	represents a specific day of a specific month, but with an unspecified year.
`month_day_last`	represents the last day of a month.
`month_weekday`	represents the n-th weekday of a month, of an as yet unspecified year. To do this, the `month_weekday` stores a `month` and a `weekday_indexed`.
`month_weekday_last`	represents the last weekday of a month, of an as yet unspecified year. To do this, the `month_weekday_last` stores a `month` and a `weekday_last`.
`year_month`	represents a specific month of a specific year, but with an unspecified day. `year_month` is a field-based time point with a resolution of months.
`year_month_day`	represents a specific year, month, and day. `year_month_day` is a field-based time point with a resolution of days. The class supports years- and months-oriented arithmetic, but not days-oriented arithmetic. For the latter, there is a conversion to `sys_days`, which efficiently supports days-oriented arithmetic.
`year_month_day_last`	represents the last day of a specific year and month. `year_month_day_last` is a field-based time point with a resolution of days, except that it is restricted to pointing to the last day of a year and month. The class supports years- and months-oriented arithmetic, but not days-oriented arithmetic.
`year_month_weekday`	represents a specific year, month, and n-th weekday of the month. `year_month_weekday` is a field-based time point with a resolution of days. The class supports years- and months-oriented arithmetic, but not days-oriented arithmetic.
`year_month_weekday_last`	represents a specific year, month, and last weekday of the month. `year_month_weekday_last` is a field-based time point with a resolution of days, except that it is restricted to pointing to the last weekday of a year and month. The class supports years- and months-oriented arithmetic, but not days-oriented arithmetic.

And some notes:

All above classes, except of last_spec, are trivially copyable and standard-layout class types.
Most types have the ok() member function that returns true if a given state is correct in terms of the type, or its subtypes.
They have an implicit constexpr constructor from sys_days and an explicit constexpr constructor from local_days for days-oriented arithmetic.
Notice that duration types end with s, while calendrical types have a singular form, like day, month, etc.

Constants

Inside the std::chrono namespace, we also have the following calendrical constants:

inline constexpr last_spec last{};

inline constexpr weekday Sunday{0};
inline constexpr weekday Monday{1};
inline constexpr weekday Tuesday{2};
inline constexpr weekday Wednesday{3};
inline constexpr weekday Thursday{4};
inline constexpr weekday Friday{5};
inline constexpr weekday Saturday{6};

inline constexpr month January{1};
inline constexpr month February{2};
inline constexpr month March{3};
inline constexpr month April{4};
inline constexpr month May{5};
inline constexpr month June{6};
inline constexpr month July{7};
inline constexpr month August{8};
inline constexpr month September{9};
inline constexpr month October{10};
inline constexpr month November{11};
inline constexpr month December{12};

Creation and Operator `/`

Constructors of chrono date types taking an integral value are explicit, so that copy initialization with an integral won’t work:

std::chrono::day d{7};           // OK
std::chrono::day anotherDay = 3; // ERROR
std::chrono::month m{12};             // OK
std::chrono::month anotherMonth = 12; // ERROR

std::chrono exposes 40 overloads for the operator /, allowing us to create various calendar types with many combinations easily. Usually, you only have to provide the first type, and then the compiler will deduce the further sequence.

For example:

auto ymdOct = std::chrono::year { 1996 } / 10 / 31; // 31st october 1996, year_month_day
auto ydmOct = std::chrono::year { 1996 } / 31 / 10; // yyyy/dd/mm not possible, so this yields an invalid date!
auto ym {std::chrono::year{2025} / 11}; // year_month
auto mw {11/Monday[3]}; // 3rd Monday in Nov, month_weekday

Literals

As Howard Hinnant pointed out in the comment - there’s even easier way to specify dates - using literals.

For example:

std::chrono::year_month_day today{ std::chrono::year{2025}/11/22 };

can be written as:

using namespace std::literals;
auto today = 2025y/11/22;

We have the following literals:

Literal	Notes
`operator""h`	`std::chrono::duration` literal representing hours (C++14)
`operator""min`	`std::chrono::duration` literal representing minutes (C++14)
`operator""s`	`std::chrono::duration` literal representing seconds (C++14)
`operator""ms`	`std::chrono::duration` literal representing milliseconds (C++14)
`operator""us`	`std::chrono::duration` literal representing microseconds (C++14)
`operator""ns`	`std::chrono::duration` literal representing nanoseconds (C++14)
`operator""d`	`std::chrono::day` literal representing a day of a month (C++20)
`operator""y`	`std::chrono::year` literal representing a particular year (C++20)

See the example:

#include <chrono>
#include <print>

using namespace std::chrono;
using namespace std::chrono_literals;

int main() {
    std::print("{}\n", 2025y/June/30);
    std::print("{}\n", 2025y/1/30);
    std::print("{}\n", 2026y/January/1);
}

Run @Compiler Explorer

Validating Dates

Calendar types allow invalid states by design. That makes them cheap, constexpr-friendly value types, but it also means validation is explicit.

#include <chrono>
#include <print>

int main()
{
    using namespace std::chrono;

    year_month_day ymd1{ year{2025} / February / 29 };
    year_month_day ymd2{ year{2024} / February / 29 };

    std::print("{:%D} ok: {}\n", ymd1, ymd1.ok());
    std::print("{} ok: {}\n", ymd1, ymd1.ok());
    std::print("{:%D} ok: {}\n", ymd2, ymd2.ok());
}

See @Compiler Explorer

Output:

02/29/25 ok: false
2025-02-29 is not a valid date ok: false
02/29/24 ok: true

The interesting part is that when printing a date with default formatter it can add extra text when the date is invalid. This is not the case with :%D specifier.

Arithmetic

Some useful operations for working with calendrical types:

`day`

// member functions:
constexpr std::chrono::day& operator+=( const std::chrono::days& d ) noexcept;
constexpr std::chrono::day& operator-=( const std::chrono::days& d ) noexcept;

// non-member:
constexpr std::chrono::day operator+( const std::chrono::day&  d,
                                      const std::chrono::days& ds ) noexcept;
constexpr std::chrono::day operator+( const std::chrono::days& ds,
                                      const std::chrono::day&  d ) noexcept;
constexpr std::chrono::day operator-( const std::chrono::day&  d,
                                      const std::chrono::days& ds ) noexcept;
// notice it returns `days`, not `day`:
constexpr std::chrono::days operator-( const std::chrono::day& x,
                                       const std::chrono::day& y ) noexcept;

Operators: ++ and -- are also supported.

Examples:

#include <chrono>
#include <print>
 
int main()
{
    std::chrono::day oneDay{ 7 };
    std::chrono::day anotherDay{ 30 };

    oneDay += std::chrono::days(20);
    std::print("7 + 20: {}\n", oneDay);

    std::chrono::day future = anotherDay + std::chrono::days{ 10 };
    std::print("oneDay + anotherDay: {}, ok: {}\n", future, future.ok());

    std::chrono::days res = oneDay - anotherDay;
    std::print("res: {}\n", res);
}

Play @Compiler Explorer

The output:

7 + 20: 27
oneDay + anotherDay: 40 is not a valid day, ok: false
res: -3d

Notice the type difference for the - operator. It’s days.

`month`

// member operations:
constexpr std::chrono::month& operator+=(const std::chrono::months& m) noexcept;
constexpr std::chrono::month& operator+=(const std::chrono::months& m) noexcept;

// non member:
constexpr std::chrono::month operator+( const std::chrono::month& m,
                                        const std::chrono::months& ms ) noexcept;
constexpr std::chrono::month operator+( const std::chrono::months& ms,
                                        const std::chrono::month& m ) noexcept;
constexpr std::chrono::month operator-( const std::chrono::month& m,
                                        const std::chrono::months& ms ) noexcept;
                                        
// type change here: `months`!
constexpr std::chrono::months operator-( const std::chrono::month& m1,
                                         const std::chrono::month& m2 ) noexcept;

Operators: ++ and -- are also supported.

The set of operations is similar to day.

std::chrono::month oneMonth{ std::chrono::September };

oneMonth += std::chrono::months(20);
std::print("Sept + 20 months: {}\n", oneMonth);

std::chrono::month future = oneMonth + std::chrono::months{ 10 };
std::print("oneMonth + months(10): {}, ok: {}\n", future, future.ok());

std::chrono::months res = std::chrono::March - std::chrono::January;
std::print("Mar - Jan: {}\n", res);

Play @Compiler Explorer

The output:

Sept + 20 months: May
oneMonth + months(10): Mar, ok: true
Mar - Jan: 2[2629746]s

`year`

// member functions:
constexpr std::chrono::year& operator+=( const std::chrono::years& y ) noexcept;
constexpr std::chrono::year& operator-=( const std::chrono::years& y ) noexcept;
	
constexpr std::chrono::year operator+() noexcept;
constexpr std::chrono::year operator-() noexcept; // negation!	
	
// non-member functions:
constexpr std::chrono::year operator+( const std::chrono::year& y,
                                       const std::chrono::years& ys ) noexcept;
constexpr std::chrono::year operator+( const std::chrono::years& ys,
                                       const std::chrono::year& y ) noexcept;
constexpr std::chrono::year operator-( const std::chrono::year& y,
                                       const std::chrono::years& ys ) noexcept;
                                       
// type change: years
constexpr std::chrono::years operator-( const std::chrono::year& y1,
                                        const std::chrono::year& y2 ) noexcept;

Operators: ++ and -- are also supported.

See a basic example:

std::chrono::year xxiCentury{ 2001 };

xxiCentury += std::chrono::years{ 21 };
std::print("2001 + 21 years: {}\n", xxiCentury);

std::print("min {}, max {} year\n", std::chrono::year::min(), std::chrono::year::max());
std::chrono::year future = xxiCentury + std::chrono::years{ 100000 };
std::print("xxiCentury + years(100000): {}, ok: {}\n", future, future.ok());

std::chrono::years res = std::chrono::years{ 2022 } - std::chrono::years{ 22 };
std::print("Mar - Jan: {}\n", res);

Run @Compiler Explorer

The output:

2001 + 21 years: 2022
min -32767, max 32767 year
xxiCentury + years(100000): -29050, ok: true
Mar - Jan: 2000[31556952]s

`year_month_day`

This structure is a pack of year, month and day structures, but it exposes the following arithmetic operations:

// member functions:
// you can add/subtract only months or years
constexpr std::chrono::year_month_day&
          operator+=( const std::chrono::years& dy ) const noexcept;
constexpr std::chrono::year_month_day&
          operator+=( const std::chrono::months& dm ) const noexcept;
constexpr std::chrono::year_month_day&
          operator-=( const std::chrono::years& dy ) const noexcept;
constexpr std::chrono::year_month_day&
          operator-=( const std::chrono::months& dm ) const noexcept;
	
// non-member functions:	
constexpr std::chrono::year_month_day operator+(
     const std::chrono::year_month_day& ymd,
     const std::chrono::months& dm) noexcept;

constexpr std::chrono::year_month_day operator+(
     const std::chrono::months& dm,
     const std::chrono::year_month_day& ymd) noexcept;
     
constexpr std::chrono::year_month_day operator+(
     const std::chrono::year_month_day& ymd,
     const std::chrono::years& dy) noexcept;

constexpr std::chrono::year_month_day operator+(
     const std::chrono::years& dy,                                            const std::chrono::year_month_day& ymd) noexcept;

constexpr std::chrono::year_month_day operator-(
     const std::chrono::year_month_day& ymd,
     const std::chrono::months& dm) noexcept;

constexpr std::chrono::year_month_day operator-(
     const std::chrono::year_month_day& ymd,
     const std::chrono::years& dy) noexcept;

This time, we can only add/subtract years or months, and there’s no operation that returns some duration type.

See an example:

std::chrono::year_month_day today{ std::chrono::year{2025}/11/22 };

auto future = today + std::chrono::years{ 10 };
std::print("2025/11/22 + 10 years: {}\n", future);

std::chrono::year_month_day again = future - std::chrono::months{ 120 };
std::print("future - months(120): {}, ok: {}\n", again, again.ok());

Run @Compiler Explorer

The output

2025/11/22 + 10 years: 2035-11-22
future - months(120): 2025-11-22, ok: true

`weekday`

// member functions:
constexpr std::chrono::weekday& operator+=(const std::chrono::days& d ) noexcept;
constexpr std::chrono::weekday& operator-=(const std::chrono::days& d ) noexcept;

// non-member functions:
constexpr std::chrono::weekday operator+(const std::chrono::weekday& wd,
                                          const std::chrono::days& d ) noexcept;
constexpr std::chrono::weekday operator+(const std::chrono::days& d,
                                          const std::chrono::weekday& wd ) noexcept;
constexpr std::chrono::weekday operator-(const std::chrono::weekday& wd,
                                          const std::chrono::days& d ) noexcept;
// return type change to days:
constexpr std::chrono::days operator-( const std::chrono::weekday& wd1,
                                       const std::chrono::weekday& wd2 ) noexcept;

We can operate with days and also two weekdays. The types use modulo arithmetic.

See the example:

std::chrono::weekday aday = std::chrono::Sunday;
std::print("{}\n", aday);
++aday;
std::print("{}\n", aday);
aday += std::chrono::days{ 22 };
std::print("after adding 22 days: {}\n", aday);
std::print("Monday - Tuesday = {}\n", std::chrono::Monday - std::chrono::Tuesday);

Play @Compiler Explorer

The output:

Sun
Mon
after adding 22 days: Tue
Monday - Tuesday = 6d

Days arithmetic for `year_` and `month_` types

As you saw in the table, types for years and months support month/year arithmetic:

The class supports years- and months-oriented arithmetic, but not days-oriented arithmetic. For the latter, there is a conversion to sys_days, which efficiently supports days-oriented arithmetic.

But in most cases, we can easily convert to sys_days, perform the computations, and convert back to the desired type.

For example:

namespace chr = std::chrono;
const auto today = chr::sys_days{ chr::floor<chr::days>(chr::system_clock::now()) };
auto importantDate = chr::year{ 2011 } / chr::July / 21;
const auto delta = (today - chr::sys_days{ importantDate }).count();
std::print("{} was {} days ago!\n", importantDate, delta);

Run @Compiler Explorer

The output:

2011-07-21 was 5238 days ago!

PS: Do you know what significant event happened on that important date? (hint: something with space exploration).

Summary

C++20 significantly enhances <chrono> with a comprehensive set of calendar types - such as day, month, year, weekday, and year_month_day- that let you represent and manipulate civil dates in a clear, type-safe way. These types support convenient construction via the / operator, simple month/year arithmetic, validation with .ok(), and easy conversion to sys_days whenever day-precision logic is needed.

Next time, we’ll look at more examples of calendar types.

Documents and Resources

C++ Standard Draft - Time section - N4868 (October 2020 pre-virtual-plenary working draft/C++20 plus editorial changes)
C++20 - The Complete Guide by Nicolai M. Josuttis @Amazon
Modern C++ Programming Cookbook: Master C++ core language and standard library features, with over 100 recipes, updated to C++20 2nd ed. Edition @Amazon
https://github.com/HowardHinnant/date/wiki/Examples-and-Recipes - examples and recipes for Howard Hinnant’s tz.h library, which is a base for chrono.

C++ Templates: How to Iterate through std::tuple: C++26 Packs and Expansion Statements

Sun, 12 Oct 2025 00:00:00 +0000

In part 1 of this mini-series, we looked at the basics of iterating over a std::tuple using index_sequence and fold expressions. In part 2, we simplified things with std::apply and even created helpers like for_each_tuple and transform_tuple.

So far, we used C++ features up to C++20/23… but now, in C++26, we finally get language-level tools that make tuple iteration straightforward and expressive. In this article, we’ll explore two new techniques:

Structured bindings can introduce a pack - P1061 - turn a tuple into a pack of variables.
Expansion statements P1306 - the ultimate “compile-time loop” syntax.

Structured binding packs

Structured bindings have been around since C++17, but in C++26 they gained the ability to introduce packs. That means you can bind an arbitrary number of elements without spelling each out.

For example:

auto [head, ...rest] = std::tuple{1, 2, 3, 4};
// head == 1, rest... expands to (2, 3, 4)

This feature gives us a way to expand tuples into parameter packs, and once we have a pack, we can use fold expressions and other familiar template techniques.

Printing a tuple

We can bind all elements of a tuple into a pack and then fold over it:

#include <tuple>
#include <iostream>
#include <utility>

template <typename Tuple>
void print_tuple(const Tuple& t) {
    const auto& [...xs] = t;
    bool first = true;
    std::cout << "(";
    ((std::cout << (std::exchange(first, false) ? "" : ", ") << xs), ...);
    std::cout << ")";
}

int main() {
    print_tuple(std::make_tuple(1, 2.2, "hello"));
}

See at Compiler Explorer

This is a direct replacement for our old index_sequence or std::apply approaches - now written in plain C++26.

(Of course, you can easily update it to work with std::print)

Transforming a tuple

We can also apply a transformation and create a new tuple:

template <typename Tuple, typename Fn>
auto transform_tuple(const Tuple& t, Fn&& fn) {
    const auto& [...xs] = t;
    return std::make_tuple(fn(xs)...);
}

std::tuple tp{1, 2, 3};
auto doubled = transform_tuple(tp, [](auto x){ return x * 2; });
// doubled == (2, 4, 6)

See @Compiler Explorer

Here the ...xs pack is expanded inside std::make_tuple. This is exactly like writing fn(x1), fn(x2), fn(x3), but without the boilerplate.

Dot product

And another example, taken from the proposal:

template <class P, class Q>
constexpr auto dot_product(const P& p, const Q& q) {
    const auto& [...ps] = p;
    const auto& [...qs] = q;

    static_assert(sizeof...(ps) == sizeof...(qs), "Mismatched sizes");

    return ((ps * qs) + ...);
}

constexpr std::array<int, 3> a{1, 2, 3};
constexpr std::array<int, 3> b{4, 5, 6};
static_assert(dot_product(a, b) == 32);

Here’s the link to Compiler Explorer

With structured binding packs, tuples and arrays become much more convenient to manipulate.

Expansion statements

While structured binding packs let us turn tuples into packs of variables, expansion statements go even further: they let us write a compile-time loop directly in the language.

In short, you can now write:

template for (...) { }

See this:

auto tp = std::make_tuple(10, 20, 3.14, "hello");

std::cout << "(";
bool first = true;
template for (auto e : tp) {
    if (!std::exchange(first, false)) std::cout << ", ";
    std::cout << e;
}
std::cout << ")";

See @Compiler Explorer

No index_sequence, no std::get, no std::apply. Just a clean template for!

Transforming tuples

We can also transform the tuple:

#include <tuple>
#include <print>
#include <utility>

template <class Tuple>
void print_tuple_expand(const Tuple& t) {
    bool first = true;
    std::print("(");
    template for (const auto& e : t) {
        if (!std::exchange(first, false)) {
            std::print(", ");
        }
        std::print("{}", e);
    }
    std::print(")\n");
}

template <class Tuple, class Fn>
void for_each_tuple_expand(Tuple&& t, Fn&& fn) {
    template for (auto&& e : std::forward<Tuple>(t)) {
        fn(std::forward<decltype(e)>(e));
    }
}

int main() {
    auto tp = std::make_tuple(10, 20, 3.14);
    print_tuple_expand(tp);
    for_each_tuple_expand(tp, [](auto& x) {x *= 2;});
    print_tuple_expand(tp);
}

See @Compiler Explorer

Summary

This article closes the mini-series about tuple iteration. It’s great to see that over the years, C++ has evolved in a way that template/meta programming looks almost the same as regular runtime code. In the first installments of this series, I showed you some tricks, like integer sequence, std::apply… but in C++26, we can remove them and take advantage of template for.

(Of course, it’s still good to know the tricks up to C++23… as C++26 won’t be used in production for a couple more years :))

See the part one here: C++ Templates: How to Iterate through std::tuple: the Basics - C++ Stories.
See the second part here: C++ Templates: How to Iterate through std::tuple: std::apply and More - C++ Stories

References:

Effective Modern C++ by Scott Meyers
C++ Templates: The Complete Guide (2nd Edition) by David Vandevoorde, Nicolai M. Josuttis, Douglas Gregor

Back to you

Do you use std::apply in your code, or do you prefer some manual techniques?
Do you see other benefits of using template for?

Share your feedback in the comments below.

C++ Templates: How to Iterate through std::tuple: std::apply and More

Sun, 21 Sep 2025 00:00:00 +0000

In the previous article on the tuple iteration, we covered the basics. As a result, we implemented a function template that took a tuple and could nicely print it to the output. There was also a version with operator <<.

Today we can go further and see some other techniques. The first one is with std::apply from C++17, a helper function for tuples. Today’s article will also cover some strategies to make the iteration more generic and handle custom callable objects, not just printing.

This is the second part of the small series. See the first article here where we discuss the basics.

And see the third part about the ultimate solution with C++26.

Updated in Sept 2025: Extended the “Return value” section.

std:apply approach

A handy helper for std::tuple is the std::apply function template that came in C++17. It takes a tuple and a callable object and then invokes this callable with parameters fetched from the tuple.

Here’s an example:

#include <iostream>
#include <tuple>
 
int sum(int a, int b, int c) { 
    return a + b + c; 
}

void print(std::string_view a, std::string_view b) {
    std::cout << "(" << a << ", " << b << ")\n";
} 

int main() {
    std::tuple numbers {1, 2, 3};
    std::cout << std::apply(sum, numbers) << '\n';

    std::tuple strs {"Hello", "World"};
    std::apply(print, strs);
}

Play @Compiler Explorer

As you can see, std::apply takes sum or print functions and then “expands” tuples and calls those functions with appropriate arguments.

Here’s a diagram showing how it works:

Ok, but how does it relate to our problem?

The critical thing is that std::apply hides all index generation and calls to std::get<>. That’s why we can replace our printing function with std::apply and then don’t use index_sequence.

The first approach - working?

The first approach that came to my mind was the following - create a variadic function template that takes Args... and pass it to std::apply:

template <typename... Args>
void printImpl(const Args&... tupleArgs) {
    size_t index = 0;
    auto printElem = [&index](const auto& x) {
        if (index++ > 0) 
            std::cout << ", ";
        std::cout << x;
    };

    (printElem(tupleArgs), ...);
}

template <typename... Args>
void printTupleApplyFn(const std::tuple<Args...>& tp) {
    std::cout << "(";
    std::apply(printImpl, tp);
    std::cout << ")";
}

Looks… fine… right?

The problem is that it doesn’t compile :)

GCC or Clang generates some general error which boils down to the following line:

candidate template ignored: couldn't infer template argument '_Fn

But how? Why cannot the compiler get the proper template parameters for printImpl?

The problem lies in the fact that out printImpl is a variadic function template, so the compiler has to instantiate it. The instantiation doesn’t happen when we call std::apply, but inside std::apply. The compiler doesn’t know how the callable object will be called when we call std::apply, so it cannot perform the template deduction at this stage.

We can help the compiler and pass the arguments:

#include <iostream>
#include <tuple>

template <typename... Args>
void printImpl(const Args&... tupleArgs) {
    size_t index = 0;
    auto printElem = [&index](const auto& x) {
        if (index++ > 0) 
            std::cout << ", ";
        std::cout << x;
        };

    (printElem(tupleArgs), ...);
}

template <typename... Args>
void printTupleApplyFn(const std::tuple<Args...>& tp) {
    std::cout << "(";
    std::apply(printImpl<Args...>, tp); // <<
    std::cout << ")";
}

int main() {
    std::tuple tp { 10, 20, 3.14};
    printTupleApplyFn(tp);
}

Play @Compiler Explorer.

In the above example, we helped the compiler to create the requested instantiation, so it’s happy to pass it to std::apply.

But there’s another technique we can do. How about helper callable type?

struct HelperCallable {
    template <typename... Args>
    void operator()(const Args&... tupleArgs)  {
        size_t index = 0;
        auto printElem = [&index](const auto& x) {
            if (index++ > 0) 
                std::cout << ", ";
            std::cout << x;
        };

        (printElem(tupleArgs), ...);
    }
};

template <typename... Args>
void printTupleApplyFn(const std::tuple<Args...>& tp) {
    std::cout << "(";
    std::apply(HelperCallable(), tp);
    std::cout << ")";
}

Can you see the difference?

Now, what we do, we only pass a HelperCallable object; it’s a concrete type so that the compiler can pass it without any issues. No template parameter deduction happens. And then, at some point, the compiler will call HelperCallable(args...), which invokes operator() for that struct. And it’s now perfectly fine, and the compiler can deduce the types. In other words, we deferred the problem.

So we know that the code works fine with a helper callable type… so how about a lambda?

#include <iostream>
#include <tuple>

template <typename TupleT>
void printTupleApply(const TupleT& tp) {
    std::cout << "(";
    std::apply([](const auto&... tupleArgs) {
                size_t index = 0;
                auto printElem = [&index](const auto& x) {
                    if (index++ > 0) 
                        std::cout << ", ";
                    std::cout << x;
                };

                (printElem(tupleArgs), ...);
            }, tp
        )
    std::cout << ")";
}

int main() {
    std::tuple tp { 10, 20, 3.14, 42, "hello"};
    printTupleApply(tp);
}

Play @Compiler Explorer.

Also works! I also simplified the template parameters to just template <typename TupleT>.

As you can see, we have a lambda inside a lambda. It’s similar to our custom type with operator(). You can also have a look at the transformation through C++ Insights: this link

Printing simplification

Since our callable object gets a variadic argument list, we can use this information and make the code simpler.

Thanks PiotrNycz for pointing that out.

The code inside the internal lambda uses index to check if we need to print the separator or not - it checks if we print the first argument. We can do this at compile-time:

#include <iostream>
#include <tuple>

template <typename TupleT>
void printTupleApply(const TupleT& tp) {    
    std::apply
        (
            [](const auto& first, const auto&... restArgs)
            {
                auto printElem = [](const auto& x) {
                    std::cout << ", " << x;
                };
                std::cout << "(" << first;
                (printElem(restArgs), ...);
            }, tp
        );
    std::cout << ")";
}

int main() {
    std::tuple tp { 10, 20, 3.14, 42, "hello"};
    printTupleApply(tp);
}

Play @Compiler Explorer.

This code breaks when tuple has no elements - we could fix this by checking its size in if constexpr, but let’s skip it for now.

Would you like to see more?
If you want to see a similar code that works with C++20's std::format, you can see my article: How to format pairs and tuples with std::format (~1450 words) which is available for C++ Stories Premium/Patreon members. See all Premium benefits here.

Making it more generic

So far we focused on printing tuple elements. So we had a “fixed” function that was called for each argument. To go further with our ideas, let’s try to implement a function that takes a generic callable object. For example:

std::tuple tp { 10, 20, 30.0 };
printTuple(tp);
for_each_tuple(tp, [](auto&& x){
    x*=2;
});
printTuple(tp);

Let’s start with the approach with index sequence:

template <typename TupleT, typename Fn, std::size_t... Is>
void for_each_tuple_impl(TupleT&& tp, Fn&& fn, std::index_sequence<Is...>) {
    (fn(std::get<Is>(std::forward<TupleT>(tp))), ...);
}

template <typename TupleT, typename Fn, 
       std::size_t TupSize = std::tuple_size_v<std::remove_cvref_t<TupleT>>>
void for_each_tuple(TupleT&& tp, Fn&& fn) {
    for_each_tuple_impl(std::forward<TupleT>(tp), std::forward<Fn>(fn), 
                        std::make_index_sequence<TupSize>{});
}

What happens here?

First, the code uses universal references (forwarding references) to pass tuple objects. This is needed to support all kinds of use cases - especially if the caller wants to modify the values inside the tuple. That’s why we need to use std::forward in all places.

But why did I use remove_cvref_t?

On std::decay and remove ref

As you can see in my code I used:

std::size_t TupSize = std::tuple_size_v<std::remove_cvref_t<TupleT>>

This is a new helper type from the C++20 trait that makes sure we get a “real” type from the type we get through universal reference.

Before C++20, you can often find std::decay used or std::remove_reference.

Here’s a good summary from a question about tuple iteration link to Stackoverflow:

As T&& is a forwarding reference, T will be tuple<...>& or tuple<...> const& when an lvalue is passed in; but std::tuple_size is only specialized for tuple<...>, so we must strip off the reference and possible const. Prior to C++20’s addition of std::remove_cvref_t, using decay_t was the easy (if overkill) solution.

Generic `std::apply` version

We discussed an implementation with index sequence; we can also try the same with std::apply. Can it yield simpler code?

Here’s my try:

template <typename TupleT, typename Fn>
void for_each_tuple2(TupleT&& tp, Fn&& fn) {
    std::apply
    (
        [&fn](auto&& ...args)
        {
            (fn(args), ...);
        }, std::forward<TupleT>(tp)
    );
}

Look closer, I forgot to use std::forward when calling fn!

We can solve this by using template lambdas available in C++20:

template <typename TupleT, typename Fn>
void for_each_tuple2(TupleT&& tp, Fn&& fn) {
    std::apply
    (
        [&fn]<typename ...T>(T&& ...args)
        {
            (fn(std::forward<T>(args)), ...);
        }, std::forward<TupleT>(tp)
    );
}

Play @Compiler Explorer

Additionally, if you want to stick to C++17, you can apply decltype on the arguments:

template <typename TupleT, typename Fn>
void for_each_tuple2(TupleT&& tp, Fn&& fn) {
    std::apply
    (
        [&fn](auto&& ...args)
        {
            (fn(std::forward<decltype(args)>(args)), ...);
        }, std::forward<TupleT>(tp)
    );
}

Play with code @Compiler Explorer.

Return value

So far, our for_each_tuple implementations focused on side effects: printing values, doubling them in place, etc. But what if you want to transform each element and collect the results into a new tuple, without modifying the original?

For example, starting with:

std::tuple tp { 10, 20, 30.0 };

we’d like to run a function on each element and get a new tuple:

auto res = transform_tuple(tp, [](const auto& x){
    return x * 2;
});

so that res holds:

(20, 40, 60.0)

How to implement this?

This is very similar to our earlier for_each_tuple2, but instead of discarding the results we return them packed into another tuple. Here’s the code:

template <typename TupleT, typename Fn>
[[nodiscard]] auto transform_tuple(TupleT&& tp, Fn&& fn) {
    return std::apply
    (
        [&fn]<typename ...T>(T&& ...args) {
            return std::make_tuple(fn(std::forward<T>(args))...);
        }, 
        std::forward<TupleT>(tp)
    );
}

Key points:

We use std::apply to expand the tuple into a parameter pack.
Inside the lambda, we forward each argument and apply fn.
Finally, we pack the results into a new std::tuple via std::make_tuple.

The [[nodiscard]] attribute is optional, but it helps warn you if you call transform_tuple and forget to use the result.

Demo

std::tuple tp { 10, 20, 30.0 };

auto res = transform_tuple(tp, [](const auto& x) {
    return x * 2;
});

for_each_tuple2(res, [](const auto& x) {
    std::cout << x << '\n';
});

Output:

20
40
60

See it live on Compiler Explorer.

Summary

It was a cool story, and I hope you learned a bit about templates.

The background task was to print tuples elements and have a way to transform them. During the process, we went through variadic templates, index sequence, template argument deduction rules and tricks, std::apply, and removing references.

I’m happy to discuss changes and improvements. Let me know in the comments below the article about your ideas.

See the part one here: C++ Templates: How to Iterate through std::tuple: the Basics - C++ Stories.
And the third part: C++ Templates: How to Iterate through std::tuple: C++26 Packs and Expansion Statements - C++ Stories

References:

Effective Modern C++ by Scott Meyers
C++ Templates: The Complete Guide (2nd Edition) by David Vandevoorde, Nicolai M. Josuttis, Douglas Gregor

Structured bindings in C++17, 8 years later

Sun, 31 Aug 2025 00:00:00 +0000

Structured binding is a C++17 feature that allows you to bind multiple variables to the elements of a structured object, such as a tuple or struct. This can make your code more concise and easier to read, especially when working with complex data structures. On this blog, we already covered this functionality, but we’ll talk about some good C++26 additions and real code use cases.

Iterating through maps

An excellent demonstration of structured bindings is an iteration through a map object.

If you have a std::map of elements, you might know that internally, they are stored as pairs of <const Key, ValueType>.

Now, when you iterate through elements, you can do:

for (const auto& elem : myMap) { ... }

You need to write elem.first and elem.second to refer to the key and the value.

std::map<KeyType, ValueType> myMap = getMap();
// C++14:
for (const auto& elem : myMap) {  
    // elem.first - is the key
    // elem.second - is the value
}

One of the coolest use cases of structured binding is that we can use it inside a range-based for loop:

// C++17:
for (const auto& [key,val] : myMap) {  
    // use key/value directly
}

In the above example, we bind to a pair of [key, val] so we can use those names in the loop. Before C++17, you had to operate on an iterator from the map - which returns a pair <first, second>. Using the real names key/value is more expressive.

The above technique can be used in the following example:

#include <map>
#include <iostream>

int main() {
    const std::map<std::string, int> mapCityPopulation {
        { "Beijing", 21'707'000 },
        { "London", 8'787'892 },
        { "New York", 8'622'698 }
    };
    
    for (const auto&[city, population] : mapCityPopulation)
        std::cout << city << ": " << population << '\n';
}

Run @Compiler Explorer

In the loop body, you can safely use the city and population variables.

The Syntax

The basic syntax for structured bindings is as follows:

auto [a, b, c, ...] = expression;
auto [a, b, c, ...] { expression };
auto [a, b, c, ...] ( expression );

The compiler introduces all identifiers from the a, b, c, ... list as names in the surrounding scope and binds them to sub-objects or elements of the object denoted by the expression.

Behind the scenes, the compiler might generate the following pseudo code:

auto tempTuple = expression;
using a = tempTuple.first;
using b = tempTuple.second;
using c = tempTuple.third;

Conceptually, the expression is copied into a tuple-like object (tempTuple) with member variables that are exposed through a, b and c. However, the variables a, b, and c are not references; they are aliases (or bindings) to the generated object member variables. The temporary object has a unique name assigned by the compiler.

For example:

std::pair a(0, 1.0f);
auto [x, y] = a;

x binds to int stored in the generated object that is a copy of a. And similarly, y binds to float.

I don’t want to repeat myself, so please see this blog post - Structured bindings in C++17, 5 years later - C++ Stories, or my book - C++17 in Detail for details about bindings.

Real code

Let’s have a look at some code in the real projects. I found the following use cases:

From chars handling

onnxruntime/migraphx_execution_provider_utils.h

inline int ToInteger(const std::string_view sv) {
  int result = 0;
  if (auto [_, ec] = std::from_chars(sv.data(), sv.data() + sv.length(), result); ec == std::errc()) {
    return result;
  }
  ORT_THROW("invalid input for conversion to integer");
}

Here, std::from_chars returns a std::from_chars_result with two fields: ptr and ec. The code only cares about the error code, so structured bindings make it easy to ignore the pointer (_) and directly test ec.

Read more about from_chars here: C++ String Conversion: Exploring std::from_chars in C++17 to C++26 - C++ Stories

Unmap

onnxruntime/env.cc

static void UnmapFile(void* addr, size_t len) noexcept {
  int ret = munmap(addr, len);
  if (ret != 0) {
    auto [err_no, err_msg] = GetErrnoInfo();
    LOGS_DEFAULT(ERROR) << "munmap failed. error code: " << err_no << " error msg: " << err_msg;
  }
}

GetErrnoInfo() returns both an error number and a message. Using structured bindings makes the call-site clean and self-documenting, with no extra boilerplate for std::pair access (.first, .second).

Softmax

onnxruntime/softmax

 auto [scale_tensor, zero_tensor] = GetQuantizationZeroPointAndScale(graph, node_unit.Outputs()[0]);
    Initializer q_scale(graph.GetGraph(), *scale_tensor, node_unit.ModelPath());
    if (fabs(q_scale.DataAsSpan<float>()[0] - 1.0f / 256.0f) > 0.0001f) {
      break;

Here, a helper function returns both the quantization scale and zero-point tensors. Structured bindings allow unpacking them directly into named variables, instead of handling them as std::pair elements or temporary structs.

Iteration

onnxruntime/prepacked_weights_container.h

 size_t GetNumberOfKeyedBlobsForWriting() const noexcept {
    size_t result = 0;
    for (const auto& [_, keys] : weight_prepacks_for_saving_) {
      result += keys.size();
    }
    return result;

The standard use case for structured bindings and iteration over a map.

Hash computation

openvinotoolkit/compilation_context.cpp

    for (const auto& [name, option] : compileOptions) {
        seed = hash_combine(seed, name + option.as<std::string>());
    }

Another use case for iteration over a container.

C++17/C++20 changes

During the work on C++20, there were several proposals that improved the initial support for structured bindings, most of them are added as defect reports (DR) against C++17:

Relaxing the structured bindings customization point finding rules - P0961

The proposal relaxes the rule so that only member template get<I> functions are considered, while plain get() members no longer interfere, making structured bindings more flexible and consistent.

Allow structured bindings to accessible members - P0969

P0969 fixed an odd inconsistency in C++17: structured bindings could only bind to public data members (or members of public bases), even if the member was actually accessible in the current scope. This meant that code which could access a private member via friendship, inheritance, or even within the class itself, failed when using structured bindings. C++20 relaxed this rule so that structured bindings follow the normal accessibility rules of C++: if you can name the member in that scope, you can also bind to it. This made structured bindings consistent with all other forms of member access.

Extending structured bindings to be more like variable declarations - P1091

P1091R3 made structured bindings feel more like “real” variable declarations in C++20. In C++17 they were restricted and somewhat “magical”: you couldn’t mark them static, thread_local, or capture them in lambdas. This proposal removed those inconsistencies — now structured bindings can carry storage-class specifiers, be captured by value in lambdas, and behave consistently with ordinary variables. It brought them closer to being true first-class citizens in the language.

#include <tuple>
#include <iostream>

int main() {
    static auto [x, y] = std::tuple{10, 20};

    auto lambda = [=] { 
        std::cout << "x = " << x << ", y = " << y << "\n";
    };

    lambda();
}

See @Compiler Explorer

Reference capture of structured bindings - P1381

The previous proposal allowed lambdas to capture structyred bindings by value, and now the missing part is to capture by reference:

struct Foo { int x; int y; };

int main() {
    Foo foo{10, 20};
    auto [a, b] = foo;

    auto by_val = [=] { return a + b; };
    auto by_ref = [&] { return a + b; }; 

    return by_ref(); // 30
}

See @Compiler Explorer

C++26 Updates

In C++26, we have at least four new features that went into the Standard:

Attributes for structured bindings - P0609R3

There are not many sensible attributes that can be applied to variables, but we can now try maybe_unused attribute on a part of structured binding:

std::pair xy { 42.3, 100.1 };
auto [x, y [[maybe_unused]]] = xy;
std::print("{}", x);

see @Compiler Explorer

The interesting thing is that the attribute is applied after the name of the binding.

Structured binding declaration as a condition - P0963R3

This paper proposes to allow structured binding declarations with initializers appearing in place of the conditions in if, while, for, and switch statements.

#include <print>

struct Point {
    int x { 0 };
    int y { 0 };
    explicit operator bool() const noexcept { return x > 0; }
};
Point createPoint(int a) { return Point { 10*a, 10*a }; }

int main() {
    if (auto [x, y] = createPoint(100))
        std::print("point is true");
}

Run @Compiler Explorer

The crucial part here is that the conversion to bool happens on the object that we bind to.

Note that the following code:

std::pair xy { 42.3, 100.1 };
if (auto [x, y] = xy; x > 40.0)
    std::print("x is larger than 40");

Worked before C++26.

Thanks to hanickadot for the comment and explanations!

Structured bindings can introduce a pack - P1061R10

This is a really powerful feature!

Before C++26, structured bindings required you to write all identifiers explicitly (auto [x, y, z] = ...;). Converting a tuple to a pack is awkward—you need std::apply or index_sequence. But now structured bindings can introduce a pack using ...identifier. This gives you a parameter-pack-like expansion directly from a tuple/array/aggregate inside a structured binding.

In short:

auto [head, ...rest] = std::tuple{1,2,3,4};
// head = 1, rest... expands to (2,3,4)

Here’s a super cool (and mind-boggling) example:

// Generic dot product for any tuple-like / array / aggregate supported by structured bindings.
template <class P, class Q>
constexpr auto dot_product(const P& p, const Q& q) {
    // Bind all elements of each input into packs.
    const auto& [...ps] = p;
    const auto& [...qs] = q;

    // (Optional) sanity check: both must have the same size
    static_assert(sizeof...(ps) == sizeof...(qs), "Mismatched sizes");

    // Elementwise multiply, then fold with +
    return ((ps * qs) + ...);
}

constexpr std::array<int, 3> a{1, 2, 3};
constexpr std::array<int, 3> b{4, 5, 6};
static_assert(dot_product(a, b) == 32);

Here’s the link to Compiler Explorer

constexpr structured bindings and references to constexpr variables - P2686R5

int main() {
    constexpr auto [a, b] = std::tuple{2, 3};
    static_assert(a * b == 6);
}

Unfortunately, as of Sept 2025, no compiler supports this.

Summary

In this text, we explored structured bindings and how they evolved over the recent C++ Standards. This feature was a big “magic” and used some compiler tricks to work, which also created some restrictions. Now (especially in C++26), those bindings can be used almost like standard variables.

Back to you

Do you use structured bindings?
What are the most common use cases for you?

Share your feedback in the comments below.

How to Avoid Thread-Safety Cost for Functions' static Variables

Sat, 23 Aug 2025 00:00:00 +0000

In this blog post, we’ll look at static variables defined in a function scope. We’ll see how they are implemented and how to use them. What’s more, we’ll discuss several cases where we can avoid extra thread-safety cost.

Let’s start.

Introduction

As you may know, C++ offers a way to define static variables in a function/block scope:

void foo() { 
    static int counter = 0;
    ++counter;
}

Above, the counter variable will be initialized and created when foo() is invoked for the first time. In other words, a static local variable is initialized lazily. The counter is kept “outside” the function’s stack space. This allows, for example, to keep the state, but limit the visibility of the global object.

Here’s the full example:

#include <iostream>

int foo() { 
    static int counter = 0;
    return ++counter;
}

int main() {
    foo();
    foo();
    foo();
    auto finalCounter = foo();
    std::cout << finalCounter;
}

Run @Compiler Explorer

If you run the program, you’ll get 4 as the output.

Static local variables, since C++11, are guaranteed to be initialized in a thread-safe way. The object will be initialized only once if multiple threads enter a function with such a variable. Have a look below:

#include <iostream>
#include <thread>

struct Value {
    Value(int x) : v(x) { std::cout << "Value(" << v << ")\n"; }
    ~Value() noexcept { std::cout << "~Value(" << v << ")\n"; }

    int v { 0 };
};

void foo() {
    static Value x { 10 };
}

int main() {
    std::jthread worker1 { foo };
    std::jthread worker2 { foo };
    std::jthread worker3 { foo };
}

Run @Compiler Explorer

The example creates three threads that call the foo() simple function.

However, on GCC, you can also try compiling with the following flags:

-std=c++20 -lpthread -fno-threadsafe-statics

And then the output might be as follows:

Value(Value(1010)
)
Value(10)
~Value(10)
~Value(10)
~Value(10)

Three static objects are created now!

On Windows, MSVC, you can play with /Zc:threadSafeInit and disable this behaviour. See /Zc:threadSafeInit (Thread-safe Local Static Initialization) | Microsoft Learn

Thread safety may be beneficial in most cases, for example, when you want to implement a singleton (Meyers Singleton, it defines the entity as a static variable in a function…).

How much does this cost?

To understand the cost of the static local variables, let’s consider this popular SQ question:

c++ - Is there a penalty for using static variables in C++11 - Stack Overflow

In short: not much :)

The answer contains the following benchmark setup:

Two versions tested:
1. Local static inside a function
2. Global static at namespace scope
Both return a const std::vector<int>& with {1, 2, 3}.
Measured over 500 million iterations, summing vector elements.
Timed using std::chrono with barriers to avoid reordering.

Results (first benchmark, indirect function calls): (warning the code is from 2014!)

Clang: local = 4618 ms, global = 4392 ms → local slower by ~0.45 ns per call.
GCC: local = 4181 ms, global = 4418 ms → local faster by ~0.47 ns per call.
Conclusion: variance is tiny, compiler-dependent.

Second benchmark (function objects for better inlining):

GCC: local = 3803 ms, global = 2323 ms → global faster by ~2.96 ns per call.
Clang: local = 4183 ms, global = 3253 ms → global faster by ~1.86 ns per call.
This matches the intuition that a global avoids the guard check entirely, while a local static still needs a fast-path check.

Takeaway:

Function-local statics in C++11+ are initialized exactly once, thread-safely.
After initialization, calls incur only a very small, predictable guard check (~1–3 ns in tight loops).
In most real-world code, the difference is negligible; choose the style that best fits your design.
If absolute lowest per-call overhead is needed in a very hot path, a namespace-scope constexpr/constinit global removes even that guard.

How to avoid the cost

We’re equipped with the basic knowledge, and now let’s consider how to limit the cost of thread-safety. Consider the following super-simple example:

#include <vector>
#include <algorithm> // std::find
#include <print>

bool is_blocked_id(int id) {
    static const std::vector<int> blocked_ids{
        101, 202, 303, 404, 505 // possibly a longer list...
    };

    return std::find(blocked_ids.begin(), blocked_ids.end(), id) != blocked_ids.end();
}

int main() {
    for (int id : {101, 150, 202, 999}) {
        std::println("{} is blocked: {}", id, is_blocked_id(id));
    }
}

Run @Compiler Explorer

The main function iterates through some ids and checks them through is_blocked_id function. The function then uses a static const vector to store “suspicious” numbers.

And here’s the generated code:

"is_blocked_id(int)":
        push    rbp
        mov     rbp, rsp
        push    r14
        push    r13
        push    r12
        push    rbx
        sub     rsp, 64
        mov     DWORD PTR [rbp-84], edi
        movzx   eax, BYTE PTR "guard variable for is_blocked_id(int)::blocked_ids"[rip]
        test    al, al
        sete    al
        test    al, al
        je      .L216
        mov     edi, OFFSET FLAT:"guard variable for is_blocked_id(int)::blocked_ids"
        call    "__cxa_guard_acquire"
        ...

As you can see, we have some extra calls to __cxa_guard_acquire and thus each time you call the function, we have some overhead.

Some issues

There are a few cases to consider.

While in Mayer’s singleton thread safety was essential, here we have a constant object, so maybe we can do something with this knowledge?
Since we use std::vector there’s extra memory allocation going on. Do we need it?

What to improve: In short, we need to move our dynamic initialization to constant initialization. Since our data is const, there’s no need to pay an extra price at runtime. The compiler can initialize all data at compile time and write it to the binary.

How to achieve this?

One way is to push the vector outside the function scope, that way we’ll lose the “locality” but the compiler will ensure it’s properly initialized before even the main function starts:

#include <vector>
#include <algorithm> // std::find
#include <print>

static const std::vector<int> blocked_ids{
        101, 202, 303, 404, 505
    };

bool is_blocked_id(int id) {
    return std::find(blocked_ids.begin(), blocked_ids.end(), id) != blocked_ids.end();
}

int main() {
    for (int id : {101, 150, 202, 999}) {
        std::println("{} is blocked: {}", id, is_blocked_id(id));
    }
}

Run @Compiler Explorer

Now the generated code is a bit better:

"is_blocked_id(int)":
        push    rbp
        mov     rbp, rsp
        push    rbx
        sub     rsp, 56
        mov     DWORD PTR [rbp-52], edi
        mov     edi, OFFSET FLAT:"blocked_ids"
        call    "std::vector<int, std::allocator<int> >::end() const"
        mov     QWORD PTR [rbp-48], rax
        mov     edi, OFFSET FLAT:"blocked_ids"
        call    "std::vector<int, std::allocator<int> >::end() const"
        mov     rbx, rax
        mov     edi, OFFSET FLAT:"blocked_ids"
        call    "std::vector<int, std::allocator<int> >::begin() const"
        mov     rcx, rax
        lea     rax, [rbp-52]
        ...

As you can see, there are no guards needed, just the vector handling. The runtime memory allocation is still there, but occurs before the main function enters.

But we can do better!

Better than a vector

As we discussed, there’s no need for dynamic allocation here. So, how about using just the std::array?

static const std::array blocked_ids{
        101, 202, 303, 404, 505
    };

bool is_blocked_id(int id) {
    return std::find(blocked_ids.begin(), blocked_ids.end(), id) != blocked_ids.end();
}

Run @Compiler Explorer

The generated code:

"is_blocked_id(int)":
        push    rbp
        mov     rbp, rsp
        push    rbx
        sub     rsp, 24
        mov     DWORD PTR [rbp-20], edi
        mov     edi, OFFSET FLAT:"blocked_ids"
        call    "std::array<int, 5ul>::end() const"
        mov     rbx, rax
        mov     edi, OFFSET FLAT:"blocked_ids"
        call    "std::array<int, 5ul>::begin() const"
        mov     rcx, rax
        lea     rax, [rbp-20]
        mov     rdx, rax
        mov     rsi, rbx
        mov     rdi, rcx
        call    "int const* std::find<int const*, int>(int const*, int const*, int const&)"
        mov     rbx, rax

Even better approach

If you still want to keep numbers in a function scope, there’s still a chance to do it, just ensure the collection is initialized at compile time:

bool is_blocked_id(int id) {
    static const std::array blocked_ids{ // or constexpr, or constinit!
        101, 202, 303, 404, 505
    };

    return std::find(blocked_ids.begin(), blocked_ids.end(), id) != blocked_ids.end();
}

The static const should be good enough for a compiler to use compile-time initialization, assuming the container is implemented in a constexpr way. But to be sure, use constinit or constexpr:

bool is_blocked_id(int id) {
    static constinit std::array blocked_ids{
        101, 202, 303, 404, 505
    };

    return std::find(blocked_ids.begin(), blocked_ids.end(), id) != blocked_ids.end();
}

Here’s the article about those keywords if you want to know more: const vs constexpr vs consteval vs constinit in C++20 - C++ Stories.

More complex types

In a real-life application, you will work not only with arrays of simple integers. Maps of strings, maps of custom data types, and much more. In that case, moving things to compile-time might not be straightforward. Yet, there are steps you can try.

Here are some examples, just to get started:

struct Point {
    int x {0};
    int y {0};
    bool operator==(const Point& pt) const {
        return pt.x == x && pt.y == y;
    }
};

bool is_blocked_point(int x, int y) {
    static constinit std::array<Point,6> blocked_pts{
        Point{101, 101}, Point{202, 202}, Point{303, 303}, 
        Point{404, 404}, Point{505, 505}, Point{606, 606}
    };

    return std::find(blocked_pts.begin(), blocked_pts.end(), {x, y}) != blocked_pts.end();
}

Run @Compiler Explorer

The point class is constexpr implicitly, and the compiler can use it at compile-time.

Or wth strings:

bool is_blocked_name(std::string_view name) {
    static constinit std::array<std::string_view, 6> blocked_names{
        "alice", "bob", "charlie", "dora", "eve", "mallory"
    };
    return std::find(blocked_names.begin(), blocked_names.end(), name) != blocked_names.end();
}

Run @Compiler Explorer

Summary

In the article, we went from a simple example that illustrates how a static variable inside a function works, to advanced scenarios where we want to limit the cost of thread-safety.

Always measure, measure, measure. In most cases, the performance hit added for a local static variable will be super small, but in your hot path, it’s best to check. Still, if you can move something to the compile time, and especially avoid dynamic runtime memory allocations, then it’s a clear win.

Back to you

Do you use static variables on a function scope?
Have you got any issues or bugs with such variables?

How to Split Ranges in C++23

Fri, 16 May 2025 00:00:00 +0000

In this blog post, we’ll continue looking at ranges and this time explore ways to split them into sub-ranges. So we’ll take a look at views::split, views::chunk, and views::chunk_by.

We’ll walk through two examples for each adaptor: one simple and one slightly more advanced, to highlight their practical uses.

Let’s go.

Splitting Ranges with `views::split`, C++20

If you want to split a range using some “delimeter,” then views::split (or ranges::split_view) will do the job.

template< ranges::forward_range V, ranges::forward_range Pattern >
// .. requires...
class split_vie  : public ranges::view_interface<split_view<V, Pattern>>

Both of the ranges V and Pattern have to be forward_range.

Notice that you need to use a pattern; it cannot be a condition.

Example 1: Splitting a Sentence into Words

#include <print>
#include <ranges>
#include <string_view>

int main() {
    using namespace std::string_view_literals;

    constexpr auto text = "C++ is powerful and elegant"sv;
    
    for (auto part : std::views::split(text, ' '))
        std::print("'{}' ", std::string_view(part));
}

Play @Compiler Explorer

The output:

'C++' 'is' 'powerful' 'and' 'elegant'

A classic example that splits a sentence into words. We convert each subrange to std::string_view for easy printing.

We can also extend this and use not just one character:

constexpr auto text = "C++breakisbreakpowerfulbreakandbreakelegant"sv;

for (auto part : std::views::split(text, "break"sv))
	std::print("'{}' ", std::string_view(part));

Example 2: Splitting Not Just Text

split can be applied to any forward range, so not just text:

#include <print>
#include <ranges>
#include <vector>
#include <utility>

int main() {
    using Point = std::pair<int, int>;

    std::vector<Point> path = {
        {0, 0}, {1, 1}, {-1, -1}, // marker: {-1, -1}
        {2, 2}, {3, 3}, {-1, -1},
        {4, 4}, {5, 5}
    };

    for (auto segment : std::views::split(path, Point{-1, -1}))
        std::print("Segment: {}\n", segment);
}

See @Compiler Explorer

And the output:

Segment: [(0, 0), (1, 1)]
Segment: [(2, 2), (3, 3)]
Segment: [(4, 4), (5, 5)]

In the example, we run through the point list and break at markers that are (-1, -1). This code also leveraged std::format support for ranges (C++23), so we don’t have to print individual points.

What About `lazy_split`?

You might also encounter views::lazy_split, introduced in C++20.

This view is specialized for input-only ranges, such as reading from streams or generators. It avoids buffering or requiring multiple passes, but its subranges aren’t contiguous and don’t expose .data() or .size().

Unless you’re processing streams in a single pass, prefer views::split for simplicity.

Here’s a question from Stack Overflow that I recently asked to clarify this view: c++ - What is the use of std::ranges::views::lazy_split when we have std::ranges::views::split? - Stack Overflow

Grouping with `views::chunk`, C++23

When you need to process data in fixed-size batches, chunk is the perfect tool.

Example 1: Fixed-Size Batches

#include <print>
#include <ranges>
#include <vector>

int main() {
    std::vector<int> data{1, 2, 3, 4, 5, 6, 7, 8};

    for (auto chunk : data | std::views::chunk(3))
        std::print("{}\n", chunk);
}

See @Compiler Explorer

And the output:

[ 1 2 3 ]
[ 4 5 6 ]
[ 7 8 ]

views::chunk splits the sequence into groups of three elements. If the number of elements isn’t divisible by 3, the last chunk will contain fewer elements.

Example 2: Processing Network Packets in Chunks

chunk can work not just with regular containers, but with just input ranges:

#include <print>
#include <ranges>
#include <sstream>
#include <vector>

int main() {
    std::istringstream stream{"AB CD EF 12 34 56 78 95 FF"};

    auto bytes = std::ranges::istream_view<std::string>(stream);

    for (auto packet : bytes | std::views::chunk(4))
        std::print("Packet: {}\n", packet);
}

See @Compiler Explorer

Output:

Packet: ["AB", "CD", "EF", "12"]
Packet: ["34", "56", "78", "95"]
Packet: ["FF"]

This example simulates processing a byte stream in fixed 2-byte packets.

Dynamic Grouping with `views::chunk_by`, C++23

When the group size isn’t fixed but defined by a condition, chunk_by comes into play. This time, the range has to model forward range at least.

Example 1: Grouping by Parity (Even/Odd)

#include <print>
#include <ranges>
#include <vector>

int main() {
    std::vector<int> values{1, 3, 5, 2, 4, 6, 7, 9, 8};

    for (auto group : values | std::views::chunk_by([](int a, int b) {
        return (a % 2) == (b % 2); // Same parity
    })) {
        std::print("size {}, {}\n", group.size(), group);
    }
}

Run @Compiler Explorer

Output:

size 3, [1, 3, 5]
size 3, [2, 4, 6]
size 2, [7, 9]
size 1, [8]

This code dynamically groups consecutive numbers based on their parity. Each group contains either only odd or only even numbers.

Example 2: Extracting Sentences from Text

We can use chunk_by and split at places where a sentence ends.

First try:

#include <print>
#include <ranges>
#include <string_view>

int main() {
    using namespace std::string_view_literals;

    constexpr auto text = "C++ is powerful. Ranges are elegant. This is fun!"sv;

    for (auto sentence : text | std::views::chunk_by([](char a, char b) {
        // Group until a dot is found; start a new group after '.'
        return a != '.' && b != '.';
    })) {
        std::print("Sentence: {}\n", sentence);
    }
}

And we get:

Sentence: ['C', '+', '+', ' ', 'i', 's', ' ', ...]
Sentence: ['.']
Sentence: [' ', 'R', 'a', 'n', 'g', 'e', 's', ...]
Sentence: ['.']
Sentence: [' ', 'T', 'h', 'i', 's', ' ', 'i', ...]

Probably not the best… but we can try cleaning the result:

int main() {
    using namespace std::string_view_literals;

    constexpr auto text = "C++ is powerful. Ranges are elegant. This is fun!"sv;

    for (auto sentence : text | std::views::chunk_by([](char a, char b) {
        // Group until a dot is found; start a new group after '.'
        return a != '.' && b != '.';
    })) {
        // Remove leading spaces if any, and skip dots-only groups
        auto view = std::string_view(&*sentence.begin(), std::ranges::distance(sentence));
        view.remove_prefix(std::min(view.find_first_not_of(' '), view.size()));

        if (!view.empty() && view != ".")
            std::print("Sentence: [{}]\n", view);
    }
}

And now we get:

Sentence: [C++ is powerful]
Sentence: [Ranges are elegant]
Sentence: [This is fun!]

Experiment @Compiler Explorer

Summary

Feature	`split`	`chunk`	`chunk_by`
C++ Standard	C++20	C++23	C++23
Fixed-size groups	❌	✅	❌
Custom grouping	❌	❌	✅
Text splitting	✅	❌	❌

In this text, we explored various ways to split ranges using split, chunk or chunk_by, we also touched a little bit about lazy_split.

Back to you

Which of these adaptors have you already tried?
Which one surprised you the most?

Let me know your thoughts in the comments!

Views as Data Members for Custom Iterators

Tue, 22 Apr 2025 00:00:00 +0000

In this blog post, we’ll write an iterator that works with a vector of vectors. We’ll explore a “manual” version as well as leverage C++20 ranges/views to do the hard work.

The problem statement

What’s the use case? See this popular interview question (found in DailyCodingProblem: https://www.dailycodingproblem.com/)

Implement a 2D iterator class. It will be initialized with an array of arrays and should implement the following methods: next() : returns the next element in the array of arrays. If there are no more elements, raise an exception. has_next() : returns whether or not the iterator still has elements left. For example, given the input [[1, 2], [3], [], [4, 5, 6]] , calling next() repeatedly should output 1, 2, 3, 4, 5, 6 .

Transforming the problem into C++, we can write the following client code:

int main() {
    std::vector<std::vector<int>> input = {{1, 2}, {3}, {}, {4, 5, 6}};
    TwoDIterator it(input);

    while (it.has_next())
        std::print("{} ", it.next());
}

The case is to implement TwoDIterator.

Initial Version

The main idea for the iterator is to track the “inner” position as well as the “outer”. When the inner position reaches the end of some inner vector, we need to check if we have more vectors to traverse.

Here’s some basic code:

#include <vector>
#include <stdexcept>
#include <print>

class TwoDIterator {
public:
    using Vec2D = std::vector<std::vector<int>>;

    TwoDIterator(const Vec2D& data) 
    : data_(data), outer_(0), inner_(0) {
        advance_to_next_valid();
    }

    bool has_next() const {
        return outer_ < data_.size();
    }

    int next() {
        if (!has_next()) throw std::out_of_range("No more elements");

        int val = data_[outer_][inner_];
        ++inner_;
        advance_to_next_valid();
        return val;
    }

private:
    void advance_to_next_valid() {
        while (outer_ < data_.size() && inner_ >= data_[outer_].size()) {
            ++outer_;
            inner_ = 0;
        }
    }

    const Vec2D& data_;
    size_t outer_, inner_;
};

int main() {
    std::vector<std::vector<int>> input = {{1, 2}, {3}, {}, {4, 5, 6}};
    TwoDIterator it(input);

    while (it.has_next())
        std::print("{} ", it.next());
}

Play @Compiler Explorer

The key part of this solution is the advance_to_next_valid() function. It checks if we can move further and if there are any more vectors in the outer range.

Applying Ranges and Views from C++20

While the solution works fine, in C++20, we can leverage ranges and views.

From my previous article (How to join or concat ranges, C++26 - C++ Stories) we know that if we have nested ranges, we can use join view to do the flattening:

#include <iostream>
#include <ranges>
#include <vector>

int main() {
    std::vector<std::vector<int>> nested{{1, 2}, {3}, {}, {4, 5, 6}};
    
    auto flat = nested | std::views::join;

    for (int i : flat)
        std::cout << i << ' ';
}

Run @Compiler Explorer

So the question is how to leverage it in our class? (We want to keep the interface of has_next() and next() as this is our requirement).

Adding C++20 Views to our class

Conceptually we want to add a join view iterators as a data member to the class TwoDIterator. However, the main issue is that it’s relatively complex to specify the correct type of such data members we cannot write:

class VecVecIt {
    auto flat = nested | std::views::join; // ??
    auto beg_ = flat.begin();  // ??
    auto end_ = flat.end(); // ??
public:
    // ...
};

Let’s try step by step:

We cannot use auto for auto-type deduction for data members; we have to specify the exact type. Later we can initialize those data members in constructors.
For simplification we can write using Vec2D = std::vector<std::vector<int>>;
std::ranges::join_view<Vec2D> doesn’t work as std::vector isn’t a view, and join requires views.

Here’s the Clang compiler error:

<source>:33:35: error: constraints not satisfied for class template 'join_view' [with _Vp = std::vector<std::vector<int>>]
   33 |     using JoinView = std::ranges::join_view<Vec2D>;  
      |                                   ^~~~~~~~~~~~~~~~
/opt/compiler-explorer/gcc-snapshot/lib/gcc/x86_64-linux-gnu/16.0.0/../../../../include/c++/16.0.0/ranges:2888:14: note: because 'std::vector<std::vector<int>>' does not satisfy 'view'
 2888 |     requires view<_Vp> && input_range<range_reference_t<_Vp>>

To “convert” a vector into a view, we can write:

std::ranges::ref_view<const Vec2D>

Now we can combine the previous steps into:

using Vec2D = std::vector<std::vector<int>>;  
using JoinView = std::ranges::join_view<std::ranges::ref_view<const Vec2D>>;  
using Iterator = std::ranges::iterator_t<JoinView>;

Here’s the final class:

#include <vector>
#include <stdexcept>
#include <print>
#include <ranges>

class TwoDIteratorJoin {  
public:  
    using Vec2D = std::vector<std::vector<int>>;  
    using JoinView = std::ranges::join_view<std::ranges::ref_view<const Vec2D>>;  
    using Iterator = std::ranges::iterator_t<JoinView>;  
  
    explicit TwoDIteratorJoin(const Vec2D& data)  
        : flattened_(data | std::views::join),  
          iter_(flattened_.begin()),  
          end_(flattened_.end()) {}  
  
    [[nodiscard]] bool has_next() const {  
        return iter_ != end_;  
    }  
  
    [[nodiscard]] int next() {  
        if (!has_next()) throw std::out_of_range("No more elements");  
        return *iter_++;  
    }  
  
private:  
    JoinView flattened_;  
    Iterator iter_, end_;  
};

int main() {
    const std::vector<std::vector<int>> input = {{1, 2}, {3}, {}, {4, 5, 6}};
    TwoDIteratorJoin it(input);

    while (it.has_next())
        std::print("{} ", it.next());
}

Run @Compiler Explorer

Summary

In this article, we explored two approaches to implementing a 2D iterator in C++. The first, a manual solution, required us to carefully manage both outer and inner indices to traverse a vector of vectors. While this approach is straightforward and works well, it can become verbose and error-prone as complexity grows.

With the advent of C++20, ranges and views offer a much more elegant and expressive alternative. By leveraging std::views::join, we can flatten nested containers and iterate over their elements with minimal boilerplate. The main challenge is specifying the correct types for view-based data members, but once addressed, the resulting code is concise, robust, and easy to maintain.

Back To you

Do you use Ranges and Views from C++20?
Have you ever needed to write a custom iterator?

How to join or concat ranges, C++26

Sun, 16 Mar 2025 00:00:00 +0000

Modern C++ continuously improves its range library to provide more expressive, flexible, and efficient ways to manipulate collections. Traditionally, achieving tasks like concatenation and flattening required manual loops, copying, or custom algorithms. With C++’s range adaptors, we now have an elegant and efficient way to process collections lazily without unnecessary allocations.

In this post, we will explore three powerful range adaptors introduced in different C++ standards:

std::ranges::concat_view (C++26)
std::ranges::join_view (C++20)
std::ranges::join_with_view (C++23)

Let’s break down their differences, use cases, and examples.

`std::ranges::concat_view` (C++26)

The concat_view allows you to concatenate multiple independent ranges into a single sequence. Unlike join_view, it does not require a range of ranges—it simply merges the given ranges sequentially while preserving their structure.

In short:

Takes multiple independent ranges as arguments.
Supports random access if all underlying ranges support it.
Allows modification if underlying ranges are writable.
Lazy evaluation: No additional memory allocations.

See the example:

#include <print>
#include <ranges>
#include <vector>
#include <array>

int main() {
    std::vector<std::string> v1{"world", "hi"};
    std::vector<std::string> v2 { "abc", "xyz" };
    std::string arr[]{"one", "two", "three"};
    
    auto v1_rev = v1 | std::views::reverse;
    auto concat = std::views::concat(v1_rev, v2, arr);

    concat[0] = "hello"; // access and write

    for (auto& elem : concat)
        std::print("{} ", elem);
}

See at @Compiler Explorer

hello world abc xyz one two three

The example below shows how to concatenate three ranges, v1 is reversed and then combined with v2 and arr. Notice that we can also access the value at position 0 and update it.

And a bit more complex example:

#include <print>
#include <ranges>
#include <vector>
#include <list>
#include <string>

struct Transaction {
    std::string type;
    double amount;
};

int main() {
    std::vector<Transaction> bank_transactions{
       {"Deposit", 100.0}, 
       {"Withdraw", 50.0}, 
       {"Deposit", 200.0}
    };
    std::list<Transaction> credit_card_transactions{
        {"Charge", 75.0}, {"Payment", 50.0}
    };
    
    auto filtered_bank = bank_transactions 
        | std::views::filter([](const Transaction& t) {
        return t.amount >= 100.0;
    });
    
    auto filtered_credit = credit_card_transactions 
        | std::views::filter([](const Transaction& t) {
        return t.amount > 60.0;
    });
    
    auto all_transactions = std::views::concat(filtered_bank, filtered_credit);
    
    for (const auto& t : all_transactions)
        std::println("{} - {}$", t.type, t.amount);
}

Run @Compiler Explorer

`std::ranges::join_view` (C++20)

The join_view is designed for flattening a single range of ranges into a single sequence. It removes the structural boundaries between nested ranges.

Works on a single range of ranges (e.g., std::vector<std::vector<int>>).
Does not support operator[] (no random access).
Eliminates boundaries between sub-ranges.
Lazy evaluation, avoiding memory copies.

A simple example:

#include <iostream>
#include <ranges>
#include <vector>

int main() {
    std::vector<std::vector<int>> nested{{1, 2}, {3, 4, 5}, {6, 7}};
    auto joined = std::views::join(nested);
    
    for (int i : joined) 
        std::println(i);
}

Run @Compiler Explorer

1 2 3 4 5 6 7

Of course, we can have different nested ranges… and this can be handy for string processing:

#include <print>
#include <ranges>
#include <vector>
#include <string>
#include <map>

int main() {
    std::vector<std::string> words { "Hello", "World", "Coding" };

    // regular:
    std::map<char, int> freq;
    for (auto &w : words)
        for (auto &c : w)
            freq[::tolower(c)]++;

    // join:
    std::map<char, int> freq2;
    for (auto &c : words | std::views::join)
        freq2[::tolower(c)]++;

    for (auto& [key, val] : freq2)
        std::println("{} -> {}", key, val);
}

Run @Compiler Explorer

As you can see, thanks to views_join we can save one nested loop and iterate through a single range of characters.

`std::ranges::join_with_view` (C++23)

The join_with_view works like join_view, but it allows inserting a delimiter between flattened sub-ranges.

Works on a single range of ranges.
Allows specifying a delimiter (single element or a range).
Does not support random access.
Useful for formatting strings or separating collections.

See the example below:

#include <iostream>
#include <ranges>
#include <vector>
#include <string>
#include <string_view>

std::string to_uppercase(std::string_view word) {
    std::string result(word);
    for (char& c : result) 
        c = std::toupper(static_cast<unsigned char>(c));
    return result;
}

int main() {
    std::vector<std::string_view> words{
        "The", "C++", "ranges", "library"
    };

    auto words_up = words | std::views::transform(to_uppercase);
    auto joined = std::views::join_with(words_up, std::string_view(" "));

    for (auto c : joined) 
        std::cout << c;
}

See at Compiler Explorer

THE C++ RANGES LIBRARY

Comparing `concat_view`, `join_view`, and `join_with_view`

Feature	`concat_view` ✅	`join_view` ✅	`join_with_view` ✅
Works on multiple independent ranges?	✅	❌	❌
Flattens nested ranges?	❌	✅	✅
Supports separators between sub-ranges?	❌	❌	✅
Random access support?	✅ (if all inputs support it)	❌	❌

Summary

C++’s range adaptors provide efficient ways to manipulate collections without unnecessary copying. Here’s a quick summary of when to use each view:

Use concat_view when merging multiple independent ranges.
Use join_view when flattening a range of ranges.
Use join_with_view when flattening a range of ranges but needing a separator between elements.

Back to you

Do you use ranges?
What are your most useful views ald algorithms on ranges?

Share your comments below

Details of std::mdspan from C++23

Thu, 27 Feb 2025 00:00:00 +0000

In this article, we’ll see details of std::mdspan, a new view type tailored to multidimensional data. We’ll go through type declaration, creation techniques, and options to customize the internal functionality.

Type declaration

The type is declared in the following way:

template<
    class T,
    class Extents,
    class LayoutPolicy = std::layout_right,
    class AccessorPolicy = std::default_accessor<T>
> class mdspan;

And it has its own header <mdspan>.

The main proposal for this feature can be found at https://wg21.link/P0009

Following the pattern from std::span, we have a few options to create mdspan: with dynamic or static extent.

The key difference is that rather than just one dimension, we can specify multiple. The declaration also gives more options in the form of LayoutPolicy and AccessorPolicy. More on that later.

Static Extent

std::vector<int> vec = {1, 2, 3, 2, 4, 5, 3, 5, 6, 7, 8, 9};
    
std::mdspan<int, std::extents<size_t, 3, 3>> mat3x3 { vec.data() };
std::print("sizeof: {}, rank: {}, ext 0: {}, ext 1: {}\n",
sizeof(mat3x3), mat3x3.rank(), mat3x3.extent(0), mat3x3.extent(1));

The output:

sizeof: 8, rank: 2, ext 0: 3, ext 1: 3

Run @Compiler Explorer

Dynamic Extent

And the dynamic option:

std::mdspan<int, std::dextents<size_t, 2>> mat2x2 { vec.data(), 2, 2 };
std::print("sizeof: {}, rank: {}, ext 0: {}, ext 1: {}\n",
sizeof(mat2x2), mat2x2.rank(), mat2x2.extent(0), mat2x2.extent(1));

The output:

sizeof: 24, rank: 2, ext 0: 2, ext 1: 2

Run @Compiler Explorer

The line:

std::mdspan<int, std::dextents<size_t, 2>>

Is equivalent:

std::mdspan<int, std::extents<size_t, std::dynamic_extents, std::dynamic_extents>>

In other words, dextents stands for dynamic extents.

And as of C++26, we can also simplify it with std::dims:

std::mdspan<int, std::dims<2>> mat2x2 { vec.data(), 2, 2 };

Sizeof mdspan

As you can see, similarly to std::span, when the size is known at compile time, the object can be smaller (it doesn’t have to store the dynamic size).

Run here: @Compiler Explorer

Construction options

Let’s review some more options to create mdspan:

1. Default Constructor

constexpr mdspan();

Example:

std::mdspan<int, std::dextents<size_t, 1>> empty_span;
std::print("is empty {}\n", empty_span.empty());

See @Compiler Explorer

2. Extents From Variadic Arguments

template< class... OtherIndexTypes >
constexpr explicit mdspan( data_handle_type p, OtherIndexTypes... exts );

Example:

int arr[12] { 0 };
std::mdspan<int, std::extents<size_t, 3, 4>> span(arr);
std::print("w: {}, h: {}\n", span.extent(0), span.extent(1));
auto ex = span.extents();
std::println("{}, {}", ex.rank(), ex.static_extent(1));

std::mdspan<int, std::dextents<size_t, 2>> dspan(arr, 3, 4);
std::print("w: {}, h: {}\n", dspan.extent(0), dspan.extent(1));
auto ex2 = dspan.extents();
std::println("{}, {}", ex2.rank(), ex2.static_extent(1));

Run @Compiler Explorer

3. Extents From `std::array`

template< class OtherIndexType, std::size_t N >
constexpr explicit(N != rank_dynamic()) mdspan( data_handle_type p, const std::array<OtherIndexType, N>& exts );

Example:

int arr[12] { 0 };
std::array<size_t, 2> my_extents = {3, 4};
std::mdspan<int, std::extents<size_t, 3, 4>> span(arr, my_extents);

or simpler with CTAD:

std::mdspan span(arr, my_extents); // but this time extends are dynamic

Run @Compiler Explorer

Layout

Since we’re dealing with multidimensional data, there are a couple of ways we can access elements. That’s why we have the “layout” policy class to represent various options.

Here are the basic layouts available as of C++23:

layout_right: C or C++ style, row major, where the rightmost index gives stride-1 access to the underlying memory;
layout_left: Fortran or Matlab style, column major, where the leftmost index gives stride-1 access to the underlying memory;
layout_stride: a generalization of the two layouts above, which stores a separate stride (possibly not one) for each extent.

Plus, as of C++26, we’ll get two more options:

layout_left_padded - column-major layout mapping policy with padding stride that can be greater than or equal to the leftmost extent
layout_right_padded - row-major layout mapping policy with padding stride that can be greater than or equal to the rightmost extent

Here’s an example:

template <typename layout>
void printMat(std::mdspan<int, std::dextents<size_t, 2>, layout> mat) {
    for (size_t i = 0; i < mat.extent(0); ++i) {
        for (size_t j = 0; j < mat.extent(1); ++j)
            std::print("{} ", mat[i, j]);
        std::println();
    }
}

int main() {
    std::vector<int> vec = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, };

    std::mdspan<int, std::dextents<size_t, 2>, std::layout_right> mat { vec.data(), 2, 3 };
    std::println("right:");
    printMat(mat);

    std::mdspan<int, std::dextents<size_t, 2>, std::layout_left> mat_left { vec.data(), 2, 3 };
    std::println("left:");
    printMat(mat_left);
}

See @Compiler Explorer

The output:

right:
0 1 2 
3 4 5 
left:
0 2 4 
1 3 5

Summary

std::mdspan provides a powerful and efficient way to interact with multidimensional data, which was a huge pain point for C++. While sometimes it is hard to spell out the full type, mdspan is very flexible and offers many handy ways to work with data.

In the text, I also showed how to use layout policy for a different way to order elements in the view. The missing part is “AccessorPolicy” which is a policy class that specifies how each element is accessed. For example we can implement a “checked” policy that would ensure the index is not out of range. But I’ll leave that for later considerations.

Back to you

Have you played with mdspan?
What do you use to work with multidimensional data?

Share your comments below

Adjacency Matrix and std::mdspan, C++23

Tue, 11 Feb 2025 00:00:00 +0000

In graph theory, an adjacency matrix is a square matrix used to represent a finite (and usually dense) graph. The elements of the matrix indicate whether pairs of vertices are adjacent or not, and in weighted graphs, they store the edge weights.

In many beginner-level tutorials, adjacency matrices are implemented using vector of vectors (nested dynamic arrays), but this approach has inefficiencies due to multiple memory allocations. C++23 introduces std::mdspan, which provides a more efficient way to handle multidimensional data structures without the overhead of nested containers.

In this blog post, we’ll explore various implementations of an adjacency matrix, starting with a basic approach and progressively improving it using std::mdspan.

Starting slow

Let’s start with a straightforward implementation using a vector of vectors to represent the adjacency matrix.

#include <iostream>
#include <vector>
#include <limits>
#include <print>

template <typename T>
class Graph {
public:
    static constexpr T INF = std::numeric_limits<T>::max();
private:
    std::vector<std::vector<T>> adjacencyMatrix;
    size_t numVertices;

public:
    Graph(size_t vertices) 
        : numVertices(vertices)
        , adjacencyMatrix(vertices, std::vector<T>(vertices, INF)) 
    {
        for (size_t i = 0; i < numVertices; i++)
            adjacencyMatrix[i][i] = 0;
    }

    void addEdge(size_t u, size_t v, T weight) {
        if (u < numVertices && v < numVertices && u != v) {
            adjacencyMatrix[u][v] = weight;
            adjacencyMatrix[v][u] = weight;
        }
    }

    bool isConnected(size_t u, size_t v) const {
        return (u < numVertices 
		     && v < numVertices 
			 && adjacencyMatrix[u][v] != INF);
    }

    const std::vector<std::vector<T>>& getAdjacencyMatrix() const {
        return adjacencyMatrix;
    }
};

template <typename T>
void printGraph(const Graph<T>& g) {
    std::print("Adjacency Matrix:\n");
    for (const auto& row : g.getAdjacencyMatrix()) {
        for (T weight : row) {
            if (weight == Graph<T>::INF)
                std::print("   ∞ ");
            else
                std::print(" {:3} ", weight);
        }
        std::println();
    }
}

The class represents an undirected weighted graph.
The adjacency matrix is initialized with INF (representing no direct connection between nodes).
The diagonal elements are set to 0 since a node is always connected to itself.
addEdge ensures bidirectional connections by setting both adjacencyMatrix[u][v] and adjacencyMatrix[v][u].
isConnected checks if a direct edge exists between two vertices.

And here’s the main function:

int main() {
    Graph<int> g(5);

    g.addEdge(0, 1, 4);
    g.addEdge(0, 2, 8);
    g.addEdge(1, 2, 2);
    g.addEdge(1, 3, 6);
    g.addEdge(2, 3, 3);
    g.addEdge(3, 4, 5);
    g.addEdge(4, 0, 7);

    printGraph(g);

    std::print("\nIs node 1 connected to node 3? {}\n", g.isConnected(1, 3));
    std::print("Is node 0 connected to node 4? {}\n", g.isConnected(0, 4));
}

Run @Compiler Explorer

Here’s a visual representation of this demo graph:

Improving Efficiency with a Single Vector

Using a vector of vectors is simple but inefficient due to separate memory allocations for each row. Instead, we can use a single contiguous vector to store the matrix elements, which improves cache locality and reduces memory fragmentation.

See the following code:

template <typename T>
class Graph {
public:
    static constexpr T INF = std::numeric_limits<T>::max();
private:
    std::vector<T> adjacencyMatrix;
    size_t numVertices;

    size_t index(size_t row, size_t col) const {
        return row * numVertices + col;
    }

public:
    Graph(size_t vertices)
        : numVertices(vertices)
        , adjacencyMatrix(vertices * vertices, INF)
    {
        for (size_t i = 0; i < numVertices; i++)
            adjacencyMatrix[index(i, i)] = 0;
    }
    
    // ...

Above, we have the function index(row, col) that does the proper indexing.

Run @Compiler Explorer

C++23 MD span

C++23 introduces std::mdspan, a lightweight non-owning view over multidimensional data. It allows indexing into a contiguous 1D array as if it were a multidimensional array without needing explicit indexing functions.

Here’s a basic example of how to use this view type:

#include <vector>
#include <mdspan> // << new header!
#include <iostream>

bool isSymmetric(std::mdspan<int, std::dextents<size_t, 2>> matrix) {
    const auto rows = matrix.extent(0);
    const auto cols = matrix.extent(1);

    if (rows != cols) return false;

    for (size_t i = 0uz; i < rows; ++i) {
        for (size_t j = i + 1; j < cols; ++j) {
            if (matrix[i, j] != matrix[j, i]) 
                return false;
        }
    }
    return true;
}

int main() {
    std::vector<int> matrix_data = {1, 2, 3, 2, 4, 5, 3, 5, 6};
    auto matrix = std::mdspan(matrix_data.data(), 3, 3);

    std::cout << isSymmetric(matrix);
}

See @Compiler Explorer

The key thing is that mdspan is just one object that you pass to a function, and it has all the information and also features to access multidimensional data. Notice the operator [] with two arguments: matrix[i, j].

I must say that the type std::mdspan<int, std::dextents<size_t, 2>> matrix is probably not the nicest to write and spell out, but it wraps all the information in a pretty flexible way.

(In C++26, have std::dims helper type that can even make things shorter for dynamic mdspan extents).

Adjacency Matrix with mdspan

Thanks to std::mdspan, we can simplify the code to calculate the index. The operator [] can now handle multiple arguments, so it’s easy to index into a multidimensional array:

#include <vector>
#include <limits>
#include <print>
#include <mdspan>

template <typename T>
class Graph {
public:
    static constexpr T INF = std::numeric_limits<T>::max();

private:
    std::vector<T> adjacencyMatrix;
    std::mdspan<T, std::dextents<size_t, 2>> matrixView;
    size_t numVertices;

public:
    Graph(size_t vertices) 
        : numVertices(vertices)
        , adjacencyMatrix(vertices * vertices, INF)
        , matrixView(adjacencyMatrix.data(), vertices, vertices) 
    {
        for (size_t i = 0; i < numVertices; i++)
            matrixView[i, i] = 0;
    }
	
	// copy ctor / move op / assignment...

    void addEdge(size_t u, size_t v, T weight) {
        if (u < numVertices && v < numVertices && u != v) {
            matrixView[u, v] = weight;
            matrixView[v, u] = weight;
        }
    }

    bool isConnected(size_t u, size_t v) const {
        return (u < numVertices 
		     && v < numVertices 
			 && matrixView[u, v] != INF);
    }

    auto getAdjacencyMatrix() const {
        return matrixView;
    }

    size_t getNumVertices() const {
        return numVertices;
    }
};

template <typename T>
void printGraph(const Graph<T>& g) {
    size_t n = g.getNumVertices();
    auto matrix = g.getAdjacencyMatrix();

    std::print("Adjacency Matrix:\n");
    for (size_t i = 0; i < n; i++) {
        for (size_t j = 0; j < n; j++) {
            T weight = matrix[i, j];
            if (weight == Graph<T>::INF)
                std::print("   ∞ ");
            else
                std::print(" {:3} ", weight);
        }
        std::println();
    }
}

Run @Compiler Explorer

Const Correctness

In the previous example, I made a mistake with the member function/getter

auto getAdjacencyMatrix() const {
    return matrixView;
}

While the auto as the return type is handy and shortened the code, the issue is that now we can write

auto matrix = g.getAdjacencyMatrix();
matrix[0, 1] = 1234;

In other words, we return a non-const view of the data, allowing modification of its elements.

It’s usually not the best approach for getters, and to preserve cost correctness, we should manually specify the return type.

std::mdspan<const T, std::dims<2>> getAdjacencyMatrix() const {
    return matrixView;
}

Now, you can access elements, but you cannot modify them.

Rule of Five

Thanks to a valuable comment at Reddit, I’ve noticed also another issue:

Rule of Five…

Since we introduced a view type as the member of the class, we have to support it through copy/move/assignment operations. Otherwise when you copy a graph object the span will point to a wrong data! See the updated version of the code in the next section.

Improvements

The code works, but it’s definitely not perfect. Here are some updates we should make:

requires std::is_arithmetic_v<T> - the graph class is a template, but we should at least narrow down the “weight” type that we support.
Rule of Zero - checked; see above.
explicit ctor - to prevent unwanted conversions
noexcept - at least in some basic functions
nodiscard - to indicate that a return value should be used, not just thrown away
error handling through exceptions

Here’s the updated version:

#include <vector>
#include <limits>
#include <print>
#include <mdspan>

template <typename T>
requires std::is_arithmetic_v<T>
class Graph {
public:
    static constexpr T INF = std::numeric_limits<T>::max();

private:
    std::vector<T> adjacencyMatrix;
    std::mdspan<T, std::dims<2>> matrixView;
    size_t numVertices;

public:
    explicit Graph(size_t vertices) 
        : numVertices(vertices)
        , adjacencyMatrix(vertices * vertices, INF)
        , matrixView(adjacencyMatrix.data(), vertices, vertices) 
    {
        for (size_t i = 0; i < numVertices; i++)
            matrixView[i, i] = 0;
    }

	// copy ctor, move ctor, assignment...

    void addEdge(size_t u, size_t v, T weight) {
        if (u >= numVertices || v >= numVertices)  
            throw std::out_of_range("Vertex index out of bounds");  
        if (u == v)  
            throw std::invalid_argument("Self-loops are not allowed");

        matrixView[u, v] = weight;
        matrixView[v, u] = weight;
    }

    [[nodiscard]] bool isConnected(size_t u, size_t v) const{
        if (u >= numVertices || v >= numVertices)  
            throw std::out_of_range("Vertex index out of bounds"); 
        return matrixView[u, v] != INF;
    }

    [[nodiscard]] std::mdspan<const T, std::dims<2>> getAdjacencyMatrix() const {
        return matrixView;
    }

    [[nodiscard]] size_t getNumVertices() const noexcept {
        return numVertices;
    }
};

template <typename T>
void printGraph(const Graph<T>& g) {
    size_t n = g.getNumVertices();
    auto matrix = g.getAdjacencyMatrix();

    std::print("Adjacency Matrix:\n");
    for (size_t i = 0; i < n; i++) {
        for (size_t j = 0; j < n; j++) {
            T weight = matrix[i, j];
            if (weight == Graph<T>::INF)
                std::print("   ∞ ");
            else
                std::print(" {:3} ", weight);
        }
        std::println();
    }
}

Run @Compiler Explorer

Implementation notes

The code shown in examples works just great in Clang 18.+! In Compiler Explorer use -stdlib=libc++ so that it uses the proper Standard Library implementation.

Summary

In this article, we discuss a simple and naive implementation of an adjacency graph and enhance it with the latest features from C++23. The use of mdspan facilitates the creation of a proper 2D view over a contiguous sequence of data.

The final code is a bit better, but of course, I’m happy to see your suggestions and improvements we can make here.

Back to you

Have you played with mdspan?
What do you use to work with multidimensional data?

Share your comments below

How to use std::span from C++20

Mon, 03 Feb 2025 00:00:00 +0000

In this article, we’ll look at std::span, which has been available since C++20. This “view” type is more generic than string_view and can help work with arbitrary contiguous collections.

Updated in Feb 2025: added section about returning spans and C++26 improvements (.at() and creatoion from initializer list).

A Motivating Example

Here’s an example that illustrates the primary use case for std::span:

In traditional C (or low-level C++), you’d pass an array to a function using a pointer and a size like this:

void process_array(int* arr, std::size_t size) {
  for(std::size_t i = 0; i < size; ++i) {
    // do something with arr[i]
  }
}

std::span simplifies the above code:

void process_array(std::span<int> arr_span) {
  for(auto& elem : arr_span) {
    // do something with elem
  }
}

The need to pass a separate size variable is eliminated, making your code less error-prone and more expressive.

And in essence: std::span<T> is:

a lightweight abstraction of a contiguous sequence of values of type T,
more or less implemented as struct { T * ptr; std::size_t length; },
a non-owning type (i.e. a “reference type” rather than a “value type”).

Construction of `std::span`

std::span lives in its own new header <span>. It’s defined as follows:

template<class T, std::size_t Extent = std::dynamic_extent>
class span;

To create this object, you have two basic options: static extent and dynamic:

Static Extent

When you know the size at compile-time:

int arr[] = {1, 2, 3, 4, 5};
std::span<int, 5> arr_span {arr};
//std::span<int, 2> arr_span2 {arr}; // error size doesn't match

Run @Compiler Explorer

Here, the 5 is an integral part of the type. You’ll get a compiler error if you try to initialize this span with an array of different sizes.

Dynamic Extent

When you only know the size at runtime, like when working with vectors:

int arr[] = {1, 2, 3, 4, 5};
std::vector v { 1, 2, 3, 4, 5};
std::span<int> arr_span {arr}; // dynamic extent
std::span<int> vec_span {v};   // also!

See @Compiler Explorer

Notice the absence of size in the type? That’s the dynamic extent in action.

Sizeof span

The interesting part about span is that when the size is static, then the type is smaller as there’s no need to store the size of the sequence:

int arr[] = {1, 2, 3, 4, 5};
std::vector<int> vec = {1, 2, 3, 4, 5};
std::span<int, 5> arr_span {arr};
std::span<int> other_span {arr};
std::span<int> vec_span{vec};

std::cout << std::format("sizeof arr_span: {}\n", sizeof(arr_span));
std::cout << std::format("sizeof other_span: {}\n", sizeof(other_span));
std::cout << std::format("sizeof vec_span: {}\n", sizeof(vec_span));

Run @Compiler Explorer

The output on GCC is:

sizeof arr_span: 8
sizeof other_span: 16
sizeof vec_span: 16

More construction options

For completeness, let’s now revise other construction options following the list of available constructors:

This article started as a preview for Patrons, sometimes even months before the publication. If you want to get extra content, previews, free ebooks and access to our Discord server, join the C++ Stories Premium membership or see more information.

Default Constructor

This constructs an empty span.

std::span<int> empty_span;

.data() returns nullptr and the size() returns 0 in this case.

From Iterators and Size

Creates a span from a starting iterator and a size.

std::vector<int> vec = {1, 2, 3};
std::span<int> from_iterator_and_size(vec.begin(), /*count*/2); // 1, 2

std::span<int> from_iterator_and_size2(vec.begin(), /*count*/3); // 1, 2, 3

And also in CTAD version:

std::vector vec = {1, 2, 3};
std::span from_iterator_and_size(vec.begin(), /*count*/2); // 1, 2
 
std::span from_iterator_and_size2(vec.begin(), /*count*/3); // 1, 2, 3

See @Compiler Explorer

From Two Iterators

Constructs a span from a range specified by two iterators.

std::span<int> from_two_iterators(vec.begin(), vec.end());

And using CTAD:

std::span from_iterator_and_end(vec.begin(), vec.end());

See @Compiler Explorer

From C-style Array

For C-style arrays.

int arr[] = {1, 2, 3};
std::span<int> from_array(arr);
std::span from_array2(arr);

.data() returns std::data(arr)

See @Compiler Explorer

From `std::array`

Can construct both from non-const and const std::array.

std::array<int, 3> std_arr = {1, 2, 3};
std::span<int, 3> from_std_array(std_arr);
// CTAD:
std::span from_std_array2(std_arr);

const std::array<int, 3> const_std_arr = {1, 2, 3};
std::span<const int> from_const_std_array(const_std_arr);

See @Compiler Explorer

From Contiguous Range

Using this constructor, you can pass in any contiguous range like std::vector.

std::vector vec {1, 2, 3, 4, 5};
std::span<int> from_range(vec); // covers entire vec
// CTAD:
std::span from_range2(vec);ec

See @Compiler Explorer

Conversion from Another Span

Can be used for type conversions if the types are compatible.

std::span<int> int_span(vec);
std::span<const int> const_span = int_span; // conversion

Passing spans

Spans are lightweight objects intended to pass by value. But we have two options to preserve the constness of its elements:

void print(span<const char> outbuf);
void transform(span<char> inbuf);

In other words, you can pass span<const T> to indicate constant elements, and “read only” access, or pass span<T> to allow read/write access.

For example:

void transform(std::span<char> outbuf) {
    for (auto& elem : outbuf) {
        elem += 1;
    }
}

void output(std::span<const char> outbuf) {
    std::cout << "contents: ";
    for (auto& elem : outbuf) {
        std::cout << elem << ", ";
//        elem = 0; // error!
    }
    std::cout << '\n';
}

int main() {
    std::string str = "Hello World";
    std::span<char> buf_span(str);

    output(str);
    transform(buf_span);
    output(buf_span);
}

Run @Compiler Explorer

The output:

contents: H, e, l, l, o,  , W, o, r, l, d, 
contents: I, f, m, m, p, !, X, p, s, m, e,

The example above shows that str nicely converts into a span and is passed to the output function. And later, the buf_span is also converted (char to const char) when passing to output.

Subspans

You can easily create subviews/subspans of existing spans:

void printSpan(std::span<const int> sp) {
    for (auto&& elem : sp)
        std::cout << elem << ' ';
    std::cout << '\n';
}

int main() {
    int arr[] = {1, 2, 3, 4, 5, 6};
    std::span<int> sp(arr, 6);

    // subspan(start, count):
    std::span<int> sub = sp.subspan(2, /*count*/2);
    printSpan(sub);

    // fist(count):
    std::span<int> subFirst3 = sp.first(3);
    printSpan(subFirst3);

    // last(count):
    std::span<int> subLast4 = sp.last(4);
    printSpan(subLast4);
}

Play @Compiler Explorer

The output:

3 4 
1 2 3 
3 4 5 6

Showing basic properties

Here’s another example that prints basic information about spans:

int main() {
    int arr[] = {1, 2, 3, 4, 5};
    std::vector<int> vec = {1, 2, 3, 4, 5};
    std::array sarr {1, 2, 3, 4, 5};
    std::span<int, 5> arr_span {arr};
    std::span<int> sarr_span {arr};
    std::span<int> vec_span{vec};

    auto span_info = [](std::string_view str, auto sp) {
        std::cout << 
           std::format("{}\n sizeof {}\n extent {}\n size in bytes: {}\n",   
           str, sizeof(sp), sp.extent == std::dynamic_extent ? "dynamic" : "static", 
                             sp.size_bytes());
    };
    span_info("arr_span", arr_span);
    span_info("sarr_span", sarr_span);
    span_info("vec_span", vec_span);
}

Run @Compiler Explorer

The output from GCC:

arr_span
 sizeof 8
 extent static
 size in bytes: 20
sarr_span
 sizeof 16
 extent dynamic
 size in bytes: 20
vec_span
 sizeof 16
 extent dynamic
 size in bytes: 20

C++26 `span::at()`

With C++26, std::span gains the at() method, which provides bounds-checked access to elements, similar to std::vector::at(). This ensures safer indexing by throwing std::out_of_range if an invalid index is accessed.

Have a look:

#include <span>
#include <vector>
#include <iostream>
#include <stdexcept>

int main() {
    std::vector<int> data = {1, 2, 3, 4, 5};
    std::span<int> span = data;

    try {
        std::cout << "Element at index 2: " << span.at(2) << "\n";
        std::cout << "Trying to access index 10...\n";
        std::cout << span.at(10) << "\n";
    } catch (const std::out_of_range& e) {
        std::cerr << "Exception caught: " << e.what() << "\n";
    }
}

See @Compiler Explorer

Key benefits of this new feature:

Safety: Prevents undefined behavior by ensuring out-of-bounds access throws an exception.
Consistency: Aligns std::span with other standard containers (std::vector, std::array, etc.) that already have at().
Debugging Aid: Makes debugging easier by immediately identifying invalid accesses.

C++26 span over an initializer list

With C++26, std::span can now be constructed from an std::initializer_list, closing a gap that previously existed between std::vector and std::span as function parameters. The change comes from the following proposal: P2447

While std::string_view can be constructed directly from a string literal, std::span<const T> couldn’t be constructed from a braced initializer list {1, 2, 3}. This led to inconsistencies when using spans in function parameters.

Consider this example before C++26:

void process(std::span<const int> sp);

process({1, 2, 3}); // Error: No viable conversion
process(std::vector<int>{1, 2, 3}); // OK but unnecessary allocation
process(std::span<const int>(std::initializer_list<int>{1, 2, 3})); // Works but verbose

The C++26 Solution

With C++26, std::span<const T> now has a constructor accepting std::initializer_list<T>, allowing direct usage of braced initializer lists:

void process(std::span<const int> sp);

process({1, 2, 3}); // Now works in C++26!
process({});        // Also works, creating an empty span

This makes std::span<const T> a better drop-in replacement for const std::vector<T>& in function parameters, improving usability and reducing unnecessary copies.

See the full example:

#include <span>
#include <vector>
#include <print>

void process(std::span<const int> sp) {
    std::println("sp size is {}", sp.size());
}

int main() {
    std::vector<int> data = {1, 2, 3, 4, 5};
    process(data);
    process({1, 2, 3}); // Now works in C++26!
    process({});        // Also works, creating an empty span
}

Check at Compiler Explorer

Important Notes on Dangling

Just like std::string_view, std::span does not extend the lifetime of temporary data. If used incorrectly, it can lead to dangling references:

std::span<const int> sp = {1, 2, 3}; // Dangles! The initializer list is a temporary.

This means std::span over an initializer list is best used for function parameters, where the span is only needed within the function’s scope.

Returning `std::span`

While spans are great as input parameters, they can also show their strength as a return type. This can be a middle ground between references and copies, with additional flexibility for handling missing values.

Consider the following example, where there’s a config manager, and we return a Config using getConfig member function:

#include <span>
#include <vector>
#include <iostream>
#include <map>

class ConfigManager {
private:
    std::map<std::string, std::vector<uint8_t>> configs;

public:
    ConfigManager() {
        configs["database"] = {
            'h','o','s','t','=','l','o','c','a','l','h','o','s','t',';',
            'p','o','r','t','=','5','4','3','2'
        };
    }

    std::span<const uint8_t> getConfig(const std::string& name) const {
        auto it = configs.find(name);
        if (it != configs.end()) 
            return it->second; 
        return {}; 
    }
};

int main() {
    ConfigManager manager;
    auto dbConfig = manager.getConfig("database");
    for (auto byte : dbConfig)
        std::cout << static_cast<char>(byte);

    auto errCfg = manager.getConfig("error");
    for (auto byte : errCfg)
        std::cout << static_cast<char>(byte);
}

See @Compiler Explorer

By using span we can clearly indicate a “view” type. Additionally, spans support many containers, so even if we change the internal representation of the config entry, we can still return a span object (assuming it’s still contiguous…).

Without spans, I’d had to return a reference to a vector, which might be tricky when there’s no entry found. See my other article about this issue: Improving Code Safety in C++26: Managers and Dangling References - C++ Stories.

Advantages of Using `std::span`

Performance: Like references, spans avoid unnecessary copies.
Safety: Being non-owning, spans make it clear that the caller doesn’t own the data.
Interoperability: Works seamlessly with other container types like std::array or raw arrays.
Simpler handling of missing data: Unlike returning a const std::vector&, where you must return a valid reference, std::span can simply return an empty span ({}).

Comparing with `std::string_view`

Generality: std::span can be used with any type, not just character types.
Mutability: Unlike std::string_view, a std::span can modify the data it views (unless you define it as a span of const).
Extent: std::span can have either static or dynamic extent, string_view is always “dynamic”.

Guidelines

Some of the guidelines related to spans:

Summary

In this article we looked at std::span introduced in C++20. This type offers a handy way to work with contiguous sequences like arrays or containers. We can say that it’s a more generic approach than std::string_view as it allows read/write access (if needed).

Back to you

Have you tried std::span?
Do you use other “Reference”/view types from the Standard Library?

Share your comments below.

Improving Code Safety in C++26: Managers and Dangling References

Mon, 20 Jan 2025 00:00:00 +0000

In this blog post, we’ll explore ways to improve the safety of a simple configuration manager. We’ll handle common pitfalls like dangling references and excessive stack usage. Additionally, we’ll see how C++26 helps enforce safer coding practices with stricter diagnostics and improved handling of large objects.

Let’s go.

Step 1: The Buggy Implementation

Below is a simple example of a manager object that stores various configs in a map and provides a method to retrieve them. When a requested configuration isn’t found, the code attempts to return a default certificate:

#include <map>
#include <string>
#include <iostream>
#include <vector>
#include <cstdint>

class ConfigManager {
private:
    std::map<std::string, std::vector<uint8_t>> configs;

public:
    ConfigManager() {
        configs["database"] = {
                'h','o','s','t','=','l','o','c','a','l','h','o','s','t',';',
                'p','o','r','t','=','5','4','3','2'
        };
    }

    const std::vector<uint8_t>& findConfig(const std::string& name) const {  
        auto it = configs.find(name);  
        if (it != configs.end()) 
            return it->second; 

        return {42};
    }  

};

int main() {
    ConfigManager configManager;

    const std::vector<uint8_t>& dbConfig = configManager.findConfig("database");  
    
    std::cout << "Database config: ";  
    for (uint8_t byte : dbConfig) {  
        std::cout << static_cast<char>(byte);  
    }  
}

See @Compiler Explorer

Do you see a potential error in this code?

. . .

At first glance, the code looks harmless. However, if the requested entry isn’t found, the function returns a reference to a temporary std::vector. Once the function exits, that temporary is destroyed—leaving you with a dangling reference and undefined behavior.

Compiler Warnings

Ok, the bug was easy… and compilers have warned about this case for a long time.

While this is helpful, it’s still possible to overlook or disable the warning, especially if your project is large.

So let’s try enabling C++26 mode… for example on GCC 14 you’ll get:

error: returning reference to temporary [-Wreturn-local-addr]

Great!

Now, we cannot skip that error, and we have to fix it.

This handy features comes from: Disallow binding a returned reference to a temporary P2748R5 - implemented in GCC 14 and Clang 19.

Step 2: Fixing the Dangling Reference

One straightforward fix is to ensure that the “default config” has a lifetime extending beyond the function scope. A static object does the trick:

class ConfigManager {
private:
    std::map<std::string, std::vector<uint8_t>> configs;

    static const std::vector<uint8_t>& getDefaultConfig() {
        static std::vector<uint8_t> def {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
        return def;
    }
public:
    ConfigManager() {
        configs["database"] = {
                'h','o','s','t','=','l','o','c','a','l','h','o','s','t',';',
                'p','o','r','t','=','5','4','3','2'
        };
    }

    const std::vector<uint8_t>& findConfig(const std::string& name) const {  
        auto it = configs.find(name);  
        if (it != configs.end()) 
            return it->second; 

        return getDefaultConfig();
    }  

};

int main() {
    ConfigManager configManager;

    const std::vector<uint8_t>& dbConfig = configManager.findConfig("database");  
    
    std::cout << "Database config: ";  
    for (uint8_t byte : dbConfig) {  
        std::cout << static_cast<char>(byte);  
    }  
}

See @Compiler Explorer

By returning a static object from getDefaultConfig, we avoid dangling references and undefined behavior. The object’s lifetime is the entire duration of the program, so it’s perfectly safe to return a reference to it.

Just to note: this solution isn’t perfect, and we could explore another approach, such as using std::optional,std::expected or std::span. But I’ll leave that for you as a homework assignment :)

The error is now gone, and our program works fine.

But…

Step 3: The Stack Overflow Problem

Now, consider what happens if requirements change, and the default config has to contain some large data, for example, a binary representation of some image… If it is substantial - say around 1…2MB of data. Storing such a large object in a single vector might look like this:

static const std::vector<uint8_t>& getDefaultConfig() {
    static std::vector<uint8_t> def {
	// 1, 2, 3, ...
	// in total it's 2MB of data
	};
    return def;
}

This seemingly harmless code can risk stack overflow, especially on systems with limited stack space.

Why is that? The object is static, so it shouldn’t take stack’s space…

However, the issue is that we used the initializer list to initialize the vector. Before C++26 the compiler creates a helper array, on the stack, and then copies the data to the vector.

See at C++Insights: https://cppinsights.io/s/85f708c6, the function might be transformed into something like this:

 static inline const std::vector<unsigned char, std::allocator<unsigned char> > & getDefaultConfig()
  {
    const unsigned char __list12_41[10]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}; << helper array!
	...

For instance, if you have 4MB of stack space and your default configuration is 1MB, you might occasionally encounter this bug depending on your call stack. This scenario can be quite tricky to catch!

Fortunately, it’s fixed in C++26:

Step 4: The C++26 Solution

Thanks to “Static storage for braced initializers P2752R3”, in C++26, similar to string literals, arrays for initializer list go into static storage!

What’s more, the fix can be implemented against C++11, so just by compiling with some of the latest compiler, your code might simply get safer.

Conclusion

In this blog post, we went through a simple scenario where we wrote a buggy code, but thanks to C++26, the code will be safer. The first bug was easy to fix, but it can be missed as current compilers only report warnings. Thanks to C++26, you’ll see a hard compiler error, so you have to address the issue of returning references to temporary objects. But with the fix, we introduced another potential issue with stack overflow. This one is tricky, and even expert coders might not be aware of this subtle issue. But again, thanks to C++26, the bug will disappear with the latest compiler versions.

Back to you

• Have you encountered similar memory management challenges in your C++ projects?
• Are you planning to switch to newer C++ standards to leverage these safety features?

Share your experiences and thoughts in the comments below!

8 More C++23 Examples

Mon, 30 Dec 2024 00:00:00 +0000

In this article, you’ll see eight larger examples that illustrate the changes in C++23.

C++23 brings a ton of cool features, as you can see in my two previous articles (here and here). So far, we explored each new addition one by one, but I’d like to share more examples that combine multiple features.

Here we go:

1. `std::ranges::to<>`

The ranges::to<> feature allows you to convert ranges into containers easily:

#include <ranges>
#include <vector>
#include <iostream>
#include <string>
#include <unordered_map>
#include <map>

int main() {
    std::vector<int> numbers = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

    std::unordered_map<int, std::string> number_to_text = {
        {1, "one"}, {2, "two"}, {3, "three"}, {4, "four"}, {5, "five"},
        {6, "six"}, {7, "seven"}, {8, "eight"}, {9, "nine"}, {10, "ten"}
    };

    auto even_numbers = numbers 
                    | std::views::filter([](int n) { return n % 2 == 0; })
                    | std::ranges::to<std::vector>();

    auto text_numbers_map = even_numbers 
                    | std::views::transform([&number_to_text](int n) { 
                        return std::pair{n, number_to_text.contains(n) ? 
                        number_to_text[n] : "unknown" }
                      })
                    | std::ranges::to<std::map>();

    std::cout << "Even numbers: ";
    for (int n : even_numbers) std::cout << n << " ";
    std::cout << "\nTextual numbers:\n";
    for (const auto& [k, v] : text_numbers_map) 
        std::cout << k << " -> " << v << '\n';
}

Run @Compiler Explorer

As you can see, there’s no issue in creating not only vectors but maps as well.

2. `std::print` and `std::println`

The new formatted output library simplifies printing. Here’s an example that demonstrates alignment, formatting, and table creation:

#include <print>
#include <unordered_map>
#include <string>
#include <numeric>

int main() {
    // Store the data in an unordered_map (country -> size in km²)
    std::unordered_map<std::string, double> country_sizes = {
        {"USA", 9833517},
        {"Canada", 9984670},
        {"Australia", 7692024},
        {"China", 9596961},
        {"Poland", 312696}
    };

    constexpr double KM_TO_MI = 0.386102; // Conversion factor

    const double total_km = std::accumulate(country_sizes.begin(), country_sizes.end(), 0.0,
              [](double sum, const auto& entry) { return sum + entry.second; });
    const double total_mi = total_km * KM_TO_MI;

    // Table headers
    std::println("{:<15} | {:>15} | {:>15}", "Country", "Size (km²)", "Size (mi²)");
    std::println("{:-<15}-+-{:-<15}-+-{:-<15}", "", "", ""); // Separator line

    // Table rows
    for (const auto& [country, size_km] : country_sizes) {
        double size_mi = size_km * KM_TO_MI;
        std::println("{:<15} | {:>15.0f} | {:>15.2f}", country, size_km, size_mi);
    }

    // Footer
    std::println("{:-<15}-+-{:-<15}-+-{:-<15}", "", "", ""); // Separator line
    std::println("{:<15} | {:>15.0f} | {:>15.2f}", "Total", total_km, total_mi);
}

Run @Compiler Explorer

Example output:

Country         |      Size (km²) |      Size (mi²)
----------------+-----------------+----------------
Poland          |          312696 |       120732.55
China           |         9596961 |      3705405.84
Australia       |         7692024 |      2969905.85
Canada          |         9984670 |      3855101.06
USA             |         9833517 |      3796740.58
----------------+-----------------+----------------
Total           |        37419868 |     14447885.87

3. `std::optional` Monadic Operations

The new and_then, transform, and or_else functions make working with std::optional more expressive. Here’s an example of chain operations that process user input.

#include <optional>
#include <iostream>
#include <string>
#include <algorithm>

std::optional<std::string> get_user_input() {
    std::string input;
    std::cout << "Enter your name: ";
    std::getline(std::cin, input);
    if (input.empty()) return std::nullopt;
    return input;
}

int main() {
    auto result = get_user_input()
        .transform([](std::string name) {
            std::transform(name.begin(), name.end(), name.begin(), ::toupper);
            return name;
        })
        .and_then([](std::string name) {
            if (name == "ADMIN") return std::optional<std::string>("Welcome, Admin!");
            return std::optional<std::string>("Hello, " + name + "!");
        })
        .or_else([] {
            return std::optional<std::string>("No input provided.");
        });

    std::cout << *result << "\n";
}

Experiment at @Compiler Explorer

4. `std::generator`

The std::generator feature simplifies creating lazy sequences. Here’s an example of reading line by line from a file:

std::string to_uppercase(std::string str) {
    std::transform(str.begin(), str.end(), str.begin(), ::toupper);
    return str;
}

struct ProcessedLine {
    std::string original;
    std::string uppercase;
};

std::generator<ProcessedLine> read_and_process_lines(const std::string& filename) {
    std::ifstream file(filename);
    std::string line;
    while (std::getline(file, line)) {
        ProcessedLine processed{
            line,
            to_uppercase(line)
        };
        co_yield processed;
    }
}

int main() {
    const std::string temp_filename = "temp_file.txt";
    {
        std::ofstream temp_file(temp_filename);
        temp_file << "Line 1: Hello, World!\n";
        temp_file << "Line 2: This is a test.\n";
        temp_file << "Line 3: C++ coroutines are cool!\n";
    }

    for (const auto& processed : read_and_process_lines(temp_filename)) {
        std::cout << "Original : " << processed.original << '\n';
        std::cout << "Uppercase: " << processed.uppercase << '\n';
    }
}

See @Compiler Explorer

5. `std::mdspan`

The std::mdspan feature is useful for multidimensional data. Here’s an example of matrix multiplication.

#include <https://raw.githubusercontent.com/kokkos/mdspan/single-header/mdspan.hpp>
#include <vector>
#include <print>

void multiply_matrices(std::mdspan<int, std::dextents<size_t, 2>> A,
                       std::mdspan<int, std::dextents<size_t, 2>> B,
                       std::mdspan<int, std::dextents<size_t, 2>> C) {
    for (size_t i = 0; i < A.extent(0); ++i) {
        for (size_t j = 0; j < B.extent(1); ++j) {
            C[i, j] = 0;
            std::print("{}{}: ", i, j);
            for (size_t k = 0; k < A.extent(1); ++k) {
                std::print("+= {} x {}, ", A[i, k], B[k, j]);
                C[i, j] += A[i, k] * B[k, j];
            }
            std::println();
        }
    }
}

int main() {
    std::vector<int> A_data = {1, 2, 3, 4, 5, 6};
    std::vector<int> B_data = {1, 2, 3, 4, 5, 6};
    std::vector<int> C_data(4);

    auto A = std::mdspan(A_data.data(), 2, 3);
    auto B = std::mdspan(B_data.data(), 3, 2);
    auto C = std::mdspan(C_data.data(), 2, 2);

    multiply_matrices(A, B, C);

    for (size_t i = 0; i < C.extent(0); ++i) {
        for (size_t j = 0; j < C.extent(1); ++j) {
            std::print("{} ", C[i, j]);
        }
        std::println();
    }
}

Experiment @Compiler Explorer

6. `cartesian_product`, `enumerate`, and `zip`

#include <ranges>
#include <print>
#include <vector>

int main() {
    std::vector<int> range1 = {1, 2, 3};
    std::vector<char> range2 = {'A', 'B', 'C'};

    auto product = std::views::cartesian_product(range1, range2);
    std::println("Cartesian Product:");
    for (const auto& [a, b] : product)
        std::println("({}, {})", a, b);

    auto enumerated = std::views::enumerate(range1);
    std::println("\nEnumerated Range:");
    for (const auto& [index, value] : enumerated)
        std::println("Index: {}, Value: {}", index, value);

    auto zipped = std::views::zip(range1, range2);
    std::println("\nZipped Ranges:");
    for (const auto& [a, b] : zipped)
        std::println("({}, {})", a, b);
}

Run @Compiler Explorer

7. `chunk`, `slide` and `stride`

#include <ranges>
#include <print>
#include <vector>

int main() {
    std::vector<int> numbers = {1, 2, 3, 4, 5, 6, 7, 8, 9};

    // Chunk: divide the range into groups of 3
    auto chunks = std::views::chunk(numbers, 3);
    std::println("Chunks of 3:");
    for (const auto& chunk : chunks) {
        for (int n : chunk)
            std::print("{} ", n);
        std::println("");
    }

    // Slide: create overlapping subranges of size 3
    auto sliding = std::views::slide(numbers, 3);
    std::println("\nSliding Window of 3:");
    for (const auto& window : sliding) {
        for (int n : window)
            std::print("{} ", n);
        std::println(""); 
    }

    // Stride: skip every 2 elements
    auto strided = std::views::stride(numbers, 2);
    std::println("\nStrided Range (step 2):");
    for (int n : strided)
        std::print("{} ", n);
}

Run @Compiler Explorer

8. `chunk`, `enumerate` and `fold_left`

Here’s an example where we process simple strings into words and sentences and provide simple stats:

Core part:

std::vector<std::string> split_into_words(const std::string& text) {
    return text 
        | std::views::split(' ')
        | std::views::transform([](auto&& word_range) {
            return std::string(word_range.begin(), word_range.end());
        })
        | std::ranges::to<std::vector>();
}

void analyze_sentence(size_t index, const std::vector<std::string>& words) {
    const size_t word_count = words.size();

    const double total_word_length = std::ranges::fold_left(words, 0.0, 
        [](double sum, const std::string& word) {
            return sum + word.size();
        });

    const double avg_word_length 
      = word_count > 0 ? total_word_length / word_count : 0.0;
    const auto longest_word = std::ranges::max(words, {}, &std::string::size);

    std::cout << "Sentence " << index + 1 << ": ";
    for (const auto& w : words)
        std::cout << w << ' ';
    std::cout << "\n  Word count: " << word_count << "\n";
    std::cout << "  Average word length: " << avg_word_length << "\n";
    std::cout << "  Longest word: " << longest_word << "\n";
}

int main() {
    std::string text = "This is the first sentence. Here is another one! "
                       "And yet another sentence. It is fun to analyze text.";

    auto all_words = split_into_words(text);
    auto sentences = all_words | std::views::chunk_by(
        [](const std::string& a, const std::string& b) {
        return !a.ends_with('.') && !a.ends_with('!');
    });

    for (const auto& [index, sentence] : std::views::enumerate(sentences)) {
        auto words = sentence | std::ranges::to<std::vector>();
        if (!words.empty() && (words.back().ends_with('.') || words.back().ends_with('!'))) {
            words.back().pop_back();  // Remove the sentence-ending punctuation for analysis
        }
        analyze_sentence(index, words);
    }
}

Run @Compiler Explorer

Back to you

Do you have some other cool examples with C++23 features?
Have you played with C++23 library features?
What are the most important features for you in this release?

Share your comments below

C++23 Library Features and Reference Cards

Sun, 08 Dec 2024 00:00:00 +0000

In this blog post, you’ll see all C++23 library features! Each with a short description and additional code example.

Prepare for a ride!

For language features please see the previous article: C++23 Language Features and Reference Cards - C++ Stories

Want your own copy to print?

If you like, I prepared PDF I packed both language and the Standard Library features. Each one has a short description and an example if possible.

All of the existing subscribers of my mailing list have already got the new document, so If you want to download it just subscribe here:

Download a free copy of C++23 Ref Card!

Please notice that along with the new ref card you’ll also get C++20 and C++17 language reference card that I initially published three years ago. With this “package” you’ll quickly learn about all of the latest parts that Modern C++ acquired over the last few years.

New Headers

expected
flat_map, flat_set
generator
mdspan
print
spanstream
stacktrace
stdfloat

Stacktrace Library

P0881

Based on Boost.Stacktrace allows for more context when debugging code. The library defines components to store the stacktrace of the current thread of execution and query information about the stored stacktrace at runtime.

#include <iostream>
#include <stacktrace>

void foo() {
    std::cout << std::stacktrace::current();
}

int main() {
    [] {
  foo();
    }();
}

Experiment @Compiler Explorer, note that GCC 14 requires to be linked with -lstdc++exp.

Possible output on GCC:

   0# foo() at /app/example.cpp:5
   1# operator() at /app/example.cpp:10
   2# main at /app/example.cpp:11
   3#      at :0
   4# __libc_start_main at :0
   5# _start at :0

`is_scoped_enum` & `to_underlying`

P1048R1 (is_scoped_enum), P1682R3 (to_underlying)

#include <print>
#include <utility>  // for std::to_underlying

enum class Color: uint8_t { Red = 1, Green = 2,  Blue = 3 };
enum Col { Red, Green, Blue };

int main() {
    Color c = Color::Blue;

    // Pre-C++23: verbose casting
    auto old_way = static_cast<std::underlying_type_t<Color>>(c);

    // C++23: clean and simple
    auto new_way = std::to_underlying(c);

    std::println("Color value: {}", new_way);
    std::println("is_scoped_enum Color {}", std::is_scoped_enum_v<Color>);
    std::println("is_scoped_enum Col   {}", std::is_scoped_enum_v<Col>);
}

Run @Compiler Explorer

The trait (underlying_type) has been available since C++11.

Bonus: First notes and ideas about to_underlying appeared in Scott Meyers’ book: Effective Modern C++: 42 Specific Ways to Improve Your Use of C++11 and C++14 @Amazon.

`std::string`/`std::string_view` Improvements

contains(char/string_view/const char*) - member function
Prohibiting std::basic_string and std::basic_string_view construction from nullptr
Range constructor for std::basic_string_view - P1989
string::resize_and_overwrite() - P1072R10 - Allows us to initialize/resize strings without clearing the buffer but filling bytes with some user operation.

See the example below:

#include <string>
#include <string_view>
#include <vector>
#include <print>

int main() {
    // 1. contains() example
    std::string str = "Hello C++23 World";

    std::println("{}", str.contains("C++23"));  
    std::println("{}", str.contains('X')); 

    // 2. resize_and_overwrite() example
    std::string numbers {"xyz"};
    numbers.resize_and_overwrite(5, [](char* buf, std::size_t n) {
        for (std::size_t i = 1; i < n; ++i) {
            buf[i] = '0' + i; 
        }
        return n; 
    });
    std::println("Numbers: {}", numbers);

    // 3. string_view range constructor
    std::vector<char> chars = {'H', 'e', 'l', 'l', 'o'};
    std::string_view sv(chars.begin(), chars.end());
    std::println("View: {}", sv);
}

Play @Compiler Explorer

`std::out_ptr()`, `std::inout_ptr()`,

P1132

Functions that wrap a smart pointer into a special type allowing to pass to low-level functions that require pointer-to-pointer parameters.

void lowLevel(int** pp) { if (pp) *pp = new int{42}; }
auto ptr = std::make_unique<int>(10);
lowLevel(std::inout_ptr(ptr));

Handy for interaction with C-style API like WindowsAPI, DirectX, Media libraries. inout_ptr additionally calls .release() on the pointer.

#include <memory>
#include <utility>  // for out_ptr/inout_ptr
#include <print>

// Simulate C-style API functions
void AllocateResource(int** pp) {
    *pp = new int{42};
}

void ModifyResource(int** pp) {
    if (*pp) {
        **pp = 100; 
    }
}

void CleanupResource(int* p) {
    delete p;
}

int main() {
    // Example 1: out_ptr for new allocation
    std::unique_ptr<int, decltype(&CleanupResource)> resource1(nullptr, CleanupResource);
    AllocateResource(std::out_ptr(resource1));
    std::println("Resource1 value: {}", *resource1); 

    // Example 2: inout_ptr for modifying existing resource
    std::unique_ptr<int> resource2 = std::make_unique<int>(42);
    std::println("Resource2 before: {}", *resource2);
    ModifyResource(std::inout_ptr(resource2));
    std::println("Resource2 after: {}", *resource2); 
}

Play @Compiler Explorer

See another example: https://godbolt.org/z/KerKM7fM8

`ranges::to<>`

P1206

A way to build containers from a view:

auto v = iota('a') | take(10);
auto vec = v | std::ranges::to<std::vector>();
auto str = v | std::ranges::to<std::string>();

When a container has a reserve() function, ranges:to will also try to use it to make creation more optimal.

#include <algorithm>
#include <concepts>
#include <iostream>
#include <ranges>
#include <vector>

int main()
{
    using namespace std::views;
    auto v = iota('a') | take(10);
    // new in C++23
    auto vec = v | std::ranges::to<std::vector>();
    for (auto x : vec) 
        std::cout << x;

    auto vec2 = vec | std::views::reverse | std::ranges::to<std::vector>();
    for (auto x : vec2) 
        std::cout << x;
}

Run @Compiler Explorer

Ranges Algorithms

ranges::starts_with() and ranges::ends_with()
ranges::iota(), ranges::shift_left/right()
ranges::find_last(), find_last_if(), find_last_if_not()
ranges::contains() and ranges::contains_subrange()
Ranges fold algorithms: ranges::fold_*()

See a simple example with fold_left and contains:

#include <ranges>
#include <print>
#include <vector>
#include <numeric> // For std::accumulate (used for comparison)
#include <algorithm> // new fold_left, ends_with

int main() {
    std::vector<int> numbers = {1, 2, 3, 4, 5};

    auto sum = std::ranges::fold_left(numbers, 0, [](int acc, int n) {
        return acc + n;
    });
    std::println("Sum of numbers: {}", sum);

    auto std_sum = std::accumulate(numbers.begin(), numbers.end(), 0);
    std::println("Sum using std::accumulate: {}", std_sum);

    bool contains_four = std::ranges::contains(numbers, 4);
    std::println("Does the range contains 4? {}", contains_four);
}

Run @Compiler Explorer

Views Additions

cartesian_product
repeat
enumerate
adjacent, adjacent_transform
stride
slide
chunk, chunk_by
join_with
zip, zip_transform
as_rvalue, as_const

For zip you can see my other article: Combining Collections with Zip in C++23 for Efficient Data Processing - C++ Stories.

And for more examples see my free article @Patreon: Seven more C++23 Library Examples | Patreon.

Heterogeneous Erasure for Associative Containers

P2077

Continuation of the work for heterogeneous operations. This time you can use transparent comparators for erase() and extract() member functions. To be backward compatible, the comparators cannot be convertible to iterator or const_iterator of a given container.

Read more about heterogeneous access in my other article: C++20: Heterogeneous Lookup in (Un)ordered Containers - C++ Stories

Monadic Operations for `std::optional`

P0798

New member functions for optional: and_then, transform, and or_else.

auto ret = userName.transform(toUpper)
.and_then([](string x) { return make_optional(x + "OK"); })
.or_else([] { return make_optional(string{"no user"}); });

For example:

#include <optional>
#include <print>
#include <algorithm>
#include <ranges>

void test(const std::optional<std::string>& userName) {
    auto up = [](std::string x) { 
        std::ranges::transform(x, x.begin(), ::toupper);
        return x;
    };

    auto ret = userName.transform(up)
       .and_then([](std::string x) { 
           std::println("x: {}", x);
           return std::optional<int>(x.size()); 
        })
        .or_else([]{ 
            std::println("empty...");
            return std::optional<int>{0}; 
        });
    std::println("ret is {}", *ret);
}

int main() {
    test(std::nullopt);
    test("john");
}

See @Compiler Explorer

output:

empty...
ret is 0
x: JOHN
ret is 4

See my full article about this extension here: How to Use Monadic Operations for `std::optional` in C++23 - C++ Stories

`<expected>` and Its Monadic Operations

P0323

A vocabulary type that allows storing either of two values: T or unexpected (in a form of some error type). It’s something between std::optional and std::variant.

enum class FuelErr { DistLarge, Neg };
std::expected<double, FuelErr> calcFuel(int dst) {
  if (dst < 0) return std::unexpected(FuelErr::Neg);
  return distance * 1.333;
}

C++23 also adds monadic operations for this type, so it’s consistent with operations for std::optional.

See my mini-series about this type:

`constexpr std::unique_ptr`

P2273

The new() operator can be used in constexpr context since C++20, and now you can wrap it also in a unique_ptr.

#include <numeric>
#include <memory>

constexpr int naiveSum(unsigned int n) {  
    auto p = std::make_unique<int[]>(n);  
    std::iota(p.get(), p.get()+n, 1);
    auto tmp = std::accumulate(p.get(), p.get()+n, 0);
    return tmp;
}

constexpr int smartSum(unsigned int n) {
    return (n*(n+1))/2;
}

int main() {
    static_assert(naiveSum(10) == smartSum(10));
    static_assert(naiveSum(11) == smartSum(11));
}

See @Compiler Explorer

`std::mdspan` Multidimensional Span, P0009

A generalization over std::span for multiple dimensions. Supports dynamic as well as static extents (compile-time constants). It also supports various mappings like column-major order, row-major, or even stride access.

#include <vector>
#include <https://raw.githubusercontent.com/kokkos/mdspan/single-header/mdspan.hpp>
#include <algorithm>
#include <iostream>

bool isSymmetric(std::mdspan<int, std::dextents<size_t, 2>> matrix) {
    const auto rows = matrix.extent(0);
    const auto cols = matrix.extent(1);

    if (rows != cols) return false;

    for (size_t i = 0uz; i < rows; ++i) {
        for (size_t j = i + 1; j < cols; ++j) {
            if (matrix[i, j] != matrix[j, i]) return false;
        }
    }
    return true;
}

int main() {
    std::vector<int> matrix_data = {1, 2, 3, 2, 4, 5, 3, 5, 6};
    auto matrix = std::mdspan(matrix_data.data(), 3, 3);

    std::cout << isSymmetric(matrix) << std::endl;
}

See at Compiler Explorer

And see my two bonus articles, available for Patreons, about this type:

`<flat_map>` and `<flat_set>`

P0429 and P1222

Drop-in replacement for maps and sets with better performance characteristics. It gives faster lookup, faster iteration, random access iteration, less memory, and better cache efficiency. But iterators might be invalidated, and insertion is slower than the tree approach. The container is actually a container adaptor with proxy iterators.

Here’s a reference implementation: https://github.com/tzlaine/flat_map/blob/master/implementation/flat_map

Formatted Output Library `<print>`

P2093

New Hello World Style for C++23!

#include <print>  
#include <string>  

int main() {  
    std::string name = "C++23";  
    int year = 2024;  
    double version = 23.0;  

    // Basic printing  
    std::println("Hello from {}!", name);  

    // Named arguments  
    std::print("Language: {0}, Version: {0}\n", name);  

    // Multiple arguments with formatting  
    std::println("Release year: {:d}, Version: {:.1f}", year, version);  
    // Alignment and width  
    std::println("{:>10}: {:>5}", "Status", "OK");  // right-aligned  
    std::println("{:<10}: {:<5}", "Error", "None"); // left-aligned  

    // Print without newline and then with newline  
    std::print("Loading");  
    std::println("... done!");  
}

See @Compiler Explorer

New functions in the <print> header: std::print, std::println (adds a new line) that uses std::format to output text to stdout. Plus lower-level routines like vprint_unicode with more parameters for output.

`constexpr` `to_chars()`, `from_chars()`

P2291

Integral versions available in constexpr context:

#include <charconv>
#include <optional>
#include <string_view>

constexpr std::optional<int> to_int(std::string_view sv)
{
    int value {};
    const auto ret = std::from_chars(sv.begin(), sv.end(), value);
    if (ret.ec == std::errc{})
        return value;

    return {};
};

int main() {
    static_assert(to_int("hello") == std::nullopt);
    static_assert(to_int("10") == 10);
}

Run @Compiler Explorer

See my article about this new feature and more example: C++ String Conversion: Exploring std::from_chars in C++17 to C++26 - C++ Stories

Standard Library Modules

P2465

C++23 introduces two standard library modules that significantly improve compilation efficiency and code organization:

`import std;`

Imports all C++ standard library components in namespace std
Includes C++ headers and C wrapper headers
Provides ::operator new and related operators
Keeps the global namespace clean
Ideal for new projects and modern C++ code

`import std.compat;`

Provides everything from import std;
Additionally, imports C functions into global namespace
Helps transition legacy code that uses unqualified C functions
Useful when working with platforms where C functions are traditionally global
Recommended for compatibility with existing codebases

Here are two examples:

import std;  

int main() {  
    std::string str = "Hello";  
    std::size_t len = std::strlen(str.c_str());  // must use std::  
    std::println("Length: {}", len);  
}

and the compat version:

import std.compat;  

int main() {  
    std::string str = "Hello";
    size_t len = strlen(str.c_str());  // works: strlen is global  
    printf("Length: %zu\n", len);      // works: printf is global  
}

Unfortunately as of December 2024, no major compiler supports the above examples.

`std::generator` Coroutine Generator

P2502

C++20 introduced coroutines, but the standard library support was minimal, leaving us to implement our own coroutine types or rely on third-party libraries. Synchronous generators are a crucial use case for coroutines, enabling efficient and lazy evaluation of sequences. Writing an efficient recursive generator is non-trivial, and the standard should provide one to simplify this task for us.

std::generator fills this gap by providing a ready-to-use, standardized coroutine type for generating sequences. This makes it significantly easier for us to adopt coroutines in our projects without the overhead of implementing custom coroutine logic.

std::generator is a coroutine-based feature that uses the co_yield keyword to define a sequence of values. Each co_yield call produces a value and suspends execution, allowing us to retrieve the value. When resumed, the coroutine continues from where it left off.

Here’s a simple generator:

std::generator<int> gen(int n) {
    for (int a = 0; a < n; ++a)
        co_yield a;
}

int main() {
    auto g = gen(5);
    auto it = g.begin();
    while (it != g.end())
    {
        std::cout << *it << " ";
        ++it;
    }
}

See @Compiler Explorer

For more examples have a look at my two bonus articles for Patreons:

Explicit Lifetime Management

P2590, P2679R2

From the proposal:

Since C++20, certain functions in the C++ standard library such as malloc, bit_cast, and memcpy are defined to implicitly create objects.

But when you use non standard techniques to obtain a memory for the object you might end up with UB:

C++23 introduces std::start_lifetime_as/start_lifetime_as_array to explicitly start the lifetime of objects in raw memory, enabling well-defined behavior without invoking constructors. This is useful for low-level tasks like custom allocators or deserialization.

For example:

#include <memory>  

struct X { int a, b; };  

void example() {  
    void* rawMemory = My_Malloc(sizeof(X));         // non standard way...
    X* obj = std::start_lifetime_as<X>(rawMemory); // Start lifetime  
    obj->a = 42; obj->b = 84;                      // Use the object  
    std::free(rawMemory);                          // Free memory  
}

(as of December no major compiler implements this functionality)

`<spanstream>` String-Stream with Span Buffers

P0448

New classes: basic_spanbuf, basic_ispanstream, basic_ospanstream, basic_spanstream analogous to existing stream classes but using std::span as the buffer. They allow explicit buffer management and improved performance.

char buffer[64] { 0 };
std::span<char> spanBuffer(buffer);
std::basic_ospanstream<char> outputStream(spanBuffer);
outputStream << "Hello, " << "C++23!";

`std::format` Improvements

Compile-time string parsing - P2216R3
Formatting Ranges - P2286R8
Improve default container formatting - P2585R1
Formatting std::thread::id and std::stacktrace - P2693R1
Add support for non-const-formattable types to std::format - P2418R2

`std::visit()` for classes derived from `std::variant`

P2162

Just see the example:

#include <iostream>
#include <variant>
#include <memory>

struct Connecting {};
struct Connected {};

struct State : std::variant<Connecting, Connected> {
    using std::variant<Connecting, Connected>::variant;

    bool is_connected() const {
        return std::holds_alternative<Connected>(*this);
    }
};

struct StateVisitor {
    void operator()(Connecting) const {
        std::cout << "State: Connecting\n";
    }
    void operator()(Connected) const {
        std::cout << "State: Connected\n";
    }
};

int main() {
    State state = Connecting{};
    std::visit(StateVisitor{}, state);

    if (state.is_connected()) 
        std::cout << "The system is connected.\n";
    else
        std::cout << "The system is not connected.\n";
}

Run @Compiler Explorer

The class State inherits from std::variant and before this proposal there was no way to use std::visit on such a class. Right now it’s possible and allows us to reduce some extra code.

This fix can be implemented against C++17.

`std::unreachable()`

P0627

The proposal introduces a new function, std::unreachable, to the C++ standard library. This function is designed to mark code locations that the programmer knows are unreachable, allowing the compiler to optimize by eliminating unnecessary checks. This function might be particularly useful for you in scenarios where the compiler cannot deduce that certain code paths are impossible, such as in fully covered switch statements or functions that never return.

The functionality also standardizes existing vendor-specific approaches like __builtin_unreachable() in GCC and Clang, or __assume(false) in MSVC.

Deprecating `std::aligned_storage` and `aligned_union`

P1413

The accepted proposal deprecates C++11’s std::aligned_storage and std::aligned_union in the C++ standard due to their poor API design, undefined behavior risks, and limited utility. These helper types require error-prone usage patterns like reinterpret_cast, lack proper size guarantees, and have inconsistent or confusing APIs, making them unsuitable for modern C++ practices.

The authors analysed libraries like Boost, Folly, and Abseil and most use cases for std::aligned_ involve repetitive patterns to deduce size and alignment manually. This can be replaced with simpler alternatives like alignas and std::byte arrays.

// deprecated:
template <typename T>  
class MyContainer {  
private:  
    std::aligned_storage_t<sizeof(T), alignof(T)> buffer;  
};  

// better:
template <typename T>  
class MyContainer {  
private:  
    alignas(T) std::byte buffer[sizeof(T)];  
};

These utilities are still available in the standard but are marked as discouraged for use. They may be removed in a future version of the standard (e.g., C++26 or later).

For example clang reports warning: 'aligned_storage_t' is deprecated when compiling in C++23 mode.

Pipe support for user-defined range adaptors,

P2387

The proposal introduces a standardized mechanism in C++23 to enable user-defined range adaptors to interoperate seamlessly with the standard library’s range adaptors and other user-defined adaptors. This is achieved by providing a base class, std::ranges::range_adaptor_closure, which simplifies the creation of custom range adaptors that support the pipe (|) operator. Additionally, the proposal introduces std::bind_back, a utility for partial function application.

#include <ranges>
#include <vector>
#include <iostream>

// Custom range adaptor closure
struct double_values_closure 
: std::ranges::range_adaptor_closure<double_values_closure> {
    template <std::ranges::viewable_range R>
    constexpr auto operator()(R&& range) const {
        return std::ranges::transform_view(std::forward<R>(range), [](auto x) { return x * 2; });
    }
};

inline constexpr double_values_closure double_values{};

int main() {
    std::vector<int> numbers = {1, 2, 3, 4, 5};
    auto doubled = numbers | double_values;

    for (int n : doubled)
        std::cout << n << " ";
}

See @Compiler Explorer

`std::move_only_function`

P0288 and more story in early proposal: N4543 (PDF)

The std::move_only_function is a move-only equivalent of std::function. It provides a type-erased wrapper for callable objects, supporting cv/ref/noexcept-qualified function types while avoiding the const-correctness issues of std::function. Unlike std::function, it does not support target or target_type accessors, ensuring a simpler and more efficient design. This feature is particularly useful for scenarios where callable objects need to be moved rather than copied, such as in asynchronous programming or resource-constrained environments. The proposal encourages small-object optimization to avoid dynamic memory allocation for small callable objects.

Books on C++23

Although the standard is fresh, there are several good books focusing on C++23 worth reading… and probably more to come :)

Title	Author(s)	Description
Modern C++ Programming Cookbook (3rd Edition)	Marius Bancila	Master Modern C++ with comprehensive solutions for C++23 and all previous standards
The C++ Programming Language (4th Edition)	Bjarne Stroustrup	The definitive guide from the creator of C++
Beginning C++23: From Beginner to Pro (7th Edition)	Ivor Horton, Peter Van Weert	A comprehensive guide for learning modern C++ from the ground up
Modern C++ for Absolute Beginners (2nd Edition)	Slobodan Dmitrović	A friendly introduction to C++ programming language and C++11 to C++23 standards
C++23 Best Practices	Jason Turner	Simple rules with specific action items for better C++
Learn C++ by Example	Frances Buontempo	A practical approach to learning C++ versions 11 to 23

Note: Links are affiliate links and may provide the site with a small commission at no extra cost to you.

Summary

I hope we covered most, if not all, C++23 library features!

You can check their implementation status at C++ Reference: https://en.cppreference.com/w/cpp/compiler_support#cpp23

For language features please see the previous article: C++23 Language Features and Reference Cards - C++ Stories

Back to you

Have you played with C++23 library features?
What are the most important features for you in this release?

C++23 Language Features and Reference Cards

Sun, 17 Nov 2024 00:00:00 +0000

C++23 Language Features

In this blog post, you’ll see all C++23 language features! Each with short description and additional code example.

Prepare for a ride!

For library features please see the next article: C++23 Library Features and Reference Cards - C++ Stories

Want your own copy to print?

If you like, I prepared PDF I packed both language and the Standard Library features. Each one has a short description and an example if possible.

All of the existing subscribers of my mailing list have already got the new document, so If you want to download it just subscribe here:

Download a free copy of C++23 Ref Card!

Timeline

Here’s a short overview about the timeline for the new standard.

February 2020 (Prague): Final C++20 meeting where initial plans for C++23 were adopted
- Key planned features: library support for coroutines, modular standard library, executors, and networking
June 2020: First planned WG21 meeting for C++23 in Varna was cancelled due to COVID-19 pandemic
November 2020: New York meeting cancelled, moved to virtual format
- Key additions: size_t literals, stacktrace library, contains() member functions, C atomic interoperability
February 2021: Virtual meeting
- Notable features: Lambda expression simplifications, variant improvements, conditionally borrowed ranges
June 2021: Virtual summer plenary meeting
- Major additions: if consteval, spanstream, out_ptr/inout_ptr, constexpr improvements
October 2021: Virtual autumn plenary meeting
- Significant features: Explicit this parameter, multidimensional subscript operator, zip ranges
February 2022: Virtual meeting
- Key additions: std::expected, ranges::to, windowing range adaptors
July 2022: Virtual meeting
- Major features: static operator(), UTF-8 source files, std::mdspan, flat containers
November 2022: First hybrid meeting
- Notable additions: Static operator[], lifetime extensions for range-based for loops
February 2023: Final hybrid meeting in Issaquah
- Technical content finalized
- Last features added: views::enumerate, formatting improvements, std::barrier guarantees
October 2024: Published with a long delay at ISO: as ISO/IEC 14882:2024 - Programming languages — C++

Now let’s jump into the features:

`if consteval { }`

P1938

The new syntax is equivalent to the following code from C++20:

if (std::is_constant_evaluated()) { }

It’s a language feature now, so no need for a separate header. It can call consteval functions and it is easier to understand.

#include <iostream>
 
constexpr int run(int i) {
    if consteval {
        return i*2;
    }
    else {
        return i;
    }
}
 
int main() {
    static_assert(run(10) == 20); // compile-time
    int a = 10;
    std::cout << run(a); // run-time
}

Run @Compiler Explorer

Deducing `this`

P0847

A way to explicitly pass the this parameter into a member function, allowing for more control and reduces code duplication. You can pass this by value, call lambdas recursively, simplify the CRTP pattern, and more.

struct Pattern {
  template <typename Self> void foo(this Self&& self) { self.fooImpl(); }
};
struct MyClass : Pattern { void fooImpl() { ... } };

See my examples at this Patreon post: C++23: Deducing This, a few examples (Available for all patrons, even at the free tier).

`auto(x)` and `auto{x}`

P0849

Replaces decay_copy (an internal library helper) with a language feature. Allows creating a decay rvalue copy of the input object.

void pop_front_alike(Container auto& x) {
    using T = std::decay_t<decltype(x.front())>;
    std::erase(x.begin(), x.end(), T(x.front()));
}
// becomes:
std::erase(x.begin(), x.end(), auto(x.front()));

const std::string& str = "hello";
auto(str);  // Creates a new std::string (decayed copy)

int arr[] = {1, 2, 3};
auto(arr);  // Creates int* (arrays decay to pointers)

Extend init-statement to allow alias-declaration

P2360

In C++20, using wasn’t allowed in the for loop, now it’s possible:

for (using T = int; T e : container) { ... }

Multidimensional Subscript Operator

P2128

Change the rules to allow multiple parameters for operator[]:

#include <vector>
 
template <typename T>
class Array2D {
    std::vector<T> m;
    size_t w, h;

public:
    Array2D(size_t width, size_t height)
        : m(width * height), w(width), h(height) {}

    T& operator[](size_t i, size_t j) { return m[i + j * w]; }
};

int main() {
    Array2D<float> arr(4, 4);
    arr[1, 2] = 0.0f;
}

Run @Compiler Explorer

This is crucial for types like std::mdspan.

`static operator()` and `static operator []`

P1169R4 and P2589R1

Allows for more optimization in the compiler. The call operator is especially handy for captureless lambdas. The compiler can optimize away passing the “this” pointer to the call. You can specify a lambda to be static.

struct Fn {
    constexpr static int operator()(int x) {
        return x*10;
    }
};

int main() {
    static_assert(Fn::operator()(10) == 100);
    Fn x;
    static_assert(x(10) == 100);
}

Run @Compiler Explorer

Features for Lambdas

Attributes on lambdas
() is more optional - P1102
The call operator can be static
Deducing this adds new capabilities like better recursion
Change scope of lambda trailing-return-type

Examples:

int main() {
    auto identity = [](int x) static { return x; };
    return identity(100);
}

Run @Compiler Explorer

Recursive lambda:

int main() {
    auto fib = [](this auto self, int n) {
       if (n < 2) return n;
       return self(n-1) + self(n-2);
    };
    static_assert(fib(7) == 13);
}

Run @Compiler Explorer

Optional ():

#include <iostream>
int main() {
    auto fn = [x = 0] mutable {
        return x++;
    };
    std::cout << fn() << fn();
}

Run @Compiler Explorer

[[assume]] New Attribute

P1774

[[assume]] specifies that an expression will always evaluate to true at a given point. It standardizes the existing vendor-specific semantics like __builtin_assume (Clang) and __assume (MSVC, ICC). Offers potential optimization opportunities for compilers. If the assumption turns out to be false during runtime, the behavior is undefined, making it crucial to use this attribute only when absolutely certain about the condition.

The attribute can help compilers eliminate unnecessary bounds checking, enable better loop optimizations, and remove redundant error handling paths.

For example:

// Basic usage - compiler can optimize sqrt calculation
void process_positive(double x) {
    [[assume(x >= 0)]];
    return std::sqrt(x); // No need for negative number checks
}

// Loop optimization example
void process_array(int* arr, size_t size) {
    [[assume(size % 4 == 0)]];  // Assume size is multiple of 4
    [[assume(size > 0)]];       // Assume non-empty array
    for (size_t i = 0; i < size; i += 4) {
        // Compiler can optimize for 4-element chunks
        // No need for remainder handling
        arr[i] = arr[i] * 2;
        arr[i + 1] = arr[i + 1] * 2;
        arr[i + 2] = arr[i + 2] * 2;
        arr[i + 3] = arr[i + 3] * 2;
    }
}

Constexpr Updates

P2448, P2647, and P2242

Relax rules for constructors and return types for constexpr functions, making them almost identical to regular functions.
Permitting static constexpr variables in constexpr functions.

Extend Lifetime of Temporaries in Range-Based For

P2718

This is a very popular and long-standing proposal that generated significant discussion in the C++ community. While initially aimed at addressing all temporary lifetime issues, it was ultimately restricted to focus on range-based for loops, where the problem was most acute and the solution most clear-cut.

The change extends the lifetime of temporary objects in the for-range-initializer until the end of the loop. This fixes a common source of undefined behavior that many developers encountered, especially when working with chain calls or complex expressions.

Before C++23:

// Undefined Behavior in C++20 and earlier:
std::vector<std::vector<int>> getVector();
for (auto e : getVector()[0]) {  // temporary vector destroyed here!
    std::cout << e << '\n';      // accessing destroyed object
}

// Workaround required in C++20:
auto temp = getVector();
for (auto e : temp[0]) {
    std::cout << e << '\n';
}

C++23 makes the first version safe and equivalent to the workaround:

// Now valid in C++23:
for (auto e : getVector()[0]) {  // temporary vector lives through the loop
    std::cout << e << '\n';
}

// More complex example that's now safe:
struct Matrix {
    auto getRow(int i) const { return /* ... */; }
};
std::vector<Matrix> matrices;

for (auto val : matrices.back().getRow(42)) {
    // Both the temporary from back() and getRow() are preserved
    process(val);
}

This change significantly improves the safety and usability of range-based for loops, eliminating a common class of bugs while maintaining the expressive power of the syntax. However, it’s important to note that this fix is specific to range-based for loops and doesn’t address temporary lifetime issues in other contexts.

New Preprocessor Directives

P2334 and P2437

C++23 introduces two sets of new preprocessor directives that improve code readability and maintain compatibility with C23. The #elifdef and #elifndef directives simplify conditional compilation chains, while #warning provides a standardized way to emit compiler warnings.

These additions reduce verbosity and make the code more maintainable compared to traditional preprocessor constructs.

// Old style:
#ifdef _WIN32
    #define PLATFORM "Windows"
#else
    #ifdef __linux__
        #define PLATFORM "Linux"
    #else
        #ifdef __APPLE__
            #define PLATFORM "macOS"
        #endif
    #endif
#endif

// New style in C++23:
#ifdef _WIN32
    #define PLATFORM "Windows"
#elifdef __linux__
    #define PLATFORM "Linux"
#elifdef __APPLE__
    #define PLATFORM "macOS"
#else
    #warning "Unknown platform detected!"
#endif

The new directives are particularly useful in cross-platform code and library compatibility checks, making conditional compilation more straightforward and easier to maintain.

Literal Suffix for (Signed) size_t – uz, UZ

P0330

A simple yet effective fix for various numerical conversions between integer types, unsigned, and size_t which is commonly returned from std:: containers. The feature introduces three new literal suffixes:

uz or UZ for size_t
z or Z for the signed counterpart (ptrdiff_t or equivalent)

This addition eliminates common warnings and potential bugs related to signed/unsigned mismatches, particularly in loop counters and container operations.

// Before C++23 - potential warnings or issues:
for (int i = 0; i < vec.size(); ++i)  // warning: comparison between signed/unsigned
for (size_t i = 0; i < vec.size(); ++i)  // verbose

// C++23 - clean and portable:
for (auto i = 0uz; i < vec.size(); ++i)  // perfect match with container's size_t
    std::cout << i << ": " << vec[i] << '\n';

The feature is particularly valuable for writing portable code that needs to work correctly across different platforms and architectures, automatically adapting to the platform’s size_t width without explicit type specifications.

CTAD from inherited constructors

P2582R1

C++23 extends Class Template Argument Deduction to work with inherited constructors, filling an important gap in CTAD functionality. This allows template arguments to be deduced when using inherited constructors through using declarations.

// Before C++23 - CTAD didn't work with inherited constructors
template<typename T>
struct Base {
    Base(T value) {}
};

template<typename T>
struct Derived : Base<T> {
    using Base<T>::Base;  // Inherit constructor
};

Derived d(42);  // Error in C++20: couldn't deduce T
                // OK in C++23: deduces Derived<int>

Run @Compiler Explorer

This feature makes template class hierarchies more intuitive to use and eliminates the need for explicit template arguments when using inherited constructors, improving code readability and maintainability.

Simpler implicit move

P2266

C++23 simplifies and fixes the implicit move rules introduced in C++20, making them more consistent and easier to implement. The new rules state that a move-eligible id-expression is always treated as an xvalue, eliminating the previous two-step overload resolution process.

// Before C++23 - inconsistent behavior
Widget&& foo(Widget&& w) {
    return w;  // Error in C++20
};

See @Compiler Explorer

DRs

Here’s the formatted list of C++23 DRs:

These Defect Reports (DRs) are actually retroactive fixes that apply to C++20 as well as C++23:

Lambda Trailing-Return-Type Scope Change (P2036R3, P2579R0)
- Changes the scope rules for lambda expression trailing return types
Meaningful Exports (P2615R1)
- Improves the semantics of export declarations in modules
Consteval Propagation (P2564R0)
- Fixes issues with consteval propagation in the language
Unicode Identifier Syntax (P1949R7)
- Updates C++ identifier syntax to align with Unicode Standard Annex 31
Duplicate Attributes (P2156R1)
- Allows multiple instances of the same attribute in declarations
Concepts Feature Test Macro (P2493R0)
- Adjusts the value of __cpp_concepts feature test macro
Wchar_t Requirements (P2460R2)
- Relaxes requirements on wchar_t to match existing implementation practices
Unknown Pointers in Constant Expressions (P2280R4)
- Allows using unknown pointers and references in constant expressions
Equality Operator Enhancement (P2468R2)
- Improves equality operator behavior
char8_t Compatibility (P2513R4)
- Enhances char8_t compatibility and portability
Diagnostic Directives Clarification (CWG2518)
- Clarifies reporting of diagnostic directives and static_assert behavior in templates

Books on C++23

Although the standard is fresh, there are several good books focusing on C++23 worth reading… and probably more to come :)

Title	Author(s)	Description
Modern C++ Programming Cookbook (3rd Edition)	Marius Bancila	Master Modern C++ with comprehensive solutions for C++23 and all previous standards
The C++ Programming Language (4th Edition)	Bjarne Stroustrup	The definitive guide from the creator of C++
Beginning C++23: From Beginner to Pro (7th Edition)	Ivor Horton, Peter Van Weert	A comprehensive guide for learning modern C++ from the ground up
Modern C++ for Absolute Beginners (2nd Edition)	Slobodan Dmitrović	A friendly introduction to C++ programming language and C++11 to C++23 standards
C++23 Best Practices	Jason Turner	Simple rules with specific action items for better C++
Learn C++ by Example	Frances Buontempo	A practical approach to learning C++ versions 11 to 23

Note: Links are affiliate links and may provide the site with a small commission at no extra cost to you.

Summary

I hope we covered most if not all C++23 language features!

You can check their implementation status at C++ Reference: https://en.cppreference.com/w/cpp/compiler_support#cpp23

And next time we’ll handle Standard Library changes - see it here: C++23 Library Features and Reference Cards - C++ Stories

Back to you

Have you played with C++23?
What are the most important features for you in this release?

Around the World in C++: Exploring Time Zones with std::chrono

Sun, 10 Nov 2024 00:00:00 +0000

While most time zones use simple hour offsets from UTC, some regions have chosen unusual time differences. In this blog post, we’ll explore how we can discover such zones using C++20’s chrono library.

We’ll use GCC 14.2 as it fully supports C++20 chrono and also std::print from C++23.

First Attempt: Basic Zone Iteration

C++20 introduced comprehensive time zone support through the <chrono> library. The implementation relies on the IANA Time Zone Database (also known as the “tz database” or “zoneinfo”), which is the de facto standard for time zone information used by most operating systems and programming languages.

The Time Zone Database

In C++20, the time zone database is represented by the tzdb class:

namespace std::chrono {
    struct tzdb {
        string version;           // IANA time zone database version
        vector<time_zone> zones;  // Time zone container
        vector<time_zone_link> links; // Alternative names
        vector<leap_second> leap_seconds; // Leap second info
    };
}

Essential aspects of the database:

It’s read-only - users cannot construct or modify a tzdb
Access is provided through two global functions:
- get_tzdb(): Returns the current database
- get_tzdb_list(): Returns the list of available database versions

Here’s how to access the database:

#include <chrono>
#include <print>
#include <cassert>

int main() {
    const auto& db = std::chrono::get_tzdb();
    std::print("Current DB version: {}\n", db.version);
    
    const auto& db_list = std::chrono::get_tzdb_list();
    auto size = std::distance(db_list.begin(), db_list.end());
    std::print("Available versions: {}\n", size);
    
    assert(&db_list.front() == &db);
}

See at Compiler Explorer

The database provides several key components:

Zones (db.zones):

Primary container of time zone information
Each zone contains complete historical and current rules
Examples: “America/New_York”, “Europe/London”

Links (db.links):

Alternative names for existing zones
Useful for backward compatibility
Example: “EST” links to “America/New_York”

Leap Seconds (db.leap_seconds):

Records of historical leap second adjustments
Used for precise astronomical calculations

Let’s start with a simple approach to explore the zones:

#include <chrono>
#include <print>

int main() {
    const auto& db = std::chrono::get_tzdb();
    auto now = std::chrono::system_clock::now();
    
    std::print("Time Zone Database Version: {}\n", db.version);
    std::print("Number of zones: {}\n", db.zones.size());
    std::print("Number of links: {}\n", db.links.size());
    std::print("Number of leap seconds: {}\n\n", db.leap_seconds.size());
    
    for (const auto& zone : db.zones) {
        try {
            auto info = zone.get_info(now);
            std::print("{:<30} offset: {:>6}s  abbrev: {:<6}  ", 
                zone.name(), 
                info.offset.count(),
                info.abbrev);
            
            // Show if DST is active
            if (info.save != std::chrono::minutes{0})
                std::print("DST: {} min", info.save.count());
 
            std::print("\n");
        }
        catch (const std::exception& e) {
            std::print("Error with zone {}: {}\n", zone.name(), e.what());
        }
    }
}

See at Compiler Explorer

Sample output:

Time Zone Database Version: 2024a
Number of zones: 447
Number of links: 150
Number of leap seconds: 27

Africa/Abidjan                 offset:      0s  abbrev: GMT     
Africa/Accra                   offset:      0s  abbrev: GMT     
Africa/Addis_Ababa             offset:  10800s  abbrev: EAT        
...

While this basic iteration gives us a good overview, the raw offset numbers make it difficult to spot unusual cases. Some zones use fractional hours, and others have unique DST rules. Let’s improve our search to find these special cases.

Finding Unusual Offsets

What makes a time zone unusual? Usually, it’s:

Non-hour offsets (like UTC+5:45)
Unusual DST rules (like 30-minute DST changes)

Here’s our improved code:

#include <chrono>
#include <print>
#include <vector>
#include <set>

void find_unusual_zones() {
    const auto& db = std::chrono::get_tzdb();
    auto now = std::chrono::system_clock::now();
    
    // Track unique unusual offsets
    std::set<std::chrono::seconds> unusual_offsets;
    
    for (const auto& zone : db.zones) {
        try {
            auto info = zone.get_info(now);
            
            // Check if offset is not a whole hour
            if (info.offset % std::chrono::hours{1} != std::chrono::seconds{0}) {
                unusual_offsets.insert(info.offset);
                std::print("Found unusual zone: {}\n", zone.name());
                std::print("  Offset: {:+.2f} hours\n", 
                    static_cast<double>(info.offset.count()) / 3600.0);
                std::print("  DST adjustment: {} minutes\n\n", 
                    info.save.count());
            }
        }
        catch (const std::exception&) {
            // Skip problematic zones
        }
    }
    
    std::print("Total unusual offsets found: {}\n", unusual_offsets.size());
}

int main() {
    find_unusual_zones();
}

See @Compiler Explorer

Running this code reveals several interesting time zones:

Found unusual zone: America/St_Johns
  Offset: -3.50 hours
  DST adjustment: 0 minutes

Found unusual zone: Asia/Colombo
  Offset: +5.50 hours
  DST adjustment: 0 minutes

Found unusual zone: Asia/Kabul
  Offset: +4.50 hours
  DST adjustment: 0 minutes

Found unusual zone: Asia/Kathmandu
  Offset: +5.75 hours
  DST adjustment: 0 minutes

...

Total unusual offsets found: 11

Exploring Unusual Zones

Now let’s examine two particularly interesting zones we found using TimeZoneExplorer:

#include <chrono>
#include <print>
#include <string_view>

class TimeZoneExplorer {
public:
    static void explore_zone(std::string_view zone_name) {
        try {
            using namespace std::chrono;
            const auto zone = locate_zone(zone_name);
            auto now = system_clock::now();
            zoned_time zt{zone, now};
            
            auto info = zone->get_info(now);
            
            std::print("\n=== {} ===\n", zone_name);
            std::print("Current time: {:%F %T %Z}\n", zt);
            std::print("Valid period: [{}, {})\n", info.begin, info.end);
            
            std::print("UTC offset: {}s ({:+.2f} hours)\n", 
                info.offset.count(),
                static_cast<double>(info.offset.count()) / 3600.0);
            
            if (info.save == 0min) {
                std::print("DST: No (Standard Time)\n");
            }
            else {
                std::print("DST: Yes (Saving: {} minutes)\n", info.save.count());
                auto standard_offset = info.offset - info.save;
                std::print("Standard Time offset would be: {}s ({:+.2f} hours)\n",
                    standard_offset.count(),
                    static_cast<double>(standard_offset.count()) / 3600.0);
            }
            
            std::print("Abbreviation: {} (Note: abbreviations are not unique)\n", 
                info.abbrev);
            
        }
        catch (const std::exception& e) {
            std::print("Error exploring {}: {}\n", zone_name, e.what());
        }
    }
};

int main() {
    TimeZoneExplorer::explore_zone("Asia/Kathmandu");
    TimeZoneExplorer::explore_zone("Australia/Lord_Howe");
}

See @Compiler Explorer

Output:

=== Asia/Kathmandu ===
Current time: 2024-11-11 02:00:42.203971363 +0545
Valid period: [1986-01-01 00:00:00, 32767-12-31 00:00:00)
UTC offset: 20700s (+5.75 hours)
DST: No (Standard Time)
Abbreviation: +0545 (Note: abbreviations are not unique)

=== Australia/Lord_Howe ===
Current time: 2024-11-11 07:15:42.209946634 +11
Valid period: [2024-10-05 15:30:00, 2025-04-05 15:00:00)
UTC offset: 39600s (+11.00 hours)
DST: Yes (Saving: 30 minutes)
Standard Time offset would be: 37800s (+10.50 hours)
Abbreviation: +11 (Note: abbreviations are not unique)

Let’s examine these unusual cases:

Nepal (Asia/Kathmandu):
- Uses UTC+5:45
- One of the few zones with a 45-minute offset
- No DST changes
- Offset hasn’t changed since 1986
Lord Howe Island (Australia/Lord_Howe):
- Base offset is UTC+10:30
- Uses 30-minute DST adjustment
- Only place in the world with a 30-minute DST change

Summary

C++20’s <chrono> library and its time zone support make it easy to explore and work with the world’s diverse time zones. While most regions use simple hour offsets from UTC, we discovered several exceptions, like Nepal’s UTC+5:45 and Lord Howe Island’s unique 30-minute DST adjustment, showing how rich and complex our world’s timekeeping really is.

What is the current time around the world? Utilizing std::chrono with time zones in C++23

Sun, 03 Nov 2024 00:00:00 +0000

In this blog post, we will explore handling dates using std::chrono, including time zones. We’ll utilize the latest features of the library to retrieve the current time across various time zones, taking into account daylight saving time changes as well. Additionally, we will incorporate new capabilities introduced in C++23, such as enhanced printing functions and more.

Let’s start with the following code that prints the current date and time:

#include <chrono>
#include <print>

int main() {
    auto now = std::chrono::system_clock::now();
    std::print("now is {}", now);
}

You can run it through Compiler Explorer

In my session, I’m getting the following results:

Now is 2024-11-01 11:44:06.374573703

It’s simple, but what time zone are we in? Can we print it in a shorter form? How can we check the time around the world? What about daylight saving time?

First things first: time output.

Improving the print

As you can see, I used std::print from C++23, which is available in GCC 14! By default, the time point obtained from ::now() is printed with full information, but we can change it and adjust it. For example:

Basic Date and Time Formatting

Show just the date: {0:%F} will format the date as YYYY-MM-DD.
Show time in hh:mm:ss format: {0:%T} will format the time as HH:MM:SS.
Show the name of the month and/or day of the week: {0:%A, %B} will format it as Day, Month.

Advanced Formatting Options

Year and Month:
- {0:%Y}: Full year (e.g., 2024).
- {0:%y}: Last two digits of the year (e.g., 24).
- {0:%B}: Full month name (e.g., November).
- {0:%b}: Abbreviated month name (e.g., Nov).
Day and Week:
- {0:%d}: Day of the month, zero-padded (e.g., 01).
- {0:%A}: Full weekday name (e.g., Friday).
- {0:%a}: Abbreviated weekday name (e.g., Fri).
Time of Day:
- {0:%H}: Hour (24-hour clock), zero-padded (e.g., 14).
- {0:%I}: Hour (12-hour clock), zero-padded (e.g., 02).
- {0:%M}: Minute, zero-padded (e.g., 05).
- {0:%S}: Second, zero-padded (e.g., 09).
- {0:%p}: AM/PM designation.
Time Zone:
- {0:%Z}: Time zone abbreviation (e.g., UTC).
- {0:%z}: Offset from UTC (e.g., +0000).

Here is an example demonstrating some of these formatting options:

#include <chrono>
#include <print>

int main() {
    auto now = std::chrono::system_clock::now();

    std::print("Full date and time: {0:%Y-%m-%d %H:%M:%S}\n", now);
    std::print("Date only: {0:%F}\n", now);
    std::print("Time only: {0:%T}\n", now);
    std::print("Day of the week: {0:%A}\n", now);
    std::print("Month name: {0:%B}\n", now);
    std::print("12-hour clock with AM/PM: {0:%I:%M:%S %p}\n", now);
    std::print("ISO 8601 format: {0:%FT%T%z}\n", now);
}

Run @Compiler Explorer

Example output:

Full date and time: 2024-11-02 19:35:16.296643881
Date only: 2024-11-02
Time only: 19:35:16.296643881
Day of the week: Saturday
Month name: November
12-hour clock with AM/PM: 07:35:16.296643881 PM
ISO 8601 format: 2024-11-02T19:35:16.296643881+0000

We can print the time, and it’s now clear that it is always in UTC. What about other time zones?

Getting time in my current time zone

C++20 chrono gives us a powerful module that handles time zones. It’s not that easy, as the library has to reach out to the operating system and get the IANA database (see more @std::chrono::tzdb - cppreference.com.

But as a user, it’s relatively easy for us: we have to use zoned_time and ask for proper time_zone:

#include <chrono>
#include <print>

int main() {
	const auto now = std::chrono::system_clock::now();		
	auto zt_local = std::chrono::zoned_time{ std::chrono::current_zone(), now };
	std::print("now is {} UTC and local is: {}\n", now, zt_local);

    try
    {
         for (auto zone: {"Europe/Warsaw", 
                           "America/New_York", 
                           "Asia/Tokyo" } ) 
        {
            std::chrono::zoned_time zt{zone, now};
            std::print("{0}: {1:%F} {1:%R}\n", zone, zt);
        }
    }
    catch (std::runtime_error& ex)
    {
        std::print("Error: {}", ex.what());
    }
}

See at Compiler Explorer

Example output:

now is 2024-11-12 08:15:22.575519666 UTC and local is: 2024-11-12 08:15:22.575519666
Europe/Warsaw: 2024-11-12 09:15
America/New_York: 2024-11-12 03:15
Asia/Tokyo: 2024-11-12 17:15

Compiler Explorer apparently runs on some servers located in the UTC time zone, so there’s no difference in time. But my time zone - Warsaw/Poland - shows different times, Similar to New York and Tokyo.

Let’s now dive into the details:

Getting the Current Time:
- We start by obtaining the current time using std::chrono::system_clock::now(). This gives us the current time in UTC.
Local Time Zone:
- We create a std::chrono::zoned_time object using std::chrono::current_zone(), which automatically detects the local time zone of the system where the code is running. This allows us to convert the UTC time to the local time zone.
Time Zone Conversion: - We define three time zones using std::string_view: Warsaw, New York, and Tokyo. These are specified using their respective IANA time zone identifiers.
- For each time zone, we create a std::chrono::zoned_time object, which converts the current UTC time to the specified time zone.
Printing the Results:
- We use std::print to display the current time in UTC and the local time zone.
- For each specified time zone (Warsaw, New York, Tokyo), we print the date and time using the format specifiers {0:%F} for the date (YYYY-MM-DD) and {0:%R} for the time (HH:MM).
Error Handling:
- The code is wrapped in a try-catch block to handle any potential runtime errors, such as invalid time zone identifiers.

Daylight saving time

We can even check some extra information related to our current time zone. We can obtain info from our zoned_time and print information like start/end, offset, and more:

#include <chrono>
#include <print>
#include <iostream>

int main() {
    try
    {
        const auto now = std::chrono::floor<std::chrono::minutes>(std::chrono::system_clock::now());        
        auto zt_local = std::chrono::zoned_time{ "Europe/Warsaw", now };
        std::print("now is {} UTC and Warsaw is: {}\n", now, zt_local);

        auto info = zt_local.get_info();
        std::print("local time info: \nabbrev: {},\n begin {}, end {}, \noffset {}, save {}\n", 
                          info.abbrev, info.begin, info.end, info.offset, info.save);
    }
    catch (std::runtime_error& ex)
    {
        std::print("Error: {}", ex.what());
    }
}

See @Compiler Explorer

and daylight info:

#include <chrono>
#include <print>

void printInfo(std::chrono::sys_days sd, std::string_view zone) {
    auto zt_local = std::chrono::zoned_time{ "Europe/Warsaw", std::chrono::sys_days{sd} };
    auto info = zt_local.get_info();
    std::print("time info for {:%F} in {}:\nabbrev: {},\nbegin {}, end {}, \noffset {}, save {}\n", 
                sd, zone, info.abbrev, info.begin, info.end, info.offset, info.save);
}

int main() {
    try
    {
        printInfo(std::chrono::year{ 2024 } / 9 / 14, "Europe/Warsaw");
        printInfo(std::chrono::year{ 2024 } / 11 / 14, "Europe/Warsaw");
    }
    catch (std::runtime_error& ex)
    {
        std::print("Error: {}", ex.what());
    }
}

Run @Compiler Explorer

I’m getting:

time info for 2024-09-14 in Europe/Warsaw:
abbrev: CEST,
begin 2024-03-31 01:00:00, end 2024-10-27 01:00:00, 
offset 7200s, save 60min
time info for 2024-11-14 in Europe/Warsaw:
abbrev: CET,
begin 2024-10-27 01:00:00, end 2025-03-30 01:00:00, 
offset 3600s, save 0min

As you can see, we get the following information:

info.abbrev - zone abbreviation name, like CET, PT, etc
info.begin, info.end - start and end date (in sys_seconds) for a given time zone, usually changes when there’s a daylight time zone switch
info.offset - offset from UTC time
info.save - if nonzero, indicates that the time zone is on daylight saving time

Summary

What a ride! We began with a simple display of the current time and progressed to more complex scenarios, including various formatting options available in C++20 and C++23, as well as time zones. In the end, we explored how to check details of a specific timepoint and examine the information related to its time zone.

Would you like to see more?
Read my other articles on std::chrono. See Exploring std::chrono in C++20 - Time Zones or Exploring std::chrono in C++20 - Calendar Types ~2700 words, which are available for C++ Stories Premium/Patreon members. See all Premium benefits here.

Back to you

Have you tried std::chrono?
Do you use the latest additions from C++20 for date support?

Share your comments below.

C++ String Conversion: Exploring std::from_chars in C++17 to C++26

Sun, 13 Oct 2024 00:00:00 +0000

With the introduction of C++17, the C++ Standard Library expanded its capabilities for converting text to numbers with the addition of std::from_chars. This low-level, high-performance API offers significant advantages over previous methods, such as atoi and stringstream. In this article, we will explore the evolution of string conversion routines from C++17 through C++26, highlighting key improvements like constexpr support and enhanced error handling. Let’s dive into the details and see how std::from_chars can transform your approach to string conversion.

Updated in Oct 2024: Added notes from constexpr support in C++23, and C++26 improvements, plus more Compiler Explorer examples.

Before C++17

C++, before C++17, offered several options when it comes to string conversion:

sprintf / snprintf
sscanf
atol
strtol
strstream
stringstream
to_string
stoi and similar functions

And with C++17 you get another option: std::from_chars! Wasn’t the old stuff good enough? Why do we need new methods?

In short: because from_chars is low-level, and offers the best possible performance.

The new conversion routines are:

non-throwing
non-allocating
no locale support
memory safety
error reporting gives additional information about the conversion outcome

The API might not be the most friendly to use, but it’s easy enough to wrap it into some facade.

A simple example:

const std::string str { "12345678901234" };
int value = 0;
std::from_chars(str.data(),str.data() + str.size(), value);
// error checking ommited...

The Series

This article is part of my series about C++17 Library Utilities. Here’s the list of the topics in the series:

Resources about C++17 STL:

C++17 In Detail by Bartek!
C++17 - The Complete Guide by Nicolai Josuttis
C++ Fundamentals Including C++ 17 by Kate Gregory
Practical C++14 and C++17 Features - by Giovanni Dicanio
C++17 STL Cookbook by Jacek Galowicz

Let’s have a look at the API now.

Converting From Characters to Numbers: `from_chars`

std::from_chars, available in the <charconv> headers, is a set of overloaded functions: for integral types and floating point types.

For integral types we have the following functions:

std::from_chars_result from_chars(const char* first, 
                                  const char* last, 
                                  TYPE &value,
                                  int base = 10);

Where TYPE expands to all available signed and unsigned integer types and char.

base can be a number ranging from 2 to 36.

Then there’s the floating point version:

std::from_chars_result from_chars(const char* first, 
                   const char* last, 
                   FLOAT_TYPE& value,
                   std::chars_format fmt = std::chars_format::general);

FLOAT_TYPE expands to float, double or long double.

chars_format is an enum with the following values: scientific,

fixed, hex and general (which is a composition of fixed and scientific).

The return value in all of those functions (for integers and floats) is from_chars_result:

struct from_chars_result {
    const char* ptr;
    std::errc ec;
};

from_chars_result holds valuable information about the conversion process.

Here’s the summary:

Return Condition	State of `from_chars_result`
Success	`ptr` points at the first character not matching the pattern, or has the value equal to `last` if all characters match and `ec` is value-initialized.
Invalid conversion	`ptr` equals `first` and `ec` equals `std::errc::invalid_argument`. `value` is unmodified.
Out of range	The number it too large to fit into the value type. `ec` equals `std::errc::result_out_of_range` and `ptr` points at the first character not matching the pattern. `value` is unmodified.

The new routines are very low level, so you might be wondering why is that. Titus Winters added a great summary in comments:

The intent of those APIs is *not* for people to use them directly, but to build more interesting/useful things on top of them. These are primitives, and we (the committee) believe they have to be in the standard because there isn’t an efficient way to do these operations without calling back out to nul-terminated C routines

Examples

Here are two examples of how to convert a string into a number using from_chars, to int andfloat.

Integral types

#include <charconv> // from_char, to_char
#include <string>
#include <iostream>

int main() {
    const std::string str { "12345678901234" };
    int value = 0;
    const auto res = std::from_chars(str.data(), 
                                     str.data() + str.size(), 
                                     value);

    if (res.ec == std::errc())
    {
        std::cout << "value: " << value 
                  << ", distance: " << res.ptr - str.data() << '\n';
    }
    else if (res.ec == std::errc::invalid_argument)
    {
        std::cout << "invalid argument!\n";
    }
    else if (res.ec == std::errc::result_out_of_range)
    {
        std::cout << "out of range! res.ptr distance: " 
                  << res.ptr - str.data() << '\n';
    }
}

The example is straightforward, it passes a string str into from_chars and then displays the result with additional information if possible.

your task is to run the code @Compiler Explorer.

Does “12345678901234” fit into the number? Or you see some errors from the conversion API?

Floating Point

Here’s the floating pointer version:

#include <charconv> // from_char, to_char
#include <string>
#include <iostream>

int main() {
    const std::string str { "16.78" };
    double value = 0;
    const auto format = std::chars_format::general;
    const auto res = std::from_chars(str.data(), 
                                 str.data() + str.size(), 
                                 value, 
                                 format);

    if (res.ec == std::errc())
    {
        std::cout << "value: " << value 
                  << ", distance: " << res.ptr - str.data() << '\n';
    }
    else if (res.ec == std::errc::invalid_argument)
    {
        std::cout << "invalid argument!\n";
    }
    else if (res.ec == std::errc::result_out_of_range)
    {
        std::cout << "out of range! res.ptr distance: " 
                  << res.ptr - str.data() << '\n';
    }
}

Run at Compiler Explorer

Here’s the example output that we can get:

`str` value	`format` value	output
`1.01`	`fixed`	`value: 1.01, distance 4`
`-67.90000`	`fixed`	`value: -67.9, distance: 9`
`20.9`	`scientific`	`invalid argument!, res.ptr distance: 0`
`20.9e+0`	`scientific`	`value: 20.9, distance: 7`
`-20.9e+1`	`scientific`	`value: -209, distance: 8`
`F.F`	`hex`	`value: 15.9375, distance: 3`
`-10.1`	`hex`	`value: -16.0625, distance: 5`

The general format is a combination of fixed and scientific so it handles regular floating point string with the additional support for e+num syntax.

Performance

I did some benchmarks, and the new routines are blazing fast!

Some numbers:

On GCC it’s around 4.5x faster than stoi, 2.2x faster than atoi and almost 50x faster than istringstream.
On Clang it’s around 3.5x faster than stoi, 2.7x faster than atoi and 60x faster than istringstream!
MSVC performs around 3x faster than stoi, ~2x faster than atoi and almost 50x faster than istringstream

You can find the results in my book on C++17: C++17 In Detail.

C++23 Updates

In C++23 we got one improvement for our functions: P2291

constexpr for integral overloads of std::to_chars() and std::from_chars().

It’s already implemented in GCC 13, Clang 16, and MSVC 19.34.

Together with std::optional that can also work in the constexpr context we can create the following example:

#include <charconv>
#include <optional>
#include <string_view>

constexpr std::optional<int> to_int(std::string_view sv)
{
    int value {};
    const auto ret = std::from_chars(sv.begin(), sv.end(), value);
    if (ret.ec == std::errc{})
        return value;

    return {};
};

int main() {
    static_assert(to_int("hello") == std::nullopt);
    static_assert(to_int("10") == 10);
}

Run @Compiler Explorer

C++26 Updates

The work is not complete, and looks like in C++26 we’ll have more additions:

See P2497R0 wchich is already accepted into the working draft of C++26:

Testing for success or failure of <charconv> functions

The feature is already implemented in GCC 14 and Clang 18.

In short from_chars_result (as well as to_chars_result) gets a bool conversion operator:

constexpr explicit operator bool() const noexcept;

And it has to return ec == std::errc{}.

This means our code can be a bit simpler:

if (res.ec == std::errc()) { ... }

can become:

if (res) { ... }

For example:

// ...
const auto res = std::from_chars(str.data(), 
                                 str.data() + str.size(), 
                                 value, 
                                 format);

if (res)
{
    std::cout << "value: " << value 
              << ", distance: " << res.ptr - str.data() << '\n';
}
// ...

Run @Compiler Explorer

Compiler Support for `std::from_chars` in C++

Visual Studio: Full support for std::from_chars was introduced in Visual Studio 2019 version 16.4, with floating-point support starting from VS 2017 version 15.7. Visual Studio 2022 includes C++23 features like constexpr for integral overloads.
GCC: Starting with GCC 11.0, std::from_chars offers full support, including integer and floating-point conversions. The latest GCC versions, such as GCC 13, incorporate constexpr integral support.
Clang: Clang 7.0 introduced initial support for integer conversions. Clang 16 and above support constexpr for integral overloads.

For the most accurate and up-to-date information see CppReference, Compiler Support.

Summary

If you want to convert text into a number and you don’t need any extra stuff like locale support then std::from_chars might be the best choice. It offers great performance, and what’s more, you’ll get a lot of information about the conversion process (for example how much characters were scanned).

The routines might be especially handy with parsing JSON files, 3d textual model representation (like OBJ file formats), etc.

What’s more the new functions can be evan used at compile-time (as of C++23), and have even better error checking in C++26.

References

If you want to read more about existing conversion routines, new ones and also see some benchmarks you can see two great posts at @fluentcpp:

Your Turn

Have you played with the new conversion routines?
What do you usually use to convert text into numbers?

std::initializer_list in C++ 2/2 - Caveats and Improvements

Sun, 29 Sep 2024 00:00:00 +0000

In this article, you’ll learn why std::initializer_list has a bad reputation in C++. Is passing values using it as efficient as “emplace”? How can we use non-copyable types? You’ll also see how to fix some of the problems.

Let’s start with the issues first:

Updated in Sept 2024: Added note about stack overflow and C++26 fixes.

This is the second part of initalizer_list mini-series. See the first part on Basics and use cases here.

1. Referencing local array

If you recall from the previous article, std::initializer_list expands to some unnamed local array of const objects. So the following code might be super unsafe:

std::initializer_list<int> wrong() { // for illustration only!
    return { 1, 2, 3, 4};
}
int main() {
    std::initializer_list<int> x = wrong();
}

The above code is equivalent to the following:

std::initializer_list<int> wrong() {
    const int arr[] { 1, 2, 3, 4}
    return std::initializer_list<int>{arr, arr+4};
}
int main() {
    std::initializer_list<int> x = wrong();
}

The example serves to highlight the error, emphasizing the importance of avoiding similar mistakes in our own code. The function returns pointers/iterators to a local object, and that will cause undefined behavior. See a demo @Compiler Explorer.

GCC or Clang reports a handy warning in this case:

GCC:
warning: returning temporary 'initializer_list' does not extend the lifetime of the underlying array [-Winit-list-lifetime]
    5 |     return { 1, 2, 3, 4};

Or in Clang:

<source>:5:12: warning: returning address of local temporary object [-Wreturn-stack-address]
    return { 1, 2, 3, 4};

We can make the following conclusion:

std::initializer_list is a “view” type; it references some implementation-dependent and a local array of const values. Use it mainly for passing into functions when you need a variable number of arguments of the same type. If you try to return such lists and pass them around, then you risk lifetime issues. Use with care.

Let’s take on another limitation:

2. The cost of copying elements

Passing elements through std::initializer_list is very convenient, but it’s good to know that when you pass it to a std::vector’s constructor (or other standard containers), each element has to be copied. It’s because, conceptually, objects in the initializer_list are put into a const temporary array, so they have to be copied to the container.

Let’s compare the following case:

std::cout << "vec... { }\n";
std::vector<Object> vec { Object("John"), Object("Doe"), Object("Jane") };

With:

std::cout << "vec emplace{}\n";
std::vector<Object> vec;
vec.reserve(3);
vec.emplace_back("John");
vec.emplace_back("Doe");
vec.emplace_back("Jane");

The Object class is a simple wrapper around std::string with some extra logging code. You can run the example @Compiler Explorer.

And we’ll get:

vec... { }
MyType::MyType John
MyType::MyType Doe
MyType::MyType Jane
MyType::MyType(const MyType&) John
MyType::MyType(const MyType&) Doe
MyType::MyType(const MyType&) Jane
MyType::~MyType Jane
MyType::~MyType Doe
MyType::~MyType John
MyType::~MyType John
MyType::~MyType Doe
MyType::~MyType Jane
vec emplace{}
MyType::MyType John
MyType::MyType Doe
MyType::MyType Jane
MyType::~MyType John
MyType::~MyType Doe
MyType::~MyType Jane

As you can see, something is wrong! In the case of the single constructor, we can spot some extra temporary objects! On the other hand, the case with emplace_back() has no temporaries.

We can link this issue with the following element:

3. Non-copyable types

The extra copy that we’d get with the initializer_list, also causes issues when your objects are not copyable. For example, when you want to create a vector of unique_ptr.

#include <vector>
#include <memory>

struct Shape {    virtual void render() const ; };
struct Circle : Shape { void render() const override; };
struct Rectangle : Shape {  void render() const override; };

int main() {
    std::vector<std::unique_ptr<Shape>> shapes {
        std::make_unique<Circle>(), std::make_unique<Rectangle>()
    };
}

Run @Compiler Explorer

The line where I want to create a vector fails to compile, and we get many messages about copying issues.

In short: the unique pointers cannot be copied. They can only be moved, and passing initializer_list doesn’t give us any options to handle those cases. The only way to build such a container is to use emplace_back or push_back:

std::vector<std::unique_ptr<Shape>> shapes;
shapes.reserve(2);
shapes.push_back(std::make_unique<Circle>());           // or
shapes.emplace_back(std::make_unique<Rectangle>());

See the working code at Compiler Explorer.

4. Stack space and large lists (fixed in C++26)

Remember that each initializer list requires a const array of objects… consider this code:

auto getImageBytes() {
    return std::vector<std::byte> v = {
        0xff, 0xaa, 0xce, ... // 2MB of soma image data
    };
}

At first glance, this code is safe, all it does is it allocates dynamic memory and puts the content of the file into that memory block.

But…

Because we have this backing array, we end up with the following code:

auto getImageBytes() {
    const std::byte bytes[] = { 
        0xff, 0xaa, 0xce, ... // 2MB of soma image data
    };
    std::vector<std::byte> v(bytes.begin(), bytes.end());
    return v;
}

This temp array is allocated on the stack. For large arrays (like our 2MB of data), this can exceed the stack size limit and cause a stack overflow, crashing your application.

You can fix this code as of C++23 by using your custom backing array:

auto getImageBytes() {
    static const std::byte bytes[] = { 
        0xff, 0xaa, 0xce, ... // 2MB of soma image data
    };
    return std::vector<std::byte>(bytes.begin(), bytes.end());
}

Now, we have control over the array and we can put it into a static storage and thus it won’t use stack space.

Fortunately, starting from C++26, thanks to P2752R3 “Static storage for braced initializers”, the compiler will handle this situation more gracefully.

The proposal allows compilers to place the temporary array backing an initializer list into static storage automatically when the array is large. This means that large initializer lists won’t cause stack overflows, and you won’t need to manually declare arrays as static.

For more details, you can read Arthur O’Dwyer’s explanation on #embed and the static storage proposal: P1967 `#embed` and D2752 “Static storage for `initializer_list`” are now on Compiler Explorer – Arthur O’Dwyer – Stuff mostly about C++.

Alternative: variadic function

If you want to reduce the number of temporary objects, or your types are not copyable, then you can use the following factory function for std::vector:

template<typename T, typename... Args>
auto initHelper(Args&&... args) {
    std::vector<T> vec;
    vec.reserve(sizeof...(Args)); 
    (vec.emplace_back(std::forward<Args>(args)), ...);
    return vec;
}

See at @Compiler Explorer

When we run the code:

std::cout << "initHelper { }\n";
auto vec = initHelper<Object>("John", "Doe", "Jane");

We’ll get the following:

initHelper { }
MyType::MyType John
MyType::MyType Doe
MyType::MyType Jane
MyType::~MyType John
MyType::~MyType Doe
MyType::~MyType Jane

What’s more, the same function template can be used to initialize from moveable objects:

std::vector<std::unique_ptr<Shape>> shapes;
shapes.reserve(2);
shapes.push_back(std::make_unique<Circle>());
shapes.emplace_back(std::make_unique<Rectangle>());

auto shapes2 = initHelper<std::unique_ptr<Shape>>(
     std::make_unique<Circle>(), std::make_unique<Rectangle>());

Run at @Compiler Explorer

Core features:

Variadic templates allow us to pass any number of arguments into a function and process it with argument pack syntax.
(vec.emplace_back(std::forward<Args>(args)), ...); is a fold expression over a comma operator that nicely expands the argument pack at compile time. Fold expressions have been available since C++17.

More proposals

The initializer list is not perfect, and there were several attempts to fix it:

From the paper N4166 by David Krauss

std::initializer_list was designed around 2005 (N1890) to 2007 (N2215), before move semantics matured, around 2009. At the time, it was not anticipated that copy semantics would be insufficient or even suboptimal for common value-like classes. There was a 2008 proposal “N2801 Initializer lists and move semantics” but C++0x was already felt to be slipping at that time, and by 2011 the case had gone cold.

There’s also a proposal by Sy Brand: WG21 Proposals - init list

Still, as of 2022, there have yet to be any improvements implemented in the standard. So we have to wait a bit.

(voted into C++26!): Recently there’s also a proposal from Arthur: P1967 `#embed` and D2752 “Static storage for `initializer_list`” are now on Compiler Explorer. This is very interesting as it allows us to put that unnamed const array into the static storage rather than on the stack. We’ll see how it goes.

Summary

In the article, I showed you three issues with initializer_list: extra copies, lack of support for non-copyable objects, and undefined behavior when returning such lists. The case with non-copyable objects can be solved by using a relatively simple factory function that accepts a variadic pack.

What’s more, we looked at an improvement from C++26 that allows the compiler to put the backing array for the initializer list in static storage space. This improvement helps with avoiding stack overflows and tricky-to-find bugs.

Resources

initializer_list class - Microsoft Learn
The cost of std::initializer_list - Andrzej Krzemieński

Back to you

Do you prefer container initialization with initializer_list or regular push_back() or emplace_back()?
Do you tend to optimize and reduce temporary copies when possible?

Share your comments below.

Boost File Scanning Speed: Query File Attributes on Windows 50x Faster

Sun, 11 Aug 2024 00:00:00 +0000

Imagine you’re developing a tool that needs to scan for file changes across thousands of project files. Retrieving file attributes efficiently becomes critical for such scenarios. In this article, I’ll demonstrate a technique to get file attributes that can achieve a surprising speedup of over 50+ times compared to standard Windows methods.

Let’s dive in and explore how we can achieve this.

Inspiration & Disclaimer

The inspiration for this article came from a recent update for Visual Assist - a tool that heavily improves Visual Studio experience and productivity for C# and C++ developers.

In one of their blog post, they shared:

The initial parse is 10..15x faster!

What’s New in Visual Assist 2024—Featuring lightning fast parser performance [Webinar] - Tomato Soup

After watching the webinar, I noticed some details about efficiently getting file attributes and I decided to give it a try on my machine. In other words I tried to recreate their results.

Disclaimer: This post was written with the support and sponsorship of Idera, the company behind Visual Assist.

Understanding File Attribute Retrieval Methods on Windows

On Windows, there are at least a few options to check for a file change:

FindFirstFile[EX]
GetFileAttributesEx
std::filesystem

Below, you can see some primary usage of each approach:

`FindFirstFileEx`

FindFirstFileEx is a Windows API function that allows for efficient searching of directories. It retrieves information about files that match a specified file name pattern. The function can be used with different information levels, such as FindExInfoBasic and FindExInfoStandard, to control the amount of file information fetched.

WIN32_FIND_DATA findFileData;
HANDLE hFind = FindFirstFileEx((directory + "\\*").c_str(), FindExInfoBasic, &findFileData, FindExSearchNameMatch, NULL, 0);

if (hFind != INVALID_HANDLE_VALUE) {
    do {
        // Process file information
    } while (FindNextFile(hFind, &findFileData) != 0);
    FindClose(hFind);
}

`GetFileAttributesEx`

GetFileAttributesEx is another Windows API function that retrieves file attributes for a specified file or directory. Unlike FindFirstFileEx, which is used for searching and listing files, GetFileAttributesEx is typically used for retrieving attributes of a single file or directory.

WIN32_FILE_ATTRIBUTE_DATA fileAttributeData;
if (GetFileAttributesEx((directory + "\\" + fileName).c_str(), GetFileExInfoStandard, &fileAttributeData)) {
    // Process file attributes
}

`std::filesystem`

Introduced in C++17, the std::filesystem library provides a modern and portable way to interact with the file system. It includes functions for file attribute retrieval, directory iteration, and other common file system operations.

for (const auto& entry : fs::directory_iterator(directory)) {
    if (entry.is_regular_file()) {
        // Process file attributes
        auto ftime = fs:last_write_time(entry);
        ...
    }
}

The Benchmark

To evaluate the performance of different file attribute retrieval methods, I developed a small benchmark. This application measures the time taken by each method to retrieve file attributes for N number of files in a specified directory.

Here’s a rough overview of the code:

The FileInfo struct stores the file name and last write time.

struct FileInfo {
    std::string fileName;
    FILETIME lastWriteTime;
};

Each retrieval technique will have to go over a directory and build a vector of FileInfo objects.

`BenchmarkFindFirstFileEx`

void BenchmarkFindFirstFileEx(const std::string& directory,     
                              std::vector<FileInfo>& files, 
                              FINDEX_INFO_LEVELS infoLevel) 
{
   WIN32_FIND_DATA findFileData;
   HANDLE hFind = FindFirstFileEx((directory + "\\*").c_str(),
                                   infoLevel, 
                                   &findFileData, 
                                   FindExSearchNameMatch, NULL, 0);

   if (hFind == INVALID_HANDLE_VALUE) {
       std::cerr << "FindFirstFileEx failed (" 
                 << GetLastError() << ")\n";
       return;
   }

   do {
       if (!(findFileData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)) {
           FileInfo fileInfo;
           fileInfo.fileName = findFileData.cFileName;
           fileInfo.lastWriteTime = findFileData.ftLastWriteTime;
           files.push_back(fileInfo);
       }
   } while (FindNextFile(hFind, &findFileData) != 0);

   FindClose(hFind);
}

`BenchmarkGetFileAttributesEx`

void BenchmarkGetFileAttributesEx(const std::string& directory,
                                  std::vector<FileInfo>& files) 
{
   WIN32_FIND_DATA findFileData;
   HANDLE hFind = FindFirstFile((directory + "\\*").c_str(),
                                &findFileData);

   if (hFind == INVALID_HANDLE_VALUE) {
       std::cerr << "FindFirstFile failed (" 
                 << GetLastError() << ")\n";
       return;
   }

   do {
       if (!(findFileData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)) {
           WIN32_FILE_ATTRIBUTE_DATA fileAttributeData;
           if (GetFileAttributesEx((directory + "\\" + findFileData.cFileName).c_str(), GetFileExInfoStandard, &fileAttributeData)) {
               FileInfo fileInfo;
               fileInfo.fileName = findFileData.cFileName;
               fileInfo.lastWriteTime = fileAttributeData.ftLastWriteTime;
               files.push_back(fileInfo);
           }
       }
   } while (FindNextFile(hFind, &findFileData) != 0);

   FindClose(hFind);
}

`BenchmarkStdFilesystem`

And the last one, the most portable technique:

void BenchmarkStdFilesystem(const std::string& directory, 
                            std::vector<FileInfo>& files) 
{
    for (const auto& entry : std::filesystem::directory_iterator(directory)) {
        if (entry.is_regular_file()) {
            FileInfo fileInfo;
            fileInfo.fileName = entry.path().filename().string();
            auto ftime = std::filesystem::last_write_time(entry);
            memcpy(&fileInfo.lastWriteTime, &ftime, sizeof(FILETIME));
            files.push_back(fileInfo);
        }
    }
}

In the code, we use the assumption that file_time_type values maps to FILETIME on Windows. Read more in this explanation std::filesystem::file_time_type does not allow easy conversion to time_t - Developer Community

The Main Function

The main function sets up the benchmarking environment, runs the benchmarks, and prints the results.

// Benchmark FindFirstFileEx (Basic)
auto start = std::chrono::high_resolution_clock::now();
BenchmarkFindFirstFileEx(directory, 
                         filesFindFirstFileExBasic, 
                         FindExInfoBasic);
auto end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double> elapsedFindFirstFileExBasic = end - start;

// Benchmark FindFirstFileEx (Standard)
start = std::chrono::high_resolution_clock::now();
BenchmarkFindFirstFileEx(directory, 
                         filesFindFirstFileExStandard, 
                         FindExInfoStandard);
end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double> elapsedFindFirstFileExStandard = end - start;

// ...

This benchmark code measures the performance of FindFirstFileEx with both FindExInfoBasic and FindExInfoStandard, GetFileAttributesEx, and std::filesystem. The results are then formatted and displayed in a table.

Performance Results

To measure the performance of each file attribute retrieval method, I executed benchmarks on a directory containing 1000, 2000 or 5000 random text files. The tests were performed on a laptop equipped with an Intel i7 4720HQ CPU and an SSD. I measured the time taken by each method and compared the results to determine the fastest approach.

Each test run consisted of two executions: the first with uncached file attributes and the second likely benefiting from system-level caching.

The speedup factor is the factor of the current result compared to the slowest technique in a given run.

1000 files:

Method                         Time (seconds)       Speedup Factor
FindFirstFileEx (Basic)        0.0131572000         17.876
FindFirstFileEx (Standard)     0.0018139000         129.665
GetFileAttributesEx            0.2351992000         1.000
std::filesystem                0.0607928000         3.869

Method                         Time (seconds)       Speedup Factor
FindFirstFileEx (Basic)        0.0009740000         61.956
FindFirstFileEx (Standard)     0.0009998000         60.358
GetFileAttributesEx            0.0602633000         1.001
std::filesystem                0.0603455000         1.000

Directory with 2000 files:

Method                         Time (seconds)       Speedup Factor
FindFirstFileEx (Basic)        0.0023182000         54.402
FindFirstFileEx (Standard)     0.0044334000         28.446
GetFileAttributesEx            0.1261137000         1.000
std::filesystem                0.1259038000         1.002

Method                         Time (seconds)       Speedup Factor
FindFirstFileEx (Basic)        0.0022301000         55.417
FindFirstFileEx (Standard)     0.0040665000         30.391
GetFileAttributesEx            0.1235858000         1.000
std::filesystem                0.1220140000         1.013

Directory with 5000 random, small text files:

Method                         Time (seconds)       Speedup Factor
FindFirstFileEx (Basic)        0.0059723000         113.144
FindFirstFileEx (Standard)     0.0125500000         53.843
GetFileAttributesEx            0.6757297000         1.000
std::filesystem                0.3098593000         2.181

Method                         Time (seconds)       Speedup Factor
FindFirstFileEx (Basic)        0.0060349000         52.300
FindFirstFileEx (Standard)     0.0136566000         23.112
GetFileAttributesEx            0.3156277000         1.000
std::filesystem                0.3075732000         1.026

The results consistently showed that FindFirstFileEx with the Standard flag was the fastest method in uncached scenarios, offering speedups up to 129x compared to GetFileAttributesEx. However, in cached scenarios, FindFirstFileEx (Basic and Standard) achieved over 50x speedup improvements.

For the directory with 2000 files, FindFirstFileEx (Basic) demonstrated a speedup factor of over 54x in the first run and maintained similar performance in the second run. In the directory with 5000 files, the Basic version achieved an impressive 113x speedup initially and 52x in the subsequent run, reflecting the impact of caching. Notably, std::filesystem performed on par with GetFileAttributesEx.

Further Techniques

Getting file attributes is only part of the story, and while important, they may contribute to only a small portion of the overall performance for the whole project. The Visual Assist team, who contributed to this article, improved their initial parse time performance by avoiding GetFileAttributes[Ex] using the same techniques as this article. But Visual Assist also improved performance through further techniques. My simple benchmark showed 50x speedups, but we cannot directly compare it with the final Visual Assist, as the tool does many more things with files.

The main item being optimised was the initial parse, where VA builds a symbol database when a project is opened for the first time. This involves parsing all code and all headers. They decided that it’s a reasonable assumption that headers won’t change while a project is being loaded, and so the file access is cached during the initial parse, avoiding the filesystem entirely. (Changes after a project has been parsed the first time are, of course, still caught.) The combination of switching to a much faster method for checking filetimes and then avoiding file IO completely contributed to the up-to-15-times-faster performance improvement they saw in version 2024.1 at the beginning of this year.

Read further details on their blog Visual Assist 2024.1 release post - January 2024 and Catching up with VA: Our most recent performance updates - Tomato Soup.

Summary

In the text, we went through a benchmark that compares several techniques for fetching file attributes. In short, it’s best to gather attributes at the same time as you iterate through the directory - using FindFirstFileEx. So if you want to do this operation hundreds of times, it’s best to measure time and choose the best technique.

The benchmark also showed one feature: while C++17 and its filesystem library offer a robust and standardized way to work with files and directories, it can be limited in terms of performance. In many cases, if you need super optimal performance, you need to open the hood and work with the specific operating system API.

The code can be found in my Github Respository: FileAttribsTest.cpp

Back to you

Do you use std::filesystem for tasks involving hundreds of files?
Do you know other techniques that offer greater performance when working with files?

Share your comments below.

Enum Class Improvements for C++17, C++20 and C++23

Sun, 04 Aug 2024 00:00:00 +0000

The evolution of the C++ language continues to bring powerful features that enhance code safety, readability, and maintainability. Among these improvements, we got changes and additions to enum class functionalities across C++17, C++20, and C++23. In this blog post, we’ll explore these advancements, focusing on initialization improvements in C++17, the introduction of the using enum keyword in C++20, and the std::to_underlying utility in C++23.

Let’s go.

Enum Class Recap

Before diving into the enhancements, let’s briefly recap what enum class is. An enum class (scoped enumeration) provides a type-safe way of defining a set of named constants. Unlike traditional (unscoped) enums, enum class does not implicitly convert to integers or other types, preventing accidental misuse. Here’s a basic example:

#include <iostream>

enum class Color {
    Red,
    Green,
    Blue
};

int main() {    
    Color color = Color::Red;

    if (color == Color::Red)
        std::cout << "The color is red.\n";

    color = Color::Blue;

    if (color == Color::Blue)
        std::cout << "The color is blue.\n";

    // std::cout << color; // error, no matching << operator
    // int i = color;      // error: cannot convert
}

Run @Compiler Explorer

Notice the two lines near the end of the main function. You’ll get compiler errors as there’s no implicit conversion to integer types.

As a comparison here’s a similar example, but with unscoped enums:

#include <iostream>

enum Color {
    Red,
    Green,
    Blue
};

int main() {    
    Color color = Red;

    if (color == Red)
        std::cout << "The color is red.\n";

    color = Blue;

    if (color == Blue)
        std::cout << "The color is blue.\n";

    std::cout << color; // fine, prints integer value!
    int i = color;      // fine, can convert...
}

Run @Compiler Explorer

In short, while enum classes give us a separate scope for all of the enum values, they also enforce better type safety. There are no implicit conversions to integral values, and thus, you have better control over your design.

While the basics are simple, let’s now go to several improvements that are handy in the latest C++ revisions.

C++17: Initialization with Brace Initialization from Underlying Type

Enum class might sometimes feel too restrictive, and having some conversions might be handy:

In P0138 accepted for C++17 we have the following example:

enum class Handle : uint32_t { Invalid = 0 }; 
Handle h { 42 }; // OK

In short, when you use enum class to define strong types, it’s helpful to allow initializing from the underlying type without any errors. This wasn’t possible before C++17.

However, the change was cautious so that enums are still safe - they can be used only for uniform/brace initialization. Have a look at this code:

#include <iostream>

enum class Handle : uint32_t { Invalid = 0 }; 

void process(Handle h) {

}

int main() {    
    Handle h { 42 }; // OK

    // process({10}); // error
    process(Handle{10});
}

You cannot just pass {10} as the parameter for the process function. You still have to be explicit about the type.

In C++14, you could use process(static_cast<Handle>(10)); so, as you can see, the C++17 version is much better.

C++20: Using Enum

C++20 introduced the using enum syntax. This feature allows you to bring all the enumerators of an enum into the current scope without losing the benefits of scoped enums.

Consider the following example:

enum class ComputeStatus {
    Ok,
    Error,
    FileError,
    NotEnoughMemory,
    TimeExceeded,
    Unknown
};

In previous C++ versions, using these enumerators required qualifying them with the enum class name:

ComputeStatus s = ComputeStatus::NotEnoughMemory;

C++20 simplifies this with the using enum declaration:

int main() {
    using enum ComputeStatus;
    ComputeStatus s = NotEnoughMemory;
}

It’s probably no sense in this silly code above, but how about the following case:

int main() {    
    ComputeStatus s = ComputeStatus::Ok;
    switch (s) {
        case ComputeStatus::Ok: 
            std::cout << "ok"; break;
        case ComputeStatus::Error: 
            std::cout << "Error"; break;
        case ComputeStatus::FileError: 
            std::cout << "FileError"; break;
        case ComputeStatus::NotEnoughMemory: 
            std::cout << "NotEnoughMemory"; break;
        case ComputeStatus::TimeExceeded: 
            std::cout << "Time..."; break;
        default: std::cout << "unknown...";
    }
}

We can convert it into:

int main() {    
    ComputeStatus s = ComputeStatus::Ok;
    switch (s) {
        using enum ComputeStatus;	// << <<
        case Ok: 
            std::cout << "ok"; break;
        case Error: 
            std::cout << "Error"; break;
        case FileError: 
            std::cout << "FileError"; break;
        case NotEnoughMemory: 
            std::cout << "NotEnoughMemory"; break;
        case TimeExceeded: 
            std::cout << "Time..."; break;
        default: std::cout << "unknown...";
    }
}

Run @Compiler Explorer

Or have a look at the following example:

struct ComputeEngine {
    enum class ComputeStatus {
        Ok,
        Error,
        FileError,
        NotEnoughMemory,
        TimeExceeded,
        Unknown
    };
    using enum ComputeStatus;
};

int main() {    
    ComputeEngine::ComputeStatus s = ComputeEngine::Ok;
}

You can bring all enumerations into the scope of ComputeEngine and still benefit from enum class features.

This C++20 improvement makes the code cleaner and reduces verbosity, especially in cases where multiple enumerators are frequently used within a scope. It allows for a more streamlined and readable approach without sacrificing the type safety provided by scoped enums.

C++23: `std::to_underlying`

C++23 further enhances enum class usability with the introduction of std::to_underlying, a utility function that converts an enum value to its underlying integral type. This feature addresses the common need to convert enum values to integers for storage, comparison, or interoperability with APIs that expect integral types.

The idea for this appeared in a wonderful book by Scott Meyers - Effective Modern C++: 42 Specific Ways to Improve Your Use of C++11 and C++14. Finally in C++23 we can enjoy this feature being standardized.

Before C++23, converting an enum to its underlying type required explicit casting:

enum class Permissions : uint8_t {
    Execute = 1,
	Write = 2,
    Read = 4
};

uint8_t value = static_cast<uint8_t>(Permissions::Read);

With std::to_underlying, this conversion becomes more straightforward and expressive:

#include <type_traits>

int main() {
    Permissions p = Permissions::Read;
    auto value = std::to_underlying(p); // C++23
}

The std::to_underlying function improves code readability and reduces the boilerplate associated with type casting. It also clarifies the intent, making it evident that the purpose is to obtain the underlying value of the enum.

If you want to read more about bitmasks, have a look at this cool article by Andreas Fertig: C++20 Concepts applied - Safe bitmasks using scoped enums - Andreas Fertig’s Blog.

Future improvements

One important update that we may receive soon is support for C++26 reflections - check out P2996 (the proposal has not been accepted yet, but is expected to be approved soon…). Reflection opens up numerous exciting possibilities, such as the ability to convert enums into strings.

See this example from the proposal:

template <typename E>
  requires std::is_enum_v<E>
constexpr std::string enum_to_string(E value) {
  template for (constexpr auto e : std::meta::enumerators_of(^E)) {
    if (value == [:e:]) {
      return std::string(std::meta::name_of(e));
    }
  }
  return "<unnamed>";
}

enum Color { red, green, blue };
static_assert(enum_to_string(Color::red) == "red");
static_assert(enum_to_string(Color(42)) == "<unnamed>");

See at EDG experimental implementation @CompilerExplorer

Of course, you don’t have to wait till C++26 and you can rely on third-party libraries like Neargye/magic_enum @Github. (Thanks to mentioning that in the comment, maxpagani).

Summary

In this article, we covered some of the new features introduced in C++17, C++20, and C++23 for enum classes. Enum classes provide type safety, but there are situations where we may need to relax some restrictions and write more concise code. We also explored the potential use case for C++26 reflection.

Back to you

Have you tried the latest improvements for enum class?
Do you use enum class for strong types or some other techniques?

Share your feedback below in the comments.

22 Common Filesystem Tasks in C++20

Sun, 14 Jul 2024 00:00:00 +0000

Working with the filesystem can be a daunting task, but it doesn’t have to be. In this post, I’ll walk you through some of the most common filesystem operations using the powerful features introduced in C++17, as well as some new enhancements in C++20/23. Whether you’re creating directories, copying files, or managing permissions, these examples will help you understand and efficiently utilize the std::filesystem library.

Let’s start.

Directory Operations

1. Creating Directories

Creating directories is a basic yet essential operation. The std::filesystem library makes this straightforward with the create_directory function.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path dir = "example_directory";
    if (std::filesystem::create_directory(dir))
        std::cout << "Directory created successfully\n";
    else
        std::cout << "Failed to create directory\n";
}

Ron @Compiler Explorer

The function returns true when the directory was newly created. But it can also throw an exception in case of an error. You can use noexcept version of this function:

bool create_directory( const std::filesystem::path& p, std::error_code& ec ) noexcept;

For example:

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path dir = "/abc/cde"; // recursive?
    std::error_code ec{};
    if (std::filesystem::create_directory(dir, ec))
        std::cout << "Directory created successfully\n";
    else
    {
        std::cout << "Failed to create directory\n";
        std::cout << ec.message() << '\n';
    }
}

Run @Compiler Explorer

Possible output:

Failed to create directory: No such file or directory
...

See more @Cppreference

2. Creating Nested Directories

When you need to create multiple nested directories, use the create_directories function.

This function creates directories recursively.

#include <filesystem>
#include <iostream>
#include <exception>

int main() {
    std::filesystem::path nested = "a/b/c";
    try {
        if (std::filesystem::create_directories(nested))
            std::cout << "Nested directories created successfully\n";
        else
            std::cout << "Failed to create nested directories\n";
    }
    catch (const std::exception& ex) {
        std::cout << ex.what() << '\n';
    }
}

Run @Compiler Explorer.

3. Removing a Directory

So far, our example directories were left in a vacuum (and I hope Compiler Explorer can safely remove them :), but we can also do it manually from code: We just need the remove function for this task.

#include <filesystem>
#include <iostream>
#include <exception>

void ls() {
   ...
}

int main() {
    std::filesystem::path dir = "test";
    try {
        if (std::filesystem::create_directory(dir))
            std::cout << "Directory created successfully\n";
        else
            std::cout << "Failed to create directory\n";

        ls();

        if (std::filesystem::remove(dir))
            std::cout << "Directory removed\n";
        else
            std::cout << "Failed to remove the directory...\n";

        ls();
    }
    catch (const std::exception& ex) {
        std::cout << ex.what() << '\n';
    }
}

Possible output @Compiler Explorer:

Directory created successfully
"./compilation-result.json"
"./output.s"
"./example.cpp"
"./test"
Directory removed
"./compilation-result.json"
"./output.s"
"./example.cpp"

4. Removing All Contents of a Directory

To remove a directory along with all its contents, use the remove_all function.

#include <filesystem>
#include <iostream>
#include <exception>
#include <format>

void ls() {
    for (const auto& entry : std::filesystem::recursive_directory_iterator(".")) 
        std::cout << entry.path() << '\n';
}

int main() {
    std::filesystem::path dir = "test";
    std::filesystem::path nested = dir / "a/b";
    std::filesystem::path more = dir / "x/y";
    try {
        if (std::filesystem::create_directories(nested) &&
            std::filesystem::create_directories(more))
            std::cout << "Directories created successfully\n";
        else
            std::cout << "Failed to create directories...\n";

        ls();

        const auto cnt = std::filesystem::remove_all(dir);
        
        std::cout << std::format("removed {} items\n", cnt);

        ls();
    }
    catch (const std::exception& ex) {
        std::cout << ex.what() << '\n';
    }
}

Possible output @Compiler Explorer:

Directories created successfully
"./compilation-result.json"
"./output.s"
"./example.cpp"
"./test"
"./test/x"
"./test/x/y"
"./test/a"
"./test/a/b"
removed 5 items
"./compilation-result.json"
"./output.s"
"./example.cpp"

Notice that we had test/x/y, test/x, test/a/b, test/a and test directories to remove in this case.

5. Listing Directory Contents

In my previous examples, I’ve used the magical ls() function, but the implementation is quite simple. It uses directory_iterator to iterate through the directory:

#include <filesystem>
#include <iostream>
#include <exception>
#include <format>

void ls() {
    for (const auto& entry : std::filesystem::directory_iterator(".")) 
        std::cout << entry.path() << '\n';
}

int main() {
    ls();
}

The iterator takes the path object and iterates through files in unspecified order (system dependent).

Here’s a possible output @Compiler Explorer:

"./compilation-result.json"
"./output.s"
"./example.cpp"

6. Listing Directory Contents Recursively

We can use a recursive version of the directory_iterator to iterate through a whole subtree:

#include <filesystem>
#include <iostream>

void ls() {
    for (const auto& entry : std::filesystem::recursive_directory_iterator(".")) 
        std::cout << entry.path() << '\n';
}

int main() {
    std::filesystem::create_directories("a/b/c");
    std::filesystem::create_directories("x/y/z");
    ls();
}

Run @Compiler Explorer. Here’s a possible output:

"./compilation-result.json"
"./output.s"
"./x"
"./x/y"
"./x/y/z"
"./example.cpp"
"./a"
"./a/b"
"./a/b/c"

If we need a nicer output, we can try implementing our own recursive version:

#include <filesystem>
#include <iostream>

void ls(const std::filesystem::path& p, unsigned tabs = 0) {
    for (const auto& entry : std::filesystem::directory_iterator(p)) {
        if (tabs > 0) std::cout << std::string(tabs, '-');
        std::cout << entry.path().stem() << '\n';
        if (entry.is_directory())
            ls(entry.path(), tabs + 1);
    }   
}

int main() {
    std::filesystem::create_directories("a/b/c");
    std::filesystem::create_directories("x/y/z");
    ls(".");
}

Run @Compiler Explorer

In this version, I used the is_directory() member function of the directory_entry to check if the item is another directory.

The output looks a bit better:

"compilation-result"
"output"
"x"
-"y"
--"z"
"example"
"a"
-"b"
--"c"

7. Creating and Using a Temporary Directory

Thus far, I have used the current directory to create new files and test dirs, but the std::filesystem library provides facilities to create and manage temporary directories in a better way. It all comes down to the temp_directory_path() function:

#include <filesystem>
#include <iostream>
#include <fstream>

void create_temp_file(const std::filesystem::path& dir, const std::string& filename) {
    std::filesystem::path file_path = dir / filename;
    std::ofstream(file_path) << "This is a temporary file.";
    std::cout << "Temporary file created at: " << file_path << '\n';
}

int main() {
    std::filesystem::path temp_dir = std::filesystem::temp_directory_path() / "my_temp_directory";
    std::filesystem::create_directory(temp_dir);
    std::cout << "Temporary directory created at: " << temp_dir << '\n';

    create_temp_file(temp_dir, "temp_file.txt");

    std::cout << "Contents of the temporary directory:\n";
    for (const auto& entry : std::filesystem::directory_iterator(temp_dir)) {
        std::cout << entry.path() << '\n';
    }
}

Run @Compiler Explorer

Possible output:

Temporary directory created at: "/tmp/my_temp_directory"
Temporary file created at: "/tmp/my_temp_directory/temp_file.txt"
Contents of the temporary directory:
"/tmp/my_temp_directory/temp_file.txt"

On Windows (Running and compiling from Visual Studio) I’m getting the following output:

Temporary directory created at: "C:\\Users\\Admin\\AppData\\Local\\Temp\\my_temp_directory"
Temporary file created at: "C:\\Users\\Admin\\AppData\\Local\\Temp\\my_temp_directory\\temp_file.txt"
Contents of the temporary directory:
"C:\\Users\\Admin\\AppData\\Local\\Temp\\my_temp_directory\\temp_file.txt"

File Operations

7. Copying Files

Copying files is a common task, especially when working with backups or file distribution. Use the copy function to achieve this.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path src = "source_file.txt";
    std::filesystem::path dest = "destination_file.txt";
    try {
        std::filesystem::copy(src, dest);
        std::cout << "File copied successfully\n";
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }
}

Run @Compiler Explorer

The code won’t run, as we have no files:

filesystem error: cannot copy: No such file or directory [source_file.txt] [destination_file.txt]

8. Copying Files Recursively

Copying directories, including their contents, can be done using the copy function with recursive options.

#include <filesystem>
#include <iostream>
#include <fstream>

void create_temp_directories_and_files() {
    std::filesystem::create_directories("source_directory/subdir1");
    std::filesystem::create_directories("source_directory/subdir2");

    std::ofstream("source_directory/file1.txt") << "This is file 1";
    std::ofstream("source_directory/subdir1/file2.txt") << "This is file 2";
    std::ofstream("source_directory/subdir2/file3.txt") << "This is file 3";
}

int main() {
    create_temp_directories_and_files();

    std::filesystem::path src = "source_directory";
    std::filesystem::path dest = "destination_directory";
    try {
        std::filesystem::copy(src, dest, std::filesystem::copy_options::recursive);
        std::cout << "Directory copied successfully\n";
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }

    for (const auto& entry : std::filesystem::recursive_directory_iterator(dest)) {
        std::cout << entry.path() << '\n';
    }
}

Run @Compiler Explorer

The interesting part is that there’s a function overload which takes std::filesystem::copy_options:

enum class copy_options {
    none = /* unspecified */,
    skip_existing = /* unspecified */,
    overwrite_existing = /* unspecified */,
    update_existing = /* unspecified */,
    recursive = /* unspecified */,
    copy_symlinks = /* unspecified */,
    skip_symlinks = /* unspecified */,
    directories_only = /* unspecified */,
    create_symlinks = /* unspecified */,
    create_hard_links = /* unspecified */
};

See more examples @CppReference

9. Moving and Renaming Files

Moving and renaming files or directories can be done using the rename function.

#include <filesystem>
#include <iostream>
#include <fstream>

void ls() {
    for (const auto& entry : std::filesystem::directory_iterator(".")) 
        std::cout << entry.path() << '\n';
}

int main() {
    std::ofstream("old_file.txt") << "This is file 1";
    std::filesystem::path old_name = "old_file.txt";
    std::filesystem::path new_name = "new_file.txt";
    try {
        ls();
        std::filesystem::rename(old_name, new_name);
        std::cout << "File renamed/moved successfully\n";
        ls();
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }
}

Run @Compiler Explorer

10. Creating Hard Links

Hard links provide multiple directory entries for a single file. Use the create_hard_link function to create them.

#include <filesystem>
#include <iostream>
#include <fstream>
#include <format>

void ls() {
    for (const auto& entry : std::filesystem::directory_iterator(".")) 
        std::cout << std::format("{}, link count {}\n", entry.path().c_str(), entry.hard_link_count());
}

int main() {
    std::ofstream("target_file.txt") << "This is file 1";
    std::filesystem::path target = "target_file.txt";
    std::filesystem::path link = "hard_link_file.txt";
    try {
        std::filesystem::create_hard_link(target, link);
        std::cout << "Hard link created successfully\n";
        ls();
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }
}

Run @Compiler Explorer

Possible output:

Hard link created successfully
./compilation-result.json, link count 1
./target_file.txt, link count 2
./hard_link_file.txt, link count 2
./output.s, link count 1
./example.cpp, link count 1

11. Creating Symbolic Links

Symbolic links (symlinks) are another way to reference files. Use the create_symlink function for this.

unix - What is the difference between a symbolic link and a hard link? - Stack Overflow

#include <filesystem>
#include <iostream>
#include <fstream>

void display_file_content(const std::filesystem::path& path) {
    if (std::filesystem::exists(path)) {
        std::ifstream file(path);
        std::string content((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
        std::cout << "Content of " << path << ": " << content << '\n';
    } else {
        std::cout << path << " does not exist.\n";
    }
}

int main() {
    std::filesystem::path original_file = "original_file.txt";
    std::filesystem::path symlink = "symlink_to_file.txt";
    std::filesystem::path hardlink = "hardlink_to_file.txt";

    // Step 1: Create the original file
    std::ofstream(original_file) << "Hello World!";
    std::cout << "Original file created.\n";
    display_file_content(original_file);

    // Step 2: Create a symbolic link to the original file
    try {
        std::filesystem::create_symlink(original_file, symlink);
        std::cout << "Symbolic link created successfully.\n";
        display_file_content(symlink);
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }

    // Step 3: Create a hard link to the original file
    try {
        std::filesystem::create_hard_link(original_file, hardlink);
        std::cout << "Hard link created successfully.\n";
        display_file_content(hardlink);
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }

    // Step 4: Delete the original file and compare...
    std::filesystem::remove(original_file);
    std::cout << "Original file deleted.\n";
    display_file_content(symlink);
    display_file_content(hardlink);
}

Run @Compiler Explorer

Possible output:

Original file created.
Content of "original_file.txt": Hello World!
Symbolic link created successfully.
Content of "symlink_to_file.txt": Hello World!
Hard link created successfully.
Content of "hardlink_to_file.txt": Hello World!
Original file deleted.
"symlink_to_file.txt" does not exist.
Content of "hardlink_to_file.txt": Hello World!

Path and Existence Checks

12. Checking File or Directory Existence

Before performing operations on files or directories, it’s often necessary to check their existence using the exists function.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path p = "example_file.txt";
    if (std::filesystem::exists(p))
        std::cout << "File or directory exists\n";
    else
        std::cout << "File or directory does not exist\n";
}

Run @Compiler Explorer

13. Checking if a Path is a File or Directory

Differentiating between files and directories is crucial for many file processing tasks. The is_regular_file and is_directory functions help with this.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path p = "example_path";
    if (std::filesystem::is_regular_file(p))
        std::cout << "It's a file\n";
    else if (std::filesystem::is_directory(p))
        std::cout << "It's a directory\n";
    else
        std::cout << "It's neither a regular file nor a directory\n";
}

Run @Compiler Explorer

14. Reading Symlink Status

If your application deals with symbolic links, you may need to read the status of these links using the is_symlink function. To get the target file, you can use read_symlink():

#include <filesystem>
#include <iostream>
#include <fstream>

int main() {
    std::filesystem::path original_file = "original_file.txt";
    std::filesystem::path symlink = "symlink_to_file.txt";

    std::ofstream(original_file) << "Hello World!";
    std::cout << "Original file created.\n";

    try {
        std::filesystem::create_symlink(original_file, symlink);
        std::cout << "Symbolic link created successfully.\n";
    } catch (std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }

    if (std::filesystem::is_symlink(symlink)) {
        std::cout << symlink << " is a symbolic link.\n";
        std::filesystem::path target = std::filesystem::read_symlink(symlink);
        std::cout << "It points to: " << target << '\n';
    } else {
        std::cout << symlink << " is not a symbolic link.\n";
    }
}

Run @Compiler Explorer

15. Getting Absolute Path

Getting the absolute path of a relative path is often necessary for various file operations. Use the absolute function for this.

#include <filesystem>
#include <iostream>

int main() {
    // Example relative paths
    std::filesystem::path relative_path1 = "example_directory";
    std::filesystem::path relative_path2 = "../parent_directory";
    std::filesystem::path relative_path3 = "subdir/another_file.txt";
    
    // Convert to absolute paths
    std::filesystem::path absolute_path1 = std::filesystem::absolute(relative_path1);
    std::filesystem::path absolute_path2 = std::filesystem::absolute(relative_path2);
    std::filesystem::path absolute_path3 = std::filesystem::absolute(relative_path3);

    // Display the absolute paths
    std::cout << "Relative path: " << relative_path1 << " -> Absolute path: " << absolute_path1 << '\n';
    std::cout << "Relative path: " << relative_path2 << " -> Absolute path: " << absolute_path2 << '\n';
    std::cout << "Relative path: " << relative_path3 << " -> Absolute path: " << absolute_path3 << '\n';
}

Run @Compiler Explorer

Possible output:

Relative path: "example_directory" -> Absolute path: "/app/example_directory"
Relative path: "../parent_directory" -> Absolute path: "/app/../parent_directory"
Relative path: "subdir/another_file.txt" -> Absolute path: "/app/subdir/another_file.txt"

16. Getting Relative Path

Sometimes, you need to convert an absolute path to a relative path. The relative function can be used for this purpose.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path base_path = "/home/user";
    std::filesystem::path absolute_path = "/home/user/example_directory/file.txt";
    std::filesystem::path relative_path = std::filesystem::relative(absolute_path, base_path);
    std::cout << "Relative path: " << relative_path << '\n';
}

Run @Compiler Explorer

17. Displaying paths without quotes

Have you noticed that when I print out a path object, it always has quotes around it?

According to CppReference the stream operator for the path class is defined in the following way:

Performs stream input or output on the path p. std::quoted is used so that spaces do not cause truncation when later read by stream input operator.

To remove the quotes, we just need to get the raw string that represents the path. For example:

void ls() {
    for (const auto& entry : std::filesystem::directory_iterator(".")) 
        std::cout << entry.path().c_str() << '\n';
}

So the output can be:

./compilation-result.json
./output.s
./example.cpp

Rather than:

"./compilation-result.json"
"./output.s"
"./example.cpp"

Miscellaneous Operations

18. Calculating Directory Size

Calculating the total size of all files within a directory can be useful for disk usage analysis and cleanup tasks.

#include <filesystem>
#include <iostream>
#include <fstream>

// Function to create a test directory with some files and subdirectories
void create_test_directory(const std::filesystem::path& dir) {
    std::filesystem::create_directories(dir / "subdir1");
    std::filesystem::create_directories(dir / "subdir2");

    std::ofstream(dir / "file1.txt") << "ABC";
    std::ofstream(dir / "subdir1/file2.txt") << "XYZ";
    std::ofstream(dir / "subdir2/file3.txt") << "123";
}

// Function to calculate the total size of a directory
std::uintmax_t calculate_directory_size(const std::filesystem::path& dir) {
    std::uintmax_t size = 0;
    for (const auto& entry : std::filesystem::recursive_directory_iterator(dir)) {
        if (std::filesystem::is_regular_file(entry.path())) {
            size += std::filesystem::file_size(entry.path());
        }
    }
    return size;
}

int main() {
    // Create a test directory with some files and subdirectories
    std::filesystem::path test_dir = "test_directory";
    create_test_directory(test_dir);
    std::cout << "Test directory created.\n";

    // Calculate the total size of the test directory
    std::uintmax_t total_size = calculate_directory_size(test_dir);
    std::cout << "Total size of directory " << test_dir << ": " << total_size << " bytes\n";

    // Clean up by removing the test directory and its contents
    std::filesystem::remove_all(test_dir);
    std::cout << "Test directory removed.\n";
}

Run @Compiler Explorer

Do you know what the size of that test directory will be?

19. Determining Free Space on a Filesystem

Knowing the available free space on a filesystem is important for managing storage and preventing errors due to insufficient disk space. The space() function returns a space_info object that contains information about the free space, available space, and capacity of the filesystem where a given path is located.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path p = "/";
    auto space_info = std::filesystem::space(p);
    std::cout << "Free space: " << space_info.free << " bytes\n";
    std::cout << "Available space: " << space_info.available << " bytes\n";
    std::cout << "Capacity: " << space_info.capacity << " bytes\n";
}

Run @Compiler Explorer

20. Checking File Permissions

File permissions determine who can read, write, or execute a file. Use the status function to check these permissions.

#include <filesystem>
#include <iostream>

int main() {
    std::filesystem::path file = "example_file.txt";
    std::filesystem::perms p = std::filesystem::status(file).permissions();

    std::cout << "Permissions for " << file << ":\n";
    std::cout << ((p & std::filesystem::perms::owner_read) != std::filesystem::perms::none ? "r" : "-")
              << ((p & std::filesystem::perms::owner_write) != std::filesystem::perms::none ? "w" : "-")
              << ((p & std::filesystem::perms::owner_exec) != std::filesystem::perms::none ? "x" : "-")
              << '\n';
}

Run @Compiler Explorer

21. Setting File Permissions

Changing file permissions can be necessary for securing files or enabling certain operations. Use the permissions function to set these permissions.

#include <filesystem>
#include <iostream>
#include <fstream>

void create_test_file(const std::filesystem::path& path) {
    std::ofstream(path) << "This is a test file.";
    std::cout << "Test file created at: " << path << '\n';
}

void display_permissions(const std::filesystem::path& path) {
    std::filesystem::perms p = std::filesystem::status(path).permissions();
    std::cout << "Permissions for " << path << ": "
              << ((p & std::filesystem::perms::owner_read) != std::filesystem::perms::none ? "r" : "-")
              << ((p & std::filesystem::perms::owner_write) != std::filesystem::perms::none ? "w" : "-")
              << ((p & std::filesystem::perms::owner_exec) != std::filesystem::perms::none ? "x" : "-")
              << ((p & std::filesystem::perms::group_read) != std::filesystem::perms::none ? "r" : "-")
              << ((p & std::filesystem::perms::group_write) != std::filesystem::perms::none ? "w" : "-")
              << ((p & std::filesystem::perms::group_exec) != std::filesystem::perms::none ? "x" : "-")
              << ((p & std::filesystem::perms::others_read) != std::filesystem::perms::none ? "r" : "-")
              << ((p & std::filesystem::perms::others_write) != std::filesystem::perms::none ? "w" : "-")
              << ((p & std::filesystem::perms::others_exec) != std::filesystem::perms::none ? "x" : "-")
              << '\n';
}

int main() {
    std::filesystem::path test_file = "test_file.txt";
    create_test_file(test_file);

    std::cout << "Initial ";
    display_permissions(test_file);

    // Set new permissions for the test file
    std::filesystem::perms new_perms = std::filesystem::perms::owner_read |
                                       std::filesystem::perms::owner_write |
                                       std::filesystem::perms::owner_exec;

    try {
        std::filesystem::permissions(test_file, new_perms);
        std::cout << "Permissions changed successfully\n";
    } catch (const std::filesystem::filesystem_error& e) {
        std::cout << e.what() << '\n';
    }

    std::cout << "Updated ";
    display_permissions(test_file);

    std::filesystem::remove(test_file);
    std::cout << "Test file removed.\n";
}

Run @Compiler Explorer

Possible output:

Test file created at: "test_file.txt"
Initial Permissions for "test_file.txt": rw-rw-r--
Permissions changed successfully
Updated Permissions for "test_file.txt": rwx------
Test file removed.

22. Listing Files in Last Modification Time Order

Sorting and listing files based on their last modification time can be useful for many applications, such as finding the most recently modified files. The following example demonstrates how to list files in a directory sorted by their last modification time.

#include <iostream>
#include <fstream>
#include <filesystem>
#include <format>
#include <map>
#include <algorithm>
#include <chrono>
#include <thread>

namespace fs = std::filesystem;

int main() {
    std::map<fs::file_time_type, fs::path> files;
    std::vector<fs::path> toDelete;
    
    // create some files...
    for (int i = 0; i < 5; ++i)
    {
        toDelete.emplace_back(std::format("example{}.txt", i));
        std::ofstream(toDelete.back().c_str()) << "Hello, World!";
        std::this_thread::sleep_for(std::chrono::seconds(1));
    }

    for (const auto& entry : fs::directory_iterator(fs::current_path())) {
        if (entry.is_regular_file())
            files.emplace(fs::last_write_time(entry), entry.path());
    }
    
    for (const auto& [time, path] : files)
        std::cout << std::format("{0:%X} on {0:%F}, {1}\n", time, path.string());

    for (auto& f : toDelete)
        fs::remove(f);
}

Run @Compiler Explorer

Summary

In this blog post, we’ve explored a variety of essential filesystem operations using the std::filesystem library in C++17, C++20, and C++23. From creating directories and files to managing symbolic and hard links, and from checking file permissions to calculating directory sizes, we’ve covered the fundamental tasks you need to handle the filesystem efficiently.

I hope this guide has provided you with valuable insights and practical examples to help you navigate the complexities of filesystem operations in C++.

Back to you

Do you use std::filesystem?
What other operations are useful when working with the filesystem?

Share your comment below.

C++ Stories

Understanding std::shared_mutex from C++17

A Simple Thread-Safe Counter with std::mutex

When std::mutex Becomes a Bottleneck

Introducing std::shared_mutex

Refactoring the Counter with std::shared_mutex

Measuring gains

A More Realistic Example: Read-Mostly Cache

Common Pitfalls

Recursive locking is undefined

You cannot upgrade a shared lock

More locks do not always mean more performance

Newer Concurrency Tools in C++20 and Above

Summary

References

Back to you

IIFE for Complex Initialization

Intro

IIFE

Improving Readability of IIFE

Should we capture the whole context with & ?

C++26 Updates

Use Case of IIFE

A Benchmark of IIFE

Summary

Back to you

References and Books

7 Practical std::chrono Calendar Examples (C++20)

1. Which day of the year is that?

2. Nth weekday of a month

3. Last day of the month

4. Computing “last” dates for the whole year

5. Adding months

6. Next business day

7. Count Fridays in a month

Summary

Documents and Resources

Back to you

Exploring C++20 std::chrono - Calendar Types

Introduction to Calendar Types

Constants

Creation and Operator /

Literals

Validating Dates

Arithmetic

day

month

year

year_month_day

weekday

Days arithmetic for year_* and month_* types

Summary

Documents and Resources

C++ Templates: How to Iterate through std::tuple: C++26 Packs and Expansion Statements

Structured binding packs

Printing a tuple

Transforming a tuple

Dot product

Expansion statements

Transforming tuples

Summary

References:

Back to you

C++ Templates: How to Iterate through std::tuple: std::apply and More

std:apply approach

The first approach - working?

Printing simplification

Making it more generic

On std::decay and remove ref

Generic std::apply version

Return value

Summary

References:

Structured bindings in C++17, 8 years later

Iterating through maps

The Syntax

Real code

From chars handling

Unmap

Softmax

A Simple Thread-Safe Counter with `std::mutex`

When `std::mutex` Becomes a Bottleneck

Introducing `std::shared_mutex`

Refactoring the Counter with `std::shared_mutex`

Should we capture the whole context with `&` ?

Creation and Operator `/`

`day`

`month`

`year`

`year_month_day`

`weekday`

Days arithmetic for `year_` and `month_` types

Generic `std::apply` version

Splitting Ranges with `views::split`, C++20

What About `lazy_split`?

Grouping with `views::chunk`, C++23

Dynamic Grouping with `views::chunk_by`, C++23

`std::ranges::concat_view` (C++26)

`std::ranges::join_view` (C++20)

`std::ranges::join_with_view` (C++23)

Comparing `concat_view`, `join_view`, and `join_with_view`

3. Extents From `std::array`