This release contains bug fixes and enhancements, the most notable of which are:

• subclasses of pyre.application have a new journal component that enables partial control of various journal aspects as part of the normal application configuration
• the pyre::grid entities now use long as the default value type
• there is now fully functional support for s3 buckets in pyre.filesystem
• pyre::memory and pyre::tensor got makeovers

What's Changed

• cuda: updated year in copyright notice by @biancagi in #119
• Redesign of {tensor} with c++ {concepts} and implementation of new {tensor} features by @biancagi in #118
• Add GCC 13 to CI matrix by @rtburns-jpl in #121
• build(deps): bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows by @dependabot[bot] in #117
• Tensor by @biancagi in #123
• Issue129: adjust the uses of the canonical packing by pyre::tensor by @aivazis in #131
• Benchmarks with google-benchmark by @biancagi in #130
• Use WITH_SOABI to simplify python module suffixes by @rtburns-jpl in #122
• All pyre::grid entities now use long as the default value type by @aivazis in #139
• Reimplemented the base decorator by @aivazis in #149
• Enable user control over journal channels by @aivazis in #140
• Added support for specifying multiple configuration files with a single --config command line argument by @aivazis in #150
• Reimplemented the S3 filesystem support to address #153 by @aivazis in #154
• Switched to S3 uris that conform to the way AWS abuses the standard by @aivazis in #155
• Update pyre_h5.cmake by @trulko in #134
• Adjusted the formatting of alerts by @aivazis in #156

New Contributors

• @trulko made their first contribution in #134

Full Changelog: v1.12.5...v1.12.6

This is an update with small bug fixes and enhancements, the most notable of which is support for enum types in h5.

What's Changed

• cmake: set the include directories for {cuda} and {cudamodule} by @biancagi in #112

Full Changelog: v1.12.4...v1.12.5

This release is mostly about enhancements and cleanup of libpyre and libjournal:

• The new pyre::flow namespace contains the basic support for building data processing workflows. It is a pure C++ implementation of the ideas in pyre.flow, and will eventually become the foundation on which the python package is built. While still in the early stages of development, it is capable of expressing complicated dependency graphs with a relatively frictionless interface. The focus has been on running large datasets through the graph, but there is support for computing efficiently with nodes that carry small data packets.
• The support for visualizing rasters in pyre::viz was reimplemented using pyre::flow nodes. The new implementation uses arbitrarily shaped data tiles at each stage of the visualization pipeline, enabling a very powerful caching mechanism when needed. The data products and factories are templated on data types and storage mechanisms, making it easy to adopt the engine to a variety of computing contexts. The old implementation got an extensive makeover and is now in pyre::viz::iterators.
• The new pyre::cuda::memory namespace contains allocators for all CUDA memory models. They are modeled after the ones in pyre::memory. This normalizes the allocation interface and allows higher level objects, such as pyre::grid instances, to be built on top of CUDA memory wherever permitted. The code was generously donated by @SebastiaanVanPaasen and @biancagi; many thanks!
• Several holes to the interface of the null_t channel in journal were plugged to eliminate errors seen when channels like debug_t and firewall_t are optimized away.

At the python level:

• The support for h5 saw several improvements: new bindings for property lists, including creation and access lists for files and datasets, as well as support for attributes. There was a thorough investigation of ways to optimize access to h5 files in S3 buckets with spectacular results.
• There is now a new smith template for bootstrapping basic projects.

On the housekeeping front:

• There is now a home for benchmarks to host code that tests complex capabilities or measures implementation performance. These are essentially test drivers that are built on demand and not a part of the normal build of the code.
• There were many enhancements in the support for building with 'mm' and 'cmake', in the generation of docker images, on the way GitHub actions run the project CI, as well as the generation of binary distributions for pypi and conda-forge

Finally, this release contains an embedding of the mm build system as it is being absorbed by merlin. The goal is to eventually automate the build for all projects that are built on pyre without the need to clone any other repositories.

What's Changed

• Added support for flow diagrams by @aivazis in #110
• Memory allocators for CUDA pinned and mapped memory by @biancagi in #98

Full Changelog: v1.12.3...v1.12.4

This is another housekeeping release, this time with focus on publishing source and wheels to PyPI. In addition, the mm action was repaired and all is once again well.

Full Changelog: v1.12.2...v1.12.3

This maintenance release fixes the cmake build that was excluding some source files and causing unresolved symbols in the h5 bindings.

It also includes some housekeeping in the GitHub actions:

• separated the pypi source upload from the building of wheels so releases are available if wheel building fails
• narrowed the build matrix for mm and cmake to the latest two support versions
• abandoned ubuntu-20.04 as a platform, since it has been left behind by recent versions of the toolchain

Full Changelog: v1.12.1...v1.12.2

The support for HDF5 now includes all the machinery necessary to control the various cache mechanisms. The motivation behind these changes was to improve performance when accessing h5 files that are stored in S3 buckets. Credentials for AWS access can now be provided by specifying the URI of a dataset using the S3 scheme:

s3:profile@region/bucket/key

@biancagi contributed pyre::tensor, a library for performing tensor algebra efficiently on small matrices

What's Changed

• Added support for CUDA to CMake build system by @biancagi in #93
• Adjusted paths of python tests in CMake build system by @biancagi in #95
• Cleaned up unused variables in CMake configuration files by @biancagi in #96
• Merlin checkpoint by @aivazis in #101
• Run pipeline on pull request by @biancagi in #100
• algebra by @biancagi in #79
• algebra by @biancagi in #103
• Added support for fine tuning the performance of h5 accesses by @aivazis in #107

New Contributors

• @biancagi made their first contribution in #93

Full Changelog: v1.12.0...v1.12.1

HDF5 support

The bulk of the work in this release is about access to HDF5 files. The central idea is the specification of the expected layout of a file, its schema, in terms of the group structure that defines the content hierarchy, and type, shape and content validators for the datasets at the leaves. The implementation of the schema leans heavily on the same ideas that help describe pyre components to the framework. The available types are user extensible and can raise the level of abstraction by leaning of HDF5 for disk content while enabling the user to 'dress up' the content, especially when accessed at the python level. A notable example is the ability to store timestamps as strings in the file but have them auto-converted to datetime objects when their values are fetched.

Given a schema, readers are able to compare the actual contents of a file against the specification and adjust its content processors to match the intended use of the information, writers have all the information necessary to build the structure of an output file, and validators can generate compliance reports to help guide the standardization of the data product generation steps.

journal improvements

There is now near perfect symmetry between the C++ and native python implementations of journal. The C++ bindings now include the necessary trampoline classes so that users can subclass devices and formatters in python. The output devices of individual channels can now be controlled from python as well, and support for indent/outdent was added to assist in formatting the output without having to pass a string with margin content around. The default margin is held by the chronicler regardless of which implementation is active. So is the maximum detail level that is forwarded to the output device, while individual channel entries can decorate their output with detail level that is compared to the maximum in order to decide whether to silence an entry or not.

Memory and grids

The bindings for pyre::memory and pyre::grid were totally restructured and now include access to template expansions for more datatypes and shapes.

User environment

The framework now respects XDG compliant user environments when deciding where the user level configuration files reside

Templates

The smith project generator was totally restructured to generate code that reflects the current usage recommendations. It is now able to bootstrap react based GUIs with built-in support for all the pyre widgets.

Pull requests

• Support for accessing HDF5 data stored in S3 buckets by @aivazis in #86
• Significant improvements to the HDF5 integration layer by @aivazis in #90

Full Changelog: v1.11.2...v1.12.0

The bulk of the changes in this maintenance release are motivated by the work on the qed visualizer: improved bindings for pyre::grid to support the flat binary data readers, preliminary support for accessing HDF5 files to support reading NISAR data, and pyre::viz enhancements with new color spaces and filters.

Another minor enhancement is the support for opening journal logfiles in append mode requested by the isce3 team.

What's Changed

• Started fleshing out deep support for HDF5 files by @aivazis in #80

Full Changelog: v1.11.1...v1.11.2

This release contains several minor updates and bug fixes

• primed the pyre::memory and pyre::grid bindings with a small set of template expansions; the intent was to exercise a strategy for minimizing the manual labor involved in generating the template expansions to bind
• primed pyre::viz, the home for a new package with support for building visualization pipelines; the package is still in a primitive state, but it can already do quite a bit; several interesting examples can be found in the pyre.lib.viz test suite
• pyre.http got a css document type that returns the correct context headers; this should quiet down the picky browsers
• extensive updates to the react project template: caught up to react 17.x, version 6.x of react-router, the latest versions of relay and graphene, updated the overall application layout, and made extensive upgrades to the widgets
• merlin got make, a new builder that generates standalone makefiles; relocated the flow builder to make room for its new sibling

Full Changelog: v1.11.0...v1.11.1

This release marks another step towards pyre 2.0 with a complete makeover of libjournal. The C++ library has been reimplemented from scratch, and many features have been added. Some of this work was part of the interim v1.10.0 release; you can find an outline of those changes in the release notes. Since then

• pyre::journal has new manipulators that can inject color into the output, associate a user defined code with a message, control the indentation level of subsequent output, and specify the marker to place on the left margin to help identify journal output
• the verbosity level was renamed to detail; the former may be the familiar name historically given to this feature by a multitude of unix tools, but it has been pointed out that it is unnecessarily judgmental. Some backwards compatibility is retained, but the name is now marked deprecated and will be removed in v2.0.
• the bland formatter was added; it displays just the injected text with no decoration or message metadata
• there is now a help channel that uses the bland formatter, intended for building help screens with minimal distractions
• the visual layout of messages generated by the C++ library and the python package had some inconsistencies; these have been fixed, the output is cleaner, and it looks the same regardless of the engine that generated it
• the python bindings provide report, a new channel method that renders a sequence of strings; the most interesting aspect of this is that report will accept a generator, so output can now be gathered while traversing hierarchical data structures

In other news:

• pyre.config now has yaml support; configuration files with the .yaml suffix are processed using pyyaml, if it is available. They are higher priority than the built in .pfg files. Please note that the # character is a comment in yaml, so certain pyre expressions that use URI syntax and have embedded #s must be placed in quotation marks
• the configuration cascade in pyre.components has improved significantly. Applications can now register a naming service so that components that are created implicitly can get custom names, components get unique ids using libuuid that can persist across sessions, and the life cycle event hooks have been streamlined. Protocols have much more say on how their implementers are discovered and instantiated
• pyre::memory got stack, a new storage strategy that uses std::array for storage; it is free of dynamic memory allocations so it is fast, but it is only appropriate for relatively small objects. It is really meant to be used to represent tensors of small rank, but it can be composed as the cell_t of a pyre::grid to store arbitrarily large tensor fields. More of this in the next release.

Notable housekeeping tasks include

• the move of the CI system to GitHub actions, motivated by the fact that the pyre travis bucks ran out. Currently, the new CI system checks both mm and cmake builds with five different compilers, two versions of python, and three different build targets. It is much more thorough. I am grateful to GitHub and Microsoft for making this possible -- and free.
• the retirement of the eoan docker images, as the basis image is no longer available; they were replaced by hirsute images. The generation of these images was streamlined a bit, but it still mostly a copy-and-paste job. There must be a better way to generate the Dockerfiles...

And finally, merlin is coming...