CFD

BLAS + LAPACK (OpenBLAS + FLAME + ATLAS + MAGMA + KBLAS) and friends

• C++ tensors with broadcasting and lazy computing

  • xtensor
  • The x template library

• Benchmarking Single- and Multi-Core BLAS Implementations and GPUs for use with R

• OpenBLAS: An optimized BLAS library

  • OpenBLAS github
  • tester for BLAS libraries including OpenBLAS and Intel MKL
  • Vector Math Library

• BLIS (flame)

  • github repo for BLIS
  • github repo for FLAME

• Matrix Template Library 4

• GEMM: From Pure C to SSE Optimized Micro Kernels ulmBLAS: parent of the above link

• GEMV kernel

• GPU Accelerated Memory-bound Linear Algebra Kernels" KBLAS + KSPARSE

  • PhD thesis 4 KBLAS+KSPARSE
  • KBLAS: An Optimized Library for Dense Matrix-Vector Multiplication on GPU Accelerators
  • KBLAS: download
  • KSPARSE: download
  • bhSPARSE: A Sparse BLAS Library
  • fmmtl: FMM Template Library -- a structured dense matrix algorithms library, GPU accelerated, Simon Layton is a coauthor
  • LAMA -- Library for Accelerated Math Applications -- GPU accelerated
    • sourceforge project for LAMA

• blaze-lib -- A high performance C++ math library -- claims to be the fastest, supports AVX

  • wiki link compares blaze vs. other C++ libraries

• The Numerical Template Toolbox - C++ Scientific Computing Made Easy

  • The Numerical Template Toolbox - C++ Scientific Computing Made Easy docs

• SIMD Vector Classes for C++

• EVE - the Expressive Vector Engine

• High-Performance Matrix-Matrix Multiplications of Very Small Matrices

• AUGEM: Automatically generate high performance Dense Linear Algebra kernels on x86 CPUs

• psBLAS -- Parallel Sparse Basic Linear Algebra Subroutines -- Fortran2003

• Ginkgo is a high-performance linear algebra library for manycore systems, with a focus on sparse solution of linear systems. It is implemented using modern C++ (you will need at least C++14 compliant compiler to build it), with GPU kernels implemented in CUDA and HIP

  • ginkgo: getting started
    tl;dr

  git clone https://github.com/ginkgo-project/ginkgo.git
  cd ginkgo
  mkdir build
  cd build
  ccmake ..
  make -j4

• GraphBLAS Repositories dedicated to Graphs in the Language of Linear Algebra

  • GraphBLAS Pointers
  • GraphBLAS ycombinator
  • SuiteSparse:GraphBLAS, Timothy A. Davis, (c) 2017-2023, All Rights Reserved.
  • ibmgraphblas
  • GraphBLAS Template Library (GBTL), v. 3.0
  • Rust wrappers to GraphBLAS
    • graphblas_sparse_linear_algebra
      • graphblas_sparse_linear_algebra
    • rustgraphblas

• Elemental is a modern C++ library for distributed-memory dense and sparse-direct linear algebra and optimization.

  • Elemental.jl

• Recommendations for a usable, fast C++ matrix library?

• The Tensor Algebra Compiler (taco) computes sparse tensor expressions on CPUs and GPUs

  • A fast and versatile library for linear and tensor algebra

• amgcl : C++ library for solving large sparse linear systems with algebraic multigrid method

  • amgl docs : AMGCL is a header-only C++ library for solving large sparse linear systems with algebraic multigrid (AMG) method
    • amgl issue : MPI & preconditioner+solver setup

The Amazing Performance of C++17 Parallel Algorithms, is it Possible?

SIMD

• The C++ scientist

Spectra - C++ Library For Large Scale Eigenvalue Problems

• A header-only C++ library for large scale eigenvalue problems
  • Spectra C++ Library For Large Scale Eigenvalue Problems
• Spectra Performance

Dimension Reduction

• A flexible and efficient С++ template library for dimension reduction
  • Tapkee an efficient dimension reduction library

Wavelet and Multiscale Library

• The Marburg Software Library is a collection of wavelet based c++ algorithms to solve multivariate partial differential equations.
• Parallel adaptive wavelet collocation method for PDEs
• Wavelet methods in computational fluid dynamics

BEM - Boundary Element Library

• Curated list of Hierarchical Matrices (H-Matrices) libraries
• BEM++ is an open-source Galerkin boundary element library that handles Laplace, Helmholtz and Maxwell problems on bounded and unbounded domains
  • BEM++ github repo
    • Bempp-rs is an (in early development) boundary element library written in Rust with a Python interface
    • ExaFMM: a high-performance fast multipole method library with C++ and Python interfaces
    • BEM++ fork
• Boundary Element Template Library
• HyENA - HyENA stands for [Hy]perbolic and [E]lliptic [N]umerical [A]nalysis and is a C++ library
• Obtaining the source code for NiHu tl;dr

git clone -b release_2.0 git://last.hit.bme.hu/toolbox/nihu.git

• Formulating various BEM problems with an open source C++ BEM template library

• fmm-bem-relaxed

• SCUFF-EM

  • PhD Thesis on SCUFF-EM

• Hierarchical matrices

  • H2Lib is an open source software library for hierarchical matrices and H²-matrices
    • H2Lib public github repository
  • AHMED - Another software library on hierarchical matrices for elliptic differential equations github repository
    • AHMED presentation

The C++ Standards Library for Concurrency and Parallelism http://stellar-group.org/libraries/hpx

• HPX (High Performance ParalleX) is a general purpose C++ runtime system for parallel and distributed applications of any scale
• HPX — рантайм-система для параллельных и распределенных вычислений

Automatic Differentiation: mostly Julia, C++, Haskell. Scala as well

• CasADi framework for nonlinear optimization

  • CasADi framework for nonlinear optimization
  • CasADi is a symbolic framework for numeric optimization implementing automatic differentiation in forward and reverse modes on sparse matrix-valued computational graphs
  • CasADi FAQ
  • CasADi docs
  • tipsandtricks for CasADi

• automatic-differentiation

  • Tensors and differentiable operations (like TensorFlow) in Rust https://docs.rs/autograd/
  • Kotlin𝛁: Differentiable Functional Programming with Algebraic Data Types https://github.com/breandan/kotlingrad

• JuliaDiff Differentiation tools in Julia

  • ForwardDiff.jl github
    • JSoC 2015 project: Automatic Differentiation in Julia with ForwardDiff.jl
    • ForwardDiff.jl docs
      • ForwardDiffNumber Types
        • ForwardDiff contributing to
    • Julia automatic differentiation with source code transformation
    • Àlex Haro Provinciale
    • Latest publications of Olivier Pironneau
  • Automatic differentiation
    • insightful intro to AD
    • /which-automatic-differentiation-tool-for-cc/
      • HyperDual Numbers: Exact numeric nth derivatives
    • Lazy Time Reversal, and Automatic Differentiation
  • Integration analog of automatic differentiation
    • Risch algorithm
      • GSoC 2013 Application Chetna Gupta: Risch algorithm for symbolic integration
      • Q: Automatic differentiation for numerical integration
      • Derivative of definite integral
        • Differentiation under the integral sign problem
    • Chebfun—numerical computing with functions
    • ApproxFun - Julia package for function approximation
      • A practical framework for infinite-dimensional linear algebra
  • Symbolic math with julia

• Haskell Automatic Differentiation

  • What is automatic differentiation, and why does it work?

• Why Automatic Differentiation Won’t Cure Your Calculus Blues

• Automatic Differentiation in Computational Science : presentation

• Reviews of AD including computational cost of FAD/BAD

  • A Hitchhiker's Guide to Automatic Differentiation - up-to-date - all you have to know about AD, Haskell focused
    • Dr Philipp Hoffmann personalia
      • National University of Ireland Maynooth Brain and Computation Lab
        • Hype: Compositional Machine Learning and Hyperparameter Optimization
  • Automatic differentiation in machine learning: a survey contains estimation for number of operations
  • AD and Sparse Matrices (Computational Complexity of Forward Mode AD)
  • Towards a Computer Algebra System with Automatic Differentiation for use with Object-Oriented modelling languages (the paper describes DAG representation for representing optimization problem)
  • Non-smooth Optimization and Automatic Differentiation (skript of the lecture)
  • PhD Thesis A Small-Perturbation Automatic-Differentiation (SPAD) Method for Evaluating Uncertainty in Computational Electromagnetics
  • Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator
    • Barak A. Pearlmutter publications
    • Jeffrey Mark Siskind's Software
  • A simple explanation of reverse-mode automatic differentiation
    • Automatic Differentiation: The most criminally underused tool in the potential machine learning toolbox?
    • Review and Unification of Methods for Computing Derivatives of Multidisciplinary Computational Models

• Applications of AD to Real World

  • Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method
  • Automatic Differentiation Adjoint of the Reynolds-Averaged Navier–Stokes Equations with a Turbulence Model AIAA 2013-2581
  • AD for Optimization
    • Ceres Solver : can solve bounds constrained robustified non-linear least squares problems

• AD in Haskell

• CS267 Project Report Towards Parallel Automatic Differentiation

• Autograd

  • autograd: Efficiently computes derivatives of numpy code
  • Autograd automatically differentiates native Torch code. Inspired by the original Python version

• Backup

  • fast automatic differentiation jacobians by compact lu factorization

Unum

• What are the biggest challenges in scientific computing?

  • John Gustafson presents: Beyond Floating Point – Next Generation Computer Arithmetic + (former Unum)Create RUST posit arithmetic library based on 21 February 2017 Posit Arithmetic by John L. Gustafson - Beating Floating Point at its Own Game

    • UNUM ... a better alternative to IEEE Floating Point?
    • The Evils of Floating Point, and the Joys of Unum
    • John Gustafson (scientist)
    • Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics
      • Unum Computing slideshare
      • Unum Computing Is A Breakthrough ... A Game Changer
      • Handbook of Floating-Point Arithmetic
      • Rump's Royal Pain
      • Official Blog - SSRLabs for UNUMs implemented in hardware
    • Unums.jl
    • Julia Implementation of Unums - (seems more mature)
    • unum - Rust library for unums, a placeholder @Jan 4, 2016

    julia> f(x,y) = 333.75*y^6 + x^2*(11*x^2*y^2-y^6-121*y^4-2)+5.5*y^8+x/(2*y)
    f (generic function with 1 method)
    
    julia> f(77617,33096)
    1.64176022256015e21
    
    julia> f(Rational(77617,1),Rational(33096,1))
    ERROR: OverflowError()
     in * at rational.jl:188
     in power_by_squaring at intfuncs.jl:96
     in f at none:1
    
    julia> f(Rational(BigInt(77617),1),Rational(BigInt(33096),1))
    -8.273960599468213681411650954798162919990331157843848199178148416727096930142628e-01
    
    julia> f(77617.0,33096.0)
    -1.1805916207174113e21
    
    julia> f(BigFloat(77617),BigFloat(33096))
    -8.273960599468213681411650954798162919990331157843848199178148416727096930142628e-01

    • (more or less) accurate floating point algorithms

    • unum in D

      • Unums.jl
      • Julia Implementation of Unums - (seems more mature)
      • Experimental Unums

    • [PosIt(new Unum) in C, C++]

      • Unum & Posit
        • Toward Floating-Point Run-time Variable Precision in CPU
        • PERCIVAL: Open-Source Posit RISC-V Core with Quire Capability
      • SoftPosit
        • Berkeley SoftFloat
      • Universal Number Arithmetic in C++

• Automatic Fortran to C++ conversion

  • Automatic Fortran to C++ conversion with FABLE

• Promising Languages for Scientific Computing (besides Julia)

  • Polyglot Programming : review several candidate languages, namely C, C++, Fortran, Cython, Numba, Nim, Rust, D, Chapel and Julia
  • SciLua: Scientific Computing with LuaJIT
  • Chapel Quickstart Instructions tl;dr

    source util/quickstart/setchplenv.bash
    make
    make check
    chpl -o hello examples/hello.chpl

• linear algebra library Eigen

  • Eigen Cheat sheet
  • Eigen Aliasing
  • Eigen 3.2.92 : Bug 1221: disable gcc 6 warning: ignoring attributes on template argument
  • Eigen disable GCC diagnostic ignored "-Wignored-attributes" has already merged
  • An evaluation of the Eigen linear algebra library for use in the aither CFD solver
  • A look at the performance of expression templates in C++: Eigen vs Blaze vs Fastor vs Armadillo vs XTensor
  • How should I initialize the contents of a large matrix in Eigen?

AeroSandbox