SciCompMod · reneSchm · Jun 28, 2024 · Jun 21, 2024 · Jun 21, 2024 · Jun 21, 2024
diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
@@ -20,6 +20,7 @@ option(MEMILIO_ENABLE_OPENMP "Enable Multithreading with OpenMP." OFF)
 option(MEMILIO_ENABLE_WARNINGS "Build memilio with warnings." ON)
 option(MEMILIO_ENABLE_WARNINGS_AS_ERRORS "Build memilio with warnings as errors." ON)
 option(MEMILIO_ENABLE_IPOPT "Enable numerical optimization with Ipopt, requires a Fortran compiler" OFF)
+option(MEMILIO_ENABLE_PROFILING "Enable runtime performance profiling of memilio" OFF)
 
 mark_as_advanced(MEMILIO_USE_BUNDLED_SPDLOG MEMILIO_SANITIZE_ADDRESS MEMILIO_SANITIZE_UNDEFINED)
 

diff --git a/cpp/README.md b/cpp/README.md
@@ -10,6 +10,7 @@ Directory structure:
 - tests: unit tests for framework and models.
 - cmake: build utility code
 - thirdparty: configuration of dependencies
+- benchmarks: applications to analyze runtime performance of the framework
 
 ## Requirements
 
@@ -53,6 +54,11 @@ Options can be specified with `cmake .. -D<OPTION>=<VALUE>` or by editing the `b
 - `MEMILIO_USE_BUNDLED_SPDLOG/_BOOST/_EIGEN/_JSONCPP`: use the corresponding dependency bundled with this project, ON or OFF, default ON.
 - `MEMILIO_BUILD_BENCHMARKS`: build the benchmarks for this project, ON or OFF, default OFF.
 - `MEMILIO_SANITIZE_ADDRESS/_UNDEFINED`: compile with specified sanitizers to check correctness, ON or OFF, default OFF.
+- `MEMILIO_ENABLE_OPENMP`: compile MEmilio with multithreading using OpenMP, ON or OFF, default OFF.
+- `MEMILIO_ENABLE_MPI`: compile MEmilio with distributed memory parallelization using MPI. ON or OFF, default OFF. Requires an MPI implementation to be installed on the system. 
+- `MEMILIO_ENABLE_WARNINGS`: enable compilation warnings (beyond those enabled in the compiler by default). ON or OFF, default ON.
+- `MEMILIO_ENABLE_WARNINGS_AS_ERRORS`: compilation warnings are treated as compilation errors. ON or OFF, default ON.
+- `MEMILIO_ENABLE_PROFILING`: compile with runtime profiling support. ON or OFF, default OFF. See [here](benchmarks/profiling.md) for information.
 
 Other important options may need:
 - `CMAKE_BUILD_TYPE`: controls compiler optimizations and diagnostics, Debug, Release, or RelWithDebInfo; not available for Multi-Config CMake Generators like Visual Studio, set the build type in the IDE or when running the compiler.

diff --git a/cpp/benchmarks/scorep_measuring.md → cpp/benchmarks/profiling.md b/cpp/benchmarks/scorep_measuring.md → cpp/benchmarks/profiling.md
@@ -1,4 +1,39 @@
-# Performance measuring with Score-P
+# Profiling
+
+This document explains how to create performance profiles, i.e., measurements of where in the code time is spent, usually aggregated by function. There is a variety of open-source or proprietary tools available. Generally, there are three types of profilers:
+- instrumenting: modify the compiled code to include code for measurement. Advantage: exact function call counts and times, no approximations. Disadvantage: instrumentation adds runtime overhead that can distort the result or disable compiler optimizations (e.g. functions aren't inlined), so short functions usually must be excluded.
+- sampling: checking the CPU stack register in regular intervals to see which code is currently being executed. Advantage: usually low measurement runtime overhead. Disadvantage: stochastic results, imprecise especially for short functions, longer runtimes improve averages.
+- emulating: running the program in a virtual environment. Advantage: exact and able to measure even small functions. Disadvantage: program can run significantly longer, no real world timing results but instruction cycle counts are generally directly translatable.
+
+These tools have been tried for memilio and are described below:
+- gperftools: sampling profiler that gives reliable results with little setup
+- Score-P: large suite that supports both sampling and instrumenting. Special support for parallel code (OpenMP and MPI), including tracing to find load imbalances in the parallelization. 
+- Valgrind: suite that includes the Callgrind emulating profiler; detailed descripton will be added soon, until then see the [official documentation](https://valgrind.org/info/tools.html).
+
+How to use these tools, the general optimization loop:
+1. Run a benchmark to get a baseline runtime and identify the need for performance optimization, e.g., a change introduced a performance regression.
+2. Create a profile to see where time is lost; maybe compare with a profile before your changes.
+3. Optimize the code.
+4. Test the functionality.
+5. Confirm the optimization with a benchmark and profile.
+6. Repeat if necessary.
+
+## gperftools
+
+gperftools (formerly google performance tools) is a suite of performance tools. Here the focus is on the [profiler](https://gperftools.github.io/gperftools/cpuprofile.html). It's a small sampling profiler that is quick to setup and use on most systems. It's integrated into our build system for even greater convenience.
+
+Basic steps:
+1. Install gperftools. It's available in many package managers (apt packages `google-perftools` and `libgoogle-perftools-dev`, spack or homebrew package `gperftools`). E.g., on the DLR-SC hpda cluster (insert the compiler version you are using): `module load spack-user; module load PrgEnv/gcc13-openmpi; spack install gperftools%[email protected]; module load gperftools`.
+2. configure memilio with cmake variable `MEMILIO_ENABLE_PROFILING=ON`. Note that compiling with profiling enabled does not incur any runtime overhead unless profling is also enabled at runtime (see step 4), so developers can just enable it always for convenience.
+3. compile memilio.
+4. run the program with environment variable `CPUPROFILE=profile.out` set.
+5. generate a human-readable annotated call graph: `pprof-symbolize --pdf <exe> profile.out > profile.pdf`. Check the documentation for other output formats.
+
+This generates a profile of the whole program. For each function, the graph contains the number of times the function was sampled and how much time is spent in the function as a percentage of the total runtime. Both sample count and percentage are displayed as exclusive numbers (i.e. the function without child functions) and inclusive numbers (the function including its child functions). 
+
+Probably not all parts of the program are interesting to profile. It's possible to filter the profile, e.g., to look at specific functions, using the `--focus` flag of `pprof-symbolize` or by starting the profiling manually using the `MEMILIO_PROFILER_START(<filename>)` and `MEMILIO_PROFILER_STOP` macros in `memilio/utils/profiler.h`. E.g., when profiling the `abm_simulation` program, it may be a good idea to exclude pre- and postprocessing by starting profiling right before the call to `Simulation::advance`.
+
+## Score-P
 
 Score-P is a tool that can be used to measure the perfomance of parallel code and supports profiling and tracing. For information on how to get it, please have a look at the [Score-P quickstart](https://scorepci.pages.jsc.fz-juelich.de/scorep-pipelines/docs/scorep-6.0/html/quickstart.html). It can be installed with the spack manager (and probably many other package managers) or loaded as a module on hpc systems.
 For the visualization of the performance data you need Cube (information on installation, see [here](https://apps.fz-juelich.de/scalasca/releases/cube/4.3/docs/CubeInstall.pdf)). It can load both Score-P profiles and Scalasca traces and you can find details about the loading and usage below or in the [User Guide](https://www.vi-hps.org/cms/upload/packages/cube/CubeGuide.pdf).
@@ -18,7 +53,7 @@ The [filter file](./scorep-filter-abm) is designed to include only the interesti
 
 To avoid the build errors about "include style" and "gcc extension" set ```-DMEMILIO_ENABLE_WARNINGS_AS_ERRORS=OFF``` during cmake configuration.
 
-## Profile measurements
+### Profile measurements
 
 To run the basic analysis with profiling:
 ```
@@ -109,7 +144,7 @@ However, there are some functions that still appear in the profile that do not a
       SCOREP         41      1   57.83    89.3    57830258.74  abm_simulation
          COM         24      1    0.00     0.0         532.58  void?mio::abm::Simulation::advance(mio::abm::TimePoint,?History&?...)??with?History?=?{mio::History<mio::abm::TimeSeriesWriter,?mio::abm::LogInfectionState>}?
 
-## Tracing
+### Tracing
 
 For deeper analysis, use Scalasca for tracing:
 

diff --git a/cpp/memilio/config_internal.h.in b/cpp/memilio/config_internal.h.in
@@ -29,5 +29,6 @@
 #cmakedefine MEMILIO_HAS_JSONCPP
 #cmakedefine MEMILIO_ENABLE_MPI
 #cmakedefine MEMILIO_ENABLE_OPENMP
+#cmakedefine MEMILIO_ENABLE_PROFILING
 
 #endif
diff --git a/cpp/memilio/utils/profiler.h b/cpp/memilio/utils/profiler.h
@@ -0,0 +1,51 @@
+/* 
+* Copyright (C) 2024 MEmilio
+*
+* Authors: Daniel Abele
+*
+* Contact: Martin J. Kuehn <[email protected]>
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+#ifndef MIO_UTILS_PROFILER_H
+#define MIO_UTILS_PROFILER_H
+
+#include "memilio/config.h"
+
+#ifdef MEMILIO_ENABLE_PROFILING
+
+#include "gperftools/profiler.h"
+
+/**
+* Start recording a runtime profile from this line until MEMILIO_PROFILER_STOP().
+* If using these macros, the `CPUPROFILE` environment variable should not be set,
+* the path to the profile output file specified by the macro argument is used instead.
+* @param file path to the file where the profile should be stored, as `const char*`.
+*/
+#define MEMILIO_PROFILER_START(file) ProfilerStart(file)
+
+/**
+* Stop recording the runtime profile.
+*/
+#define MEMILIO_PROFILER_STOP() ProfilerStop()
+
+#else //MEMILIO_ENABLE_PROFILING
+
+//empty definitions so the macros can remain in the code if profiling is disabled.
+#define MEMILIO_PROFILER_START(file)
+#define MEMILIO_PROFILER_STOP()
+
+#endif //MEMILIO_ENABLE_PROFILING
+
+#endif //MIO_UTILS_PROFILER_H
diff --git a/cpp/thirdparty/CMakeLists.txt b/cpp/thirdparty/CMakeLists.txt
@@ -8,6 +8,18 @@ set(MEMILIO_JSONCPP_VERSION "1.9.5")
 set(MEMILIO_RANDOM123_VERSION "v1.14.0")
 set(MEMILIO_IPOPT_VERSION "3.14.12")
 
+# Gperftools for profiling; must be first, so that libraries are included in the profile
+if(MEMILIO_ENABLE_PROFILING)
+    if(NOT UNIX)
+        message(FATAL_ERROR "Profiling with gperftools currently only supported on unix systems.")
+    endif()
+
+    find_package(PkgConfig REQUIRED)
+    pkg_check_modules(GPERFTOOLS REQUIRED libprofiler)
+    link_libraries(${GPERFTOOLS_LINK_LIBRARIES}) # link globally so all libs are included
+    include_directories(${GPERFTOOLS_INCLUDE_DIRS})
+endif()
+
 # ## SPDLOG
 set(SPDLOG_INSTALL ON)