Skip to content

Commit 8616f71

Browse files
authored
652 Add serialization to ABM (#1072)
- Add a default serialization feature to make adding (de)serialization more convenient. - Add a TimeSeriesFunctor to replace std::functions in ABM parameters. - Make several objects serializable, especially from the ABM.
1 parent 0134012 commit 8616f71

39 files changed

Lines changed: 1856 additions & 645 deletions

cpp/CMakeLists.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,8 @@ if(CMAKE_BUILD_TYPE STREQUAL "Debug" OR CMAKE_BUILD_TYPE STREQUAL "DEBUG")
7676
message(STATUS "Coverage enabled")
7777
include(CodeCoverage)
7878
append_coverage_compiler_flags()
79-
# In addition to standard flags, disable elision and inlining to prevent e.g. closing brackets being marked as uncovered.
80-
79+
# In addition to standard flags, disable elision and inlining to prevent e.g. closing brackets being marked as
80+
# uncovered.
8181
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-elide-constructors -fno-default-inline")
8282
setup_target_for_coverage_lcov(
8383
NAME coverage

cpp/memilio/CMakeLists.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ add_library(memilio
2727
compartments/simulation.h
2828
compartments/flow_simulation.h
2929
compartments/parameter_studies.h
30+
io/default_serialize.h
31+
io/default_serialize.cpp
3032
io/io.h
3133
io/io.cpp
3234
io/hdf5_cpp.h
@@ -57,6 +59,8 @@ add_library(memilio
5759
math/matrix_shape.cpp
5860
math/interpolation.h
5961
math/interpolation.cpp
62+
math/time_series_functor.h
63+
math/time_series_functor.cpp
6064
mobility/metapopulation_mobility_instant.h
6165
mobility/metapopulation_mobility_instant.cpp
6266
mobility/metapopulation_mobility_stochastic.h

cpp/memilio/io/README.md

Lines changed: 53 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,25 @@ This directory contains utilities for reading and writing data from and to files
44

55
## The Serialization framework
66

7-
## Main functions and types
7+
### Using serialization
8+
9+
In the next sections we will explain how to implement serialization (both for types and formats), here we quickly show
10+
how to use it once it already is implemented for a type. In the following examples, we serialize (write) `Foo` to a
11+
file in Json format, then deserialize (read) the Json again.
12+
```cpp
13+
Foo foo{5};
14+
mio::IOResult<void> io_result = mio::write_json("path/to/foo.json", foo);
15+
```
16+
```cpp
17+
mio::IOResult<Foo> io_result = mio::read_json("path/to/foo.json", mio::Tag<Foo>{});
18+
if (io_result) {
19+
Foo foo = io_result.value();
20+
}
21+
```
22+
There is also support for a binary format. If you want to use a format directly, use the
23+
`serialize_json`/`deserialize_json` and `serialize_binary`/`deserialize_binary` functions.
24+
25+
### Main functions and types
826

927
- functions serialize and deserialize:
1028
Main entry points to the framework to write and read values, respectively. The functions expect an IOContext
@@ -14,7 +32,38 @@ This directory contains utilities for reading and writing data from and to files
1432
- IOStatus and IOResult:
1533
Used for error handling, see section "Error Handling" below.
1634

17-
## Concepts
35+
### Default serialization
36+
37+
Before we get into the details of the framework, this feature provides an easy and convenient alternative to the
38+
serialize and deserialize functions. To give an example:
39+
40+
```cpp
41+
struct Foo {
42+
int i;
43+
auto default_serialize() {
44+
return Members("Foo").add("i", i);
45+
}
46+
};
47+
```
48+
The default serialization is less flexible than the serialize and deserialize functions and has additional
49+
requirements:
50+
- The class must be default constructible.
51+
- If there is a default constructor that is *private*, it can still be used by marking the struct `DefaultFactory` as
52+
a friend. For the example above, the line `friend DefaultFactory<Foo>;` would be added to the class definition.
53+
- Alternatively, you may provide a specialization of the struct `DefaultFactory`. For more details,
54+
view the struct's documentation.
55+
- Every class member must be added to `Members` exactly once, and the provided names must be unique.
56+
- The members must be passed directly, like in the example. No copies, accessors, etc.
57+
- It is recommended, but not required, to add member variables to `Members` in the same order they are declared in
58+
the class, using the variables' names or something very similar.
59+
- Every class member itself must be serializable, deserializable and assignable.
60+
61+
As to the feature set, default-serialization only supports the `add_element` and `expect_element` operations defined in
62+
the Concepts section below, where each operation's arguments are provided through the `add` function. Note that the
63+
value provided to `add` is also used to assign a value during deserialization, hence the class members must be used
64+
directly in the function (i.e. as a non-const lvalue reference).
65+
66+
### Concepts
1867
1968
1. IOContext
2069
Stores data that describes serialized objects of any type in some unspecified format and provides structured
@@ -66,7 +115,7 @@ for an IOObject `obj`:
66115
value or it may be empty. Otherwise returns an error. Note that for some formats a wrong key is indistinguishable from
67116
an empty optional, so make sure to provide the correct key.
68117
69-
## Error handling
118+
### Error handling
70119
71120
Errors are handled by returning error codes. The type IOStatus contains an error code and an optional string with additional
72121
information. The type IOResult contains either a value or an IOStatus that describes an error. Operations that can fail return
@@ -78,7 +127,7 @@ inspected, so `expect_...` operations return an IOResult. The `apply` utility fu
78127
of multiple `expect_...` operations and use the values if all are succesful. See the documentation of `IOStatus`, `IOResult`
79128
and `apply` below for more details.
80129
81-
## Adding a new data type to be serialized
130+
### Adding a new data type to be serialized
82131
83132
Serialization of a new type T can be customized by providing _either_ member functions `serialize` and `deserialize` _or_ free functions
84133
`serialize_internal` and `deserialize_internal`.

cpp/memilio/io/binary_serializer.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -278,7 +278,7 @@ class BinarySerializerContext : public SerializerBase
278278
"Unexpected type in stream:" + type_result.value() + ". Expected " + type);
279279
}
280280
}
281-
return BinarySerializerObject(m_stream, m_status, m_flags);
281+
return obj;
282282
}
283283

284284
/**
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
/*
2+
* Copyright (C) 2020-2024 MEmilio
3+
*
4+
* Authors: Rene Schmieding
5+
*
6+
* Contact: Martin J. Kuehn <[email protected]>
7+
*
8+
* Licensed under the Apache License, Version 2.0 (the "License");
9+
* you may not use this file except in compliance with the License.
10+
* You may obtain a copy of the License at
11+
*
12+
* http://www.apache.org/licenses/LICENSE-2.0
13+
*
14+
* Unless required by applicable law or agreed to in writing, software
15+
* distributed under the License is distributed on an "AS IS" BASIS,
16+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17+
* See the License for the specific language governing permissions and
18+
* limitations under the License.
19+
*/
20+
#include "memilio/io/default_serialize.h"

cpp/memilio/io/default_serialize.h

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
/*
2+
* Copyright (C) 2020-2024 MEmilio
3+
*
4+
* Authors: Rene Schmieding
5+
*
6+
* Contact: Martin J. Kuehn <[email protected]>
7+
*
8+
* Licensed under the Apache License, Version 2.0 (the "License");
9+
* you may not use this file except in compliance with the License.
10+
* You may obtain a copy of the License at
11+
*
12+
* http://www.apache.org/licenses/LICENSE-2.0
13+
*
14+
* Unless required by applicable law or agreed to in writing, software
15+
* distributed under the License is distributed on an "AS IS" BASIS,
16+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17+
* See the License for the specific language governing permissions and
18+
* limitations under the License.
19+
*/
20+
#ifndef MIO_IO_DEFAULT_SERIALIZE_H_
21+
#define MIO_IO_DEFAULT_SERIALIZE_H_
22+
23+
#include "memilio/io/io.h"
24+
#include "memilio/utils/metaprogramming.h"
25+
26+
#include <tuple>
27+
#include <type_traits>
28+
#include <utility>
29+
30+
namespace mio
31+
{
32+
33+
/**
34+
* @brief A pair of name and reference.
35+
*
36+
* Used for default (de)serialization.
37+
* This object holds a char pointer to a name and reference to value. Mind their lifetime!
38+
* @tparam ValueType The (non-cv, non-reference) type of the value.
39+
*/
40+
template <class ValueType>
41+
struct NamedRef {
42+
using Reference = ValueType&;
43+
44+
const char* name;
45+
Reference value;
46+
47+
/**
48+
* @brief Create a named reference.
49+
*
50+
* @param n A string literal.
51+
* @param v A non-const lvalue reference to the value.
52+
*/
53+
explicit NamedRef(const char* n, Reference v)
54+
: name(n)
55+
, value(v)
56+
{
57+
}
58+
};
59+
60+
namespace details
61+
{
62+
63+
/**
64+
* @brief Helper type to detect whether T has a default_serialize member function.
65+
* Use has_default_serialize.
66+
* @tparam T Any type.
67+
*/
68+
template <class T>
69+
using default_serialize_expr_t = decltype(std::declval<T>().default_serialize());
70+
71+
/// Add a name-value pair to an io object.
72+
template <class IOObject, class Member>
73+
void add_named_ref(IOObject& obj, const NamedRef<Member> named_ref)
74+
{
75+
obj.add_element(named_ref.name, named_ref.value);
76+
}
77+
78+
/// Unpack all name-value pairs from the tuple and add them to a new io object with the given name.
79+
template <class IOContext, class... Members>
80+
void default_serialize_impl(IOContext& io, const char* name, const NamedRef<Members>... named_refs)
81+
{
82+
auto obj = io.create_object(name);
83+
(add_named_ref(obj, named_refs), ...);
84+
}
85+
86+
/// Retrieve a name-value pair from an io object.
87+
template <class IOObject, class Member>
88+
IOResult<Member> expect_named_ref(IOObject& obj, const NamedRef<Member> named_ref)
89+
{
90+
return obj.expect_element(named_ref.name, Tag<Member>{});
91+
}
92+
93+
/// Read an io object and its members from the io context using the given names and assign the values to a.
94+
template <class IOContext, class DefaultSerializable, class... Members>
95+
IOResult<DefaultSerializable> default_deserialize_impl(IOContext& io, DefaultSerializable& a, const char* name,
96+
NamedRef<Members>... named_refs)
97+
{
98+
auto obj = io.expect_object(name);
99+
100+
// we cannot use expect_named_ref directly in apply, as function arguments have no guarantueed order of evaluation
101+
std::tuple<IOResult<Members>...> results{expect_named_ref(obj, named_refs)...};
102+
103+
return apply(
104+
io,
105+
[&a, &named_refs...](const Members&... result_values) {
106+
// if all results are successfully deserialized, they are unpacked into result_values
107+
// then all class variables are overwritten (via the named_refs) with these values
108+
((named_refs.value = result_values), ...);
109+
return a;
110+
},
111+
results);
112+
}
113+
114+
} // namespace details
115+
116+
/**
117+
* @brief List of a class's members.
118+
*
119+
* Used for default (de)serialization.
120+
* Holds a char pointer to the class name as well as a tuple of NamedRefs with all added class members.
121+
* Initially, the template parameter pack should be left empty. It will be filled by calling Members::add.
122+
* @tparam ValueTypes The (non-cv, non-reference) types of member variables.
123+
*/
124+
template <class... ValueTypes>
125+
struct Members {
126+
// allow other Members access to the private constructor
127+
template <class...>
128+
friend struct Members;
129+
130+
/**
131+
* @brief Initialize Members with a class name. Use the member function `add` to specify the class's variables.
132+
* @param[in] class_name Name of a class.
133+
*/
134+
Members(const char* class_name)
135+
: name(class_name)
136+
, named_refs()
137+
{
138+
}
139+
140+
/**
141+
* @brief Add a class member.
142+
*
143+
* Use this function consecutively for all members, e.g. `Members("class").add("a", a).add("b", b).add...`.
144+
*
145+
* @param[in] member_name The name used for serialization. Should be the same as or similar to the class member.
146+
* For example, a good option a private class member `m_time` is simply `"time"`.
147+
* @param[in] member A class member. Always pass this variable directly, do not use getters or accessors.
148+
* @return A Members object with all previous class members and the newly added one.
149+
*/
150+
template <class T>
151+
[[nodiscard]] Members<ValueTypes..., T> add(const char* member_name, T& member)
152+
{
153+
return Members<ValueTypes..., T>{name, std::tuple_cat(named_refs, std::tuple(NamedRef{member_name, member}))};
154+
}
155+
156+
const char* name; ///< Name of the class.
157+
std::tuple<NamedRef<ValueTypes>...> named_refs; ///< Names and references to members of the class.
158+
159+
private:
160+
/**
161+
* @brief Initialize Members directly. Used by the add function.
162+
* @param[in] class_name Name of a class.
163+
* @param[in] named_references Tuple of added class Members.
164+
*/
165+
Members(const char* class_name, std::tuple<NamedRef<ValueTypes>...> named_references)
166+
: name(class_name)
167+
, named_refs(named_references)
168+
{
169+
}
170+
};
171+
172+
/**
173+
* @brief Creates an instance of T for later initialization.
174+
*
175+
* The default implementation uses the default constructor of T, if available. If there is no default constructor, this
176+
* class can be spezialized to provide the method `static T create()`. If there is a default constructor, but it is
177+
* private, DefaultFactory<T> can be marked as friend instead.
178+
*
179+
* The state of the object retured by `create()` is completely arbitrary, and may be invalid. Make sure to set it to a
180+
* valid state before using it further.
181+
*
182+
* @tparam T The type to create.
183+
*/
184+
template <class T>
185+
struct DefaultFactory {
186+
/// @brief Creates a new instance of T.
187+
static T create()
188+
{
189+
return T{};
190+
}
191+
};
192+
193+
/**
194+
* @brief Detect whether T has a default_serialize member function.
195+
* @tparam T Any type.
196+
*/
197+
template <class T>
198+
using has_default_serialize = is_expression_valid<details::default_serialize_expr_t, T>;
199+
200+
/**
201+
* @brief Serialization implementation for the default serialization feature.
202+
* Disables itself (SFINAE) if there is no default_serialize member or if a serialize member is present.
203+
* Generates the serialize method depending on the NamedRefs given by default_serialize.
204+
* @tparam IOContext A type that models the IOContext concept.
205+
* @tparam DefaultSerializable A type that can be default serialized.
206+
* @param io An IO context.
207+
* @param a An instance of DefaultSerializable to be serialized.
208+
*/
209+
template <class IOContext, class DefaultSerializable,
210+
std::enable_if_t<has_default_serialize<DefaultSerializable>::value &&
211+
!has_serialize<IOContext, DefaultSerializable>::value,
212+
DefaultSerializable*> = nullptr>
213+
void serialize_internal(IOContext& io, const DefaultSerializable& a)
214+
{
215+
// Note that the following cons_cast is only safe if we do not modify members.
216+
const auto members = const_cast<DefaultSerializable&>(a).default_serialize();
217+
// unpack members and serialize
218+
std::apply(
219+
[&io, &members](auto... named_refs) {
220+
details::default_serialize_impl(io, members.name, named_refs...);
221+
},
222+
members.named_refs);
223+
}
224+
225+
/**
226+
* @brief Deserialization implementation for the default serialization feature.
227+
* Disables itself (SFINAE) if there is no default_serialize member or if a deserialize meember is present.
228+
* Generates the deserialize method depending on the NamedRefs given by default_serialize.
229+
* @tparam IOContext A type that models the IOContext concept.
230+
* @tparam DefaultSerializable A type that can be default serialized.
231+
* @param io An IO context.
232+
* @param tag Defines the type of the object that is to be deserialized (i.e. DefaultSerializble).
233+
* @return The restored object if successful, an error otherwise.
234+
*/
235+
template <class IOContext, class DefaultSerializable,
236+
std::enable_if_t<has_default_serialize<DefaultSerializable>::value &&
237+
!has_deserialize<IOContext, DefaultSerializable>::value,
238+
DefaultSerializable*> = nullptr>
239+
IOResult<DefaultSerializable> deserialize_internal(IOContext& io, Tag<DefaultSerializable> tag)
240+
{
241+
mio::unused(tag);
242+
DefaultSerializable a = DefaultFactory<DefaultSerializable>::create();
243+
auto members = a.default_serialize();
244+
// unpack members and deserialize
245+
return std::apply(
246+
[&io, &members, &a](auto... named_refs) {
247+
return details::default_deserialize_impl(io, a, members.name, named_refs...);
248+
},
249+
members.named_refs);
250+
}
251+
252+
} // namespace mio
253+
254+
#endif // MIO_IO_DEFAULT_SERIALIZE_H_

0 commit comments

Comments
 (0)