template<template< typename, typename Enable=void > class S>
class fpmas::io::datapack::BasicObjectPack< S >
Base object used to implement the FPMAS binary serialization technique.
The purpose of the BasicObjectPack is to provide an high level API to efficiently read/write data from/to a low level fpmas::api::communication::DataPack.
The BasicObjectPack is the container that provides access to the serialized data. Serialization / deserialization is ensured by the provided serializer S specializations.
The BasicObjectPack provides two serialization techniques.
- The Serializer technique is used by BasicObjectPack(const T&), operator=(const T&), put() and get(). In this case, the
S<T>::to_datapack() and S<T>::from_datapack() methods are used to perform those serializations, so the corresponding S<T> specializations must be available.
- The native
std::memcpy technique is used by read() and write(). This method is only valid for members that can safely be copied as raw binary data, i.e. scalar types or associated arrays.
- Note
- Examples presented here are based on the default ObjectPack implementation, i.e.
BasicObjectPack<Serializer>, however the same considerations are valid for any custom Serializer implementation passed as the S template parameter.
allocate(), put() and get()
A BasicObjectPack can be manually allocated using the allocate() and size() methods. put() and get() can then be used to serialize and unserialize data using the Serializer specializations. Notice that specializations are already provided for fundamental types and other C++ standard types and containers.
int main() {
std::vector<int> vec {4, -12, 17};
std::string str = "Hello";
pack.allocate(pack.size(vec) + pack.size(str));
pack.put(vec);
pack.put(str);
std::cout << pack << std::endl;
std::cout << "vec: ";
for(auto& item : pack.get<std::vector<int>>())
std:cout << item << " ";
std::cout << std::endl;
std::cout << "str: " << pack.get<std::string>() << std::endl;
}
BasicObjectPack< Serializer > ObjectPack
Definition: datapack.h:1393
Expected output:
pack: 0x3 0 0 0 0 0 0 0 0x4 0 0 0 0xfff4 0xffff 0xffff 0xffff 0x11 0 0 0 0x5 0 0 0 0 0 0 0 0x48 0x65 0x6c 0x6c 0x6f
vec: 4 -12 17
str: Hello
The interest of the BasicObjectPack serialization technique is that the buffer can be allocated in a single std::malloc operation even for very complex data schemes, what is very efficient in terms of memory usage and execution time.
put() and get() must be called in the same order to retrieve data at the appropriate place. Indeed, a get() call automatically advances the internal cursor to point to the next piece of data. readOffset() and seekRead() can eventually be used to handle very specific cases.
In particular, it might be tempting to call get() several times in a constructor for example, but the following example will produce an unexpected behavior, since nothing guarantees in C++ that pack.get<int>() will be called before pack.get<std::string>() (see https://stackoverflow.com/questions/2934904/order-of-evaluation-in-c-function-parameters):
int i = 12;
std::string str = "hello";
pack.allocate(pack.size<int>() + pack.size(str));
pack.put(i);
pack.put(str);
MyData
data = {pack.get<
int>(), pack.get<std::string>()}
const DataPack & data() const
Definition: datapack.h:390
MyData can be constructed as follows instead, to respect call orders:
int u_i = pack.get<int>();
std::string u_str = pack.get<std::string>();
MyData
data = {u_i, u_str};
Constructor and operator=
The allocation operator can be performed automatically using the BasicObjectPack(const T&) constructor or the operator=(const T&).
int main() {
std::string str = "Hello world";
std::cout << "pack: " << pack << std::endl;
std::cout << "str: " << pack.get<std::string>() << std::endl;
}
Expected output:
pack: 0xb 0 0 0 0 0 0 0 0x48 0x65 0x6c 0x6c 0x6f 0x20 0x77 0x6f 0x72 0x6c 0x64
str: Hello world
The disadvantage of this method is that nothing can be added a priori to the BasicObjectPack once it as been assigned. But this method can be efficiently combined with put() and get() to define custom data serialization rules.
Custom Serializer
Custom serialization rules can be specified thanks to Serializer specializations. For any type T, the static methods specified in the Serializer documentation must be implemented.
struct Data {
std::unordered_map<int, std::string> map {{4, "hey"}, {2, "ho"}};
double x = 127.4;
};
namespace fpmas {
namespace io {
namespace datapack {
template<>
struct Serializer<Data> {
return pack.size(item.map) + pack.size(item.x);
}
static void to_datapack(
ObjectPack& pack,
const Data& item) {
pack.put(item.map);
pack.put(item.x);
}
static Data from_datapack(
const ObjectPack& pack) {
Data item;
item.map = pack.get<std::unordered_map<int, std::string>>();
item.x = pack.get<double>();
return item;
}
};
}}}
int main() {
Data item;
std::cout << "pack: " << pack << std::endl;
Data unserial_item = pack.get<Data>();
std::cout << "item.map: ";
for(auto& item : unserial_item.map)
std::cout << "(" << item.first << "," << item.second << ") ";
std::cout << std::endl;
std::cout << "item.x: " << unserial_item.x << std::endl;
}
std::size_t size() const
Definition: datapack.h:367
Expected output:
pack: 0x2 0 0 0 0 0 0 0 0x2 0 0 0 0x2 0 0 0 0 0 0 0 0x68 0x6f 0x4 0 0 0 0x3 0 0 0 0 0 0 0 0x68 0x65 0x79 0xff9a 0xff99 0xff99 0xff99 0xff99 0xffd9 0x5f 0x40
item.map: (4,hey) (2,ho)
item.x: 127.4
Once a Serializer specialization has been specified for a type T, it can then be implicitly used to serialize compound types that use T. For example std::vector<Data> can be serialized. But even in this case, an ObjectPack pack = std::vector<Data>(...) operation only performs a single contiguous memory allocation, thanks to the size() method trick.
write() and read() operations
An read() / write() serialization technique can be used for types that can be directly copied using an std::memcpy operation. This is a low level feature that is not required in general use cases.
int main() {
std::uint64_t i = 23000;
char str[6] = "abcde";
pack.allocate(sizeof(i) + sizeof(str));
pack.write(i);
pack.write(str, 3);
std::cout << "pack: " << pack << std::endl;
std::uint64_t u_i;
char u_str[3];
pack.read(u_i);
pack.read(u_str, 3);
std::cout << "u_i: " << u_i << std::endl;
std::cout << "u_str: ";
for(std::size_t i = 0; i < 3; i++)
std::cout << u_str[i];
std::cout << std::endl;
}
Expected output:
pack: 0xffd8 0x59 0 0 0 0 0 0 0x61 0x62 0x63 0 0 0
u_i: 23000
u_str: abc
- Note
- The BasicObjectPack design is widely insprired by the
nlohmann::basic_json class.
- Template Parameters
-
1.9.3