DataStorm supports custom user types for the following:
- Update tags
- Key or sample filter criteria
These data values must support binary encoding to be transmitted over the wire. Keys should also support the
std::ostream& operator<<. If you intend to use regular expression filters on values, values should also support this operator.
On this page:
By default, DataStorm uses the Ice encoding to transmit data values over-the-wire.
The Ice encoding supports encoding a number of C++ types out-of-the box:
long long int,
std::vector<T>where T is a type supported by the Ice encoding
std::map<K, V>where K and V are types supported by the Ice encoding
A DataStorm application can also define data types using the Slice language. The Slice definitions can be compiled with slice2cpp to generate C++ source and header files. The generated code will contain the necessary code to allow encoding and decoding the type.
The Slice generated code contains support for both the Ice for C++98 and C++11 language mappings. When used with DataStorm, you should make sure to compile the generated code with
-DICE_CPP11_MAPPING to build the C++11 language mapping generated code.
You can also provide encoding and decoding function if you don't want to rely on Slice for defining your data types. DataStorm uses the
Decoder templates for encoding. You will need to provide template specializations for your data type. The decoder and encoder template declarations are shown below:
For example, you can specialize these two templates and implement the encode and decode static methods for the
chrono::system_clock::time_point type as follow:
DataStorm can also provide the Ice communicator to the encoding and decoding methods. If you need the Ice communicator to perform the encoding and decoding, you should instead implement the following static methods:
DataStorm uses the
std::ostream& operator<< for two purposes:
If you enable session or data tracing, DataStorm can print out the key values. For custom key types, if no streaming operator is provided, DataStorm will print the name of the C++ typeid and the address of the value. If you want to print out a user friendly value, you should override the streaming operator.
DataStorm provides a pre-defined regular expression filter. The regular expression is matched against the value transformed to a string with the streaming operator.
The streaming operator can be defined as follow:
DataStorm needs to copy values to handle partial updates. By default, values are copied using the default copy constructor. This might not always be appropriate depending on the value type. For example the value type could be a
shared_ptr<T> or a complex type that requires deep-cloning. DataStorm uses the Cloner template to perform the clone. The template declaration is shown below:
You can specialize this template to implement a custom cloning for a given data type: