Data elements are required to have a size prefix so elements like map, array, raw, etc. are finite. However, the number of top-level elements does not need to be calculated in advance. Indefinite streams can be achieved by sending a series of top-level elements. Each top-level element will have a finite size, but the number of elements is unlimited.
This allows a perpetual e-NON event stream to be sent over a connection. However, consider a higer-level transport such as RSocket to implement the streaming. With RSocket, e-NON can be used to format the payload data.
Encrypted and/or compressed data can be added directly, without Base64 expansion.
As in the case for streaming, e-NON requires a size to be known in advance. If the encrypted and/or compressed size is not known up front, the data can be chunked into blocks of known size. The chunks can be streamed into a single e-NON package as a series of top-level byte[] elements.
There are 2 features that provide optimization:
Tightly packed arrays
Large sequences of numeric data can be represented without overhead, just a maximum of 10 bytes in a prefix. This works best for primitive data types (int, double, etc.) and requires the entire block to be of the same type.
Glossary
For tagged records containing name-value pairs, the tag names will be cached into a glossary. Each tag name appears just once in the data stream, the first time it is encountered. Subsequent appearances of the tag names will be replaced by a small glossary-reference token. When the data is received the tag names will be restored.
The first 250 distinct keys can be reduced to a 1-byte glossary-reference. Up to 65,535 distinct keys can be to 3-byte glossary references.