Chunk large messages to avoid size limitations
This MR implements message-chunking in cases when large (usually, > 1MiB) messages are being exchanged via one of the current network communication protocols (WebSockets or gRPC). The code additions are framework-dependent, but are implemented in coherence, following these common specs:
- When a (string-serialized) message reaches a size that exceeds the communication protocol's (default) message-size limitation, it is sent as a stream of ordered chunks, obtained by merely splitting the string into fixed-size chunks.
- This may be used either on the client or server side, and should be invisible to the user (including developers): chunking is automated as part of the backend of public message-sending methods.
- When an unregistered client attempts to send a chunked message to the server, its message should be refused, without collection nor parsing of the actual message contents.
The practical implementations are the following:
- WebSockets:
- When chunking, send a conventional flag as first message; expect an acceptance (or denial) flag; sequentially send chunks; send an ending flag.
- When receiving, check for the "stream start" flag; decide whether to accept (based on registration status) and send a flag; receive and assemble chunks until the ending flag is received.
- In practice, util functions have been implemented, that are shared by the Client and Server classes (but not made available to the end-users).
- gRPC:
- Turn
send
into a server streaming RPC, enabling the server to send replies as an ordered sequence of message chunks. - Add the
send_large
bidirectional-streaming RPC, that is similar tosend
but is used when the client performs message-chunking. - Refuse
send_large
requests from unregistered clients (send back anError
message). - Automate the choice of
send_large
oversend
in the client backend code.
- Turn
This MR closes issue #12 (closed).