Mentions légales du service

Skip to content

Chunk large messages to avoid size limitations

This MR implements message-chunking in cases when large (usually, > 1MiB) messages are being exchanged via one of the current network communication protocols (WebSockets or gRPC). The code additions are framework-dependent, but are implemented in coherence, following these common specs:

  • When a (string-serialized) message reaches a size that exceeds the communication protocol's (default) message-size limitation, it is sent as a stream of ordered chunks, obtained by merely splitting the string into fixed-size chunks.
  • This may be used either on the client or server side, and should be invisible to the user (including developers): chunking is automated as part of the backend of public message-sending methods.
  • When an unregistered client attempts to send a chunked message to the server, its message should be refused, without collection nor parsing of the actual message contents.

The practical implementations are the following:

  • WebSockets:
    • When chunking, send a conventional flag as first message; expect an acceptance (or denial) flag; sequentially send chunks; send an ending flag.
    • When receiving, check for the "stream start" flag; decide whether to accept (based on registration status) and send a flag; receive and assemble chunks until the ending flag is received.
    • In practice, util functions have been implemented, that are shared by the Client and Server classes (but not made available to the end-users).
  • gRPC:
    • Turn send into a server streaming RPC, enabling the server to send replies as an ordered sequence of message chunks.
    • Add the send_large bidirectional-streaming RPC, that is similar to send but is used when the client performs message-chunking.
    • Refuse send_large requests from unregistered clients (send back an Error message).
    • Automate the choice of send_large over send in the client backend code.

This MR closes issue #12 (closed).

Merge request reports