GitLab

* guix/jupyter/proxy.scm (unregister-proxied, terminate-proxied-kernel):
New procedures.
* guix-jupyter-kernel.scm (reply-execute-request): When 'lookup-proxied'
returns true, call 'unmonitor-client' and 'terminate-proxied-kernel',
and then 'create-environment'.

* guix/jupyter/proxy.scm (terminate-proxied-kernels): Add call to
'unmonitor-client'.

* tests/kernels.scm ("run-kernel python3"): Increase timeout to 5000ms
to increase the chances of getting a "starting" status message.

* jupyter/servers.scm (unmonitor-client): New procedure.
(serve-kernels)[handle-abort]: Change to simply accumulate requests.
[handle-requests]: New procedure.
Adjust call to 'handle-message' accordingly.
* tests/servers.scm ("ping pong, many clients leaving dynamically,
shell"): New test.

The kernel has access to the manifest, so it can display its actual
contents (including version numbers) upfront, before the environment is
built.

* guix/jupyter/inner-proxy.scm (kernel-info->shtml)
(kernel-info->html): Remove.
(reply-execute-request): Simply reply "Done!".
* guix-jupyter-kernel.scm (manifest->shtml)
(reply-for-environment, create-environment): New procedures.
(reply-execute-request): Call 'create-environment'.

* guix-jupyter-kernel.scm (%store-property): New variable.
(proxy-state-store): New procedure.
(reply-execute-request): Use it instead of using 'with-store'.
(kernel): Use 'with-store' and pass the store as a state property.

* guix/jupyter/inner-proxy.scm (%context): Remove.
(%zmq-context-property): New variable.
(reply-execute-request): Use 'proxy-state-property' instead of referring
to '%context'.
(run-inner-proxy): Call 'zmq-create-context' here, and save the context
as a %ZMQ-CONTEXT-PROPERTY property of the state.

* guix/jupyter/proxy.scm (<proxy-state>)[properties]: New field.
(proxy-state): Adjust accordingly.
(proxy-state-property, set-proxy-state-property): New procedures.

* guix-jupyter-container.scm: Rename to...
* guix/jupyter/inner-proxy.scm: ... this, and turn into a module.
* Makefile.am (SOURCES): Add 'inner-proxy.scm' and remove
'guix-jupyter-container.scm'.
* guix-jupyter-kernel.scm (container-path): Remove.
(start-container)[spawn]: Use (guix jupyter inner-proxy).

* guix/jupyter/environ.scm: Remove.
* guix/jupyter/environment.scm, guix/jupyter/containers.scm: New files.
* tests/environ.scm: Remove.
* Makefile.am (SOURCES): Adjust accordingly.
(SCM_TESTS): Remove tests/environ.scm.
* guix-jupyter-container.scm <top level>: Turn into...
(run-inner-proxy): ... this.  New procedure.
* guix-jupyter-kernel.scm (%inferior): New variable.
(reply-execute-request): Use 'string-tokenize' and
'specifications->manifest'.  Pass the manifest to 'start-container'.
(module-to-import?): New procedure.
(start-container): Rewrite to take a manifest and to evaluate code with
'eval/container*'.  Set the 'GUIX_ENVIRONMENT' and 'JUPYTER_PATH'
environment variables.
* jupyter/kernels.scm (jupyter-kernel-path): Remove "GUIX_PROFILE"
hack.

* Makefile.am (GUILD_COMPILE_FLAGS): Remove.
* guile.am (GUILD_COMPILE_FLAGS): New variable.
(.scm.go): Use $(GUILD) and $(GUILD_COMPILE_FLAGS).

Until now '.go' files of all the dependencies would not be found,
and thus building would take a long time.

* env.in: Append to the user's $GUILE_LOAD_COMPILED_PATH.  Don't set
GUILE_WARN_DEPRECATED.

* configure.ac: Ensure we have the (gcrypt mac) module.
* jupyter/kernels.scm: Use (gcrypt mac) instead of (gcrypt hmac).
(generate-key): Call 'generate-signing-key'.
* jupyter/messages.scm: Use (gcrypt mac).
(get-signature): Use 'mac-algorithm' instead of a symbol.

* jupyter/json.scm <top level>: Remove workaround for Guile-JSON 3.1.0.

* guix-kernel/environ.scm: Rename to...
* guix/jupyter/environ.scm: ... this.
* guix-kernel/magic.scm: Rename to...
* guix/jupyter/magic.scm: ... this
* guix-jupyter-container.scm, guix-jupyter-kernel.scm,
tests/environ.scm, tests/magic.scm: Adjust accordingly.
* Makefile.am (SOURCES): Adjust accordingly.

* jupyter/messages.scm (%current-protocol-version): New variable.
(<language-info>, <kernel-info-reply>): New record types.
* guix/jupyter/kernel.scm: New file.
* Makefile.am (SOURCES): Add it.
* guix-jupyter-kernel.scm (KERNEL-INFO): Remove.
(reply-kernel-info-request): Use 'kernel-info-reply->json'.
* tests/kernels.scm ("kernel_info_request"): Use
'json->kernel-info-reply' and associated record types.

* jupyter/json.scm (json, <=>): New variables.
(define-json-reader): Adjust to the new syntax for SPEC.
(define-json-writer): Likewise.
(define-json-mapping): Rebase on top of 'define-record-type*'.  Move
JSON field spec under the 'json' keyword.
* jupyter/kernels.scm (<kernel-specs>, <connection>): Adjust
accordingly.
(find-kernel-specs, allocate-connection): Use new syntactic
constructors.

* guix-jupyter-container.scm (general-handler, dispatch): Remove.
<top level>: Use 'proxy-request-handler' instead.
* guix-jupyter-kernel.scm (general-handler, dispatch): Remove.
<top level>: Use 'proxy-request-handler' instead.
* guix/jupyter/proxy.scm (proxy-request-handler): New procedure.

* jupyter/kernels.scm (reply-html): Pass #:recipient only when SOCKET is
'kernel-shell'.

* guix/jupyter/proxy.scm: New file.
* Makefile.am (SOURCES): Add it.
* guix-jupyter-container.scm (%main-kernel, %main-kernel-id)
(register-kernel, kernel-by-name): Remove.
(local-eval): No longer return COUNTER.
(general-handler): Rewrite to expect a <proxy-state> and to pass a
<proxy-state> to all the message type handlers.
(ignore-request, reply-execute-request, shutdown): Adjust accordingly.
<top level>: Create a <proxy-state> and pass it to 'server-kernels'.
* guix-jupyter-kernel.scm (%main-kernel, %main-kernel-id)
(register-proxy, proxy-by-name): Remove.
(general-handler): Rewrite to expect a <proxy-state> and to pass a
<proxy-state> to all the message type handlers.
(ignore-request, reply-execute-request, shutdown): Adjust accordingly.
(kill-containers): Remove.
<top level>: Create a <proxy-state> and pass it to 'server-kernels'.

* jupyter/kernels.scm (reply-html): Pass #:recipient to 'send-message'.

* jupyter/kernels.scm (generate-key): New procedure.
(relay-message): Remove.
* jupyter/messages.scm (<message>): Use 'define-immutable-record-type'
and define 'set-message-content'.
* guix-jupyter-container.scm (local-eval): Rewrite in terms of
'send-message'.
(reply-html-to-kernel): Remove.
(general-handler): Copy from *-kernel.scm.
(execute-request-sans-magic): New procedure.
(ignore-request, reply-execute-request, shutdown): Adjust to new calling
convention.
(%main-kernel, %main-kernel-id): New variables.
<top level>: Use 'serve-kernels'.
* guix-jupyter-kernel.scm (%main-kernel, %main-kernel-id): New
variables.
(general-handler): Check whether we're talking to %MAIN-KERNEL.
(reply-kernel-info-request, reply-execute-request, shutdown): Add 'kind'
parameter.
(new-container-connect): Remove.
(start-container): Return a <kernel>.
<top level>: Pass both the "default" container kernel and our client to
'server-kernels'.  Set %MAIN-KERNEL.

* guix/jupyter/logging.scm: New file.
* Makefile.am (SOURCES): Add it.

* guix-jupyter-kernel.scm (heartbeat-handler, start-kernel): Remove.
(general-handler): Adjust prototype to match what 'serve-kernels'
expects.
<top level>: Use 'serve-kernels'.

* jupyter/kernels.scm (allocate-connection): #:first-port defaults to #f.
[allocate-port]: New procedure
[try-connect]: New procedure.
Use it in lieu of 'test-bind', and remove the extra 'zmq-connect'
calls.
(test-bind): Remove.

* jupyter/kernels.scm (kernel-socket-kind): New procedure.
(zmq-poll*, allocate-connection): Export.
* jupyter/servers.scm (%server-scheduler-prompt, %server-exit-prompt):
New variables.
* jupyter/servers.scm (monitor-client, leave-server-loop)
(serve-kernels): New procedures.
* tests/servers.scm: New file.

* jupyter/kernels.scm (EINTR-safe): New procedure.
(read-message): Use it.

* jupyter/kernels.scm (read-message): Remove call to non-existent
'zmq-get-msg-bytevector'.

* jupyter/servers.scm (connection-file->kernel): Rename to...
(connection->kernel): ... this, and take a CONNECTION instead of a
PORT.
(connection-file->kernel): Rewrite as a composition.

* jupyter/kernels.scm (<kernel>): Use 'define-immutable-record-type'.
Add 'set-kernel-name' and 'set-kernel-pid'.
(find-kernel-by-name): Rename to...
(find-kernel-specs-file): ... this.
(find-kernel-specs): New procedure.
(<connection>): New record type.
(new-connection-file, kernel-arguments, exec-kernel): Remove.
(allocate-connection, spawn-kernel): New procedure.
(run-kernel): Rewrite to take a <kernel-spec> and to use
'allocate-connection'.
* tests/kernels.scm (%python3-specs): New variable.
("run-kernel python3"): Adjust accordingly.
* guix-jupyter-container.scm (reply-execute-request): Likewise.

* guix-kernel/environ.scm (guile-current-version->path): Don't require a
specific Guile version.

* jupyter/json.scm <top level>: Override (@@ (json builder) json-build).

* jupyter/json.scm (define-json-writer): New macro.
(define-json-mapping): Add support for defining writers.

* tests/kernels.scm ("kernel_info_request"): Rewrite to remove assumptions
about ordering, and to account for all the replies.

* jupyter/kernels.scm (<kernel-specs>): New record type.
(kernel-arguments): Use 'json->kernel-specs' and 'kernel-specs-arguments'.

Code taken from (guix swh) as of Guix commit
81c3dc32244a17241d74eea9fa265edfcb326f6d.

* jupyter/json.scm: New file.
* Makefile.am (SOURCES): Add it.

* tests/kernels.scm ("execute_request"): Rewrite to remove assumptions
about ordering, and to account for all the replies.

* guix-kernel/jupyter-client.scm: Rename to...
* jupyter/servers.scm: ... this.
(KERNEL-INFO, guile-version): Remove.
(json->kernel): Rename to...
(connection-file->kernel): ... this.
* guix-jupyter-container.scm: Adjust accordingly.
* guix-jupyter-kernel.scm (KERNEL-INFO): New variable.
(kernel): Use new 'connection-file->kernel' name.
* Makefile.am (SOURCES): Adjust file list accordingly.

* jupyter/kernels.scm (read-message): Return heartbeat messages.

Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on
until now: it maps JSON dictionaries to alists (instead of hash tables),
and JSON arrays to vectors (instead of lists).  This commit is about
adjusting all the existing code to this new mapping.

* jupyter/kernels.scm (kernel-arguments): Use 'assoc-ref' instead of
'hash-ref'; pass JSON array through 'vector->list'.
(relay-message, reply-html): Likewise.
* jupyter/messages.scm (string->header): Likewise.
* guix-jupyter-container.scm (local-eval, reply-execute-request): Likewise.
* guix-jupyter-kernel.scm (reply-execute-request): Likewise.
* guix-kernel/jupyter-client.scm (json->kernel): Likewise.
* tests/kernels.scm: Likewise.
* environment.scm (propagated-inputs): Replace GUILE-JSON with
GUILE-JSON-3.