A GitLab upgrade is scheduled for Monday, April 7, 2025. Service interruptions are expected in the morning. We'll keep you posted on the progress of the upgrade on the Mattermost channel: https://mattermost.inria.fr/devel/channels/gitlab. We recommend that you do not work on the platform until an announcement indicates that maintenance is complete.
"This notebook is meant to be run in google colab. You can find import your local copy of the file in the the [colab welcome page](https://colab.research.google.com/)."
"This notebook is meant to be run in google colab. You can find import your local copy of the file in the the [colab welcome page](https://colab.research.google.com/)."
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {
"id": "s9bpLdH5ThpJ"
"id": "s9bpLdH5ThpJ"
},
},
"source": [
"source": [
"# Setting up your declearn "
"# Setting up your declearn "
]
]
},
},
{
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {
"metadata": {
"id": "Clzf4NTja121"
"id": "Clzf4NTja121"
},
},
"source": [
"source": [
"We first clone the repo, to have both the package itself and the `examples` folder we will use in this tutorial, then naviguate to the package directory, and finally install the required dependencies"
"We first clone the repo, to have both the package itself and the `examples` folder we will use in this tutorial, then naviguate to the package directory, and finally install the required dependencies"
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m106.5/106.5 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.9/dist-packages (from cryptography>=35.0->declearn==2.1.0) (1.15.1)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.9/dist-packages (from fire>=0.4->declearn==2.1.0) (1.16.0)\n",
"Requirement already satisfied: termcolor in /usr/local/lib/python3.9/dist-packages (from fire>=0.4->declearn==2.1.0) (2.2.0)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.2->declearn==2.1.0) (2022.7.1)\n",
"Requirement already satisfied: numpy>=1.20.3 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.2->declearn==2.1.0) (1.22.4)\n",
"Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.2->declearn==2.1.0) (2.8.2)\n",
"Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (2.0.12)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (1.26.15)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (2022.12.7)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (3.4)\n",
"Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.9/dist-packages (from scikit-learn>=1.0->declearn==2.1.0) (1.10.1)\n",
"Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from scikit-learn>=1.0->declearn==2.1.0) (3.1.0)\n",
"Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from scikit-learn>=1.0->declearn==2.1.0) (1.2.0)\n",
"Requirement already satisfied: pycparser in /usr/local/lib/python3.9/dist-packages (from cffi>=1.12->cryptography>=35.0->declearn==2.1.0) (2.21)\n",
"Building wheels for collected packages: fire, declearn\n",
" Building wheel for fire (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116952 sha256=ab01943c400d3267450974ec56a6572193bed40710845edd44623e56c7757799\n",
" Stored in directory: /root/.cache/pip/wheels/f7/f1/89/b9ea2bf8f80ec027a88fef1d354b3816b4d3d29530988972f6\n",
" Building wheel for declearn (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for declearn: filename=declearn-2.1.0-py3-none-any.whl size=276123 sha256=4969a91ded8b704c8c9497bcda8f514f847c49098715d659cc8e96a947ec887f\n",
" Stored in directory: /tmp/pip-ephem-wheel-cache-fgkx9jiw/wheels/cc/79/79/6586306a117d40a1f8b251a22e50583b8abb2d7e855a62ecf7\n",
"We are going to train a common model between three simulated clients on the classic [MNIST dataset](http://yann.lecun.com/exdb/mnist/). The input of the model is a set of images of handwritten digits, and the model needs to determine which number between 0 and 9 each image corresponds to."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KlY_vVtFHv2P"
},
"source": [
"## The model\n",
"\n",
"To do this, we will use a simple CNN, defined in `examples/mnist_quickrun/model.py`"
"/content/declearn2/declearn/model/tensorflow/utils/_gpu.py:66: UserWarning: Cannot use a GPU device: either CUDA is unavailable or no GPU is visible to tensorflow.\n",
"We start by splitting the MNIST dataset between 3 clients and storing the output in the `examples/mnist_quickrun` folder. For this we use an experimental utility provided by `declearn`. "
" Splitting scheme(s) to use. In all cases, shards contain mutually-\n",
" exclusive samples and cover the full raw training data.\n",
" - If \"iid\", split the dataset through iid random sampling.\n",
" - If \"labels\", split into shards that hold all samples associated\n",
" with mutually-exclusive target classes.\n",
" - If \"biased\", split the dataset through random sampling according\n",
" to a shard-specific random labels distribution.\n",
" perc_train: float, default= 0.8\n",
" Train/validation split in each client dataset, must be in the\n",
" ]0,1] range.\n",
" seed: int or None, default=None\n",
" Optional seed to the RNG used for all sampling operations.\n",
" \n"
]
}
],
"source": [
"print(split_data.__doc__)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kZtbxlwUftKd"
},
"source": [
"## Quickrun\n",
"\n",
"We can now run our experiment. As explained in the section 2.1 of the [quickstart documentation](https://magnet.gitlabpages.inria.fr/declearn/docs/2.1/quickstart), using the mode `declearn-quickrun` requires a configuration file, some data, and a model:\n",
"\n",
"* A TOML file, to store your experiment configurations. Here: \n",
"`examples/mnist_quickrun/config.toml`.\n",
"* A folder with your data, split by client. Here: `examples/mnist_quickrun/data_iid`\n",
"* A model file, to store your model wrapped in a `declearn` object. Here: `examples/mnist_quickrun/model.py`.\n",
"\n",
"We then only have to run the `quickrun` util with the path to the TOML file:"
"The output obtained is the combination of the CLI output of our server and our clients, going through: \n",
"\n",
"* `INFO:Server:Starting clients registration process.` : a first registration step, where clients register with the server\n",
"* `INFO:Server:Sending initialization requests to clients.`: the initilization of the object needed for training on both the server and clients side.\n",
"* `Server:INFO: Initiating training round 1`: the training starts, where each client makes its local update(s) and send the result to the server which aggregates them\n",
"* `INFO: Initiating evaluation round 1`: the model is evaluated at each round\n",
"* `Server:INFO: Stopping training`: the training is finalized "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wo6NDugiOH6V"
},
"source": [
"## Results \n",
"\n",
"You can have a look at the results in the `examples/mnist_quickrun/result_*` folder, including the metrics evolution during training. "
"# Install the package, with TensorFlow and Websockets extra dependencies.\n",
"If you run the model as is, you should see a drop of performance\n",
"# You may want to work in a dedicated virtual environment.\n",
"\n"
"!pip install .[tensorflow,websockets]"
]
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hC8Fty8YTy9P"
},
"source": [
"# Running your first experiment"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rcWcZJdob1IG"
},
"source": [
"We are going to train a common model between three simulated clients on the classic [MNIST dataset](http://yann.lecun.com/exdb/mnist/). The input of the model is a set of images of handwritten digits, and the model needs to determine which number between 0 and 9 each image corresponds to."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KlY_vVtFHv2P"
},
"source": [
"## The model\n",
"\n",
"To do this, we will use a simple CNN, defined in `examples/mnist_quickrun/model.py`"
"We start by splitting the MNIST dataset between 3 clients and storing the output in the `examples/mnist_quickrun` folder. For this we use an experimental utility provided by `declearn`. "
"Now try modifying the `examples/mnist_quickrun/config.toml` file like this, to implement the [scaffold algorithm](https://arxiv.org/abs/1910.06378) and running the experiment again. \n",
"Splitting data into 3 shards using the 'iid' scheme.\n"
"\n",
]
"```\n",
}
" [optim]\n",
],
"\n",
"source": [
" [optim.client_opt]\n",
"from declearn.dataset import split_data\n",
" lrate = 0.005 \n",
"\n",
" modules = [\"scaffold-client\"] \n",
"split_data(folder=\"examples/mnist_quickrun\")"
"\n",
]
" [optim.server_opt]\n",
},
" lrate = 1.0 \n",
{
" modules = [\"scaffold-client\"]\n",
"cell_type": "markdown",
"```"
"metadata": {},
]
"source": [
"The python code above is equivalent to running `declearn-split examples/mnist_quickrun/` in a shell command-line."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3-2hKmz-2RF4"
},
"source": [
"Here is what the first image of the first client looks like:"
" Splitting scheme(s) to use. In all cases, shards contain mutually-\n",
" exclusive samples and cover the full raw training data.\n",
" - If \"iid\", split the dataset through iid random sampling.\n",
" - If \"labels\", split into shards that hold all samples associated\n",
" with mutually-exclusive target classes.\n",
" - If \"biased\", split the dataset through random sampling according\n",
" to a shard-specific random labels distribution.\n",
" perc_train: float, default= 0.8\n",
" Train/validation split in each client dataset, must be in the\n",
" ]0,1] range.\n",
" seed: int or None, default=None\n",
" Optional seed to the RNG used for all sampling operations.\n",
" \n"
]
}
}
],
"source": [
"print(split_data.__doc__)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kZtbxlwUftKd"
},
"source": [
"## Quickrun\n",
"\n",
"We can now run our experiment. As explained in the section 2.1 of the [quickstart documentation](https://magnet.gitlabpages.inria.fr/declearn/docs/latest/quickstart), using the `declearn-quickrun` entry-point requires a configuration file, some data, and a model:\n",
"\n",
"* A TOML file, to store your experiment configurations. Here: \n",
"`examples/mnist_quickrun/config.toml`.\n",
"* A folder with your data, split by client. Here: `examples/mnist_quickrun/data_iid`\n",
"* A model python file, to declare your model wrapped in a `declearn` object. Here: `examples/mnist_quickrun/model.py`.\n",
"\n",
"We then only have to run the `quickrun` util with the path to the TOML file:"
"The python code above is equivalent to running `declearn-quickrun examples/mnist_quickrun/config.toml` in a shell command-line."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "O0kuw7UxJqKk"
},
"source": [
"The output obtained is the combination of the CLI output of our server and our clients, going through: \n",
"\n",
"* `INFO:Server:Starting clients registration process.` : a first registration step, where clients register with the server\n",
"* `INFO:Server:Sending initialization requests to clients.`: the initilization of the object needed for training on both the server and clients side.\n",
"* `Server:INFO: Initiating training round 1`: the training starts, where each client makes its local update(s) and send the result to the server which aggregates them\n",
"* `INFO: Initiating evaluation round 1`: the model is evaluated at each round\n",
"* `Server:INFO: Stopping training`: the training is finalized "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wo6NDugiOH6V"
},
"source": [
"## Results \n",
"\n",
"You can have a look at the results in the `examples/mnist_quickrun/result_*` folder, including the metrics evolution during training. "
"Now try modifying the `examples/mnist_quickrun/config.toml` file like this, to implement the [scaffold algorithm](https://arxiv.org/abs/1910.06378) and running the experiment again. \n",
This notebook is meant to be run in google colab. You can find import your local copy of the file in the the [colab welcome page](https://colab.research.google.com/).
This notebook is meant to be run in google colab. You can find import your local copy of the file in the the [colab welcome page](https://colab.research.google.com/).
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
# Setting up your declearn
# Setting up your declearn
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
We first clone the repo, to have both the package itself and the `examples` folder we will use in this tutorial, then naviguate to the package directory, and finally install the required dependencies
We first clone the repo, to have both the package itself and the `examples` folder we will use in this tutorial, then naviguate to the package directory, and finally install the required dependencies
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m106.5/106.5 kB[0m [31m3.5 MB/s[0m eta [36m0:00:00[0m
[?25hRequirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.9/dist-packages (from cryptography>=35.0->declearn==2.1.0) (1.15.1)
Requirement already satisfied: six in /usr/local/lib/python3.9/dist-packages (from fire>=0.4->declearn==2.1.0) (1.16.0)
Requirement already satisfied: termcolor in /usr/local/lib/python3.9/dist-packages (from fire>=0.4->declearn==2.1.0) (2.2.0)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.2->declearn==2.1.0) (2022.7.1)
Requirement already satisfied: numpy>=1.20.3 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.2->declearn==2.1.0) (1.22.4)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas>=1.2->declearn==2.1.0) (2.8.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (2.0.12)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests~=2.18->declearn==2.1.0) (3.4)
Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.9/dist-packages (from scikit-learn>=1.0->declearn==2.1.0) (1.10.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from scikit-learn>=1.0->declearn==2.1.0) (3.1.0)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from scikit-learn>=1.0->declearn==2.1.0) (1.2.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/dist-packages (from cffi>=1.12->cryptography>=35.0->declearn==2.1.0) (2.21)
Building wheels for collected packages: fire, declearn
Building wheel for fire (setup.py) ... [?25l[?25hdone
Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116952 sha256=ab01943c400d3267450974ec56a6572193bed40710845edd44623e56c7757799
Stored in directory: /root/.cache/pip/wheels/f7/f1/89/b9ea2bf8f80ec027a88fef1d354b3816b4d3d29530988972f6
Building wheel for declearn (pyproject.toml) ... [?25l[?25hdone
Created wheel for declearn: filename=declearn-2.1.0-py3-none-any.whl size=276123 sha256=4969a91ded8b704c8c9497bcda8f514f847c49098715d659cc8e96a947ec887f
Stored in directory: /tmp/pip-ephem-wheel-cache-fgkx9jiw/wheels/cc/79/79/6586306a117d40a1f8b251a22e50583b8abb2d7e855a62ecf7
We are going to train a common model between three simulated clients on the classic [MNIST dataset](http://yann.lecun.com/exdb/mnist/). The input of the model is a set of images of handwritten digits, and the model needs to determine which number between 0 and 9 each image corresponds to.
We are going to train a common model between three simulated clients on the classic [MNIST dataset](http://yann.lecun.com/exdb/mnist/). The input of the model is a set of images of handwritten digits, and the model needs to determine which number between 0 and 9 each image corresponds to.
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## The model
## The model
To do this, we will use a simple CNN, defined in `examples/mnist_quickrun/model.py`
To do this, we will use a simple CNN, defined in `examples/mnist_quickrun/model.py`
/content/declearn2/declearn/model/tensorflow/utils/_gpu.py:66: UserWarning: Cannot use a GPU device: either CUDA is unavailable or no GPU is visible to tensorflow.
warnings.warn(
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## The data
## The data
We start by splitting the MNIST dataset between 3 clients and storing the output in the `examples/mnist_quickrun` folder. For this we use an experimental utility provided by `declearn`.
We start by splitting the MNIST dataset between 3 clients and storing the output in the `examples/mnist_quickrun` folder. For this we use an experimental utility provided by `declearn`.
Splitting scheme(s) to use. In all cases, shards contain mutually-
Splitting scheme(s) to use. In all cases, shards contain mutually-
exclusive samples and cover the full raw training data.
exclusive samples and cover the full raw training data.
- If "iid", split the dataset through iid random sampling.
- If "iid", split the dataset through iid random sampling.
- If "labels", split into shards that hold all samples associated
- If "labels", split into shards that hold all samples associated
with mutually-exclusive target classes.
with mutually-exclusive target classes.
- If "biased", split the dataset through random sampling according
- If "biased", split the dataset through random sampling according
to a shard-specific random labels distribution.
to a shard-specific random labels distribution.
perc_train: float, default= 0.8
perc_train: float, default= 0.8
Train/validation split in each client dataset, must be in the
Train/validation split in each client dataset, must be in the
]0,1] range.
]0,1] range.
seed: int or None, default=None
seed: int or None, default=None
Optional seed to the RNG used for all sampling operations.
Optional seed to the RNG used for all sampling operations.
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Quickrun
## Quickrun
We can now run our experiment. As explained in the section 2.1 of the [quickstart documentation](https://magnet.gitlabpages.inria.fr/declearn/docs/2.1/quickstart), using the mode `declearn-quickrun` requires a configuration file, some data, and a model:
We can now run our experiment. As explained in the section 2.1 of the [quickstart documentation](https://magnet.gitlabpages.inria.fr/declearn/docs/latest/quickstart), using the `declearn-quickrun`entry-point requires a configuration file, some data, and a model:
* A TOML file, to store your experiment configurations. Here:
* A TOML file, to store your experiment configurations. Here:
`examples/mnist_quickrun/config.toml`.
`examples/mnist_quickrun/config.toml`.
* A folder with your data, split by client. Here: `examples/mnist_quickrun/data_iid`
* A folder with your data, split by client. Here: `examples/mnist_quickrun/data_iid`
* A model file, to store your model wrapped in a `declearn` object. Here: `examples/mnist_quickrun/model.py`.
* A model python file, to declare your model wrapped in a `declearn` object. Here: `examples/mnist_quickrun/model.py`.
We then only have to run the `quickrun` util with the path to the TOML file:
We then only have to run the `quickrun` util with the path to the TOML file:
The python code above is equivalent to running `declearn-quickrun examples/mnist_quickrun/config.toml` in a shell command-line.
%% Cell type:markdown id: tags:
The output obtained is the combination of the CLI output of our server and our clients, going through:
The output obtained is the combination of the CLI output of our server and our clients, going through:
*`INFO:Server:Starting clients registration process.` : a first registration step, where clients register with the server
*`INFO:Server:Starting clients registration process.` : a first registration step, where clients register with the server
*`INFO:Server:Sending initialization requests to clients.`: the initilization of the object needed for training on both the server and clients side.
*`INFO:Server:Sending initialization requests to clients.`: the initilization of the object needed for training on both the server and clients side.
*`Server:INFO: Initiating training round 1`: the training starts, where each client makes its local update(s) and send the result to the server which aggregates them
*`Server:INFO: Initiating training round 1`: the training starts, where each client makes its local update(s) and send the result to the server which aggregates them
*`INFO: Initiating evaluation round 1`: the model is evaluated at each round
*`INFO: Initiating evaluation round 1`: the model is evaluated at each round
*`Server:INFO: Stopping training`: the training is finalized
*`Server:INFO: Stopping training`: the training is finalized
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Results
## Results
You can have a look at the results in the `examples/mnist_quickrun/result_*` folder, including the metrics evolution during training.
You can have a look at the results in the `examples/mnist_quickrun/result_*` folder, including the metrics evolution during training.
Now try modifying the `examples/mnist_quickrun/config.toml` file like this, to implement the [scaffold algorithm](https://arxiv.org/abs/1910.06378) and running the experiment again.
Now try modifying the `examples/mnist_quickrun/config.toml` file like this, to implement the [scaffold algorithm](https://arxiv.org/abs/1910.06378) and running the experiment again.
