Update README

README.md

@@ -109,19 +109,20 @@ pip3 install kwollect
 Kwollect needs a [PostgreSQL](https://www.postgresql.org/) database with
 [TimescaleDB](https://www.timescale.com/) extension to store metrics.
 
-For example, use these commands to install them on Debian Buster:
+For example, use these commands to install them on Debian Bullseye:
 
 ```
-sudo sh -c "echo 'deb https://packagecloud.io/timescale/timescaledb/debian/ `lsb_release -c -s` main' > /etc/apt/sources.list.d/timescaledb.list"
-wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | sudo apt-key add -
-sudo apt update
-sudo apt-get install -y postgresql postgresql-client timescaledb-postgresql-11 postgresql-plpython3-11
+wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | apt-key add -
+echo 'deb https://packagecloud.io/timescale/timescaledb/debian/ bullseye main' > /etc/apt/sources.list.d/timescaledb.list
+apt update
+apt-get install -y --no-install-recommends postgresql postgresql-client libpq-dev timescaledb-2-postgresql-13 timescaledb-tools postgresql-plpython3-13
 
-# TimescaleDB comes with a script to tune Postgres configuration that you might want to use:
-sudo cp /etc/postgresql/11/main/postgresql.conf /etc/postgresql/11/main/postgresql.conf-timescaledb_tune.backup
-sudo timescaledb-tune -yes -quiet
-echo 'timescaledb.telemetry_level=off' | sudo tee -a /etc/postgresql/11/main/postgresql.conf
-sudo systemctl restart postgresql
+## TimescaleDB comes with a script to tune Postgres configuration that you might want to use:
+cp /etc/postgresql/13/main/postgresql.conf /etc/postgresql/13/main/postgresql.conf-timescaledb_tune.backup
+timescaledb-tune -yes -quiet
+echo 'timescaledb.telemetry_level=off' >> /etc/postgresql/13/main/postgresql.conf
+
+systemctl restart postgresql
 ```
 
 Then, you can setup Kwollect database using the `kwollect-setup-db` tool. It is
@@ -166,7 +167,7 @@ jwt-secret = "changemechangemechangemechangemechangeme"
 *kwollector-setup-db* also outputs an *API token* that is needed to perform
 write access to the database.
 
-Finally, don't forget to start Postgres with `postgrest
+Finally, don't forget to start Postgrest with `postgrest
 <path_to_configuration_file>`
 
 
@@ -319,8 +320,6 @@ connected to your Kwollect's database and
 [import](https://grafana.com/docs/grafana/latest/dashboards/export-import/#importing-a-dashboard)
 our dashboard.
 
-Some more specialized visualization dashboard [have also been developed](https://gitlab.inria.fr/grid5000/kwollect-dashboard).
-
 
 # Advanced topics
 
@@ -360,12 +359,28 @@ requests on `metrics_by_job`, e.g.:
 SELECT timestamp, device_id, metric_id, value FROM metrics_by_job WHERE job_id = 1234;
 ```
 
-It is also possible to provide the "job_id" argument when calling API:
+It is also possible to provide the "job_id" argument when calling API's `get_metrics`:
 
 ```
 curl http://kwollect.host:3000/rpc/get_metrics?job_id=1234
 ```
 
+Once job scheduler integration configured, an additional `get_job_metrics` API function is available to provide "job-wide" metrics:
+
+```
+curl http://kwollect.host:3000/rpc/get_job_metrics?job_id=1234
+```
+
+The list of SQL requests to be performed to obtain "job-wide" metrics must be defined in a configuration file (one request per-line, performed on `jobmetrics` table), which must be provided to `--jobmetrics_requests_path` option of `kwollect-setup-db`. Here is an example of such file:
+```
+SELECT SUM(jobnode_energy_watthour) AS job_energy_watthour FROM (SELECT AVG(value) * EXTRACT(EPOCH FROM AGE(MAX(timestamp), MIN(timestamp)))/3600 AS jobnode_energy_watthour FROM jobmetrics WHERE metric_id = 'bmc_node_power_watt' GROUP BY device_id) s
+SELECT MAX(value) AS jobnode_maxpower_watt FROM jobmetrics WHERE metric_id = 'bmc_node_power_watt'
+SELECT AVG(1-value) AS job_cpu_avgusage_percent FROM jobmetrics WHERE metric_id = 'prom_node_cpu_seconds_total' AND labels->>'mode' = 'idle'
+SELECT MAX(1-value) AS job_cpu_maxusage_percent FROM jobmetrics WHERE metric_id = 'prom_node_cpu_seconds_total' AND labels->>'mode' = 'idle'
+SELECT AVG(value)*8/1024/1024 AS job_network_avginput_mbps FROM jobmetrics WHERE metric_id = 'network_ifaceout_bytes_total'
+SELECT MAX(value)*8/1024/1024 AS jobnode_network_maxinput_mbps FROM jobmetrics WHERE metric_id = 'network_ifaceout_bytes_total'
+```
+
 
 ## Optional metrics
 
@@ -390,6 +405,16 @@ currently existing jobs. It is called by OAR at the beginning and at the end of
 the jobs.
 
 
+## Metrics summary
+
+Metrics stored in the database are automatically averaged over a 5 minutes
+period. These summarized metrics are used by Grafana dashboard when the
+timerange to be displayed is greater than 30 minutes, to avoid overload it.
+
+If required, summarized metrics can be accessed using the `summary=1`
+parameter of the `get_metrics` API call.
+
+
 ## Wattmetre
 
 A specific kwollector, called *kwollector-wattmetre*, is available to read and
@@ -426,3 +451,15 @@ wattmetre, may be provided to associate metrics collected from a wattmetre port
 to the corresponding device. The file should contain one `device_id:
 [wattmetre_id-portX, wattmetre_id-portY]` line per device, containing the
 device identifier followed by the list of wattmetres and ports which power it.
+
+
+## Other features
+
+- Prometheus metrics with name "kwollect_custom" are managed specially: if they
+  include a `_metric_id` and/or `_timestamp` in their label, they will be
+  inserted with specified metric_id and/or timestamp in the Kwollect database
+- Automatic archiving of oldest metrics in a different filesystem is available
+  using `--archive_path` option of `kwollector-setup-db`
+- Increase rate of counter-type metrics can be obtained by adding to API call
+  `get_metrics` the parameter `as_rate=1` (or `as_rate=auto` to only process
+  metrics ending with `_total`) .