Probe support : protocol draft
This is the first step towards %Convenient and powerful probes. Goal is to implement base protocol messages to support probe creation and basic information retrieval. Only basic features will be supported for now, but the message definition should be designed in such a way that features are easy to add later on.
protocol
Note about the current state of probe support in the- scheduler may issue a QUERY event to ask for the energy consumption of the whole platform from time 0 to now. Batsim replies with a ANSWER event right away.
- periodic queries are not implemented directly. this can be done by issuing periodic CALL_ME_LATER events. when the corresponding REQUESTED_CALL is received, scheduler can issue a QUERY request and issue another CALL_ME_LATER to create periodic calls.
New design overview
- Scheduler can create probes at runtime
- Probes must be named
- Probes have a trigger mechanism that defines when they should generate new data. for now, only the
one-shot
trigger should be supported, which sends data right away and destroys the probe (probes have no lifetime). Future trigger mechanisms may include:- every T seconds
- whenever the scheduler is called
- whenever a certain type of Batsim event occurs
- whenever another probe generates data
- whenever another probe sends data to the scheduler (same as before, but only triggers when the other probe's filter predicate is true)
- Probes have a filter predicate, that selects which information should be forwarded to the scheduler. for now, only the trivial predicate
true
should be supported, which forwards all information. Future filter predicates may include:- thresholding mechanism: only forward data if value is in a given range (e.g., load > 0.8)
- Probes are on a single metrics type. for now, only the instantaneous
power consumption
should be supported - Probes can send their data
raw
or via anexponential moving average
. for now, onlyraw
should be supported - Probes are set on resources. For now, the only resource type that should be supported is
hosts
. For now, the only supported values ofhosts
are intervalsets of size 1 (AKA one host)
- Batsim sends all probe-related information in a single protocol event
- Data is written in a per-probe format, not on a per-resource one. Rationale is that in the end, we will want complex probes that can aggregate data from various sources
New protocol events proposal
Scheduler sends a ADD_PROBE
event to add a new probe. Event example:
{
"timestamp": 0.0,
"type": "ADD_PROBE",
"data": {
"name": "my-amazing-probe",
"trigger": "one-shot",
"metrics": "power consumption",
"filter": "true",
"smoothing": "none",
"aggregation": "addition",
"object": "host",
"resources": {
"hosts": "10"
}
}
}
{
"timestamp": 0.0,
"type": "ADD_PROBE",
"data": {
"name": "my-amazing-probe",
"trigger": "periodic",
"metrics": "power consumption",
"filter": "true",
"smoothing": "none",
"aggregation": "addition",
"object": "host",
"resources": {
"period": "5"
"hosts": "10"
}
}
}
Batsim sends PROBE_DATA
events to gather the desired probe information. Event example:
{
"timestamp": "0.01",
"type": "PROBE_DATA",
"data": {
"name": "my-amazing-probe",
"aggregation": "addition",
"metrics": "power consumption",
"value": 20
}
}
{
"timestamp": "0.01",
"type": "PROBE_DATA",
"data": {
"name": "my-amazing-probe",
"aggregation": "none",
"metrics": "power consumption",
"value" : [ {"resource":"1", "value":20.0}, {"resource":"2", "value":20.0} ]
}
}
Implementation
-
Protocol parsing (Batsim side) -
Protocol parsing (sched side) -
Toy scheduler (FCFS + probes or similar) -
Batsim core to return desired value