Migration to python-grid5000
Python-grid5000 is a new wrapper for the Grid'5000 API. I'd like to migrate from execo (g5k api module + oar*) to python-grid5000 exclusively.
The first difficulty is that enoslib used to work at the grid level (using raw call to oargrid). There is no way to query Oargrid states through G5K API. But I'd like to keep the same granularity (grid) in enoslib and keep the stateless way of doing this (meaning we currently don't keep any state of the grid job since we use oargrid for that)
-
⛈ : critical and require lot of work -
🌩 : a bit of work is needed -
🌟 : should be straightforward -
🌪 : probably unsupported :'( after the migration
I've created a branch to_pyg5k
as the placeholder for the migration.
Feel free to submit pull requests there corresponding to the below work items.
Work Items:
-
enos_g5k.utils
-
⛈ enos_g5k.utils.grid_get_or_create_job [Addressed in !7 (merged)]- multisite job synchronisation ? (they all should start at the same time approx.)
- If several site are involved we should make sure the different jobs starts at the same time approx.
- We'll have to reimpplement oargrid on the client side, using the name as identifier for the set of jobs (one per site).
- multisite job synchronisation ? (they all should start at the same time approx.)
-
🌩 enos_g5k.utils.get_network_info_from_job_id- replace :
vlan_ids = ex5.get_oar_job_kavlan(job_id, site)
- replace:
ipmac, info = ex5.get_oar_job_subnets(job_id, site)
- or rewrite the whole
- replace :
-
🌪 enos_g5k.utils.grid_reload_from_id (non sense with python-grid5000 -- from_name maybe)[Addressed in !7 (merged)] -
🌟 enos_g5k.utils.oar_reload_from_id -
🌩 enos_g5k.utils._deploy-
Initial support -
Retry with failed nodes (max 3 times) [Done in 6b1a6362] -
Incorporate ssh connectivity test (execo-like). If the check is successful, the deployment will not occur. Execo is using this by default 'check_deployed_command': "! (mount | grep -E '^/dev/[[:alpha:]]+2 on / ')",
[Done in 11237f95]
-
-
🌟 enos_g5k.utils._mount_secondary_nics [Addressed in 0d48bf9a] -
🌟 enos_g5k.utils.get_cluster_site [Addressed by a280a1fc] -
🌟 enos_g5k.utils.get_cluster_interfaces [Addressed in 0d48bf9a] -
🌩 enos_g5k.utils._do_grid_make_reservation [Addressed in !7 (merged)] -
🌩 enos_g5k.utils.grid_destroy_from_name [Addressed in !7 (merged)] -
🌪 enos_g5k.utils.grid_destroy_from_id [removed by 36d35c57]- we introduced a more general grid_destroy_from_ids
-
🌟 enos_g5k.utils.oar_destroy_from_id [Addressed by 36d35c57]- we introduced a more general grid_destroy_from_ids
-
-
enos_g5k.driver
-
🌪 enos_g5k.driver.OargridStaticDriver [kept by 36d35c57]- we kept it, it accepts a list of jobs to reload the deployment from
-
-
enos_vmong5k.provider
-
🌟 enos_vmong5k.provider.get_host_core [Addressed by f51adf25]- Let's move this in a common place with the other utils/api calls
-
-
Misc.
-
🌩 Expose the python-grid5000 to the external world [Done with bb423e38]e.g: grid5000 = G5KProvider(conf); gk = grid5000.client
-
Misc
Ip to Mac
Getting a subnet from the API leads to get the cidr of the subnet. Grid'5000 also associates a list of corresponding mac (e.g for booting vm) but is a bit hidden from the API.
Hopefully looking at the g5k-subnets
, we can find the mapping function:
G5KMACPREFIX = '00:16:3E'
def ip2mac(ip)
check_prefix(ip, G5KIPPREFIX)
split_ip = ip.split('.')
raise "Invalid IP address: not 4 bytes long" unless split_ip.length == 4
"#{G5KMACPREFIX}:#{split_ip[1..-1].map{|i|'%02X'%i}.join(':')}"
end