Mentions légales du service

Skip to content
Snippets Groups Projects
Verified Commit 6b1419db authored by Philippe Virouleau's avatar Philippe Virouleau
Browse files

Add test for overriding cpu affinity for gpu

parent bd5c0bf2
Branches
No related tags found
1 merge request!599oar-properties: introduce cpu_affinity_override and use it
Pipeline #885139 passed
This diff is collapsed.
......@@ -141,4 +141,10 @@ describe 'OarProperties' do
check_oar_properties({ :oar => 'oar_but_chassis_unset', :data => 'data', :case => 'chassis_unset' })
end
end
context 'generating with an overriden CPU affinity for the GPUs' do
it 'should work correctly' do
check_oar_properties({ :oar => 'oar_empty', :data => 'data_grue_overridden_cpu_affinity', :case => 'grue_overridden_affinity' })
end
end
end
Output format: [ '-', 'key', 'value'] for missing, [ '+', 'key', 'value'] for added, ['~', 'key', 'old value', 'new value'] for changed
clustera-1: new node !
["+", "besteffort", "YES"]
["+", "chassis", "Dell Inc. PowerEdge R7425 FVJVY03"]
["+", "cluster", "clustera"]
["+", "cluster_priority", 201911]
["+", "core_count", 32]
["+", "cpu_count", 2]
["+", "cpuarch", "x86_64"]
["+", "cpucore", 16]
["+", "cpufreq", "2.4"]
["+", "cputype", "AMD EPYC 7351"]
["+", "deploy", "YES"]
["+", "disk_reservation_count", 0]
["+", "disktype", "SAS/HDD"]
["+", "eth_count", 1]
["+", "eth_kavlan_count", 1]
["+", "eth_rate", 10]
["+", "exotic", "NO"]
["+", "gpu_count", 4]
["+", "gpu_model", "Tesla T4"]
["+", "ib", "NO"]
["+", "ib_count", 0]
["+", "ib_rate", 0]
["+", "ip", "172.16.77.1"]
["+", "maintenance", "NO"]
["+", "max_walltime", 86400]
["+", "memcore", 4096]
["+", "memcpu", 65536]
["+", "memnode", 131072]
["+", "mic", "NO"]
["+", "myri", "NO"]
["+", "myri_count", 0]
["+", "myri_rate", 0]
["+", "nodemodel", "Dell PowerEdge R7425"]
["+", "opa_count", 0]
["+", "opa_rate", 0]
["+", "production", "YES"]
["+", "switch", "gw"]
["+", "thread_count", 64]
["+", "virtual", "amd-v"]
["+", "wattmeter", "NO"]
Properties that need to be created on the fakesite server: ip, cluster, nodemodel, switch, besteffort, deploy, virtual, cpuarch, cpucore, cpu_count, core_count, thread_count, cputype, cpufreq, disktype, chassis, eth_count, eth_kavlan_count, eth_rate, ib_count, ib_rate, ib, opa_count, opa_rate, myri_count, myri_rate, myri, memcore, memcpu, memnode, gpu_model, gpu_count, exotic, mic, wattmeter, cluster_priority, max_walltime, production, maintenance, disk_reservation_count
#! /usr/bin/env bash
set -eu
set -x
set -o pipefail
echo '================================================================================'
property_exist () {
[[ $(oarproperty -l | grep -e "^$1$") ]]
}
node_exist () {
[[ $(oarnodes --sql "host='$1' and type='default'") ]]
}
disk_exist () {
[[ $(oarnodes --sql "host='$1' and type='disk' and disk='$2'") ]]
}
# if [ $(oarnodes -Y | grep " cpu:" | awk '{print $2}' | sort -nr | wc -c) == "0" ]; then
# NEXT_AVAILABLE_CPU_ID=0
# else
# MAX_CPU_ID=$(oarnodes -Y | grep " cpu:" | awk '{print $2}' | sort -nr | head -n1)
# let "NEXT_AVAILABLE_CPU_ID=MAX_CPU_ID+1"
# fi
#
# if [ $(oarnodes -Y | grep " core:" | awk '{print $2}' | sort -nr | wc -c) == "0" ]; then
# NEXT_AVAILABLE_CORE_ID=0
# else
# MAX_CORE_ID=$(oarnodes -Y | grep " core:" | awk '{print $2}' | sort -nr | head -n1)
# let "NEXT_AVAILABLE_CORE_ID=MAX_CORE_ID+1"
# fi
#############################################
# Create OAR properties that were created by 'oar_resources_add'
#############################################
property_exist 'host' || oarproperty -a host --varchar
property_exist 'cpu' || oarproperty -a cpu
property_exist 'core' || oarproperty -a core
property_exist 'gpudevice' || oarproperty -a gpudevice
property_exist 'gpu' || oarproperty -a gpu
property_exist 'disk' || oarproperty -a disk --varchar
property_exist 'diskpath' || oarproperty -a diskpath --varchar
#############################################
# Create OAR properties if they don't exist
#############################################
property_exist 'ip' || oarproperty -a ip --varchar
property_exist 'cluster' || oarproperty -a cluster --varchar
property_exist 'nodemodel' || oarproperty -a nodemodel --varchar
property_exist 'switch' || oarproperty -a switch --varchar
property_exist 'virtual' || oarproperty -a virtual --varchar
property_exist 'cpuarch' || oarproperty -a cpuarch --varchar
property_exist 'cpucore' || oarproperty -a cpucore
property_exist 'cpu_count' || oarproperty -a cpu_count
property_exist 'core_count' || oarproperty -a core_count
property_exist 'thread_count' || oarproperty -a thread_count
property_exist 'cputype' || oarproperty -a cputype --varchar
property_exist 'cpufreq' || oarproperty -a cpufreq --varchar
property_exist 'disktype' || oarproperty -a disktype --varchar
property_exist 'chassis' || oarproperty -a chassis --varchar
property_exist 'eth_count' || oarproperty -a eth_count
property_exist 'eth_kavlan_count' || oarproperty -a eth_kavlan_count
property_exist 'eth_rate' || oarproperty -a eth_rate
property_exist 'ib_count' || oarproperty -a ib_count
property_exist 'ib_rate' || oarproperty -a ib_rate
property_exist 'ib' || oarproperty -a ib --varchar
property_exist 'opa_count' || oarproperty -a opa_count
property_exist 'opa_rate' || oarproperty -a opa_rate
property_exist 'myri_count' || oarproperty -a myri_count
property_exist 'myri_rate' || oarproperty -a myri_rate
property_exist 'myri' || oarproperty -a myri --varchar
property_exist 'memcore' || oarproperty -a memcore
property_exist 'memcpu' || oarproperty -a memcpu
property_exist 'memnode' || oarproperty -a memnode
property_exist 'gpu_model' || oarproperty -a gpu_model --varchar
property_exist 'gpu_count' || oarproperty -a gpu_count
property_exist 'exotic' || oarproperty -a exotic --varchar
property_exist 'mic' || oarproperty -a mic --varchar
property_exist 'wattmeter' || oarproperty -a wattmeter --varchar
property_exist 'cluster_priority' || oarproperty -a cluster_priority
property_exist 'max_walltime' || oarproperty -a max_walltime
property_exist 'production' || oarproperty -a production --varchar
property_exist 'maintenance' || oarproperty -a maintenance --varchar
property_exist 'disk_reservation_count' || oarproperty -a disk_reservation_count
###################################
# clustera-1.fakesite.grid5000.fr
###################################
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=1 -p cpuset=0 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=2 -p cpuset=2 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=3 -p cpuset=4 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=4 -p cpuset=6 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=5 -p cpuset=8 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=6 -p cpuset=10 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=7 -p cpuset=12 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=8 -p cpuset=14 -p gpu=1 -p gpu_model='Tesla T4' -p gpudevice=0 # This GPU is mapped on /dev/nvidia0
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=9 -p cpuset=16 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=10 -p cpuset=18 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=11 -p cpuset=20 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=12 -p cpuset=22 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=13 -p cpuset=24 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=14 -p cpuset=26 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=15 -p cpuset=28 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=1 -p core=16 -p cpuset=30 -p gpu=2 -p gpu_model='Tesla T4' -p gpudevice=1 # This GPU is mapped on /dev/nvidia1
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=17 -p cpuset=1 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=18 -p cpuset=3 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=19 -p cpuset=5 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=20 -p cpuset=7 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=21 -p cpuset=9 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=22 -p cpuset=11 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=23 -p cpuset=13 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=24 -p cpuset=15 -p gpu=3 -p gpu_model='Tesla T4' -p gpudevice=2 # This GPU is mapped on /dev/nvidia2
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=25 -p cpuset=17 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=26 -p cpuset=19 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=27 -p cpuset=21 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=28 -p cpuset=23 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=29 -p cpuset=25 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=30 -p cpuset=27 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=31 -p cpuset=29 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
oarnodesetting -a -h 'clustera-1.fakesite.grid5000.fr' -s Absent -p host='clustera-1.fakesite.grid5000.fr' -p cpu=2 -p core=32 -p cpuset=31 -p gpu=4 -p gpu_model='Tesla T4' -p gpudevice=3 # This GPU is mapped on /dev/nvidia3
echo; echo 'Setting properties for clustera-1.fakesite.grid5000.fr:'; echo
oarnodesetting --sql "host='clustera-1.fakesite.grid5000.fr' and type='default'" -p ip='172.16.77.1' -p cluster='clustera' -p nodemodel='Dell PowerEdge R7425' -p switch='gw' -p besteffort='YES' -p deploy='YES' -p virtual='amd-v' -p cpuarch='x86_64' -p cpucore=16 -p cpu_count=2 -p core_count=32 -p thread_count=64 -p cputype='AMD EPYC 7351' -p cpufreq='2.4' -p disktype='SAS/HDD' -p chassis='Dell Inc. PowerEdge R7425 FVJVY03' -p eth_count=1 -p eth_kavlan_count=1 -p eth_rate=10 -p ib_count=0 -p ib_rate=0 -p ib='NO' -p opa_count=0 -p opa_rate=0 -p myri_count=0 -p myri_rate=0 -p myri='NO' -p memcore=4096 -p memcpu=65536 -p memnode=131072 -p gpu_model='Tesla T4' -p gpu_count=4 -p exotic='NO' -p mic='NO' -p wattmeter='NO' -p cluster_priority=201911 -p max_walltime=86400 -p production='YES' -p maintenance='NO' -p disk_reservation_count=0
echo '================================================================================'
+---------- + -------------------- + ----- + ----- + -------- + ---- + -------------------- + ------------------------------ + ------------------------------+
| cluster | host | cpu | core | cpuset | gpu | gpudevice | cpumodel | gpumodel |
+---------- + -------------------- + ----- + ----- + -------- + ---- + -------------------- + ------------------------------ + ------------------------------+
| clustera | clustera-1 | 1 | 1 | 0 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 2 | 2 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 3 | 4 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 4 | 6 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 5 | 8 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 6 | 10 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 7 | 12 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 8 | 14 | 1 | 0 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 9 | 16 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 10 | 18 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 11 | 20 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 12 | 22 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 13 | 24 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 14 | 26 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 15 | 28 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 1 | 16 | 30 | 2 | 1 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 17 | 1 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 18 | 3 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 19 | 5 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 20 | 7 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 21 | 9 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 22 | 11 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 23 | 13 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 24 | 15 | 3 | 2 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 25 | 17 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 26 | 19 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 27 | 21 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 28 | 23 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 29 | 25 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 30 | 27 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 31 | 29 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
| clustera | clustera-1 | 2 | 32 | 31 | 4 | 3 | AMD EPYC 7351 | Tesla T4 |
+---------- + -------------------- + ----- + ----- + -------- + ---- + -------------------- + ------------------------------ + ------------------------------+
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment