Skip to content
GitLab
Commit 3f57ec3f authored by Jérémie Gaidamour's avatar Jérémie Gaidamour
Browse files

[dev] Merged dev, keeping current generators, data etc.

parent 06a031fd
Hide whitespace changes
Inline Side-by-side
Gemfile 0 → 100644
View file @ 3f57ec3f
source :rubygems
gem 'json_pure', '~> 1.5'
gem 'restfully','1.0.5'
gem "rake", ">= 0.9.2.2"
gem "net-admin-common",
:git => "ssh://g5kadmin@git.grid5000.fr/srv/git/repos/net-admin-common",
:tag => "2.5.3"
gem "cacti-play",
:git => "ssh://g5kadmin@git.grid5000.fr/srv/git/repos/cacti-play",
:tag => "2.8"
gem "weathermap",
:git => "ssh://g5kadmin@git.grid5000.fr/srv/git/repos/weathermap",
:tag => "5.8.4"
Gemfile.lock 0 → 100644
View file @ 3f57ec3f
GIT
remote: ssh://g5kadmin@git.grid5000.fr/srv/git/repos/cacti-play
revision: 4a2b6cb6ae2d9f791a4529737491ea02eeeb58c8
tag: 2.8
specs:
cacti-play (2.7.4)
net-admin-common (>= 2.5.3)
GIT
remote: ssh://g5kadmin@git.grid5000.fr/srv/git/repos/net-admin-common
revision: 3985600064a0a583ee9a47b4d51c97bc94f291a2
tag: 2.5.3
specs:
net-admin-common (2.5.3)
net-ssh (>= 2.3.0)
net-ssh-multi (>= 1.1)
rest-client (>= 1.6.7)
GIT
remote: ssh://g5kadmin@git.grid5000.fr/srv/git/repos/weathermap
revision: a258691fb1bdfa8db04897fd593bc2f9d3264b53
tag: 5.8.4
specs:
weathermap (5.8.2)
cacti-play (>= 2.7.4)
interpolator (>= 0.15)
ipaddress (>= 0.8.0)
json (>= 1.7.0)
net-admin-common (>= 2.5.3)
GEM
remote: http://rubygems.org/
specs:
addressable (2.3.2)
backports (2.6.3)
bond (0.4.2)
interpolator (0.15)
ipaddress (0.8.0)
json (1.7.4)
json_pure (1.7.4)
mime-types (1.19)
net-ssh (2.6.2)
net-ssh-gateway (1.1.0)
net-ssh (>= 1.99.1)
net-ssh-multi (1.1)
net-ssh (>= 2.1.4)
net-ssh-gateway (>= 0.99.0)
rack (1.4.1)
rack-cache (1.2)
rack (>= 0.4)
rake (0.9.2.2)
rb-readline (0.4.2)
rest-client (1.6.7)
mime-types (>= 1.16)
rest-client-components (1.2.0)
rack (>= 1.0.1)
rest-client (>= 1.6.0, < 1.7.0)
restfully (1.0.5)
addressable
backports
json (~> 1.5)
rack-cache
rb-readline
rest-client (~> 1.6)
rest-client-components
ripl (= 0.6.1)
ripl-color_streams
ripl-multi_line
ripl-play (~> 0.2.1)
ripl-short_errors
ripl (0.6.1)
bond (~> 0.4.0)
ripl-color_streams (0.1.2)
ripl (>= 0.2.8)
ripl-multi_line (0.3.0)
ripl (>= 0.3.6)
ripl-play (0.2.1)
ripl (>= 0.3.0)
ripl-short_errors (0.1.0)
ripl (>= 0.2.8)
PLATFORMS
ruby
DEPENDENCIES
cacti-play!
json_pure (~> 1.5)
net-admin-common!
rake (>= 0.9.2.2)
restfully (= 1.0.5)
weathermap!
README deleted 100644 → 0
View file @ 06a031fd
To get JSON input data:
./generator/input_loader.rb
To generate Ref API data in /tmp/data:
./generator/input_loader.rb | ./generators/reference-api/reference-api.rb
README.wiki 0 → 100644
View file @ 3f57ec3f
{{See also|[[Network_Description]] &#124; [[Reference_Repository]] &#124; [[OAR_properties_2.0]]}}
{{Maintainer|Sebastien Badia}}
{{Author|Cyril Rohr}}
{{Status|Draft}}
{{Portal|Admin}}
== Synopsis ==
The reference data is stored in a Git repository as JSON files, organized into hierarchical folders. These files can be manually written but the Git repository comes with a "/generators" folder which contains a script to ease their generation, based on high-level description files written in Ruby. Given one or more input files that describe the data you want to add, it will generate the required JSON files, directories and symlinks.
== Requirements ==
* Ruby >= 1.8.6
* Bundler (<code>gem install bundler</code>)
* Git
* g5kadmin account
== Workflow ==
The general overview of the workflow between git repositories is as follows:
ADMIN REPOSITORY <-- pull/push --> MASTER REPOSITORY
|
|
pull
(every minute)
/ \
/ \
| |
API REPOSITORY SITE X <--| |--> API REPOSITORY SITE Y
Each site administrator must first clone the remote MASTER REPOSITORY located on the git.grid5000.fr server and store it on a local machine (this is what I call the ADMIN REPOSITORY and has to be done once):
{{Term|location=g5kadminlaptop|cmd=<code class="command">git</code> clone <code class="file">ssh://g5kadmin@git.grid5000.fr/srv/git/repos/reference-repository.git</code>}}
When there is a need for change, the site administrator PULLs from the MASTER REPOSITORY to get the latest changes:
{{Term|location=reference-repo|cmd=<code class="command">git</code> pull}}
Then she manually adds/edits/removes the raw JSON files or uses the generator (more on that later). When she's done, she COMMITs her changes and PUSHes them to the MASTER REPOSITORY:
... editing ...
{{Term|location=reference-repo|cmd=<code class="command">git</code> commit -a -m "<code class="replace">list of modifications</code>"}}
{{Term|location=reference-repo|cmd=<code class="command">git</code> push}}
Finally, these changes are automatically replicated every minute to each API REPOSITORY (one per site), that are used by the Reference API.
{{Warning|text=Please note that manually edited files may be overwritten by a generator if it contains instructions on how to generate them. The recommended way of editing the data is to use the generators.}}
== Getting started ==
First, clone the remote MASTER REPOSITORY if it is not already done:
{{Term|location=g5kadminlaptop|cmd=<code class="command">git</code> clone <code class="file">ssh://g5kadmin@git.grid5000.fr/srv/git/repos/reference-repository.git</code>}}
From the newly created reference-repository folder, run the following command to install the required dependencies:
=== RVM Users ===
{{Term|location=reference-repo|cmd=<code class="command">rvm</code> use 1.8.7}}
{{Term|location=reference-repo|cmd=<code class="command">rvm</code> create gemset reference-repository}}
{{Term|location=reference-repo|cmd=<code class="command">rvm</code> gemset use reference-repository}}
=== Others Users ===
{{Term|location=reference-repo|cmd=<code class="command">bundle</code> install}}
== g5k-generator ==
Right now, the easiest way to get started is to look at some existing input files in the "generators/input" directory. There you can see how you can define sites, clusters, nodes and environments programmatically.
Then you may create a new input file or change an existing one and run it in simulation mode: (run <code class="command">rake -T</code> to see the list of available tasks):
{{Term|location=reference-repo|cmd=<code class="command">rake</code> g5k:generate DRY=yes}}
Your changes won't be applied but you'll see what would have been changed. Thus, the simulation mode is useful to review your changes before committing and check the ruby syntax of the input files.
When you are happy with your changes, you can then run the command without the DRY flag:
{{Term|location=reference-repo|cmd=<code class="command">rake</code> g5k:generate SITE=*}}
Please be aware that config files (YAML format) may be passed on the command line, so that the values can be used in the input files via the <code>lookup(config_filename, key)</code> function. To tell the generator to include one or more config files, you must pass them in your command arguments:
Finally, commit your changes with a meaningful message in ENGLISH (you SHOULD first review the changes that will be committed by running the <code>git diff</code> command) and push them immediately to the MASTER REPOSITORY:
{{Term|location=reference-repo|cmd=<code class="command">git</code> commit -a -m "[<code class="replace">TAGS</code>] message"}}
{{Note|text=<code class="replace">TAGS</code> is a comma-separated list of tags (e.g. <code class="replace">[rennes,lyon]</code>) that add parseable semantics to the commit message.}}
{{Term|location=reference-repo|cmd=<code class="command">git</code> push}}
== Synchronizing the OAR database ==
As of 2010/09/08, a synchronization task has been added that allows you to generate the diff between 2 commits (not necessarily consecutive).
Once you've committed your changes, run the <code>oar:generate</code> rake task to generate the corresponding oaradmin lines (run <code>$ rake -D</code> to see the list of available tasks):
{{Term|location=g5kadminlaptop|cmd=<code class="command">rake</code> oar:generate -s FROM=<PREVIOUS-COMMIT-ID> TO=<LATEST-COMMIT-ID>}}
By default, <code class="replace">TO</code> is set to <code class="replace">HEAD</code>.
The oaradmin lines are sent to STDOUT, the logging data to STDERR.
== Environments generator ==
To generate the environments descriptions of Grid5000 a rake task is available
{{Term|location=reference-repo|cmd=<code class="command">rake</code> env:generate DRY=<code class="replace">yes</code> ENV_NAME=<code class="replace">lenny-x64-base-2.5.rb</code>}}
== Manage deadnodes ==
=== Deadnodes to fix ===
Check the comments of nodes, (comment if dead, 'ok' il alive).
{{Term|location=reference-repo|cmd=<code class="command">rake</code> deadnodes:tofix SITE=<code class="replace">nancy</code>}}
=== Deadnodes reasons ===
Check comments of nodes (including phoenix).
{{Term|location=reference-repo|cmd=<code class="command">rake</code> deadnodes:reasons SITE=<code class="replace">reims</code>}}
I, [2012-07-25T17:05:58.997802 #8056] INFO -- : Node 'stremi-14.reims.grid5000.fr' is dead because 'Attente d une intervention de HP : Soucis BIOS'
{{Term|location=reference-repo|cmd=<code class="command">rake</code> deadnodes:reasons SITE=<code class="replace">sophia</code>}}
I, [2012-07-25T17:06:57.985664 #8081] INFO -- : Node 'sol-36.sophia.grid5000.fr' is dead because '[phoenix] Could not respawn this node at Wed Jul 25 12:30:02 +0200 2012'
I, [2012-07-25T17:06:57.985794 #8081] INFO -- : Node 'sol-40.sophia.grid5000.fr' is dead because '[phoenix] Could not respawn this node at Wed Jul 25 12:30:01 +0200 2012'
I, [2012-07-25T17:06:57.986003 #8081] INFO -- : Node 'suno-21.sophia.grid5000.fr' is dead because 'Chassis is on, but the node never get up. No dhcp request received.'
== Filling the reference - Guidelines ==
=== <code>network_adapters</code> ===
Many machines have several network interfaces, which are not always all configured. We have identified 4 cases in G5K clusters:
# The interface is not connected to any cable.
# The interface is the admin interface (e.g. IPMI).
# The interface is not mounted in the production environment, but users may use it in their own deployed environment.
# The interface is mounted in the production environment.
After several discussions inside the PS team, we have fixed some attributes. All of them are mandatory, but the ones between square brackets are only <font color="#FF0000">mandatory under conditions</font>. Those conditions follow the field name, in <font color="#FF0000">red</font>.
* '''interface''': the type of network interface, &isin; {"Ethernet", "Myrinet", "InfiniBand"}
:: NB: It is useless to define "Myrinet 10G" or "Myri-2000" values, because the '''rate''' will differentiate them.
* '''rate''': speed of the interface in b/s
* '''mac''':
:if '''interface''' &isin; {"Ethernet", "Myrinet"}, the MAC address of this interface,
:if '''interface'''=="InfiniBand", its GUID.
* '''vendor''': the company which made the device
* '''version''': its version according to the company nomenclatura
* '''enabled''': <code>true</code> if there is any cable connected to this interface
:* ['''management''']''<font color="#FF0000">(if '''enabled'''==true)</font>'': <code>true</code> if this interface is on the administration network (IPMI,...)
::* ['''network_address''']''<font color="#FF0000">(if '''mounted'''==true or '''management'''==true)</font>'': the DNS entry of the machine by this interface
:* ['''mountable''']''<font color="#FF0000">(if '''enabled'''==true)</font>'': <code>true</code> if it is usable by any user (even if it possibly requires a customized environment)
::* ['''driver''']''<font color="#FF0000">(if '''mountable'''==true)</font>'': name of the driver for the device in the linux kernel
::* ['''mounted''']''<font color="#FF0000">(if '''mountable'''==true)</font>'': <code>true</code> if the production environment mounts, configures this interface
:::* ['''network_address''']''<font color="#FF0000">(if '''mounted'''==true or '''management'''==true)</font>'': the DNS entry of the machine by this interface
:::* ['''device''']''<font color="#FF0000">(if '''mounted'''==true)</font>'': name of this interface in the production environment
:* ['''ip''']''<font color="#FF0000">(if '''enabled'''==true)</font>'': the IP of this interface.
::*''<font color="#FF0000">(if '''enabled'''==true AND '''ip''' is empty)</font>'' : The dhcp provides no ip adress for this interface
:* ['''ip6''']: the IPv6 of this interface, for future use...
* '''bridged''': is the interface used for the KVM bridge in the production environment ? (true | false)
==== script reaching IP/MAC addresses of cluster ====
Some scripts have been created to ease the retrieving of MAC/IP addresses on cluster. Get them
[https://scm.gforge.inria.fr/svn/grid5000/admin/trunk/refapi_fillers/fill-api-nw/?root=grid5000 here]
==== how to retrieve the guid on Infiniband card ====
Here a sample of an ohai plugin (included on the useful gem [https://helpdesk.grid5000.fr/redmine/projects/reference-helper/wiki/Wiki reference-helper] ! ) :
<pre class="brush: ruby">
#
# Author:: Pascal Morillon <pascal.morillon@irisa.fr>
provides "infiniband"
infiniband Mash.new
interfaces = Dir['/sys/class/net/*'].collect { |c| File.basename(c) }.select { |s| s =~ /ib.*/ }
interfaces.each do |interface|
infiniband[:"#{interface}"] = Mash.new
if File.exist?(File.join('/sys/class/net', interface, 'address'))
if File.exist?('/sys/class/infiniband/mthca0/ports')
guid_prefix = "20:00:55:04:01:"
elsif File.exist?('/sys/class/infiniband/mlx4_0/ports')
guid_prefix = "20:00:55:00:41:"
end
guid_part2 = File.read(File.join('/sys/class/net', interface, 'address')).chomp
infiniband[:"#{interface}"][:guid] = guid_prefix + guid_part2.split(":")[5..20].join(":")
else
exit 1
end
end
</pre>
== Resources ==
* [http://cheat.errtheblog.com/s/git Git Cheat Sheet]
== FAQ ==
=== Resolv::ResolvError occurred when I tried to generate ===
At runtime, the generator resolves the hostname of each node to obtain its IP address. As a consequence it should be run from within Grid5000 so that it can query the Grid5000 DNS.
=== What kind of commit message ? ===
After each modification to the repository, you should immediately commit your changes with a meaningful message, so that people can easily understand what has changed (latest changes will be displayed in a syndication feed). Your commits should also be site-specific, or even cluster-specific to avoid merge conflicts. Try to avoid putting a lot of changes in only one commit.
You should also check that your name and email are correctly configured in your Git configuration:
$ git config --get user.name
$ git config --get user.email
Otherwise you can set them by issuing:
$ git config --global user.name 'John Doe'
$ git config --global user.email johndoe@example.com
=== My commit has been rejected, why ? ===
Since users will make queries such as: "give me the description of that site at this date", the time between the date of the commit and the effective replication of the changes to the APIs must be as low as possible.
That's why, right after your commit, you should push your changes to the remote MASTER REPOSITORY. Please note that commits whose committed date is older than 60 seconds will be rejected (if you encounter an error, check that your system clock is correctly synchronized with a time server).
Rakefile 0 → 100644
View file @ 3f57ec3f
#
require 'fileutils'
require 'json'
require 'logger'
require 'restfully'
ROOT_DIR = File.expand_path File.dirname(__FILE__)
LIB_DIR = File.join(ROOT_DIR, "generators", "lib")
$LOAD_PATH.unshift(LIB_DIR) unless $LOAD_PATH.include?(LIB_DIR)
require 'grid5000'
task :environment do
Dir.chdir(ROOT_DIR)
@logger = Logger.new(STDERR)
@logger.level = Logger.const_get((ENV['DEBUG'] || 'INFO').upcase)
end
task :api_sites do
api_logger = Logger.new("/dev/null")
api_logger.level = Logger::FATAL
@api = Restfully::Session.new(:configuration_file => File.expand_path("~/.restfully/api.grid5000.fr.yml"),:logger => api_logger,:base_uri => 'https://api.grid5000.fr/sid')
@api_sites = if ENV['SITE']
[@api.root.sites[ENV['SITE'].to_sym]]
else
@api.root.sites
end
end
task :hosts do
# HOSTS=gw.lille
# HOSTS=*.lille
# SITES=lille => HOSTS=*.lille
# SITES=* => HOSTS=*.*
site = ENV['SITE']
host = ENV['HOST']
if site != nil
host = "*.#{site}"
elsif host != nil
abort "HOST must be on the form <hostname>.<site>. You provided '#{host}'." if host.scan(/^(\S+)\.(\S+)$/).empty?
else
abort "You must provide SITE= , (SITE=lille, or SITE=*), or a HOST, (HOST=gw.lille, or HOST=*.lille, or HOST=sw-*.lille)"
end
abort "You must provide HOST= , (HOST=gw.lille, or HOST=*.lille, or HOST=sw-*.lille)" if host.nil?
@host = host
end
namespace :g5k do
desc "Generates the JSON files based on the generators, for all sites.\nUse SITE=<SITE-NAME> if you wish to restrict the generation to a specific site.\nUse DRY=yes to simulate the execution."
task :generate => [:environment,:hosts] do
host,site = @host.scan(/(\S+)\.(\S+)/).flatten
root_dir_input = "#{ROOT_DIR}/generators/input/sites"
command = File.join(ROOT_DIR, "generators", "grid5000")
command += " " + File.join(root_dir_input, site,"#{site}.rb")
command += " " + File.join(root_dir_input, site,"clusters", "*", "#{host}.rb")
command += " " + File.join(root_dir_input, site,"clusters", "*", "nodes", "#{host}*.yaml")
command += " " + File.join(root_dir_input, site,"clusters", "*", "#{host}_manual.yaml")
command += " " + File.join(root_dir_input, site,"pdus.rb")
command << " -s" if ENV['DRY'] == "yes"
# puts command
sh command
end
end
# rake deadnodes:reasons
# rake deadnodes:tofix
namespace :deadnodes do
desc "List all dead nodes and the reason why they are dead. (SITE=)"
task :reasons => [:environment,:api_sites] do
@logger.level = Logger::INFO
@reasons = true
Rake::Task["deadnodes:browse"].execute
end
desc "List all nodes which have they state not in synch with they comment. (SITE=)"
task :tofix => [:environment,:api_sites] do
@logger.level = Logger::ERROR
@tofix = true
Rake::Task["deadnodes:browse"].execute
end
task :browse do
def comment_ok?(comment)
comment == "OK"
end
phoenix = []
@api_sites.each do |site|
reg = /^([^-]+)-(\d+)/
site.status["nodes"].sort{|a,b|
a_cluster,a_id = a[0].scan(reg).flatten
b_cluster,b_id = b[0].scan(reg).flatten
[a_cluster,a_id.to_i] <=> [b_cluster,b_id.to_i]
}.each do |uid,status|
comment = status["comment"]
state = status["hard"].downcase
if comment_ok?(comment)
if state == "dead"
@logger.error "Node '#{uid}' of state '#{state}' should not have comment '#{comment}'" if @tofix
else
# nothing, good state
end
else
if state == "dead"
@logger.info "Node '#{uid}' is dead because '#{comment}'" if @reasons
else
@logger.error "Node '#{uid}' should have the not-dead-comment 'OK', since its state is '#{state}'. Instead, it has comment '#{comment}'." if @tofix
phoenix.push uid if comment.match(/^\[phoenix\]/) != nil
end
end
end
end
puts phoenix if ENV["PHOENIX"] == "yes"
end
end
# TESTS
# Deletion:
# rake -s oar:generate FROM=4cfebf92e9cce05315782b51e05eded4ab4f0e7e TO=7d2648eaad7dbbc6f1fdb9c0279f73d374ccd47a
#
# Update:
# rake -s oar:generate FROM=7d2648eaad7dbbc6f1fdb9c0279f73d374ccd47a TO=bb528643003757942521942eaeab74b15aaa976d
#
# Add:
# rake -s oar:generate FROM=be9f7338b9750ce675447c13d172157992041ec1 TO=7dc3a4101a657230b7ad0534025a7ca93c905411
#
# All:
# rake -s oar:generate FROM=be9f7338b9750ce675447c13d172157992041ec1 TO=7d2648eaad7dbbc6f1fdb9c0279f73d374ccd47a
#
namespace :oar do
desc "Generates the oaradmin lines to update the OAR database after a change in the reference repository.\nUse FROM=<SHA-ID> and TO=<SHA-ID> to specify the starting and ending commits.\n Use -s to suppress the 'in directory' announcement.\n Use SUDO=sudo to prefix the commandes.\n use PROPS=<comma-separated-list> to limit the update to some properties"
task :generate => :environment do
if ENV['FROM'].nil? || ENV['FROM'].empty?
@logger.fatal "You MUST specify a commit id from where to start using the FROM=<SHA-ID> argument. Ex: rake -s oar:generate FROM=be9f7338b9750ce675447c13d172157992041ec1 TO=7dc3a4101a657230b7ad0534025a7ca93c905411 2> /dev/null"
exit(1)
end
ENV['TO'] ||= 'HEAD'
ENV['SUDO'] ||= ''
ENV['PROPS'] ||= ''
properties = ENV['PROPS'].split(',')
@logger.info "Analysing changes between #{ENV['FROM']}..#{ENV['TO']}..."
commands = []
diff = `git diff --name-status #{ENV['FROM']}..#{ENV['TO']}`
diff.split("\n").each do |line|
action, filename = line.split("\t")
next unless filename =~ %r{.+/nodes/.+json}
node_uid, site_uid, grid_uid = filename.gsub(/\.json/,'').split("/").values_at(-1, -5, -7)
cluster_uid = node_uid.split("-")[0]
host = [node_uid, site_uid, grid_uid, "fr"].flatten.join(".")
if ENV['SITE'] && !ENV['SITE'].split(",").include?(site_uid)
@logger.info "Skipping #{host} since you only want changes that occured on #{ENV['SITE'].inspect}"
next
end
command = ""
case action
when "A", "C", "M"
node_properties = JSON.parse(File.read(filename))
cluster_properties = JSON.parse(File.read(filename.gsub(%r{/nodes.*}, "/#{cluster_uid}.json")))
cluster = Grid5000::Cluster.new(cluster_properties)
node = Grid5000::Node.new(cluster, node_properties)
begin
export = node.export("oar-2.4")
rescue Grid5000::MissingProperty => e
@logger.warn "Error when exporting #{host}: #{e.message}. Skipped."
next
end
if action == "M" # modification of a file
command.concat("#{ENV["SUDO"]} oarnodesetting -h #{host} ")
command.concat(" -p ").concat( export.to_a.map{|(k,v)|
if v.nil?
nil
elsif properties.size > 0
if properties.include?(k)
"#{k}=#{v.inspect.gsub("'", "\\'").gsub("\"", "'")}"
else
nil
end
else
"#{k}=#{v.inspect.gsub("'", "\\'").gsub("\"", "'")}"
end
}.compact.join(" -p ") )
else # new file
node_number = node_uid.split("-")[1]
command.concat("oar_resources_add -H 1 --host0 #{node_number} --host-prefix #{cluster_uid}- --host-suffix .#{site_uid}.#{grid_uid}.fr -C #{node.properties['architecture']['smp_size']} -c #{export['cpucore']}")
command.concat(" -a")
# Add other properties
command.concat(' -A "')
if ENV['MAINTENANCE'] && ENV['MAINTENANCE']=='NO'
command.concat(' -p maintenance=\'NO\'')
else
# by default, maintenance is YES when creating new resources
command.concat(' -p maintenance=\'YES\'')
end
# by default, an Alive node has comment "OK"
command.concat(' -p comment=\'OK\'')
command.concat(" -p ").concat( export.to_a.map{|(k,v)|
if v.nil?
nil
else
"#{k}=#{v.inspect.gsub("'", "\\'").gsub("\"", "'")}"
end
}.compact.join(" -p ") )
command.concat('"')
end
when "D" # deletion of a file
command.concat("oarnodesetting -s Dead -h #{host}")
else
@logger.warn "Don't know what to do with #{line.inspect}. Ignoring."
next
end
if ENV['COMMIT'] && ENV['COMMIT']=='YES'
command.concat(' -c')
end
commands << command
end
commands.each do |command|
puts command
end
end
end
namespace :netlinks do
desc "Generates network API JSON files based on net-links yaml files.\nUse DRY=yes to simulate the execution. "
task :generate => [:environment,:hosts] do
host,site = @host.scan(/(\S+)\.(\S+)/).flatten
root_dir_input = File.join(ROOT_DIR, "generators","input")
command = File.join(ROOT_DIR, "generators", "grid5000")
command += " " + File.join(root_dir_input, "net-links.rb")
command += " " + File.join(root_dir_input,"sites", site,"#{site}.rb")
command += " " + File.join(root_dir_input,"sites", site,"net-links","#{host}.yaml")
command << " -s" if ENV['DRY'] == "yes"
# puts command
sh command
end
end
namespace :env do
desc "Generates environment JSON files .\nUse DRY=yes to simulate the execution. "
task :generate => [:environment] do
env_name = ENV["ENV_NAME"]
abort "You must provide ENV_NAME=" if env_name.nil?
root_dir_input = "#{ROOT_DIR}/generators/input"
command = File.join(ROOT_DIR, "generators", "grid5000")
command += " " + File.join(root_dir_input, "environments","#{env_name}")
command << " -s" if ENV['DRY'] == "yes"
sh command
end
end
namespace :weathermap do
@weathermap_options = ""
task :hosts do
@weathermap_site = ENV['SITE']
@weathermap_host = ENV['HOST']
abort "You must provide the SITE= " if @weathermap_site.nil?
abort "You must provide the HOST= name (uid) in its site " if @weathermap_host.nil? or @weathermap_host.match(/\.grid5000\.fr/) != nil
end
task :execute => ["weathermap:hosts"] do
cmd = "bundle exec weathermap"
cmd += " --site '#{@weathermap_site}' --host '#{@weathermap_host}' --api-path #{ROOT_DIR} #{@weathermap_options}"
sh cmd
end
desc "Create weathermaps for host HOST without data."
task :testing => ["weathermap:hosts"] do
@weathermap_options.replace("--action write --use-cacti no")
Rake::Task['weathermap:execute'].invoke
end
desc "Create weathermaps for host HOST with RRD from cacti."
task :production => ["weathermap:hosts"] do
@weathermap_options.replace("--action write --use-cacti yes")
Rake::Task['weathermap:execute'].invoke
end
desc "Display network links description amongst network equipments."
task :display do
@weathermap_options.replace("--action display")
Rake::Task['weathermap:execute'].invoke
end
end
data/grid5000/environments/etch-x64-base-1.0.json 0 → 100644
View file @ 3f57ec3f
{
"accounts": [
{
"login": "root",
"password": "grid5000"
},
{
"login": "g5k",
"password": "grid5000"
}
],
"applications": [
"Vim",
"XEmacs",
"JED",
"nano",
"JOE",
"Perl",
"Python",
"Ruby"
],
"based_on": "Debian version etch for amd64",