Mentions légales du service

Skip to content
Snippets Groups Projects
Commit bbc38acc authored by Faure Adrien's avatar Faure Adrien
Browse files

fix intro tuto nix

parent 14a335d1
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 125 additions and 212 deletions
slides.tex
+ 125
− 212
View file @ bbc38acc
......@@ -14,7 +14,7 @@
\usepackage{array,booktabs}
\usepackage{pgfpages}
\usepackage{listings}
\setbeameroption{show notes on second screen=right}
%\setbeameroption{show notes on second screen=right}
\lstset{language=Bash}
\definecolor{identifiercolor}{rgb}{0,0,0}
......@@ -23,7 +23,7 @@
\definecolor{stringcolor}{rgb}{0.5,0.6,0.9}
\lstdefinelanguage{Nix}{
% Anything between $ becomes LaTeX math mode
% Anything between $ becomes LaTeX math modse
mathescape=false,
% Comments may or not include Latex commands
texcl=false,
......@@ -70,7 +70,7 @@
\setbeamertemplate{footline}[frame number]
\setbeamertemplate{navigation symbols}{}
\title{\textbf{A Way Toward Reproducible Experiment Environment with Nix}}
\title{\textbf{A Way Towards Reproducible Experiment Environments with Nix}}
\date{Rennes 2019, \today}
\author{
\underline{Adrien Faure}, Millian Poquet\\
......@@ -98,219 +98,117 @@
\section{Experimentation From a Software Point of View}
\begin{frame}{Experimentation Codes}
\begin{itemize}
\item Experimentations codes are often \emph{ad hoc}.
\begin{frame}{Experimentations}
Experimentation Codes:
\begin{itemize}
\item \emph{ad hoc} experimentation code
\note{Publish and perish\\}
\item Few or no indications describing how to build them
\note{Relevant compilation options, dependencies...}
\item Few informations about the execution.
\note{Read scripts, runtime dependencies etc}
\item No documentations
\note{Relevant compilation options, dependencies...\\}
\note{Read scripts, runtime dependencies etc\\}
\end{itemize}
\end{frame}
\begin{frame}{The Problem}
\vspace{1em}
The Problem:
\begin{itemize}
\item Hinder the repeatability of the experimentations.
\item Difficult replay
\note{Hard to verify what you did, for reviewer for instance.}
\item Hard to continue someone else work.
\item Hard to maintain
\note{Trying other inputs, fixing bugs etc}
\note{Encourage restarting from scratch}
\end{itemize}
\end{frame}
\section{Nix}
\begin{frame}{How can it be Improved ?}
The experimentation code should be documented:
\begin{frame}{How to improve it?}
More documentation:
\begin{itemize}
\item Documenting the building procedures.
\note{How to build it, relevant compilation flags etc\\}
\item What are the dependencies of the project.
\item Build process
\note{Which libraries etc\\}
\item Dependencies, requirements
\note{How to run, inputs etc\\}
% \item the dependencies.
% \item Using literate programming ?
\end{itemize}
\vspace{1em}
\pause
The running environment should be explicit:
Running environments:
\begin{itemize}
\item Which runtime is used?
\note{
How(had-hoc script, complex runtime etc), \\
Where(your laptop, your old laptop etc) you run you experiment.\\}
\item How you did your final data analysis,
\item How the visualisations are constructed.
% \item Using literate programming ?
\item Runtime, libraries
\item Data analysis scripts
\item Visualisation scripts
\end{itemize}
\end{frame}
\begin{frame}
% What does Nix offer to Experimenters ?
We propose to use Nix :)
% Why not using Nix ?
% \begin{itemize}
% \item Explicit build process,
% \note{Every deps need to be stated\\}
% \note{The building process is explicit in Nix\\}
% \item reconstructible environment.
% \note{Build Once, build twice (six month later)\\}
% \note{Possibility to describing full software environment\\}
% \note{From building environment, to runtime and data analysis.\\}
% \item Application Portability
% \note{Run on my laptop, run on g5k}
% \end{itemize}
\end{frame}
\section{Nix}
\begin{frame}{What is Nix ?}
\only<1>{%
Nix is a Package Manager with a lot of good properties. \\
% Why for experimentations ?
\vspace{1em}
\begin{itemize}
\item Reproducibility,
\note{Reproducibility of software\\}
% \item Portability,
\note{Run on my laptop, run of g5k\\}
\item Multiple version cohabitations,
% \item Composability,
\note{why\\}
\item Decentralized package repository,
\note{Packages can come from different places, thus you can have yours.\\}
\item Explicit build process,
\note{Every deps need to be stated\\}
\note{The building process is explicit in Nix\\}
\item Define reconstructible software environments,
\note{Build Once, build twice (six month later)\\}
\note{Possibility to describing full software environment\\}
\note{From building environment, to runtime and data analysis.\\}
\item Application portability
\note{build on my laptop, run on g5k}
\end{itemize}
Nix is a Package Manager: \\
% Why for experimentations ?
\vspace{1em}
\begin{itemize}
\item Reproducible packages
\item Reproducible software environments
\item Multiple versions
\item Decentralized package repositories
\note{Packages can come from different places, thus you can have yours.\\}
\item Explicit build process
\note{Every deps need to be stated\\
The building process is explicit in Nix\\
Build Once, build twice (six month later)\\
Possibility to describing full software environment\\
From building environment, to runtime and data analysis\\}
\item Portable packages
\note{build on my laptop, run on g5k}
\end{itemize}
}
% \center\Large\sc The Nix Ecosystem. {\url{https://nixos.org}}
\end{frame}
\begin{frame}{Nix Ecosystem}
Core projects:
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Nix - The Expression Language (2003)
\end{tcolorbox}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Nix - The Nix package manager (2003)
\end{tcolorbox}
Satellites:
\only<1>{
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
NixOS - The Purely Functional Linux Distribution
\end{tcolorbox}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Hydra - Nix-based continuous build system (2008)
\end{tcolorbox}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
NixOps - The NixOS Deployment Tool (2011)
\end{tcolorbox}
}
\only<2>{
\begin{tcolorbox}[colframe=gray!50,colback=gray!1,coltext=gray!100]
NixOS - The Purely Functional Linux Distribution
\end{tcolorbox}
\begin{tcolorbox}[colframe=gray!50,colback=gray!1,coltext=gray!100]
Hydra - Nix-based continuous build system (2008)
\end{tcolorbox}
\begin{tcolorbox}[colframe=gray!50,colback=gray!1,coltext=gray!100]
NixOps - The NixOS Deployment Tool (2011)
\end{tcolorbox}
}
\end{frame}
\subsection{Expression Language}
\begin{frame}{Nix - Language}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
{Nix - The Expression Language }
\end{tcolorbox}
What?
\begin{itemize}
\item Is a Domain Specific Language (DSL)
\item Functional, Turing complete language.
\item Dynamically typed. int, bool, path, string, set, list, lambda.
\item Large built-in and standard lib. \texttt{stdenv, fetchTarball, fromJson, fromGitHub, assert, test..}
\end{itemize}
\end{frame}
\begin{frame}{Nix - Language}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
{Nix - The Expression Language }
\end{tcolorbox}
Why?
\begin{itemize}
\item Packaging is complex.
\item Abstraction layers.
\item Better reusability, factorization.
\item (Readable and Maintainable)
\end{itemize}
\end{frame}
\subsection{Package Manager}
\begin{frame}{Nix}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
{Nix - The Package Manager }
\end{tcolorbox}
\begin{itemize}
\item Packages are defined in Nix expressions,
% \item Independent of the system,
\item Atomic upgrades and rollbacks
\item Several version of the same package on the same system,
\item Unprivileged package installation,
\item Provides isolated build \& runtime environments,
\item Reproducible build from source,
\item Binary cache,
\item Garbage collection,
% \item Declarative \& Imperative use.
\end{itemize}
\end{frame}
% \begin{frame}{Nix - Programming Language}
%
% What?
% \begin{itemize}
% \item Is a Domain Specific Language (DSL)
% \item Functional, Turing complete language.
% \item Dynamically typed. int, bool, path, string, set, list, lambda.
% \item Large built-in and standard lib. \texttt{stdenv, fetchTarball, fromJson, fromGitHub, assert, test..}
% \end{itemize}
% \end{frame}
%
% \begin{frame}{Nix - Language}
% \begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
% {Nix - The Expression Language }
% \end{tcolorbox}
% Why?
% \begin{itemize}
% \item Packaging is complex.
% \item Abstraction layers.
% \item Better reusability, factorization.
% \item (Readable and Maintainable)
% \end{itemize}
% \end{frame}
\begin{frame}{}
\center
How does it work?
\end{frame}
\section{Nix Structure}
\section{Nix Concepts}
\begin{frame}{Main Concepts}
% Nix borrows its concept from Functional Languages. \\
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Nix applies the concept of functional programming languages to package management.
\end{tcolorbox}
\vspace{1em}
Each software building action is described through functions.
\note{Function from a functional paradigm point of view}
\vspace{1em}
Package dependencies are also functions given as inputs.
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Functional paradigm applied to package management
\end{tcolorbox}
\vspace{1em}
Such functions are called \textbf{Derivations}
\begin{itemize}
\item Functions build packages
\item Functions inputs are package dependencies
\end{itemize}
\end{frame}
......@@ -323,18 +221,16 @@ The running environment should be explicit:
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
A derivation is a Nix build action.
\end{tcolorbox}
\begin{onlyenv}<2>
\vspace{1em}
It is a Nix attribute set describing how to build a package.
A derivation is defined in Nix, as a set of attributes:
% It is a Nix attribute set describing how to build a package.
\note{A python dict, or java hashtable\\}
\end{onlyenv}
\pause
\vspace{-1em}
\begin{onlyenv}<3>
\begin{lstlisting}[language=Nix,basicstyle=\small]
stdenv.mkDerivation {
......@@ -346,13 +242,9 @@ stdenv.mkDerivation {
};
buildInputs = [ simgrid boost cmake ];
meta = ..;
}
\end{lstlisting}
\end{onlyenv}
\end{frame}
\begin{frame}[fragile]{Store}
......@@ -374,8 +266,8 @@ The result of the build action is automatically placed into the nix store.
\begin{frame}[fragile]{Store}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Every packages (or derivations) are located into a common place: The store.
% Nix uses a single directory to store every packages.
Every package is located into a common place: The store.
% Nix uses a single directory to store every packages.
\end{tcolorbox}
\vspace{1em}
......@@ -405,11 +297,11 @@ The result of the build action is automatically placed into the nix store.
\end{tcolorbox}
\begin{lstlisting}[basicstyle=\small]
/nix/store/hash-packagename/..
/bin/package
/lib/package.so
/share/
.
/nix/store/hash-packag/..
/bin/package
/lib/package.so
/share/
.
\end{lstlisting}
\end{frame}
......@@ -419,7 +311,7 @@ The result of the build action is automatically placed into the nix store.
\begin{column}{0.5\textwidth}
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
Packages can reference/depend on other packages.
Package can reference on other packages.
\end{tcolorbox}
\end{column}
......@@ -447,26 +339,16 @@ The result of the build action is automatically placed into the nix store.
\end{lstlisting}
\vspace{1em}
Missing dependencies can be identified and downloaded from a distant store.
Missing dependencies can be identified and downloaded from a remote store.
\end{frame}
\begin{frame}
A derivation is a build action written in Nix. \\
\vspace{1em}
Used to build package. \\
\vspace{1em}
The resulting package is stocked in the store, \\
identified with a unique path. \\
\vspace{1em}
In which it can refer to other packages.
\vspace{1em}
\end{frame}
\begin{frame}[fragile]{Nixpkgs}
How and where can we get derivations?
How and where can we get a derivation?
\pause
\vspace{1em}
......@@ -497,8 +379,7 @@ The result of the build action is automatically placed into the nix store.
\end{tcolorbox}
\vspace{1em}
Channels are useful to downloaded latest version of a software.
Channels are useful to downloaded latest packaged version of a software. \\
\vspace{1em}
......@@ -507,6 +388,10 @@ The result of the build action is automatically placed into the nix store.
\item nixpkgs-19.03 (current stable)
\item nixpkgs-18.09 (outdated)
\end{itemize}
\vspace{1em}
(Channels benefit from binary cache.)
}
\only<2>{
......@@ -521,14 +406,28 @@ The result of the build action is automatically placed into the nix store.
\end{frame}
% \begin{frame}{}
% A derivation is a build action written in Nix. \\
% \vspace{1em}
% Used to build package. \\
% \vspace{1em}
% The resulting package is stocked in the store, \\
% identified with a unique path. \\
% \vspace{1em}
% In which it can refer to other packages.
% \vspace{1em}
% \end{frame}
\begin{frame}{Key points}
\begin{enumerate}
\item Applies functional paradigm to package management system.
\item Packages are called derivations, written in Nix lang.
\item The list of existing packages are into NixPkgs repository.
\item Such derivation defines a building process.
\item A built derivation is located to the nix store into a dedicated directory.
\item The folder name is generated from a hash of its inputs and its nix file.
\item The directory name is generated from a hash of its inputs and its nix file.
\item Packages can have reference on other packages from the store.
\item The list of existing packages are into NixPkgs repository.
\end{enumerate}
\end{frame}
......@@ -606,11 +505,25 @@ src = fetgit {
\begin{frame}[fragile]
With Nix you can define and share complex experiment environment.
Pros:
\begin{tcolorbox}[colframe=black,colback=gray!10,coltext=black]
\begin{itemize}
\item It will be reconstructible.
\item The dependencies will be clearly stated.
\item Defining and sharing complex environments easily.
\item Environments are reconstructible .
\item Writing derivations requires to state every dependencies.
\end{itemize}
\end{tcolorbox}
\vspace{1em}
Cons:
\begin{tcolorbox}[colframe=red!37,colback=red!10,coltext=black]
\begin{itemize}
\item The entry cost is high.
\item It is contaminating (heavily depends on the nix store).
\end{itemize}
\end{tcolorbox}
% Nix ensures the reproducibility of software environments.
\vspace{1em}
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment