From a95f7fa35ec0d8403a80613ccfabfd5806c8af5f Mon Sep 17 00:00:00 2001 From: Rodrigo Arias Mallo Date: Mon, 6 Jul 2020 11:19:20 +0200 Subject: [PATCH] Add details for xeon07 --- README | 194 ++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 116 insertions(+), 78 deletions(-) diff --git a/README b/README index 666376b..fbdbf98 100644 --- a/README +++ b/README @@ -1,6 +1,6 @@ - BSC Nixpkgs: User guide + BSC Nixpkgs: User guide 1 Introduction @@ -11,34 +11,50 @@ Some preliminary steps must be done manually to be able to build and install packages (derivations in nix jargon). + This guide is specific for the nix installation in the xeon07 node, + accessed by the ssfhead.bsc.es login node, but it may be used in other + machines as well. + + To easily connect to xeon07 in one step, setup the SSH (for version + 7.3 and upwards) configuration file in ~/.ssh/config adding these + lines: + + Host cobi + HostName ssflogin.bsc.es + User your-username-here + + Host xeon07 + ProxyJump cobi + HostName xeon07 + User your-username-here + + You should be able to connect with: + + $ ssh xeon07 + 1.1 Network access - In order to use nix you need to be able to download the sources from - Internet. Usually the download requires the ports 22, 80 and 443 to be - open for outgoing traffic. + In order to use nix would you need to be able to download the sources + from Internet. Usually the download requires the ports 22, 80 and 443 + to be open for outgoing traffic. Unfortunately, in some clusters (as is the case in xeon07) access to Internet is disabled. However you can tunnel the connection by SSH to your local machine, and then reach the Internet. - There are some guides on how to prepare the proxy server and the - tunnel in SSH such as: + In order to tell nix to use the proxy connection, you will need to + export the "https_proxy" and "http_proxy" variables. A proxy + connection is already configured in xeon07 and you can automatically + set those variables to the correct address by loading: - https://www.seniorlinuxadmin.co.uk/ssh-over-proxy.html - - In order to instruct nix to use the proxy connection, you will need to - export the https_proxy and http_proxy variables. In the xeon07 node is - already configured and you can automatically set those variables to - the correct address by loading: - - $ . /scratch/nix/internet + xeon07$ . /scratch/nix/internet Consider adding the command to your ~/.bashrc file so you don't need to do it every time you want to use nix. Now you should be able to reach the outside world by running: - $ curl google.com + xeon07$ curl google.com 301 Moved

301 Moved

@@ -53,13 +69,13 @@ prompt. Most repositories at https://pm.bsc.es/gitlab are open to read for - logged users, but there are some exceptions for example the nanos6 - repository where you must have explicitly granted read access. + logged in users, but there are some exceptions (for example the nanos6 + repository) where you must have explicitly granted read access. - If you don't have a ssh key at ~/.ssh/*.pub create a new one without - password protection by running: + If you don't have a ssh key at ~/.ssh/*.pub in xeon07 create a new one + without password protection by running: - $ ssh-keygen + xeon07$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (~/.ssh/id_rsa): Enter passphrase (empty for no passphrase): @@ -74,17 +90,15 @@ https://pm.bsc.es/gitlab/profile/keys - If you want to select another key rather than the default - ~/.ssh/id_rsa then you must configure it for use in the ~/.ssh/config - file, adding: + Then, configure it for use in the ~/.ssh/config file, adding: Host bscpm02.bsc.es - IdentityFile ~/.ssh/your-private-key + IdentityFile ~/.ssh/id_rsa Finally verify the SSH connection to the server works and you get a greeting from the GitLab server with your username: - $ ssh git@bscpm02.bsc.es + xeon07$ ssh git@bscpm02.bsc.es PTY allocation request failed on channel 0 Welcome to GitLab, @rarias! Connection to bscpm02.bsc.es closed. @@ -97,32 +111,32 @@ Finally, you should be able to download the rarias/nanos6 git repository without any password interaction by running: - $ git clone git@bscpm02-bsc-es:rarias/nanos6.git + xeon07$ git clone git@bscpm02-bsc-es:rarias/nanos6.git 1.3 Prepare the bsc-nixpkgs repo Once you have Internet and you have granted access to the PM GitLab repositories you can begin down the rabbit hole of nix. First ensure - that the nix binaries are available in your machine: + that the nix binaries are available from your shell in xeon07: - $ nix --version + xeon07$ nix --version nix (Nix) 2.3.6 Now you are set to install packages with nix. Clone the bsc-nixpkgs repository: - $ git clone git@bscpm02-bsc-es:rarias/bsc-nixpkgs.git + xeon07$ git clone git@bscpm02-bsc-es:rarias/bsc-nixpkgs.git Nix looks in the current folder for a file named "default.nix" for packages, so go to the repo: - $ cd bsc-nixpkgs + xeon07$ cd bsc-nixpkgs Now you should be able to build nanos6 from the git repository: - $ nix-build -A bsc.nanos6-git + xeon07$ nix-build -A bsc.nanos6-git - The output is placed in the "result" symlink. + The output is placed in the "result" symbolic link. 2. Basic usage of nix @@ -138,10 +152,10 @@ All nix packages are stored under the /nix directory. When you need to "install" some binary from nix, a symlink is added to a folder - included in the $PATH variable. In particular, you should have the + included in the $PATH variable. In particular, you should have something similar added to your $PATH: - $ echo $PATH | sed 's/:/\n/g' | grep nix + xeon07$ echo $PATH | sed 's/:/\n/g' | grep nix /home/Computational/rarias/.nix-profile/bin /nix/var/nix/profiles/default/bin @@ -150,17 +164,35 @@ which contains the nix tools (which are installed in the /nix directory as well). + Use `nix search` to look for official packages in the "nixpkgs" + channel (the default repository of packages): + + xeon07$ nix search cowsay + warning: using cached results; pass '-u' to update the cache + * cowsay (cowsay) + A program which generates ASCII pictures of a cow with a message + + * neo-cowsay (neo-cowsay) + Cowsay reborn, written in Go + + * ponysay (ponysay-3.0.3) + Cowsay reimplemention for ponies + + * tewisay (tewisay-unstable-2017-04-14) + Cowsay replacement with unicode and partial ansi escape support + When you need a program that is not available in your environment, much like when you use "module load ..." you can use nix-env to modify what is currently loaded. For example: - $ nix-env -iA nixpkgs.cowsay + xeon07$ nix-env -iA nixpkgs.cowsay - The command will download (if not found already in the nix store), - compile (if necessary) and load the program `cowsay` from the nixpkgs + Notice that you should specify the prefix "nixpkgs." before. The + command will download (if not found already in the nix store), compile + (if necessary) and load the program `cowsay` from the nixpkgs repository in the environment. You should be able to run it as: - $ cowsay "hello world" + xeon07$ cowsay "hello world" _____________ < hello world > ------------- @@ -170,17 +202,17 @@ ||----w | || || - You can inspect now the ~/.nix-profile/bin folder, and see that a new + You can now inspect the ~/.nix-profile/bin folder, and see that a new symlink was added to the actual installation of the binary: - $ file ~/.nix-profile/bin/cowsay + xeon07$ file ~/.nix-profile/bin/cowsay /home/Computational/rarias/.nix-profile/bin/cowsay: symbolic link to `/nix/store/673gczmhr5b449521srz2n7g1klykz6n-cowsay-3.03+dfsg2/bin/cowsay' You can list the current packages installed in your environment by running: - $ nix-env -q + xeon07$ nix-env -q cowsay-3.03+dfsg2 nix-2.3.6 @@ -191,7 +223,7 @@ You can remove any package from the environment using: - $ nix-env -e cowsay + xeon07$ nix-env -e cowsay See the manual with `nix-env --help` if you want to know more details. @@ -203,63 +235,67 @@ any binary cache server yet. Nix will handle the build process without any user interaction (with a - few exceptions stated later, which you shouldn't have to worry). If - any other user has already built the package then the build process is - not needed, and the package is used as is. + few exceptions which you shouldn't have to worry). If any other user + has already built the package then the build process is not needed, + and the package is used as is. In order to build a BSC package go to the `bsc-nixpkgs` directory, and run: - $ nix-build -A bsc.dummy + xeon07$ nix-build -A bsc.dummy - The package will be built and installed in the /nix directory, then a - symlink is placed in the result directory: + Notice the "bsc." prefix for BSC packages. The package will be built + and installed in the /nix directory, then a symlink is placed in the + result directory: - $ find result/ -type f + xeon07$ find result/ -type f result/ result/bin result/bin/dummy The way in which nix handles the packages and dependencies ensures - that the output of a compilation of any package is exactly the same. + that the environment of the build process of any package is exactly + the same, so the generated output should be the same if the builds are + deterministic. + You can check the reproducibility of the build by adding the "--check" flag, which will rebuild the package and compare the checksum of every - file with the ones installed: + file with the ones previously built: - $ nix-build -A bsc.dummy --check + xeon07$ nix-build -A bsc.dummy --check ... - $ echo $? + xeon07$ echo $? 0 A return code of zero ensures the output is bit by bit identical to - the one installed. There are some packages that try to include - non-reproducible information in the build process as the timestamp - which will produce an error. Those packages must be patched to ensure - the output is deterministic. + the one installed. There are some packages that include + indeterministic information in the build process (such as the + timestamp of the current time) which will produce an error. Those + packages must be patched to ensure the output is deterministic. - Notice that if you "cd" into the result/ directory you will be at /nix - directory (as you follow the symlink) where you don't have write - permission. Therefore if your program attempts to write to the current - directory it will fail. It is recommended to instead run your program - from the top directory: + Notice that if you "cd" into the "result/" directory you will be at + /nix directory (as you have follow the symlink) where you don't have + write permission. Therefore if your program attempts to write to the + current directory it will fail. It is recommended to instead run your + program from the top directory: - $ result/bin/dummy + xeon07$ result/bin/dummy Hello world! Or you can install it in the environment: - $ nix-env -i ./result + xeon07$ nix-env -i ./result And "cd" into any directory where you want to output some files and just run it by the name: - $ cd /tmp - $ dummy + xeon07$ cd /tmp + xeon07$ dummy Hello world! Finally, you can remove it from the environment if you don't need it: - $ nix-env -e dummy + xeon07$ nix-env -e dummy If you want to know more details use "nix-build --help" to see the manual. @@ -267,18 +303,20 @@ 2.3 The build process Each package is built following a programmable configuration - description in the nix language. Build in nix are performed under very - strict conditions. No access any file in the file system is allowed, - unless stated in the dependencies of the package which are then - available in the build environment. + description in the nix language. Builds in nix are performed under + very strict conditions. No access to any file in the file system is + allowed, unless stated in the dependencies, which are in the /nix + store only. There is no network access in the build process and other restrictions - are enforced that the build is deterministic. See more details here: + are enforced so that the build environment is reproducible. See more + details here: https://nixos.wiki/wiki/Nix#Sandboxing - In the "default.nix" file of the bsc-nixpkgs you can see the - definition for each package, for example the nbody app: + The top level "default.nix" file of the bsc-nixpkgs serves as a index + of all BSC packages. You can see the definition for each package, for + example the nbody app: nbody = callPackage ./bsc/apps/nbody/default.nix { stdenv = pkgs.gcc9Stdenv; @@ -304,12 +342,12 @@ Then you can rebuild the nbody package: - $ nix-build -A bsc.nbody + xeon07$ nix-build -A bsc.nbody ... And verify that the binary is indeed linked to MPICH now: - $ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi + xeon07$ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi libmpi.so.12 => /nix/store/dwkkcv78a5bs8smflpx9ppp3klhz3i98-mpich-3.3.2/lib/libmpi.so.12 (0x00007f6be0f07000) If you modify a package which another package requires as a @@ -333,9 +371,9 @@ The build process now is not required: - $ nix-build -A bsc.nbody + xeon07$ nix-build -A bsc.nbody /nix/store/rbq7wrjcmg6fzd6yhrlnkfvzcavdbdpc-nbody - $ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi + xeon07$ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi libmpifort.so.12 => /nix/store/jvsjvxj2a08340fpdrqbqix9z3mpp3bd-intel-mpi-2019.7.217/lib/libmpifort.so.12 (0x00007f3a00402000) libmpi.so.12 => /nix/store/jvsjvxj2a08340fpdrqbqix9z3mpp3bd-intel-mpi-2019.7.217/lib/libmpi.so.12 (0x00007f39fed34000)