Skip to content

Fundamentals

Main.nix format

Each main.nix file under the makes/ directory should be a function that receives one or more arguments and returns a derivation:

1
2
3
4
5
6
{
  argA,
  argB,
  ...
}:
doSomethingAndReturnADerivation

Derivations

A Nix derivation is the process of:

  • taking zero or more inputs
  • transforming them as we see fit
  • placing the results in the output path

Derivation outputs live in the /nix/store. Their locations in the filesystem are always in the form: /nix/store/hash123-name where hash123 is computed by hashing the derivation's inputs.

Derivation outputs are:

  • A regular file
  • A regular directory that contains arbitrary contents

For instance the derivation output for Bash is: /nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23 which contains, among other files:

1
2
3
4
/nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23
├── bin
│   ├── bash
│   └── sh

makeSearchPaths

On Linux software dependencies can be located anywhere in the file system.

We can control where programs find other programs, dependencies, libraries, etc, through special environment variables.

Below we describe shortly the purpose of the environment variables we currently support.

makeSearchPaths helps you write code like this:

1
2
3
makeSearchPaths {
  bin = [ inputs.nixpkgs.git ];
}

Instead of this:

1
export PATH="/nix/store/m5kp2jhiga25ynk3iq61f4psaqixg7ib-git-2.32.0/bin${PATH:+:}${PATH:-}"

Types:

  • makeSearchPaths (function { ... } -> package):
    • bin (listOf coercibleToStr): Optional. Append /bin of each element in the list to PATH. Defaults to [ ].
    • rpath (listOf coercibleToStr): Optional. Append /lib and /lib64 of each element in the list to LD_LIBRARY_PATH. Defaults to [ ].
    • source (listOf coercibleToStr): Optional. Source (as in Bash's source command) each element in the list. Defaults to [ ].

Types specific to Crystal:

  • makeSearchPaths (function { ... } -> package):
    • crystalLib (listOf coercibleToStr): Optional. Append /lib of each element in the list to CRYSTAL_LIBRARY_PATH. Defaults to [ ].

Types specific to Java:

  • makeSearchPaths (function { ... } -> package):
    • javaClass (listOf coercibleToStr): Optional. Append each element in the list to CLASSPATH. Defaults to [ ].

Types specific to Kubernetes:

  • makeSearchPaths (function { ... } -> package):
    • kubeConfig (listOf coercibleToStr): Optional. Append each element in the list to KUBECONFIG. Defaults to [ ].

Types specific to pkg-config:

  • makeSearchPaths (function { ... } -> package):
    • pkgConfig (listOf coercibleToStr): Optional. Append /lib/pkgconfig of each element in the list to PKG_CONFIG_PATH. Defaults to [ ].

Types specific to OCaml:

  • makeSearchPaths (function { ... } -> package):
    • ocamlBin (listOf coercibleToStr): Optional. Append /bin of each element in the list to PATH. Defaults to [ ].
    • ocamlLib (listOf coercibleToStr): Optional. Append / of each element in the list to OCAMLPATH. Defaults to [ ].
    • ocamlStublib (listOf coercibleToStr): Optional. Append /stublib of each element in the list to CAML_LD_LIBRARY_PATH. Defaults to [ ]

Types specific to Python:

  • makeSearchPaths (function { ... } -> package):
    • pythonMypy (listOf coercibleToStr): Optional. Append / of each element in the list to MYPYPATH. Defaults to [ ].
    • pythonMypy39 (listOf coercibleToStr): Optional. Append /lib/python3.9/site-packages of each element in the list to MYPYPATH. Defaults to [ ].
    • pythonMypy310 (listOf coercibleToStr): Optional. Append /lib/python3.10/site-packages of each element in the list to MYPYPATH. Defaults to [ ].
    • pythonMypy311 (listOf coercibleToStr): Optional. Append /lib/python3.11/site-packages of each element in the list to MYPYPATH. Defaults to [ ].
    • pythonPackage (listOf coercibleToStr): Optional. Append / of each element in the list to PYTHONPATH. Defaults to [ ].
    • pythonPackage39 (listOf coercibleToStr): Optional. Append /lib/python3.9/site-packages of each element in the list to PYTHONPATH. Defaults to [ ].
    • pythonPackage310 (listOf coercibleToStr): Optional. Append /lib/python3.10/site-packages of each element in the list to PYTHONPATH. Defaults to [ ].
    • pythonPackage311 (listOf coercibleToStr): Optional. Append /lib/python3.11/site-packages of each element in the list to PYTHONPATH. Defaults to [ ].

Types specific to Node.js:

  • makeSearchPaths (function { ... } -> package):
    • nodeBin (listOf coercibleToStr): Optional. Append /.bin of each element in the list to PATH. Defaults to [ ].
    • nodeModule (listOf coercibleToStr): Optional. Append / of each element in the list to NODE_PATH. Defaults to [ ].

Types specific to Ruby:

  • makeSearchPaths (function { ... } -> package):
    • rubyBin (listOf coercibleToStr): Optional. Append /bin of each element in the list to PATH. Defaults to [ ].
    • rubyGemPath (listOf coercibleToStr): Optional. Append / of each element in the list to GEM_PATH. Defaults to [ ].

Types for non covered cases:

  • makeSearchPaths (function { ... } -> package):

    • export (listOf (tuple [ str coercibleToStr str ])): Optional. Export (as in Bash's export command) each tuple in the list.

      Defaults to [ ].

      Tuples elements are:

      • Name of the environment variable to export.
      • Base package to export from.
      • Relative path with respect to the package that should be appended.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{
  makeSearchPaths,
  ...
}:
makeSearchPaths {
  bin = [ inputs.nixpkgs.git ];
  source = [
    [ ./template.sh "a" "b" "c" ]
    # add more as you need ...
  ];
  export = [
    [ "PATH" inputs.nixpkgs.bash "/bin"]
    [ "CPATH" inputs.nixpkgs.glib.dev "/include/glib-2.0"]
    # add more as you need ...
  ];
}

1
2
# /path/to/my/project/makes/example/template
echo "${@}"

1
2
3
4
5
6
7
8
export PATH"/nix/store/...-git/bin${PATH:+:}${PATH:-}"
export PATH="/nix/store/...-bash/bin${PATH:+:}${PATH:-}"
export CPATH="/nix/store/...-glib-dev/include/glib-2.0${CPATH:+:}${CPATH:-}"

if test -e "/nix/store/...-template/template"
then source "/nix/store/...-template/template" '1' '2' '3'
else source "/nix/store/...-template" '1' '2' '3'
fi

makeDerivation

Perform a build step in an isolated environment:

  • External environment variables are not visible by the builder script. This means you can't use secrets here.
  • Search Paths as in makeSearchPaths are completely empty.
  • The HOME environment variable is set to /homeless-shelter.
  • Only GNU coreutils commands (cat, echo, ls, ...) are present by default.
  • An environment variable called out is present and represents the derivation's output. The derivation must produce an output, may be a file, or a directory.

  • Convenience bash functions are exported:

    • echo_stderr: Like echo but to standard error.
    • debug: Like echo_stderr but with a [DEBUG] prefix.
    • info: Like echo_stderr but with a [INFO] prefix.
    • warn: Like echo_stderr but with a [WARNING] prefix.
    • error: Like echo_stderr but with a [ERROR] prefix. Returns exit code 1 to signal failure.
    • critical: Like echo_stderr but with a [CRITICAL] prefix. Exits immediately with exit code 1, aborting the entire execution.
    • copy: Like cp but making paths writeable after copying them.

    • require_env_var: errors when the specified env var is not set, or set to an empty value.

      1
      require_env_var USERNAME

  • After the build, for all paths in $out:

    • User and group ownership are removed
    • Last-modified timestamps are reset to 1970-01-01T00:00:00+00:00.

Types:

  • makeDerivation (function { ... } -> package):
    • builder (either str package): A Bash script that performs the build step.
    • env (attrsOf str): Optional. Environment variables that will be propagated to the builder. Variable names must start with env. Defaults to { }.
    • local (bool): Optional. Should we always build locally this step? Thus effectively ignoring any configured binary caches. Defaults to false.
    • name (str): Custom name to assign to the build step, be creative, it helps in debugging.
    • searchPaths (asIn makeSearchPaths): Optional. Arguments here will be passed as-is to makeSearchPaths. Defaults to makeSearchPaths's defaults.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# /path/to/my/project/makes/example/main.nix
{
  inputs,
  makeDerivation,
  ...
}:
makeDerivation {
  env = {
    envVersion = "1.0";
  };
  builder = ''
    debug Version is $envVersion
    info Running tree command on $PWD
    mkdir dir
    touch dir/file
    tree dir > $out
  '';
  name = "example";
  searchPaths = {
    bin = [ inputs.nixpkgs.tree ];
  };
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
$ m . /example

    [DEBUG] Version is 1.0
    [INFO] Running tree command on /tmp/nix-build-example.drv-0
    /nix/store/30hg7hzn6d3zmfva1bl4zispqilbh3nm-example

$ cat /nix/store/30hg7hzn6d3zmfva1bl4zispqilbh3nm-example
    dir
    `-- file

    0 directories, 1 file

makeTemplate

Replace placeholders with the specified values in a file of any format.

Types:

  • makeTemplate (function { ... } -> package):
    • local (bool): Optional. Should we always build locally this step? Thus effectively ignoring any configured binary caches. Defaults to true.
    • name (str): Custom name to assign to the build step, be creative, it helps in debugging.
    • replace (attrsOf strLike): Optional. Placeholders will be replaced in the script with their respective value. Variable names must start with __arg, end with __ and have at least 6 characters long. Defaults to { }.
    • template (either str package): A string, file, output or package in which placeholders will be replaced.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# /path/to/my/project/makes/example/main.nix
{
  inputs,
  makeTemplate,
  ...
}:
makeTemplate {
  name = "example";
  replace = {
    __argBash__ = inputs.nixpkgs.bash;
    __argVersion__ = "1.0";
  };
  template = ''
    Bash is: __argBash__
    Version is: __argVersion__
  '';
}

1
2
3
4
$ m . /example

    Bash is: /nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23
    Version is: 1.0

makeScript

Wrap a Bash script that runs in a almost-isolated environment.

  • The file system is not isolated, the script runs in user-space.
  • External environment variables are visible by the script. You can use this to propagate secrets.
  • Search Paths as in makeSearchPaths are completely empty.
  • The HOME_IMPURE environment variable is set to the user's home directory.
  • The HOME environment variable is set to a temporary directory.
  • Only GNU coreutils commands (cat, echo, ls, ...) are present by default.
  • An environment variable called STATE points to a directory that can be used to store the script's state (if any). That state can be optionally persisted. That state can be optionally shared across repositories.

  • Convenience bash functions are exported:

    • running_in_ci_cd_provider: Detects if we are running on the CI/CD provider (gitlab/github/etc).

      1
2
3
4
5
      if running_in_ci_cd_provider; then
  # ci/cd logic
else
  # non ci/cd logic
fi

    • prompt_user_for_confirmation: Warns the user about a possibly destructive action that will be executed soon and aborts if the user does not confirm aproppriately.

      This function assumes a positive answer when running on the CI/CD provider because there is no human interaction. - prompt_user_for_input: Ask the user to type information or optionally use a default value by pressing ENTER.

      This function assumes the default value when running on the CI/CD provider because there is no human interaction.

      1
2
3
      user_supplied_input="$(prompt_user_for_input "default123123")"

info Supplied input: "${user_supplied_input}"

  • After the build, the script is executed.

Types:

  • makeScript (function { ... } -> package):
    • entrypoint (either str package): A Bash script that performs the build step.
    • name (str): Custom name to assign to the build step, be creative, it helps in debugging.
    • replace (attrsOf strLike): Optional. Placeholders will be replaced in the script with their respective value. Variable names must start with __arg, end with __ and have at least 6 characters long. Defaults to { }.
    • searchPaths (asIn makeSearchPaths): Optional. Arguments here will be passed as-is to makeSearchPaths. Defaults to makeSearchPaths's defaults.
    • persistState (bool): Optional. If true, state will not be cleared before each script run. Defaults to false.

    • globalState (bool): Optional. If true, script state will be written to globalStateDir and to projectStateDir otherwise. Defaults to false, if projectStateDir is specified or derived.

      Note
      • It is implicitly true, if projectStateDir == globalStateDir.
      • projectStateDir == globalStateDir is the default if projectIdentifier is not configured.
      • Hence, generally enable project local state by
        • either setting projectIdentifier
        • or projectStateDir different from globalStateDir.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# /path/to/my/project/makes/example/main.nix
{
  inputs,
  makeScript,
  ...
}:
makeScript {
  replace = {
    __argVersion__ = "1.0";
  };
  entrypoint = ''
    debug Version is __argVersion__
    info pwd is $PWD
    info Running tree command on $STATE
    mkdir $STATE/dir
    touch $STATE/dir/file
    tree $STATE
  '';
  name = "example";
  searchPaths = {
    bin = [ inputs.nixpkgs.tree ];
  };
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
$ m . /example

    [DEBUG] Version is 1.0
    [INFO] pwd is /data/github/fluidattacks/makes
    [INFO] Running tree command on /home/user/.cache/makes/state/example
    /home/user/.cache/makes/state/example
    └── dir
        └── file

    1 directory, 1 file

projectPath

Copy a path from the current Makes project being evaluated to the Nix store in the most pure and reproducible way possible.

Types:

  • projectPath (function str -> package):
    • (str): Absolute path, assumming the repository is located at "/".

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# Consider the following path within the repository: /src/nix

# /path/to/my/project/makes/example/main.nix
{
  makeScript,
  projectPath,
  ...
}:
makeScript {
  replace = {
    __argPath__ = projectPath "/src/nix";
  };
  entrypoint = ''
    info Path is: __argPath__
    info Path contents are:
    ls __argPath__
  '';
  name = "example";
}

1
2
3
4
5
$ m . /example

    [INFO] Path is: <nix-store-path>
    [INFO] Path contents are:
    packages.nix  sources.json  sources.nix