Applying Nix best practices and useful paradigms

[brianmcgee]

• introduces flake.parts for a modular approach to composing the flake outputs
• configures treefmt for nix fmt which enables the following:
  • alejandra to format .nix files
  • deadnix to remove dead nix code
  • statix to avoid nix anti-patterns
  • shellcheck to lint shell scripts
  • mixes in a flake check
• adds reuse-lint to check for copyright header issues when running nix flake check
• adds flake-compat to ensure non-flake users can work with the repo and moves the devshell definition into the flake devShells.default output
• adds a .envrc file for direnv users to automatically drop into the default dev shell when cd'ing into the repo
• consolidates the various derivations for scripts etc into a single sbomnix derivation as they shared the same source and dependencies

[henrirosten]

@brianmcgee : thanks for this PR, this is great work!

Few issues I think we need to address before we merge this. I noticed vulnix and nix-visualize are no longer in the $PATH after this change, which makes vulnxscan and nix_outdated fail.

vulnxscan failure example:

$ vulnxscan  $(nix eval github:numtide/devshell#apps.x86_64-linux.default.program | xargs) 
  CRITICAL command 'vulnix' is not in PATH

nix_outdated failure example:

$ nix_outdated $(nix eval github:numtide/devshell#apps.x86_64-linux.default.program | xargs) 
INFO     Checking runtime dependencies referenced by '/nix/store/c0ah7myj7ska9np268scry4phi2znh4k-devshell-dir/entrypoint'
INFO     Generating SBOM for target '/nix/store/c0ah7myj7ska9np268scry4phi2znh4k-devshell-dir/entrypoint'
INFO     Loading runtime dependencies referenced by '/nix/store/c0ah7myj7ska9np268scry4phi2znh4k-devshell-dir/entrypoint'
INFO     Using SBOM '/tmp/nixdeps_1t6e1mzr.cdx.json'
INFO     Running repology_cli
INFO     Using repology out: '/tmp/repology_b7re5x25.csv'
INFO     Running nix-visualize
Traceback (most recent call last):
  File "/home/hrosten/projects/sbomnix/venv/bin/nix_outdated", line 33, in <module>
    sys.exit(load_entry_point('sbomnix', 'console_scripts', 'nix_outdated')())
  File "/home/hrosten/projects/sbomnix/scripts/nixupdate/nix_outdated.py", line 267, in main
    nix_visualize_out = _run_nix_visualize(target_path_abs)
  File "/home/hrosten/projects/sbomnix/scripts/nixupdate/nix_outdated.py", line 106, in _run_nix_visualize
    exec_cmd(cmd.split())
  File "/home/hrosten/projects/sbomnix/sbomnix/utils.py", line 116, in exec_cmd
    return subprocess.run(cmd, capture_output=True, encoding="utf-8", check=True)
  File "/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'nix-visualize'

[brianmcgee]

@henrirosten

Few issues I think we need to address before we merge this. I noticed vulnix and nix-visualize are no longer in the $PATH after this change, which makes vulnxscan and nix_outdated fail.

Fixed in f61703c

[henrirosten]

Thanks @brianmcgee. One more thing I noticed in testing nix run with the flake apps:

When I run the below command outside sbomnix nix-shell (direnv), it fails with the error shown below:

$ nix run github:brianmcgee/sbomnix?rev=f61703c4458d593b9153003c4ab09c4799662fce#nix_outdated -- $(nix eval github:numtide/devshell#apps.x86_64-linux.default.program | xargs)
Traceback (most recent call last):
  File "/nix/store/7c73b4rhinp09c43kyllqkw2np9pm1d4-python3.10-sbomnix-1.4.6/bin/..nix_outdated-wrapped-wrapped", line 6, in <module>
    from scripts.nixupdate.nix_outdated import main
ModuleNotFoundError: No module named 'scripts.nixupdate'

Also repology_cli and repology_cve fail similarly:

$ nix run github:brianmcgee/sbomnix?rev=f61703c4458d593b9153003c4ab09c4799662fce#repology_cli -- --pkg_search 'firef' --repository 'nix_unstable'
Traceback (most recent call last):
  File "/nix/store/7c73b4rhinp09c43kyllqkw2np9pm1d4-python3.10-sbomnix-1.4.6/bin/.repology_cli-wrapped", line 6, in <module>
    from scripts.repology.repology_cli import main
ModuleNotFoundError: No module named 'scripts.repology'

$ nix run github:brianmcgee/sbomnix?rev=f61703c4458d593b9153003c4ab09c4799662fce#repology_cve -- openssl 3.1.0
Traceback (most recent call last):
  File "/nix/store/7c73b4rhinp09c43kyllqkw2np9pm1d4-python3.10-sbomnix-1.4.6/bin/.repology_cve-wrapped", line 6, in <module>
    from scripts.repology.repology_cve import main
ModuleNotFoundError: No module named 'scripts.repology'

[brianmcgee]

@henrirosten the nix run issues are resolved in fbe307d.

There were some packages missing in setup.py.

nix_outdated

❯ nix run github:brianmcgee/sbomnix/fbe307d15908d7e13fbd827c33d6155c6aa469c3#nix_outdated -- $(nix build --print-out-paths nixpkgs#graphviz)

INFO     Checking runtime dependencies referenced by '/nix/store/7v2hwwxhfklcp2plkw23qh7ak8baq9l9-graphviz-7.1.0'
INFO     Generating SBOM for target '/nix/store/7v2hwwxhfklcp2plkw23qh7ak8baq9l9-graphviz-7.1.0'
INFO     Loading runtime dependencies referenced by '/nix/store/7v2hwwxhfklcp2plkw23qh7ak8baq9l9-graphviz-7.1.0'
INFO     Using SBOM '/tmp/nixdeps_91fskes_.cdx.json'
INFO     Running repology_cli
INFO     Using repology out: '/tmp/repology_8gjhdj5m.csv'
INFO     Running nix-visualize
INFO     Using nix-visualize out: '/tmp/nix-visualize_ohswb3jc.csv'
INFO     Writing console report
INFO     Dependencies that need update in nixpkgs (in priority order based on how many other packages depend on the potentially outdated package):

|  priority  | nix_package   | version_local   | version_nixpkgs   | version_upstream   |
|------------+---------------+-----------------+-------------------+--------------------|
|     7      | gcc           | 12.2.0          | 12.3.0            | 13.2.0;13.2        |
|     7      | xz            | 5.4.3           | 5.4.4             | 5.4.5              |
|     7      | libdeflate    | 1.18            | 1.18              | 1.19               |
|     7      | libjpeg-turbo | 2.1.5.1         | 2.1.5.1           | 3.0.1              |
|     6      | libunwind     | 1.6.2           | 1.6.2             | 1.7.2              |
|     6      | tiff          | 4.5.1           | 4.5.1             | 4.6.0              |
|     5      | openexr       | 2.5.8           | 2.5.8             | 3.2.1              |
|     5      | openexr       | 2.5.8           | 3.2.0             | 3.2.1              |
|     5      | gperftools    | 2.10            | 2.10              | 2.13;2.13.0        |
|     4      | libselinux    | 3.3             | 3.3               | 3.5                |
|     3      | glib          | 2.76.2          | 2.76.4            | 2.78.1             |
|     3      | libglvnd      | 1.6.0           | 1.6.0             | 1.7.0              |
|     3      | dav1d         | 1.2.0           | 1.2.1             | 1.3.0              |
|     2      | harfbuzz      | 7.2.0           | 7.3.0             | 8.2.2              |


INFO     Wrote: nix_outdated.csv

repology_cli

❯ nix run github:brianmcgee/sbomnix/fbe307d15908d7e13fbd827c33d6155c6aa469c3#repology_cli -- --pkg_search 'firef' --repository 'nix_unstable'

INFO     GET: https://repology.org/projects/?search=firef&inrepo=nix_unstable
INFO     Repology package info, packages:20

| repo         | package                                                | version               | status    |  potentially_vulnerable  | newest_upstream_release   | repo_version_classify   |
|--------------+--------------------------------------------------------+-----------------------+-----------+--------------------------+---------------------------+-------------------------|
| nix_unstable | emacs:exwm-firefox-core                                | 20190812.2110         | newest    |            0             | 20190812.2110             |                         |
| nix_unstable | emacs:exwm-firefox-evil                                | 20231026.309          | newest    |            0             |                           |                         |
| nix_unstable | emacs:firefox-javascript-repl                          | 0.9.5                 | newest    |            0             | 0.9.5                     |                         |
| nix_unstable | emacs:helm-firefox                                     | 20220420.1346         | untrusted |            0             | 1.3                       |                         |
| nix_unstable | faust2firefox                                          | 2.59.6                | unique    |            0             |                           |                         |
| nix_unstable | firefly-desktop                                        | 2.1.8                 | newest    |            0             | 2.1.8                     |                         |
| nix_unstable | firefox                                                | 115-unwrapped-115.4.0 | legacy    |            1             | 119.0.1                   |                         |
| nix_unstable | firefox                                                | 115.4.0               | legacy    |            1             | 119.0.1                   |                         |
| nix_unstable | firefox                                                | 118.0b9               | outdated  |            1             | 119.0.1                   | repo_pkg_needs_update   |
| nix_unstable | firefox                                                | 119.0                 | outdated  |            0             | 119.0.1                   | repo_pkg_needs_update   |
| nix_unstable | firefox                                                | 119.0b6               | outdated  |            1             | 119.0.1                   | repo_pkg_needs_update   |
| nix_unstable | firefox                                                | 119.0b9               | legacy    |            1             | 119.0.1                   |                         |
| nix_unstable | firefox                                                | 119.0b9               | outdated  |            1             | 119.0.1                   | repo_pkg_needs_update   |
| nix_unstable | firefox-decrypt                                        | 1.1.0                 | newest    |            0             |                           |                         |
| nix_unstable | gnome:firefox-pip-always-on-top                        | 4                     | unique    |            0             |                           |                         |
| nix_unstable | haskell:firefly                                        | 0.2.1.0               | newest    |            0             | 0.2.1.0                   |                         |
| nix_unstable | haskell:firefly-example                                | 0.1.0.0               | newest    |            0             | 0.1.0.0                   |                         |
| nix_unstable | himitsu-firefox                                        | 0.3                   | outdated  |            0             | 0.4                       | repo_pkg_needs_update   |
| nix_unstable | python:fireflyalgorithm                                | 0.3.4                 | outdated  |            0             | 0.4.1                     | repo_pkg_needs_update   |
| nix_unstable | vscode-extension-firefox-devtools-vscode-firefox-debug | 2.9.10                | unique    |            0             |                           |                         |

For more details, see: https://repology.org/projects/?search=firef&inrepo=nix_unstable

INFO     Wrote: repology_report.csv

repology_cve

❯ nix run github:brianmcgee/sbomnix/fbe307d15908d7e13fbd827c33d6155c6aa469c3#repology_cve -- openssl 3.1.0                                   

INFO     GET: https://repology.org/project/openssl/cves?version=3.1.0
INFO     Repology affected CVE(s)

| package   | version   | cve           |
|-----------+-----------+---------------|
| openssl   | 3.1.0     | CVE-2023-0464 |
| openssl   | 3.1.0     | CVE-2023-0465 |
| openssl   | 3.1.0     | CVE-2023-0466 |
| openssl   | 3.1.0     | CVE-2023-1255 |
| openssl   | 3.1.0     | CVE-2023-2650 |
| openssl   | 3.1.0     | CVE-2023-2975 |
| openssl   | 3.1.0     | CVE-2023-3446 |
| openssl   | 3.1.0     | CVE-2023-3817 |
| openssl   | 3.1.0     | CVE-2023-4807 |
| openssl   | 3.1.0     | CVE-2023-5363 |


INFO     Wrote: repology_cves.csv