Work better with Haskell.nix?

[shajra]

Hi!

I tried to integrate two things new to me recently, Cabal-doctest and Haskell.nix. As you might expect, Doctests are already pretty hacky when it comes to Cabal integration. It only gets trickier when you throw in Nix. Actually, the default way of building Haskell projects with Nix (Nixpkgs) didn't give me a problem with Doctests, even with Cabal-doctest. But Haskell.nix, specifically, causes me issues.

I have a couple ideas on how we might improve the state of things.

Background on Nix and Haskell.nix

I don't know much you already know about Nix and Haskell.nix, but I figured it would be good to spell out some relevant context, even if for others finding this ticket.

In Nix, everything is cached at the level of a Nix derivation. A derivation sets up a very hygenic/controlled environment, with all of the files/folders/permissions/envvars explicitly specified. Then the crank is turned, and specific outputs are cached as the output of the derivation. These can then be used to set up the environment for subsequent derivations. This is important, because it means that transient build artifacts from a previous derivation are not just lying around for the next derivation. They have to be explicitly copied over.

What Haskell.nix does that breaks the normal Cabal-doctest workflow is have a separate derivation for each component in a Cabal project. So the libraries, executables, and test suites each get their own derivation. Furthermore, there is even a separate derivation to run a test that is different from the one that builds the test code. This is kind of nice because it gives developers more control. A developer can make sure all these tests build without actually running the tests. But then, because in Nix everything is cached at the derivation-level, we can run the tests later and use the same artifacts we cached previously. Stack/Cabal users take this for granted, but in Nix we have to make Nix expressions to regain this flexibility, which is one of the reasons the Haskell.nix project exists as a alternative to the default way of building things with Nix (Nixpkgs).

What's going wrong with Haskell.nix + Cabal-doctest

Components are seen only one at a time

The code for Cabal-doctest assumes that most of the flags we need are derived from the library and executable components. But in a Haskell.nix derivation for a test component, there are no library or executable components. They have already been built and made available in the package database provided to the derivation's Cabal run. However, the source code of these components can be made available (obviously needed for the authored doctests themselves).

Right now, Cabal-doctest when running in a Haskell.nix derivation won't output any flags, packages, or module sources. Because these flags are output only for libraries and executables, which are not available when compiling the test component. What would be useful is to get the at least the packages flags (possibly the rest too) from the test component. The code I need to load up in my doctests have already been shuttled into a package database. And I can access this package database for normal dependencies from my doctest. This does mean to list dependencies I need for doctests on my test component, not just the dependencies needed for the doctest fixture. But that seems like an okay compromise to me.

Some references to missing artifacts

All the references to build/ and dist/, but also Build_doctests aren't useful for a Haskell.nix derivation. All of this stuff is in the package database.

Possible solutions (not mutually exclusive)

1. At least output flags for the test component

I made a patch of cabal-doctest that does this (not cleaned up for a PR yet, and not sure if you want to go this direction anyway), and it makes this following Build_doctests.hs file:

module Build_doctests where

import Prelude

data Name = NameLib (Maybe String) | NameExe String deriving (Eq, Show)
data Component = Component Name [String] [String] [String] deriving (Eq, Show)

pkgs_test_doctests :: [String]
pkgs_test_doctests = ["-package-id=base-4.12.0.0","-package-id=base-compat-0.11.0-BTru338SRTmL3dK24Mui3j","-package-id=doctest-0.16.2-6z3pUedJOHdCkF4jrPAXAU","-package=exceptions-checked-0.0.1"]

flags_test_doctests :: [String]
flags_test_doctests = ["-i","-i/build/exceptions-checked/dist/build/doctests/autogen","-i/build/exceptions-checked/dist/build","-i/build/exceptions-checked/test","-i/build/exceptions-checked/test","-package-env=-","-hide-all-packages","-clear-package-db","-package-db=/nix/store/ryxw8lj748jbmvriz3w8k4vw7brnann4-exceptions-checked-0.0.1-test-doctests-config/package.conf.d","-package-db=dist/package.conf.inplace","-optP-include","-optPdist/build/doctests/autogen/cabal_macros.h"]

module_sources_test_doctests :: [String]
module_sources_test_doctests = ["Build_doctests", "Doctest.Checked.Catch","Doctest.Checked.Throw"]

-- [NameLib Nothing]
components :: [Component]
components = []

2. Eliding problematic missing references

Although there's a few references to artifacts that are missing:

• -i/build/exceptions-checked/dist/build/doctests/autogen
• -i/build/exceptions-checked/dist/build
• -i/build/exceptions-checked/test
• -i/build/exceptions-checked/test
• -package-db=dist/package.conf.inplace
• -optPdist/build/doctests/autogen/cabal_macros.h
• Build_doctests

The only ones that really cause my Haskell.nix build trouble are the two bolded above. The Build_doctests file should be elided from module_sources_test_* always, I think. If we did nothing with the missing dist/package.conf.inplace I can always do a ghc-pkg init dist/package.conf.inplace in my Haskell.nix configuration. It's a small one-liner.

We could possibly make an option to elide the "inplace" package flag. But we'd need this flag for non-Haskell.nix builds. I'd much more prefer to have a static Cabal configuration and source code that works with all builds systems. Small tweaks seem better on the Nix side than that Cabal side.

3. Nothing

I list this for completeness. I kind of understand the argument that Cabal-doctest is complicated enough as it stands. Is Haskell.nix breaking an invariant expected of Cabal?

I'm probably going to point some Haskell.nix developers at this ticket. Maybe there's something about the Haskell.nix implementation that can be changed to allows all the components to show up in the LocalBuildInfo when Setup.hs runs.

[phadej]

I have to read this properly, but I have a quick comment:

cabal-install doesn’t use per-component build for build-type: Custom packages for related reasons: the custom Setup.hs can do virtually anything (particularly the Cabal hooks interface is not refined to support per component builds)

cabal-doctest is a hack, and I really really don’t want to make it more hacky. So option 3: Nothing is very tempting.

[shajra]

Okay, let me put that PR up anyway, and you can decide if it makes the code noticeably worse. I'll go out of my way to not break any prior functionality. It will just generate new things in Build_doctests.hs that it didn't before.