thomasjm Github contribution chart
thomasjm Github Stats
thomasjm Most Used Languages

Activity

04 Oct 2022

Issue Comment

Thomasjm

Introduce mkBinaryCache function

Description of changes

This introduces a new function mkBinaryCache which can be used to construct a flat-file binary cache (i.e. one you can use with the file:// prefix) from a derivation. It's inspired by the closureInfo function and uses the same techniques to construct .narinfo files.

My use-case that I'd like to test Nix builds in a containerized CI environment, without downloading a bunch of stuff on every pipeline run. Thus, I want to make a binary cache of some kind available inside the container. The most efficient way to do this is with a bind-mount.

Initially I tried just mounting /nix/store to a path on the container, but I soon ran into a problem: you can either mount read/write, in which case the container might potentially mess with the integrity of the store, especially when run as root. Or, you can mount read-only, in which case you run into https://github.com/NixOS/nix/issues/6835. Some discussion on the Nix hackers Matrix channel showed that mounting the store read-only is unlikely to be solvable because of how the Sqlite DB works.

So, making a flat-file binary cache and then bind-mounting it seemed like the best approach. I initially tried just running nix copy inside a derivation but that didn't work well, so I came up with this function.

Things done
  • Built on platform(s)
    • [X] x86_64-linux
    • [ ] aarch64-linux
    • [ ] x86_64-darwin
    • [ ] aarch64-darwin
  • [ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • [ ] Tested, as applicable:
  • [N/A] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • [N/A] Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • [ ] (Package updates) Added a release notes entry if the change is major or breaking
    • [ ] (Module updates) Added a release notes entry if the change is significant
    • [ ] (Module addition) Added a release notes entry if adding a new NixOS module
    • [ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • [X] Fits CONTRIBUTING.md.

Forked On 04 Oct 2022 at 11:06:25

Thomasjm

Indeed haha our beautiful pure functional abstraction is built on a thing that loves to randomly chmod stuff.

Any chance you'd be willing to review this?

Commented On 04 Oct 2022 at 11:06:25
Issue Comment

Thomasjm

Introduce mkBinaryCache function

Description of changes

This introduces a new function mkBinaryCache which can be used to construct a flat-file binary cache (i.e. one you can use with the file:// prefix) from a derivation. It's inspired by the closureInfo function and uses the same techniques to construct .narinfo files.

My use-case that I'd like to test Nix builds in a containerized CI environment, without downloading a bunch of stuff on every pipeline run. Thus, I want to make a binary cache of some kind available inside the container. The most efficient way to do this is with a bind-mount.

Initially I tried just mounting /nix/store to a path on the container, but I soon ran into a problem: you can either mount read/write, in which case the container might potentially mess with the integrity of the store, especially when run as root. Or, you can mount read-only, in which case you run into https://github.com/NixOS/nix/issues/6835. Some discussion on the Nix hackers Matrix channel showed that https://github.com/NixOS/nix/issues/6835 is unlikely to be solvable because of how the Sqlite DB works.

So, making a flat-file binary cache and then bind-mounting it seemed like the best approach. I initially tried just running nix copy inside a derivation but that didn't work well, so I came up with this function.

Things done
  • Built on platform(s)
    • [X] x86_64-linux
    • [ ] aarch64-linux
    • [ ] x86_64-darwin
    • [ ] aarch64-darwin
  • [ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • [ ] Tested, as applicable:
  • [N/A] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • [N/A] Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • [ ] (Package updates) Added a release notes entry if the change is major or breaking
    • [ ] (Module updates) Added a release notes entry if the change is significant
    • [ ] (Module addition) Added a release notes entry if adding a new NixOS module
    • [ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • [X] Fits CONTRIBUTING.md.

Forked On 04 Oct 2022 at 10:16:56

Thomasjm

disable locking, which means promising that nothing else has read-write access to the same database

You're right that SQLite can operate in that mode, but it's not really viable when you're talking about mounting the main store of the CI machine which is busily building other things.

FWIW, I did come up with one (fragile and intricate) solution where you manually create a writable overlay filesystem on /nix/var/nix/db and mount that. This allows Nix to work with the DB normally, but probably still leads to undefined behavior if the host database changes unexpectedly.

There is probably also the possibility for weirdness because IIRC Nix sometimes creates store paths while performing apparently side-effect-free operations, which would not work on the read-only mount--I'm not sure if this is a possibility when the store is being used as a substituter.

Lastly I forgot to mention, but there were also other problems with read-only mounts, like the one I described in the linked issue. I spent a while trying to hack Nix to operate in a read-only mode where it wouldn't try to do things like chmod directories, but it was kind of a mess.

Commented On 04 Oct 2022 at 10:16:56
Create Branch
Thomasjm In codedownio/nixpkgs Create Branchrelease-22.05-codedown

Thomasjm

Nix Packages collection

On 04 Oct 2022 at 08:03:36
Pull Request

Thomasjm

Introduce mkBinaryCache function

Created On 04 Oct 2022 at 04:17:17
Create Branch
Thomasjm In codedownio/nixpkgs Create Branchbinary-cache

Thomasjm

Nix Packages collection

On 04 Oct 2022 at 04:07:14
Issue Comment

Thomasjm

System.FSNotify does not export type WatchConfig

I understand the API change from 0.4.0.0 to cease to export the data constructors for the data type WatchConfig.

However, is it intentional not to export the data type too, despite exporting defaultConfig :: WatchConfig?

Stack depends on fsnotify and has a function:

fileWatchConf
  :: (HasLogFunc env, HasTerm env)
  => WatchConfig
  -> ((Set (Path Abs File) -> IO ()) -> RIO env ())
  -> RIO env () 

However, if type WatchConfig is not available, this function can't compile.

Forked On 04 Oct 2022 at 01:44:03

Thomasjm

It's because I like to use the TUI interface of Sandwich to run tests in development, which works better as an executable (https://github.com/codedownio/sandwich/issues/35).

We could backtrack on this if it's causing problems though.

Commented On 04 Oct 2022 at 01:44:03
Create Branch
Thomasjm In codedownio/nixpkgs Create Branchmultiple-github-runners-oct3

Thomasjm

Nix Packages collection

On 03 Oct 2022 at 09:26:05
Issue Comment

Thomasjm

Can't build with Nix 2.4 when Stack extra-deps present

I wanted to upgrade to the newly released Nix 2.4, but started running into build problems with Haskell.nix. It seems to only happen when your stack.yaml contains extra-deps. It causes failures like the following:

> nix-build -A stack-nix.passthru.updateMaterialized --no-out-link

trace: Checking materialization in /home/tom/tools/haskell-nix24-repro/materialized
building '/nix/store/1l7a0yxrg356q1n4i7mj7fxh0smgqa1z-haskell-project-stack-to-nix-pkgs.drv'...
building '/nix/store/6phz0dyzndmln3wav97x75irz3d7an5s-haskell-project-stack-to-nix-pkgs.drv'...
error: path '/nix/store/012r76kyra8z09qq9d6f40s7qc2m64fa-haskell-project-stack-to-nix-pkgs' is not valid 

Random theorizing: Nix 2.4 handles hashes differently and so the process to construct this path in Haskell.nix no longer works?

I made a repro here: https://github.com/thomasjm/haskell-nix24-repro. If you try building this project with Nix 2.3.16 it will work, but with Nix 2.4 it will fail.

Forked On 03 Oct 2022 at 05:18:20

Thomasjm

Awesome, thanks!

Commented On 03 Oct 2022 at 05:18:20
Issue Comment

Thomasjm

[TextField] Bad behavior when user edits box before hydration with SSR

Duplicates

  • [X] I have searched the existing issues

Latest version

  • [X] I have tested the latest version

Current behavior 😯

I'm using MUI's TextField to make a simple SSR login page with a username and password.

The problem is when the user is working over a slow network connection, so it takes some time for the Javascript to download and hydrate the page. As a result, they can edit the text box before hydration occurs.

In the picture below, the user has typed "bob" for the username prior to hydration. The helper text remains in place and it looks bad:

Screenshot from 2022-09-01 19-47-09

There is also a second problem: once hydration does complete, the TextField reverts to blank, losing the user's edits, which is not good UX.

Expected behavior 🤔

The TextField should behave in a reasonable way prior to hydration. Reasonable could mean either a) don't allow edits, or b) handle the edits gracefully, with good styling and without losing them upon hydration.

Steps to reproduce 🕹

I am doing my SSR setup with Razzle.js. I made a fork here to showcase this.

  1. Clone and build the repro:
git clone git@github.com:thomasjm/razzle.git
cd razzle/examples/with-material-ui
git checkout textfield-ssr
npm install
npm run build # Press Y
cd build
node server.js 
  1. Open localhost:3000 in Chrome. Open the Chrome DevTools, go to the Network tab, and set the speed to "Slow 3G".
  2. Refresh the page. Before the JS finishes downloading, try typing in the text box.

Context 🔦

SSR is a popular technique for achieving fast page loads, and TextField should work gracefully with it.

Your environment 🌎

 System:
    OS: Linux 5.15 Ubuntu 22.04.1 LTS 22.04.1 LTS (Jammy Jellyfish)
  Binaries:
    Node: 16.15.0 - ~/.nvm/versions/node/v16.15.0/bin/node
    Yarn: 1.22.17 - ~/codedown/frontend/node_modules/.bin/yarn
    npm: 8.5.5 - ~/.nvm/versions/node/v16.15.0/bin/npm
  Browsers:
    Chrome: 104.0.5112.79
    Firefox: 104.0.1
  npmPackages:
    @emotion/react: ^11.8.2 => 11.8.2
    @emotion/styled: ^11.8.1 => 11.8.1
    @mui/base:  5.0.0-alpha.95
    @mui/core-downloads-tracker:  5.10.3
    @mui/icons-material:  5.0.5
    @mui/material: ^5.10.3 => 5.10.3
    @mui/private-theming:  5.10.3
    @mui/styled-engine:  5.10.3
    @mui/styles:  5.5.3
    @mui/system:  5.10.3
    @mui/types:  7.2.0
    @mui/utils:  5.10.3
    @types/react: ^17.0.43 => 17.0.43
    react: ^17.0.1 => 17.0.2
    react-dom: ^17.0.1 => 17.0.2
    typescript: ^4.8.2 => 4.8.2 

Forked On 03 Oct 2022 at 04:11:27

Thomasjm

@eps1lon just checking in, is the above sufficient to reopen the React issue?

Commented On 03 Oct 2022 at 04:11:27

Thomasjm

Release 0.1.1.2

Pushed On 03 Oct 2022 at 01:55:27
Issue Comment

Thomasjm

System.FSNotify does not export type WatchConfig

I understand the API change from 0.4.0.0 to cease to export the data constructors for the data type WatchConfig.

However, is it intentional not to export the data type too, despite exporting defaultConfig :: WatchConfig?

Stack depends on fsnotify and has a function:

fileWatchConf
  :: (HasLogFunc env, HasTerm env)
  => WatchConfig
  -> ((Set (Path Abs File) -> IO ()) -> RIO env ())
  -> RIO env () 

However, if type WatchConfig is not available, this function can't compile.

Forked On 03 Oct 2022 at 12:41:57

Thomasjm

e2a2ed391a94f70a21b1988dfee9a219728ce8af

Commented On 03 Oct 2022 at 12:41:57

Thomasjm

Pick up cabal file update

Pushed On 03 Oct 2022 at 12:41:47

Thomasjm

Consistent exports formatting

Pushed On 03 Oct 2022 at 12:41:47

Thomasjm

Export WatchConfig type (not constructors)

Pushed On 03 Oct 2022 at 12:41:47

Thomasjm

Tweak haddocks

Pushed On 03 Oct 2022 at 12:41:47

Thomasjm

Reformat pragmas

Pushed On 03 Oct 2022 at 12:41:47
Issue Comment

Thomasjm

System.FSNotify does not export type WatchConfig

I understand the API change from 0.4.0.0 to cease to export the data constructors for the data type WatchConfig.

However, is it intentional not to export the data type too, despite exporting defaultConfig :: WatchConfig?

Stack depends on fsnotify and has a function:

fileWatchConf
  :: (HasLogFunc env, HasTerm env)
  => WatchConfig
  -> ((Set (Path Abs File) -> IO ()) -> RIO env ())
  -> RIO env () 

However, if type WatchConfig is not available, this function can't compile.

Forked On 03 Oct 2022 at 12:29:45

Thomasjm

Stack currently has an fsnotify constraint of < 0.4, are you trying to update Stack to the new version?

Sure, I'd be fine with exporting WatchConfig.

Commented On 03 Oct 2022 at 12:29:45

Thomasjm

Improve semantics of multiple --filter arguments

Pushed On 02 Oct 2022 at 10:48:16

Thomasjm

Add sandwich constraint on windows

Pushed On 30 Sep 2022 at 06:45:43

Thomasjm

Better message on exit due to signal + handle sigTERM

Pushed On 30 Sep 2022 at 06:43:27

Thomasjm

Try bumping stack cache key

Pushed On 30 Sep 2022 at 06:02:52

Thomasjm

Add LTS labels

Pushed On 30 Sep 2022 at 05:28:14

Thomasjm

Drop lts-15 (GHC 8.8.2) since lts-16 is 8.8.4

Pushed On 30 Sep 2022 at 05:28:14

Thomasjm

Add a liftIO for windows

Pushed On 30 Sep 2022 at 05:28:14

Thomasjm

Fix OS_Win32 flag

Pushed On 30 Sep 2022 at 05:11:45

Thomasjm

More extra-deps for lts 12 and 13

Pushed On 30 Sep 2022 at 08:13:24

Thomasjm

Try fixing LTS 12 and 13

Pushed On 30 Sep 2022 at 07:21:12

Thomasjm

Try forcing bash shell on windows github runner

Pushed On 30 Sep 2022 at 07:21:12

Thomasjm

More stack lts stuff for 12-14

Pushed On 30 Sep 2022 at 12:15:49

Thomasjm

Drop support for lts-9 and lts-11

Pushed On 30 Sep 2022 at 12:15:49

Thomasjm

More extra-deps for lts 14 and 15

Pushed On 30 Sep 2022 at 12:01:41

Thomasjm

Custom stack.yaml for lts 14 and 15

Pushed On 29 Sep 2022 at 11:37:37

Thomasjm

Custom stack.yaml for lts-16 in CI

Pushed On 29 Sep 2022 at 11:16:17
Issue Comment

Thomasjm

Having Trouble Running Demo On Windows

I end up with:

PS C:\sandwich> stack run demo-landing -- --tui

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for brick-0.58.1(+demos):
    unix is a library dependency, but the package provides no library
needed due to demo-landing-0.1.0.0 -> brick-0.58.1

In the dependencies for sandwich-0.1.0.8:
    unix is a library dependency, but the package provides no 

needed due to demo-landing-0.1.0.0 -> sandwich-0.1.0.8

In the dependencies for terminfo-0.4.1.4:
    base-4.14.1.0 from stack configuration does not match >=4.9 && <4.14  (latest matching version        
                  is 4.13.0.0)
needed due to demo-landing-0.1.0.0 -> terminfo-0.4.1.4

In the dependencies for vty-5.32:
    unix is a library dependency, but the package provides no library
needed due to demo-landing-0.1.0.0 -> vty-5.32

Some different approaches to resolving this:

  * Build requires unattainable version of base. Since base is a part of GHC, you most likely need to use 
    a different GHC version with the matching base.

Plan construction failed. 

full log

Forked On 29 Sep 2022 at 08:36:56

Thomasjm

A little late but Windows is supported now in sandwich-0.1.1.1

Commented On 29 Sep 2022 at 08:36:56

Thomasjm

Improve fixupName to handle disallowed chars on windows

Pushed On 29 Sep 2022 at 08:17:47

Thomasjm

Ready to upload 0.1.1.1

Pushed On 29 Sep 2022 at 08:17:47

Thomasjm

Fix up backslash for windows paths

Pushed On 29 Sep 2022 at 07:56:53

Thomasjm

Bump versions of hedgehog, slack, webdriver, sandwich

Pushed On 29 Sep 2022 at 07:01:15