I am very excited to watch this new space unfold. Its a huge hole in platform engineering. We need a lot of config but there often arent robust enough config languages so people end up resorting to using Turing complete languages and thats just lighting a fuse for your future self.
Im really excited about cuelang what is everyone else keeping an eye on?
From what I observe we haven’t found a good model for configuration languages yet, in the space between pure data on the one end and Turing-complete languages on the other. In particular if the languages are also aimed at non-developers, like traditional, mostly key-value configuration files are (INI files, many Unix configuration file formats).
As is often noted, the goal of avoiding Turing-completeness is a red herring, because restricting a language to (say) primitive-recursive functions still easily allows programs that don’t halt in any remotely practical time, and effectively still allows to emulate Turing machines up to some large if finite upper bound of execution steps.
We want some expressiveness and some ability for functional abstraction in our configuration languages (variable substitutions, simple macros, and the like), but we also don’t want them to become full-blown programming languages allowing arbitrarily complex programs. However, it’s not clear that there really is a sensible and principled middle ground.
Does total functional programming not do it for you as a middle ground? In particular look at Dhall and its theoretical properties, but there are more possibilities where that came from. There are a lot of typed lambda calculi with various normalizing properties, which I believe can be read as nice analyzability, halting but still somewhat expressive.
It's a statically typed data language that takes inspiration from many things, including Python/Starlark.
Unlike Jsonnet etc., it encourages separating pure data from transformation. You can define a type-safe schema for your data, and then write code that "renders" this into the final schema.
It comes with schemas for things like Kubernetes, and it's been adopted by the CNCF. I would be really happy if this becomes the de facto way to write Kubernetes manifests.
Dhall[1] also usually gets mentioned in the same sentence as CUE.
As for Nickel, I have mixed feelings about it of the same kind as about ES6 classes and perhaps modules—we’re taking a simple language (an extremely simple one, in the case of Nix) and baking into it some stuff that the original had as a completely satisfactory library, in the name of discoverability resp. uniformity. Seems a bit sad and unnecessary to be honest. (The problem of Nixpkgs and NixOS, IMO, is not Nix the language, which is fine for the most part, but the fact that they’re a single giant monorepo with comparatively very little documentation even for the parts that basically serve as the Nix standard library.) Looks like my design sensibilities just align much better with PhD-student Eelco than with Tweag-engineer Eelco.
I agree. If only they wrote down the typechecking/evaluation semantics, even if it’s in the traditional form of a page’s worth of cryptic deduction rules.
I think [1] is a good summary of the types and [2] to be a good explanation of their philosophy, but I had some experience with lattice theory and typed feature structure grammars which helped a lot. There are a couple of papers cited in [2] that would help a lot in providing background.
Not quite a one-pager! Maybe it would make a good blog post...
Is there a particular feature in Nickel you object to? When I think of Nickel I imagine the merging of contracts (and not much else really - that seems like its big flagship feature to me), which seems to me like a pretty minimal necessary feature to model Nixpkgs idioms; is that one of the features you don't like?
I'm excited too. I took a half-hearted stab at the problem when I was 16 or something and seeing a giant computed-JSON-shaped hole in the landscape, so it's been nice to see my intuition confirmed by the arrival of JSonnet, Cue, Dhall, Nickel, etc, even if I never actually played a role in it (so far).
I think the reason the hole persists is due to the lack of production-ready client libraries for these configuration languages (e.g. Nickel) in Turing-complete languages and popular tools (e.g. Go, Kubectl, IDEs like VSCode and Jetbrains). Everything needs to be wired together with shell scripts and task runners. IDEs that offer autocompletion, smart functionality, and other first-class support for Kubernetes, Helm, Terraform only at best offer support for generic CUE or Dhall or Nickel via their LSPs.
I think if you could kubectl apply -f production.cue, momentum would start to shift.
Every time I see Nickel, I get teleported back to my days unraveling thousands of lines of Nix code in a company that used it to configure everything. I know they are trying to improve the situation, but I really don't see how this doesn't encourage the same behavior of over-engineering configuration.
Honestly i agree. It feels wrong to me that i can't easily ask the system "What is the value of the object here?" because it's a lazily evaluated almost infinitely deep stack.
Still, that's what i want. Give me types (like Nickel) to validate APIs, but also give me easy inspection of values without manually jumping through pointers (function calls) like it's 1985 or something.
Ps, please for the love of god give me the ability to use Nickel instead of Nix. It's been such a long wait.
First I've heard of Nickel - a more user friendly version of the Nix language would be amazing but I skimmed their syntax and in seconds found some really bizarre choices.
Multiline strings are m%" ... "% - why? having to match reversed difficult to remember magic glyph sequences is such an arse. Like C++'s raw strings. Awful. Python clearly got this right. Or you could just copy Rust and not even have special syntax for multiline strings. Why do you need it?
String interpolation is "... %{foo} ..." - why? Do they take delight in avoiding what other people chose? What was wrong with "... ${foo} ..."? You just wanted to make me have to remember 3 different string interpolation syntaxes?
They still inexplicably use `let ... in` rather than the much more beginner friendly and just plain nicer `let ...;`
I've only scratched the surface of course but the surface wasn't encouraging. Let me see if you still declare functions in a weird way... checks Well, it's a lot better but they still have inexplicably avoided separating arguments with commas which is obviously going to bite them in the arse when they want to add type annotations.
Still, I've been following Dhall, Jsonnet, Cue and all of them seem to have big issues too so I won't count Nickel out.
> JSON data model: records (objects), arrays, booleans, numbers and strings
Is this the JSON data model? There are places I’ve seen this fall apart: very large and very small numbers, nulls, empty objects / arrays. How confident are you that every participant in your data interchange handles all these cases the same way? I’m suspicious of libraries that assume only they will handle your JSON.
Edit: just remembered another one. I had to deal with the fallout from a JSON serializer that dropped leading zeros from floating point numbers smaller than one, to save space.
How could you name a language after an element and then not use its chemical symbol as the file extension?? (Yes I know NCL means "Nickel configuration language" and can be pronounced like Nickel, but come on)
I am very excited to watch this new space unfold. Its a huge hole in platform engineering. We need a lot of config but there often arent robust enough config languages so people end up resorting to using Turing complete languages and thats just lighting a fuse for your future self.
Im really excited about cuelang what is everyone else keeping an eye on?
From what I observe we haven’t found a good model for configuration languages yet, in the space between pure data on the one end and Turing-complete languages on the other. In particular if the languages are also aimed at non-developers, like traditional, mostly key-value configuration files are (INI files, many Unix configuration file formats).
As is often noted, the goal of avoiding Turing-completeness is a red herring, because restricting a language to (say) primitive-recursive functions still easily allows programs that don’t halt in any remotely practical time, and effectively still allows to emulate Turing machines up to some large if finite upper bound of execution steps.
We want some expressiveness and some ability for functional abstraction in our configuration languages (variable substitutions, simple macros, and the like), but we also don’t want them to become full-blown programming languages allowing arbitrarily complex programs. However, it’s not clear that there really is a sensible and principled middle ground.
Does total functional programming not do it for you as a middle ground? In particular look at Dhall and its theoretical properties, but there are more possibilities where that came from. There are a lot of typed lambda calculi with various normalizing properties, which I believe can be read as nice analyzability, halting but still somewhat expressive.
Total functional programming corresponds to what I wrote with regard to primitive-recursive functions, so no.
I think KCL [1] looks really promising.
It's a statically typed data language that takes inspiration from many things, including Python/Starlark.
Unlike Jsonnet etc., it encourages separating pure data from transformation. You can define a type-safe schema for your data, and then write code that "renders" this into the final schema.
It comes with schemas for things like Kubernetes, and it's been adopted by the CNCF. I would be really happy if this becomes the de facto way to write Kubernetes manifests.
[1] https://www.kcl-lang.io/
Dhall[1] also usually gets mentioned in the same sentence as CUE.
As for Nickel, I have mixed feelings about it of the same kind as about ES6 classes and perhaps modules—we’re taking a simple language (an extremely simple one, in the case of Nix) and baking into it some stuff that the original had as a completely satisfactory library, in the name of discoverability resp. uniformity. Seems a bit sad and unnecessary to be honest. (The problem of Nixpkgs and NixOS, IMO, is not Nix the language, which is fine for the most part, but the fact that they’re a single giant monorepo with comparatively very little documentation even for the parts that basically serve as the Nix standard library.) Looks like my design sensibilities just align much better with PhD-student Eelco than with Tweag-engineer Eelco.
[1] https://dhall-lang.org/
Dhall does seem to have more integrations but I really like the amount of thought that has gone into Cue's types and operational semantics.
I agree. If only they wrote down the typechecking/evaluation semantics, even if it’s in the traditional form of a page’s worth of cryptic deduction rules.
I think [1] is a good summary of the types and [2] to be a good explanation of their philosophy, but I had some experience with lattice theory and typed feature structure grammars which helped a lot. There are a couple of papers cited in [2] that would help a lot in providing background.
Not quite a one-pager! Maybe it would make a good blog post...
[1]: https://cuelang.org/docs/concept/the-logic-of-cue/
[2]: https://github.com/cue-lang/cue/blob/master/doc/ref/impl.md
Is there a particular feature in Nickel you object to? When I think of Nickel I imagine the merging of contracts (and not much else really - that seems like its big flagship feature to me), which seems to me like a pretty minimal necessary feature to model Nixpkgs idioms; is that one of the features you don't like?
I'm excited too. I took a half-hearted stab at the problem when I was 16 or something and seeing a giant computed-JSON-shaped hole in the landscape, so it's been nice to see my intuition confirmed by the arrival of JSonnet, Cue, Dhall, Nickel, etc, even if I never actually played a role in it (so far).
I think the reason the hole persists is due to the lack of production-ready client libraries for these configuration languages (e.g. Nickel) in Turing-complete languages and popular tools (e.g. Go, Kubectl, IDEs like VSCode and Jetbrains). Everything needs to be wired together with shell scripts and task runners. IDEs that offer autocompletion, smart functionality, and other first-class support for Kubernetes, Helm, Terraform only at best offer support for generic CUE or Dhall or Nickel via their LSPs.
I think if you could kubectl apply -f production.cue, momentum would start to shift.
Every time I see Nickel, I get teleported back to my days unraveling thousands of lines of Nix code in a company that used it to configure everything. I know they are trying to improve the situation, but I really don't see how this doesn't encourage the same behavior of over-engineering configuration.
Honestly i agree. It feels wrong to me that i can't easily ask the system "What is the value of the object here?" because it's a lazily evaluated almost infinitely deep stack.
Still, that's what i want. Give me types (like Nickel) to validate APIs, but also give me easy inspection of values without manually jumping through pointers (function calls) like it's 1985 or something.
Ps, please for the love of god give me the ability to use Nickel instead of Nix. It's been such a long wait.
First I've heard of Nickel - a more user friendly version of the Nix language would be amazing but I skimmed their syntax and in seconds found some really bizarre choices.
Multiline strings are m%" ... "% - why? having to match reversed difficult to remember magic glyph sequences is such an arse. Like C++'s raw strings. Awful. Python clearly got this right. Or you could just copy Rust and not even have special syntax for multiline strings. Why do you need it?
String interpolation is "... %{foo} ..." - why? Do they take delight in avoiding what other people chose? What was wrong with "... ${foo} ..."? You just wanted to make me have to remember 3 different string interpolation syntaxes?
They still inexplicably use `let ... in` rather than the much more beginner friendly and just plain nicer `let ...;`
I've only scratched the surface of course but the surface wasn't encouraging. Let me see if you still declare functions in a weird way... checks Well, it's a lot better but they still have inexplicably avoided separating arguments with commas which is obviously going to bite them in the arse when they want to add type annotations.
Still, I've been following Dhall, Jsonnet, Cue and all of them seem to have big issues too so I won't count Nickel out.
For me i'd be "okay" with Nix, i just want types and validation natively. The big blob of unknown types that i get with Nix is just too much.
> JSON data model: records (objects), arrays, booleans, numbers and strings
Is this the JSON data model? There are places I’ve seen this fall apart: very large and very small numbers, nulls, empty objects / arrays. How confident are you that every participant in your data interchange handles all these cases the same way? I’m suspicious of libraries that assume only they will handle your JSON.
Edit: just remembered another one. I had to deal with the fallout from a JSON serializer that dropped leading zeros from floating point numbers smaller than one, to save space.
How could you name a language after an element and then not use its chemical symbol as the file extension?? (Yes I know NCL means "Nickel configuration language" and can be pronounced like Nickel, but come on)
There’s an issue for this :)
https://github.com/tweag/nickel/issues/357
You're right, "k" is a slop tier letter.
Seems to be in the same tradition as GNU being "GNU is Not Unix".