I just dug through a lot of the issues and PRs in Goja, and eventually found that the grafana/k6 team recently forked Goja as Sobek [1], because the Goja dev has not been able to dedicate sufficient time to their PRs - namely ES Modules support [2], which was one of the only modern (ES6+) JS features outstanding [3]
I did see that. I'm going to let the fork simmer for awhile. I didn't require module support for my runtime. Since its meant to be very basic ETL runtime.
Fair enough! Though they're quite clearly a much more serious/focused/resourced team than the goja dev, and they've fully integrated it into the K6 codebase now. I expect it'll have a bright future.
Since you're doing ETL, did you consider something like Conduit, which is written in Golang and also allows for JS processing in the pipeline? Or is that overkill for what youre doing?
While I'm sure Goja is great for just JavaScript, using something like wazero allows your app to support more languages (anything that compiles to WASM) while keeping the benefits of CGO.
I work on the D2 project ([0]) and we switched from other Javascript runners (v8go) to Goja. Using a JS runtime with a dependency on cgo means your Go program loses the (huge) benefit of cross-compiling to different architectures, since at build time it gets linked to the current system's libc.
If you're interested in some production code with Goja, this is our code for calling RoughJS ([1]) from Go in order to produce the hand-drawn diagram look: [2]
In short: Goja is an ES5 interpreter written in pure Go and seamlessly integrating with it. You pass a struct, the JS side receives an object, you update it, and the Go side seamlessly gets an updated struct. No cgo overhead.
Calling a C function with Cgo has a certain overhead if you compare it to calling a Go function. But cgo overheads are completely negligible as soon as the function does any kind of real work. (A bigger reason to avoid cgo would be if you want easier cross-compilation.)
cgo execution overhead is a nothingburger, but the programmer overhead to convert between Go types and C->Javascript types, and vice versa, as required by the use of a third-party engine (v8, etc.) is still significant as compared to what Goja offers. That is what the parent is talking about.
It's definitely cool, and and accomplishment. I think it's a bit strange to compare it to "no cgo overhead" though
I say that because it's an interpreter rather than a jit, and doesn't have nearly as many person hours put into it as v8.
I can see the argument that if you just need to evaluate a quick expression or maybe some user supplied script or something, it could be very handy, but I have a hard time believing that the cgo overhead is a factor when using cgo to invoke v8 compared to the parse/jit overhead that v8 will do.
This is all armchair speculation of course. I use go regularly, but have no taste for javascript, so I have no dogs in that race. Definitely cool that there's an interpreter focused on correctness though.
PocketBase is a ton of fun. I’ve been using it for building my business’s store, and integrating with stripe was such a breeze. I’m using it sort of like a headless CMS/store. I considered other things like Payload CMS (which is awesome too) but the simplicity and productivity of pocket was such a draw. I don’t regret it.
It's certainly a good idea to know golang while using something like pocketbase.
However, one of the strengths of pocketbase, I think, is that the creator chose to add bindings for a very well known scripting language (js) to allow projects to be rapidly developed. It feels like django rethought.
Using go for the base of your project and then gluing things together via scripting seems like a good idea. It's quite a big feature of the framework.
performance seems pretty solid too, removing one of my prime motivators for switching to Go for extension work. PocketBase just seems to have made a lot of great decisions based on the conceptual integrity of the project goals.
The examples in the article seem awfully contrived. Can someone please explain what a real-world use case for something like this would be? Why not just do it all in Go?
Or is the point of it to be able to use existing JS scripts within a Go application?
I just found this excellent comment in an relevant Reddit post[1], which clarifies things quite nicely for me.
---
Let’s set up the scenario. You have a program that a user can manage via a config file. That tune knobs and dials, and everything works. They then ask you one day, “I’d like to add some custom logic for they have.”
Do you add another option to the configuration file to support their one use case? Or provide a mechanism for them actually to write some code?
If it’s the latter, do you want them to have to recompile your program to be able to run that custom code? Or do you have a runtime in your application for that custom code?
The latter is what a browser does. You can write JavaScript for a website without recompiling the browser just for the website. The user doesn’t have to worry about your event loops, SDKs, APIs, wiring code in rust/C, etc. This feature allows others to add custom logic to your application's runtime.
This is the purpose of embedding JavaScript into an application. It’s not the whole program, but it allows someone to provide code in your program event loop to do something custom. Not every program needs this. Sometimes, a config file of YAML or command line arguments works, too.
Not doing it all, Go allows my API endpoint to accept sandboxed javascript code to enable users to run more complex queries against my data. It is also faster because they don't have to make individual API calls for each data set. Therefore, more data is required to be sent over the wire to them.
I've been using goja in a logical replication tool to make it field-programmable by the end users. No two deployments look alike, so a great deal of flexibility is needed for ETL uses. There's a tendency for configuration languages to become Turing-complete, so we started with a Turing-complete language for configuration.
JS, or in Replicator's case, TypeScript (shout-out to esbuild), is sufficiently well-known that any dev group will have some experience with it. On the whole, I've been very impressed with how straightforward it's been to have user-scripts integrated into the processing pipeline.
Ah, that makes a lot of sense - far easier to expose a JS api for user-extension than Golang (which is probably not even all that possible, except for via wasm, which would still be challenging)
And, I suppose my initial hunch is probably correct - for the dev/application to be able to use existing JS libraries (be it part of your broader application/system or external) within a Go app.
Check out my other comment [1] in this post for a newer, better version/fork of Goja - from the grafana/k6 team.
On top of the previous comments describing how to use JS as an embedded scripting language, I've seen in production (series B startup) an architecture where:
a) non-technical users describe, in plain English, what they would like from the data;
b) an LLM with lots and lots of prompt-engineering massaging translates the English into JS,
c) the JS gets executed in a sandbox,
d) the results get returned to the user
For a real-worl use case, look towards the 'script' processor in Elastic' Beats family of data shippers [1] (example is from Metricbeat but it exists in other Beats too)
They can be configured to mutate the data with some basic DSL syntax, but if you have more advanced needs, you can break out to JavaScript and transform the JSON any way you want.
This is very useful because now any transformation becomes possible. And because this is the user of your product who writes that JavaScript more as configuration than code.
And since there's no build step, it is just part if the configuration loaded at startup.
I am using otto[0] (similar to goja) in the headless cms I maintain [1] in "actions" api [2]. So one can customize actions according to needs by writing JS. The JS is executed inside a transaction. Sort of a mini-lambda kind of api
I didn't use Goja (this was maybe a decade ago) but I used Otto, which is the same concept.
I used it for game scripting. Lots of stuff is just easier to write in a high level language. I was able to use inheritance to compose behaviors for entities, and I could restart entities with new code without having to restart the server.
You know, back when I got started as a Schemer, I remember people complaining about how many implementations of Scheme there were and how difficult it was to port between them. Today in JavaScript land we have Node, Deno, Bun, Goja, Sobek, Nashorn, whatever the browsers have, QML, GNOME, all of them subtly different. But instead of complaining, which people seem inclined to do when they encounter nested parens, people bro down and wrote code necessary to make them all interoperate if necessary -- despite JS having even less of a core language standard than Scheme does.
I guess it's the Lisp Curse or something, I dunno.
I just dug through a lot of the issues and PRs in Goja, and eventually found that the grafana/k6 team recently forked Goja as Sobek [1], because the Goja dev has not been able to dedicate sufficient time to their PRs - namely ES Modules support [2], which was one of the only modern (ES6+) JS features outstanding [3]
So, Sobek seems to be the way forward...
[1] https://github.com/grafana/sobek/
[2] https://github.com/dop251/goja/pull/430
[3] https://github.com/dop251/goja/milestone/1
I did see that. I'm going to let the fork simmer for awhile. I didn't require module support for my runtime. Since its meant to be very basic ETL runtime.
Fair enough! Though they're quite clearly a much more serious/focused/resourced team than the goja dev, and they've fully integrated it into the K6 codebase now. I expect it'll have a bright future.
Since you're doing ETL, did you consider something like Conduit, which is written in Golang and also allows for JS processing in the pipeline? Or is that overkill for what youre doing?
https://conduit.io/docs/processors/builtin/custom.javascript
While I'm sure Goja is great for just JavaScript, using something like wazero allows your app to support more languages (anything that compiles to WASM) while keeping the benefits of CGO.
https://wazero.io/
I've used Wazero myself on C++ -> WASM code but I'm sure you could use Emscripten or something to compile JavaScript to WASM.
I work on the D2 project ([0]) and we switched from other Javascript runners (v8go) to Goja. Using a JS runtime with a dependency on cgo means your Go program loses the (huge) benefit of cross-compiling to different architectures, since at build time it gets linked to the current system's libc.
If you're interested in some production code with Goja, this is our code for calling RoughJS ([1]) from Go in order to produce the hand-drawn diagram look: [2]
[0] https://github.com/terrastruct/d2
[1] https://roughjs.com/
[2] https://github.com/terrastruct/d2/blob/master/d2renderers/d2...
It is quite easy to cross-compile cgo program to most architectures - just use Zig for this:
In short: Goja is an ES5 interpreter written in pure Go and seamlessly integrating with it. You pass a struct, the JS side receives an object, you update it, and the Go side seamlessly gets an updated struct. No cgo overhead.
Calling a C function with Cgo has a certain overhead if you compare it to calling a Go function. But cgo overheads are completely negligible as soon as the function does any kind of real work. (A bigger reason to avoid cgo would be if you want easier cross-compilation.)
cgo execution overhead is a nothingburger, but the programmer overhead to convert between Go types and C->Javascript types, and vice versa, as required by the use of a third-party engine (v8, etc.) is still significant as compared to what Goja offers. That is what the parent is talking about.
It's definitely cool, and and accomplishment. I think it's a bit strange to compare it to "no cgo overhead" though
I say that because it's an interpreter rather than a jit, and doesn't have nearly as many person hours put into it as v8.
I can see the argument that if you just need to evaluate a quick expression or maybe some user supplied script or something, it could be very handy, but I have a hard time believing that the cgo overhead is a factor when using cgo to invoke v8 compared to the parse/jit overhead that v8 will do.
This is all armchair speculation of course. I use go regularly, but have no taste for javascript, so I have no dogs in that race. Definitely cool that there's an interpreter focused on correctness though.
This is the runtime pocketbase uses to allow javascript interop.
Pocketbase is the framework that makes me want to switch to golang. It just makes a lot of sense.
One thing that concerned me, though. How do I debug goja? It doesn't seem like I can set breakpoints as usual.
https://pocketbase.io/
PocketBase is a ton of fun. I’ve been using it for building my business’s store, and integrating with stripe was such a breeze. I’m using it sort of like a headless CMS/store. I considered other things like Payload CMS (which is awesome too) but the simplicity and productivity of pocket was such a draw. I don’t regret it.
Actually I decided to learn golang for extending using Golang for better DX (type hints and compiler errors)
It's certainly a good idea to know golang while using something like pocketbase.
However, one of the strengths of pocketbase, I think, is that the creator chose to add bindings for a very well known scripting language (js) to allow projects to be rapidly developed. It feels like django rethought.
Using go for the base of your project and then gluing things together via scripting seems like a good idea. It's quite a big feature of the framework.
performance seems pretty solid too, removing one of my prime motivators for switching to Go for extension work. PocketBase just seems to have made a lot of great decisions based on the conceptual integrity of the project goals.
I've used Pocketbase for some other projects. A quick small web app, but it was so easy.
The examples in the article seem awfully contrived. Can someone please explain what a real-world use case for something like this would be? Why not just do it all in Go?
Or is the point of it to be able to use existing JS scripts within a Go application?
I just found this excellent comment in an relevant Reddit post[1], which clarifies things quite nicely for me.
---
Let’s set up the scenario. You have a program that a user can manage via a config file. That tune knobs and dials, and everything works. They then ask you one day, “I’d like to add some custom logic for they have.”
Do you add another option to the configuration file to support their one use case? Or provide a mechanism for them actually to write some code?
If it’s the latter, do you want them to have to recompile your program to be able to run that custom code? Or do you have a runtime in your application for that custom code?
The latter is what a browser does. You can write JavaScript for a website without recompiling the browser just for the website. The user doesn’t have to worry about your event loops, SDKs, APIs, wiring code in rust/C, etc. This feature allows others to add custom logic to your application's runtime.
This is the purpose of embedding JavaScript into an application. It’s not the whole program, but it allows someone to provide code in your program event loop to do something custom. Not every program needs this. Sometimes, a config file of YAML or command line arguments works, too.
[1] https://www.reddit.com/r/golang/comments/1d78d3j/what_script...
Not doing it all, Go allows my API endpoint to accept sandboxed javascript code to enable users to run more complex queries against my data. It is also faster because they don't have to make individual API calls for each data set. Therefore, more data is required to be sent over the wire to them.
Thanks. Ive been exploring all of this for the past many hours and definitely see the appeal and various use cases for it now.
Thanks for opening me up to it all - it will definitely solve a problem that I'm about to start working on!
I've been using goja in a logical replication tool to make it field-programmable by the end users. No two deployments look alike, so a great deal of flexibility is needed for ETL uses. There's a tendency for configuration languages to become Turing-complete, so we started with a Turing-complete language for configuration.
JS, or in Replicator's case, TypeScript (shout-out to esbuild), is sufficiently well-known that any dev group will have some experience with it. On the whole, I've been very impressed with how straightforward it's been to have user-scripts integrated into the processing pipeline.
https://github.com/cockroachdb/replicator/wiki/User-Scripts
Ah, that makes a lot of sense - far easier to expose a JS api for user-extension than Golang (which is probably not even all that possible, except for via wasm, which would still be challenging)
And, I suppose my initial hunch is probably correct - for the dev/application to be able to use existing JS libraries (be it part of your broader application/system or external) within a Go app.
Check out my other comment [1] in this post for a newer, better version/fork of Goja - from the grafana/k6 team.
[1] https://news.ycombinator.com/item?id=41470601
On top of the previous comments describing how to use JS as an embedded scripting language, I've seen in production (series B startup) an architecture where:
a) non-technical users describe, in plain English, what they would like from the data; b) an LLM with lots and lots of prompt-engineering massaging translates the English into JS, c) the JS gets executed in a sandbox, d) the results get returned to the user
For a real-worl use case, look towards the 'script' processor in Elastic' Beats family of data shippers [1] (example is from Metricbeat but it exists in other Beats too)
They can be configured to mutate the data with some basic DSL syntax, but if you have more advanced needs, you can break out to JavaScript and transform the JSON any way you want.
This is very useful because now any transformation becomes possible. And because this is the user of your product who writes that JavaScript more as configuration than code. And since there's no build step, it is just part if the configuration loaded at startup.
[1] https://www.elastic.co/guide/en/beats/metricbeat/current/pro...
I am using otto[0] (similar to goja) in the headless cms I maintain [1] in "actions" api [2]. So one can customize actions according to needs by writing JS. The JS is executed inside a transaction. Sort of a mini-lambda kind of api
[0] https://github.com/robertkrimen/otto [1] https://github.com/daptin/daptin [2] https://daptin.github.io/docs/actions/actions/
I didn't use Goja (this was maybe a decade ago) but I used Otto, which is the same concept.
I used it for game scripting. Lots of stuff is just easier to write in a high level language. I was able to use inheritance to compose behaviors for entities, and I could restart entities with new code without having to restart the server.
You know, back when I got started as a Schemer, I remember people complaining about how many implementations of Scheme there were and how difficult it was to port between them. Today in JavaScript land we have Node, Deno, Bun, Goja, Sobek, Nashorn, whatever the browsers have, QML, GNOME, all of them subtly different. But instead of complaining, which people seem inclined to do when they encounter nested parens, people bro down and wrote code necessary to make them all interoperate if necessary -- despite JS having even less of a core language standard than Scheme does.
I guess it's the Lisp Curse or something, I dunno.
I've been a pretty happy Otto[1] user for a number of years now.
The article makes no comparisons seemingly? I don't think there's even a never. I find that pretty odd.
I'm curious if there'd be any reason to switch.
https://github.com/robertkrimen/otto
I just wanted to share the details of my experience with Goja. It was not relevant to compare Otto.
The name looks a obvious choice but... Ouch. It's a very funny name for Korean. It means eunuch. Lol
Then there’s Ejacs
https://github.com/emacsattic/ejacs
http://steve-yegge.blogspot.com/2008/11/ejacs-javascript-int...
Someone on Reddit mentioned in Norwegian, it means "nothing."
I actually came to comments to say that.. lol.
Great article. I was looking for something like this last week.
I'm glad it was helpful. It was a list of things I had known before I used it. So it was meant to be a quick getting started like doc.