Leyden has been relying pretty heavily on training runs which I get, but man is that a pretty hard burden to setup.
The tooling for doing these sorts of training runs doesn't really exist, you end up needing to do something more bespoke as part of your build pipeline if you want it there. That can be especially tricky with more complicated applications like the ones I maintain.
I like what these can deliver, but dislike the effort needed to get it going.
The complexity of training depends heavily on your goals. If your goal is to improve startup time, it's actually pretty simple.
If your goal is peak performance across the board and you can't wait for runtime optimizations to kick in, you're going to have a harder time. But in many of the things I've worked on, startup time has been a bigger issue than JIT->peak.
I completely agree. I know that there has been some work on command line ergonomics for this stuff, but my hope is that after the bulk of the Leyden work is done, another project starts up aimed at unifying all of the various disparate tools that modern Java now has.
You have training runs to generate bytecode/compiled code, you have jlink to create a distributable JRE for your app, leaning on jdeps and various other tools to correctly create it, you have the venerable jar file, but all in all the process of actually getting a java project ready to distribute in a way that doesn't presuppose a relevant pre-installed JRE on the user's side, and which takes advantage of the various facilities available to modern Java is still a pretty bespoke and painful process.
Compared to Go and Rust, and just throwing a static binary to someone or some server, Java kind of sucks. And that's unfortunate because the language, the runtime, and even the dependency situation all feels pretty competent, there's not some huge gulf that makes Java "old". Its really just the every about setting up a project and then distributing it, so the "start" and "end" of a project, that currently sucks, and I fear that so much of the great work the Java team is doing is going to go largely unused if they don't work to make it easier for the average developer to utilize it.
Are you describing jpackage? I'm not sure how you would make it less bespoke, given you are essentially creating a Java runtime for your program. Maybe the Java folks could tree-shake the runtime for you so you wouldn't have to care about their weird module system, but it's not the worst thing to learn.
jpackage is heavily oriented around installable artifacts, it creates a .deb or .rpm for Linux for example. So if I wanted to throw someone a cli utility I can literally just send them the binary if I’m working in Go, whereas Java they need to use dpkg to install the deb. This somewhat complicates building a docker image as well. It’s the same story for windows and Mac as well. If I’m creating a Swing app or something similar this is very convenient but anything else and jpackage isn’t ideal. Not to mention that I don’t know what the integration is currently with jpackage and the aot cache that Leyden produces but I’m guessing it isn’t super straight forward.
I’m sure that all of this stuff can be strung together after spending a suitable amount of time trolling through the documentation of the individual tools, but it’s far cry from go build and cargo build, and I think it would be great if Java could narrow the gap in this aspect.
Don't forget GCJ. It wasn't removed from GCC until about 2016, a nearly 20-year long run, though interest had waned many years prior.
I don't remember if GCJ supported loading and running code dynamically with a built-in JIT or interpreter. I think it was just pure AOT, which caused some compatibility headaches.
I worked on this years ago. Gcj did support loading and interpreting bytecode alongside the AOT-compiled code. libffi's closures were originally implemented to support calling interpreted methods from AOT-compiled code, because a class's vtable could include a mix of AOT-compiled and interpreted methods. You could also compile jar files directly into .so files, which would get loaded and used at runtime if a classloader ever loaded the corresponding jar.
GCJ was great - I wrote a handset UI entirely in java for a startup in the early 2000's, basically android before android existed, using GCJ. I remember their native interface was somewhat more convenient to use than JNI.
I wish we had GCJ resurrected, now that the java libraries are GPL'd.
They were not the only ones, see PTC, Aicas, IBM, and a few others.
The big difference until GraalVM and OpenJ9 became available as open source, was that all those vendors made AOT compilation a commercial feature, and thus most devs never cared they existed.
In fact GraalVM was one of the main reasons Excelsior JET eventually closed doors.
That would place significant limitations on the applications that can be compiled. As the JEP explains: “Features such as dynamic class loading, dynamic linkage, dynamic dispatch, and dynamic reflection bring vast expressive power, and have been fundamental to the platform's success. HotSpot handles these features naturally, while static compilers struggle with them. Even heroic amounts of static analysis cannot make up for the fact that these features require many decisions to be made at run time. Implementors of static compilers for Java code have therefore resorted to incompatible constraints, such as closed-world assumptions, and to putting significant burdens on developers, such as having to identify in advance the classes eligible for reflection.”
Therefore I don’t see AOT-only becoming an integral part of standard Java in the foreseeable future.
It works pretty well for greenfield projects on .NET, so long as you can live without third-party libraries that don't support AOT.
Existing code bases can be a pain, though, especially applications that heavily rely on things like C++/CLI and COM Interop that basically need to be rewritten from scratch.
Which is not to say that it's intended to replace the traditional JIT runtime in applications where it isn't troublesome, because it's not.
we are not talking about fully supporting dynamic loading, because it's not needed in all the cases, and in some cases, the list of allowed classes in the application can (or must) be limited.
i guess what i am saying is that AOT-only case is not "standard java", but something that will make a (more) useful replacement for things like c/c++.
If you haven't checked it out yet, GraalVM's Native Image [1] already let's do AOT compilation suitable for distribution. You have to do some configuration to deal with reflection and dynamic class loading since there's no JVM in the produced build, but it comes with tooling to simplify that. And that restriction is being addressed by Project Crema [2].
The slight drawback is that AOT compiled Java code's performance isn't as good as the JIT compiled form running on the JVM.
There's a reason why folks keep coming back to JIT compilation. It's hard to beat the value that can be gained from actually gathering data on how the code is actually used, which leads to a whole set of potential optimisations (which is the problem Profile Guided Optimisation attempts to solve for AOT compiled code.) The JVM is arguably one of the most advanced and capable JITing runtimes.
As with anything it's a trade off. Fast start times, pretty fast running (I think maybe less memory usage?); vs the full JIT speed you can get on the JVM at the cost of start-up speed and memory consumption. All depends on what you want to use the application for.
If you're talking something like a serverless function, go native. If you're talking production server where you're measuring runtime in more than dozens of minutes, probably better to stick to the JVM & JIT.
Full agreement with everything you said. Just one detail to add on serverless (and potentially in a scheduled environment like k8s), startup time can be an order of magnitude or more faster in a native build. For instance, I have some quarkus applications that can take 10 seconds to ready running via the jre that take 10ms or less to start as a graalvm built binary.
Data point from a small side project: I run user code in a Piston sandbox for a coding challenge site. A trivial Java program costs 2.5-3 s wall clock on a 4 vCPU box, almost all of it javac plus JVM startup. Go takes about 1.7 s for build plus run, and Swift running through the interpreter is at 0.4 s. Java is the reason I had to put a global rate limit on compiled languages at all. If AOT gets the JVM side down to a few hundred ms, that changes what a small box can serve.
I suppose write once run anywhere is no longer a goal either.
"It is not a goal to support all CPU architectures currently supported by HotSpot."
This pretty much validates the point of view that VM design is now baggage, as a sandbox it has been flawed, for performance it's been prohibitive, and cross platform portability by virtue of being virtual, was just a convenient byproduct.
AI now handles the portability, sandbox security hasn't changed (cf. docker still has sandbox problems, LLMs have sandbox problems, it's always an ongoing concern) and that just leaves Java, as always, chasing performance.
- "Improve startup and warmup time by making optimized native code for an application instantly available when the HotSpot Java Virtual Machine starts."
- "If the workload changes in production, regenerate native code dynamically for continued peak performance, providing the best of both ahead-of-time (AOT) and just-in-time (JIT) compilation."
- "Ensure that shifting from AOT-compiled code to JIT-compiled code is invisible to applications."
That is JEP544 doesn't substitute JVM and JIT optimizations. It just changes the nature of the JIT's outcome.
The very next sentence after the one you posted is "We expect normal porting activities to eventually add support for all major architectures." Goals and non-goals inform the scope of the proposal. Given this is effectively an "add-on", I don't think it's unreasonable to prioritize the most popular platforms. Others will continue to execute Java perfectly fine in JVM mode.
Startup time being the main benefit means simple training runs get you most of the value. Going for peak throughput is where the setup gets heavy.
Leyden has been relying pretty heavily on training runs which I get, but man is that a pretty hard burden to setup.
The tooling for doing these sorts of training runs doesn't really exist, you end up needing to do something more bespoke as part of your build pipeline if you want it there. That can be especially tricky with more complicated applications like the ones I maintain.
I like what these can deliver, but dislike the effort needed to get it going.
The complexity of training depends heavily on your goals. If your goal is to improve startup time, it's actually pretty simple.
If your goal is peak performance across the board and you can't wait for runtime optimizations to kick in, you're going to have a harder time. But in many of the things I've worked on, startup time has been a bigger issue than JIT->peak.
I completely agree. I know that there has been some work on command line ergonomics for this stuff, but my hope is that after the bulk of the Leyden work is done, another project starts up aimed at unifying all of the various disparate tools that modern Java now has. You have training runs to generate bytecode/compiled code, you have jlink to create a distributable JRE for your app, leaning on jdeps and various other tools to correctly create it, you have the venerable jar file, but all in all the process of actually getting a java project ready to distribute in a way that doesn't presuppose a relevant pre-installed JRE on the user's side, and which takes advantage of the various facilities available to modern Java is still a pretty bespoke and painful process.
Compared to Go and Rust, and just throwing a static binary to someone or some server, Java kind of sucks. And that's unfortunate because the language, the runtime, and even the dependency situation all feels pretty competent, there's not some huge gulf that makes Java "old". Its really just the every about setting up a project and then distributing it, so the "start" and "end" of a project, that currently sucks, and I fear that so much of the great work the Java team is doing is going to go largely unused if they don't work to make it easier for the average developer to utilize it.
Are you describing jpackage? I'm not sure how you would make it less bespoke, given you are essentially creating a Java runtime for your program. Maybe the Java folks could tree-shake the runtime for you so you wouldn't have to care about their weird module system, but it's not the worst thing to learn.
jpackage is heavily oriented around installable artifacts, it creates a .deb or .rpm for Linux for example. So if I wanted to throw someone a cli utility I can literally just send them the binary if I’m working in Go, whereas Java they need to use dpkg to install the deb. This somewhat complicates building a docker image as well. It’s the same story for windows and Mac as well. If I’m creating a Swing app or something similar this is very convenient but anything else and jpackage isn’t ideal. Not to mention that I don’t know what the integration is currently with jpackage and the aot cache that Leyden produces but I’m guessing it isn’t super straight forward.
I’m sure that all of this stuff can be strung together after spending a suitable amount of time trolling through the documentation of the individual tools, but it’s far cry from go build and cargo build, and I think it would be great if Java could narrow the gap in this aspect.
What are the architectural differences between this and Android's ahead-of-time runtime?
A little late, isn't it? .NET has had this for over 20 years (pre-compile).
Excelsior JET did that 25 years ago: https://en.wikipedia.org/wiki/Excelsior_JET
It's a shame that Sun/Oracle never partnered with them to bring native apps. This could have saved Java on the desktop.
Don't forget GCJ. It wasn't removed from GCC until about 2016, a nearly 20-year long run, though interest had waned many years prior.
I don't remember if GCJ supported loading and running code dynamically with a built-in JIT or interpreter. I think it was just pure AOT, which caused some compatibility headaches.
I worked on this years ago. Gcj did support loading and interpreting bytecode alongside the AOT-compiled code. libffi's closures were originally implemented to support calling interpreted methods from AOT-compiled code, because a class's vtable could include a mix of AOT-compiled and interpreted methods. You could also compile jar files directly into .so files, which would get loaded and used at runtime if a classloader ever loaded the corresponding jar.
GCJ was great - I wrote a handset UI entirely in java for a startup in the early 2000's, basically android before android existed, using GCJ. I remember their native interface was somewhat more convenient to use than JNI.
I wish we had GCJ resurrected, now that the java libraries are GPL'd.
Just ask claude to port it to latest gcc :P, pls don't
Yeah! I used it around 2003 to integrate a Java PDF parsing library into our C++ app.
Its biggest downside was the use of Boehm GC that had some issues with large heaps.
They were not the only ones, see PTC, Aicas, IBM, and a few others.
The big difference until GraalVM and OpenJ9 became available as open source, was that all those vendors made AOT compilation a commercial feature, and thus most devs never cared they existed.
In fact GraalVM was one of the main reasons Excelsior JET eventually closed doors.
IBM's J9 had AOT for a long time too. I wonder how JEP 544 compares.
The only thing that could possibly save Java on the desktop would require Oracle not owning it.
Welcome to ngen and GAC, Java
I hope at some point we'll get AOT-only mode (or compile to native) and maybe even cross-compilation.
That would place significant limitations on the applications that can be compiled. As the JEP explains: “Features such as dynamic class loading, dynamic linkage, dynamic dispatch, and dynamic reflection bring vast expressive power, and have been fundamental to the platform's success. HotSpot handles these features naturally, while static compilers struggle with them. Even heroic amounts of static analysis cannot make up for the fact that these features require many decisions to be made at run time. Implementors of static compilers for Java code have therefore resorted to incompatible constraints, such as closed-world assumptions, and to putting significant burdens on developers, such as having to identify in advance the classes eligible for reflection.”
Therefore I don’t see AOT-only becoming an integral part of standard Java in the foreseeable future.
It works pretty well for greenfield projects on .NET, so long as you can live without third-party libraries that don't support AOT.
Existing code bases can be a pain, though, especially applications that heavily rely on things like C++/CLI and COM Interop that basically need to be rewritten from scratch.
Which is not to say that it's intended to replace the traditional JIT runtime in applications where it isn't troublesome, because it's not.
> so long as you can live without third-party libraries that don't support AOT.
Yes, and that’s a significant limitation in the Java library and framework ecosystem.
But the possibility of AOT compilation is what drives libraries to support it. That's why .NET libraries are adapting over time to support AOT.
we are not talking about fully supporting dynamic loading, because it's not needed in all the cases, and in some cases, the list of allowed classes in the application can (or must) be limited.
i guess what i am saying is that AOT-only case is not "standard java", but something that will make a (more) useful replacement for things like c/c++.
If you haven't checked it out yet, GraalVM's Native Image [1] already let's do AOT compilation suitable for distribution. You have to do some configuration to deal with reflection and dynamic class loading since there's no JVM in the produced build, but it comes with tooling to simplify that. And that restriction is being addressed by Project Crema [2].
[1] -- https://www.graalvm.org/latest/reference-manual/native-image... [2] -- https://github.com/oracle/graal/issues/11327
The slight drawback is that AOT compiled Java code's performance isn't as good as the JIT compiled form running on the JVM.
There's a reason why folks keep coming back to JIT compilation. It's hard to beat the value that can be gained from actually gathering data on how the code is actually used, which leads to a whole set of potential optimisations (which is the problem Profile Guided Optimisation attempts to solve for AOT compiled code.) The JVM is arguably one of the most advanced and capable JITing runtimes.
As with anything it's a trade off. Fast start times, pretty fast running (I think maybe less memory usage?); vs the full JIT speed you can get on the JVM at the cost of start-up speed and memory consumption. All depends on what you want to use the application for.
If you're talking something like a serverless function, go native. If you're talking production server where you're measuring runtime in more than dozens of minutes, probably better to stick to the JVM & JIT.
Full agreement with everything you said. Just one detail to add on serverless (and potentially in a scheduled environment like k8s), startup time can be an order of magnitude or more faster in a native build. For instance, I have some quarkus applications that can take 10 seconds to ready running via the jre that take 10ms or less to start as a graalvm built binary.
progress, not perfection! Java is on fire right now with new stuff landing. We'll get there!
We have that in Codename One, yes it's not really "Java" but it compiles bytecode AOT and cross compiles to some of the platforms.
Data point from a small side project: I run user code in a Piston sandbox for a coding challenge site. A trivial Java program costs 2.5-3 s wall clock on a 4 vCPU box, almost all of it javac plus JVM startup. Go takes about 1.7 s for build plus run, and Swift running through the interpreter is at 0.4 s. Java is the reason I had to put a global rate limit on compiled languages at all. If AOT gets the JVM side down to a few hundred ms, that changes what a small box can serve.
So we've gone full circle again?
I suppose write once run anywhere is no longer a goal either.
"It is not a goal to support all CPU architectures currently supported by HotSpot."
This pretty much validates the point of view that VM design is now baggage, as a sandbox it has been flawed, for performance it's been prohibitive, and cross platform portability by virtue of being virtual, was just a convenient byproduct.
AI now handles the portability, sandbox security hasn't changed (cf. docker still has sandbox problems, LLMs have sandbox problems, it's always an ongoing concern) and that just leaves Java, as always, chasing performance.
I don't get your point. This is what JEP says:
- "Improve startup and warmup time by making optimized native code for an application instantly available when the HotSpot Java Virtual Machine starts."
- "If the workload changes in production, regenerate native code dynamically for continued peak performance, providing the best of both ahead-of-time (AOT) and just-in-time (JIT) compilation."
- "Ensure that shifting from AOT-compiled code to JIT-compiled code is invisible to applications."
That is JEP544 doesn't substitute JVM and JIT optimizations. It just changes the nature of the JIT's outcome.
The very next sentence after the one you posted is "We expect normal porting activities to eventually add support for all major architectures." Goals and non-goals inform the scope of the proposal. Given this is effectively an "add-on", I don't think it's unreasonable to prioritize the most popular platforms. Others will continue to execute Java perfectly fine in JVM mode.
Not at all, this is OpenJDK getting JIT cache feature like OpenJ9 has since 2008, or ART since Android 7.