Cmd+Shift+V - Stacked clipboard, you can start typing to search or hit a number to choose what to paste (keeps everything you've copied/cut inside jetbrains for a while)
Cmd+Shift+E - Recent locations, you can start typing to search - shows little buffers of where you've been recently
Cmd+Shift+A - Action tab of the command palette - fuzzy search for any command (really the only shortcut you need, other than maybe Shift+Shift for main command palette shortcut)
--- Through the Action bar...
Local History / Local History of Selection - you can start typing to search quite far back the history of all changes of the current file or selection - you can also right click a folder or the project and do the same. Much finer grained than git.
The general concept of being able to search for something and edit directly in the buffer of the search results.
Hero! I had not done my homework/have not been aware, but these all look fantastic! The stacked clipboard is something I periodically mentally complain about (Why is clipboard on every OS/tool I've used single item?)
I will add one that are possibly more well-known:
- ctrl + shift + F: Find text in any file
- ctrl + N: Find types (structs, classes etc)
- ctrl + shift + N: Find any file by name or path
Windows has had a pretty usable stacking clipboard for a while! You just have to activate it. Since you can pin thing into it it’s also quite useful as a rough and ready way to type special characters you use frequently.
Mathematica is the earliest thing I am aware of with this feature where it was Alt+. to expand selection in their notebook interface starting in the early 90s. But the thing I miss most that I still can't shake the muscle memory of after almost a decade of not using much Mathematica, is that single/double/triple/n-click scaled this way as well. So double-click selected a whole word (as in all editors), triple-click selected all the comma-separated multiple args of a function, 4-click for f(a,r,g,s), and so on.
I work on AST based revision control. I have a stack of ideas on how to achieve the same Ctrl+W effect with commits/diffs/cherry-picks. All still in flux. If you have some thoughts to share, please do.
I use it constantly in helix too. The vscode one is meh. I think I saw a discussion in github once about switching to tree-sitter, which would improve AST-related actions. I don't think it went anywhere though.
I love AST aware editing. I think it's one reason it's always been so nice to edit lisps. Stuff that is complicated to describe in javascript (and doesn't have LSP support) p much requires a whole AST parser, but in lisp it's just a simple list operation. When I go back typescript after a weekend of clojure, I reeaally miss slurp! and other paredit commands
I tried it, but it just was too clumsy. Sometimes refactoring/editing needs to go through phases where the AST is invalid, and MPS makes that just too clumsy.
To me, it feels like Zed and VsCode perform most operations in a general way on the text; they don't seem to (in Python and Rust at least) have an understanding of the code structure in the way JB does. (And based on some digging on Ki the way it does as well?) So, I would bet they are using that text-based model, which would be hit/miss here.
I had this issue too, so I remapped Ctrl-W/Shift-Ctrl-W to Ctrl-\/Shift-Ctrl-\ .
(Also git operations became two-key sequences, starting with Ctrl-G and that damn Ctrl-K stopped being the shortcut for commit.)
Many years ago, I created an editor operating on syntax trees that I think is more "hard-core" than this - that is, only tree-oriented operations are done. There is no parsing of text, since entering plain text, rather than a tree, is impossible. Hence, there can be no syntactically invalid programs.
The challenge is getting this to be a useable way of entering programs. I think I made progress on this, but the feasibility varies with the programming language.
> The challenge is getting this to be a useable way of entering programs.
Well exactly.
When the path between Program A and Program B can only be valid programs, you are going to end up with either a much longer, less intuitive path, or deleting everything and starting again. It can also be quite possible to invent structures which are valid but have no valid path to creating them.
Well, I think most common transformations work reasonably well. One usually doesn't want to do things completely contrary to the AST, such as convert "while a<b" to "whilea = b".
And it's certainly possible to create any valid AST in the editor I describe. The set of valid trees is extended to those with "holes" in places, which one fills in when entering a program, and it's always possible to do this.
The challenge is one of finding an intuitive user interface, not whether it's possible at all. One issue is that infix notation is unnatural for entering trees (prefix is more natural).
> It can also be quite possible to invent structures which are valid but have no valid path to creating them.
I'm curious if you have an example of such a structure?
Pedantically: if, for every valid tree, there exists a bidirectional path to the empty root node, there's always at least one path between all given pairs of valid trees ... albeit one that no developer would ever take.
You are quite right to call this out - and quite right in general. With AST verification alone your statement holds strongly.
I was remembering a professor I had who was very into formal verification and had an IDE that would only accept valid programs - however his validity checks were more than just tree based, they involved passing a type check. Which now you've called me on it, is quite definitely beyond valid AST editing.
The base case: if you have a structure with a mandated cyclical reference that doesn't accept a second (e.g. nil) type, you cannot construct it without creating an intermediate invalid program. This doesn't tend to show up if the cyclical reference is all the same type (A1 -> A2 -> A1) because you can often cheat and self reference, but it does if it is different types (A1 -> B1 -> A1). You can't construct an A without a B, which cannot be constructed without an A.
But that's still not a problem you cry! And quite right, you can edit the type itself to remove and re-add the reference.
That is until the type is built into the language, or a library.
OP clarified, but I understood that as a valid tree without a path to empty node. I don't have an example, but with grammar weird enough it sounds plausible. Especially if some form of type checking is involved, as OP mentioned.
Pantograph[0] seems to be a more recent attempt to implement the same idea. It is still not a general editor but generalizing it to ranges of tree selections looks promising
I actually used a language and editor like this in a previous large company.
It was an experimental language that they wanted to replace their current application level language and was built ontop of JetBrains MPS https://www.jetbrains.com/mps/ which has that feature among others.
My personal opinion after working with that lang is all of this is that its theoretically interesting. But a dead end in practice along with visual coding etc.
The universality, simplicity, and legibility of text as interface for both humans and machines is too hard to beat. I think LLMs is just the most recent example of this.
Things that don't work in practice:
- You need a special editor, its heavy and slow, you lose all the ecosystem around them.
- You can't just cat or inspect the raw file, you can't see it in terminal at all.
- You need a new version control system, review system, and people need to learn it.
- You can't use any existing tools for working with code, you are essentially starting from scratch, and lose all the benefits of all the work everyone else is doing around tooling, dev saas etc.
- humans don't think in trees, they don't think in syntactically correct programs. Its actually absurdly frustrating writing a program in only syntactically correct edits. 99% of the time you are coding, your work is probably syntactically and semantically incorrect.
The tooling that is able to either make the right tradeoff around strictness or allow the users to make the that tradeoff is what ends up being used in practice. A good example of this typescript with gradual typing, python with type annotations, etc.
I think these editors just fall in a bit too far right on the strictness scale.
These are good points, but are mostly pragmatic, resulting from the rest of the system not being designed around a tree representation. Of course, that can be decisive in practice...
A more fundamental issue (which I mention in the document) is that there is a discordance between the 2D textual display and the underlying tree. I think this becomes more apparent when input is not only keystrokes, but also mouse positioning.
Of course, these disadvantages do not necessarily outweigh the advantages of editing in terms of trees!
What was the issue with MPS? MPS also looks like "custom widgets in normal text-based java" i.e. mixed paradigms and not "ground up ast-based editing".
I'm not sure if I can agree with your other points. I can't think of the last time I cat'd or maniuplated rust/java/go outside a heavy and slow editor. Many languages have more or less "one" editor with all the features that most people use, and often use it for change review too. And it seems weird to characterize needing new tools as a "dead end"... all languages need new tools and won't have them at the start.
Plenty of people use go, rust, and java which are not gradually typed.
Nothing wrong with MPS. I think it was just the language itself was design around ast editing, and projectional editor.
I was really excited about the concept and read some papers on it at the time. Then I ran a real world application of it, and writing code in it was just a bad experience.
I think I didn't explain the gradual typing properly. I mean the broader concept of moving developers from one way of doing things to another.
When the vast majority of people and tools are working a certain way. Introducing a whole new thing that requires them to work a whole different way is a hard sell.
So having a way for people to move gradually from point A to point B at their own pace and pick up benefits along the way is really nice.
One example of this is Typescript with gradual typing. Swapping form js to typescript is fairly easy. You can still use your old tools and over time you can start to adapt new features into your code, and over time everyone gets used to this new stricter/better approach.
I think ast based code like this essentially requires you to throw out everything all at once, not just editor, but literally everyone's workflow today and rebuild from the ground up.
I just don't think it will ever be more than a niche thing unless it can loosen up on strictness and play nice with existing tooling.
If you want to relive it then simh¹ with mame² might be an option. There appears to be some support for VT11³, along with docs⁴ to use as a starting point.
No, I'm not claiming to have read all one hundred pages already. However, from what I have read I'd love to see a functional demo.
Now you're out of the realm of Ki, but what you're talking about is still being worked on in the modern era, by me! I'm building BABLR which is a modern follow-up to your idea, built on top of a powerful, generic system of parsers which has gaps/holes but no error recovery, so that it works with trees which may be incomplete but which must not be invalid.
The hard part is that we need to be able to talk about these structures. Even just here on this forum we need to be able to communicate precisely about them. I often use · as a typesetting symbol so that I can easily write and read expressions like 2 + · which you would read as "two plus gap". The · symbol is only for typesetting as I say thought because you it's not safe to assume that any one character is reserved for our use in every programming language. Instead we wrap the parts that aren't syntactic in quotes and we use <//> as the symbol for a gap so that it looks more like this:
<*> "2 + " <//> </>
The * there is a flag on the node, it means this node a leaf of the tree -- a token.
And yes, BABLR can really parse this. If we've piqued your curiosity, our Discord server is currently the hub of our community and we'd love to see you. https://discord.gg/NfMNyYN6cX
Wow! I stumbled onto your paper a while ago when I was looking into structural editing and thinking of a masters in CS, exploring editor interfaces / feedback loops.
(Maybe your paper is famous and it’s not wild that I read it, but it was wild to see after so many years)
I never took that path, spent time in tech industry confused why people didn’t seem interested in structural editing and better editing tools.
Out of curiosity, how do you think LLMs and genai affect the value of structural editors and similar tooling?
Part of me wants to stay disciplined— of course it’s valuable to work efficiently and work on the AST and with a repl. The other part of me gets paid to work on essentially a punch card system (building dev, ship to prod and see what happens)
I feel so illiterate when it comes to AST editing sometimes. I understand what an AST is from a computer science perspective. But I've never worked on a huge software refactor before that required direct AST textobject editing. Maybe an indication of my skill level...
The extent of my usage is having nice textobjects to easily interact with arglists and functions which aren't native to (neo)vim. Very cute and nice to just write "daf" somewhere in a function and just have it "just delete". Or hook it up with basic macros: search for regex, "daf".
I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).
As for LSP's, I do use go to definition and rename all references, which is nice. But the huge structural refactoring part I have never really done. I don't really use many LSP features besides those two either...
> I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).
This is actually what's nice about tools like ast-grep. The pattern language reads almost like the code itself so you can see the transformation right in front of you (at least for small-scale cases) and reason about it. TypeScript examples:
The $X, $A etc. are metavariables that match any AST node and if the same metavariable appears twice (e.g. $X = $X ?? $Y), it requires both occurrences to bind to the same code so `x = x ?? y` will match but `x = y ?? z` won't. You can do way more sophisticated stuff via yaml rules but those are less visually intuitive.
Sadly coding agents are still pretty bad at writing ast-grep patterns probably due to sparse training data. Hopefully that improves. The tool itself is solid!
If your editor of choice supports an extension (vscode does for example) it's a very easy on-ramp for a better search/replace than regex offers. It's syntax-aware so you don't need care about whitespace, indentation etc. Very easy to dip your toes in where a regex would get complex fast, or require multiple passes.
I converted a codebase from commonjs to esm trivially with a few commands right after first installing it. Super useful.
I hope LLMs eventually start operating at this level rather than raw text. And likewise for them to leverage the language server to take advantage of built in refactorings etc
I don't know which editors support this but there's also, for lack of a better word, context aware grep. For example, search and replace foo with bar but only inside strings, or only inside comments, or only variables, or only methods, not class names (maybe ast-grep does this).
TBH, it's actually not as hard as you think, most of the time, what I do is just select the whole syntax node and delete it, copy it, or replace it, and only 20% of the time would actually require deliberate understanding of how the AST is structured in the current language I'm coding in.
Maybe, or maybe not. I think there are many different kinds of work all lumped together under the "software engineering" umbrella, and tools which are indispensable for some people can seem unimpressive or unnecessary to others. We're not all spending our time in the same way, working on the same kinds of tasks.
I don't find myself doing huge structural refactoring often enough that it would be worth my while to learn a specialized tool for doing it; I'm a quick typist, and it's easy enough to just blat through the changes by hand on the rare occasion such a thing is necessary. But I don't work on giant line-of-business apps or sprawling web services, and I can see someone who did a lot of that sort of work having a different take on the matter.
I feel the same way. Learning Elixir and getting into macro writing was really helpful for me. Lisp is also the same way though for me the syntax of Elixir resonated a lot better for me. Many paths to the destination and all that, but figured I'd share in case it's helpful to someone
Hold on. Operating systems and long-running esoteric history are always on topic here, right?
(This is spoken with something close to affection; I look fondly on my former Emacs days. I'm probably more likely than not to enjoy the company of a human who has at least tried Emacs and had some thoughtful reaction to it. I may not use it now, but the ideas live on in how I think and what I expect. Editors that push the limits, whatever those limits are, are part of the old school ethos I love about programming.)
I think the "Scripting language? Check!" is missing an important distinctions between Emacs and most other extensible editors. The proportion of the code editor features that are written in the configuration/extension language matters
Looking at the Emacs and NeoVim codebases in GitHub: Emacs is 74.6% ELisp or CommonLisp and NeoVim is 71.9% Lua or VimScript. Never mind, NeoVim is close!
But, can you at runtime modify the NeoVim core functions? Honest question because I've only used vim and VimScript was definately limited compared to elisp. If so, then I say NeoVim and Emacs are both highly-extensible editors.
None of those capture the essence of emacs, which is that it is a scripting language repl where the vast majority of functionality is implemented in that language and can be inspected, debugged and modified at runtime.
Emacs is a general purpose programming/computing environment that ships with a text editor and some other nice features. Vim is a text editor and some other nice features that ships with a scripting mechanism. Not the same thing.
Vim has stopped before that particular threshold. But if ever Neovim get a GUI version, I believe the community will soon have its own Gnus, Eww, and EMMS.
I am working on a modal code editor project that you might find interesting then. It also operates on an AST directly, which is represented as UI nodes which closely resemble normal text layout. Email in profile if you’d like to give it a try and possibly give early feedback (still in early development).
I made the vscode integration for this. I feel bad that I haven’t contributed much since, it’s a really cool project. IMO it’s important to try to innovate in the foundational tools of our craft (editors, languages, tooling, OS, etc) which Ki does.
Thank you for that! I find it really daunting to try a new editor, but having a vs code extension as an entry point really helps me and increases the chances of trying it out soon!
I wrote the vscode extension for Ki. It would almost certainly be usable with Zed since it talks with the editor over a websocket. Check out the code in the Ki repo.
I can't wait to try it. Love that it's keyboard layout agnostic. A lot of other good sounding ideas in the docs. Especially inspiration from Emacs as everything being an editable buffer. There's always some massive tradeoffs between editors though. Guess I'll have to see.
It's a shame that editors are build around the editing model, so much time has been wasted by recreating everything. Neovim is a perfect editor in many aspects but editing model is not one of them, in my opinion. Helix or even Ki would not be needed if Neovim allowed to completely replace the vim part of it with plugins
I didn't really spent time thinking about it, because from reading their issue tracker, I very much doubt neovim would consider something like that. Partially because of how much of it is already vim specific and it would be very difficult to abstract that away, but mostly because of ideology.
You can already to a lot in Neovim with plugins, including recreating Helix, Ki, whatever. But the problem is that the vim state machine is always present underneath it, resulting in unwanted behaviors or impossibility to implement some things. I would envision it to pretty much allow to listen on raw keyboard and mouse events. However cursed it sounds, it should be possible to fully implement vscode or any other editing model in neovim
> I would envision it to pretty much allow to listen on raw keyboard and mouse events.
This is pretty much how the BRIEF editor for programmers worked, which was used in MS-DOS around 1990. In my opinion this was one of the best programs that I have ever encountered and I have never encountered again some of the good UI features that existed in the BRIEF text editor and in the XTree file manager, both for MS-DOS.
BRIEF had a tiling-window user interface with optional menus and it came with a default behavior, but it could be customized extremely. For each kind of key press or mouse event you could bind a function to be executed, which replaced the default behavior. For instance, the default behavior for pressing most keys was to invoke a function "self_insert", which inserted in the current text buffer and the current cursor position the character corresponding to the pressed key, but it was easy to change the behavior to anything else, e.g. to do auto-completion or inserting some program template.
The functions bound to events were written in a custom language similar to LISP, so in this respect BRIEF resembled Emacs. A great number of text manipulation functions were provided, so it was easy to describe complex actions in the event handling functions.
This was what I liked most at BRIEF, that by rewriting all the event handling functions it was very easy to convert it in an editor that no longer resembled the default BRIEF editor at all, but which behaved in whatever way you believed that an editor for program sources should behave.
So the text editor provided an extensive library of functions for handling text, text buffers, windows and files and a LISP-like language for gluing all these into a text editor. It also had a default implementation of a text editor, to be able to use it immediately, but that could be replaced partially or totally, when you had more experience and you knew what kind of behavior you want.
I assume that Emacs can be customized in a similar manner with BRIEF, but when I looked at Emacs it required a much greater initial effort than BRIEF, to reach a stage where you are able to modify it.
The default editing mode of BRIEF was very intuitive, at least for someone familiar with the IBM PC keyboard, so you could use it easily out of the box, before learning to customize it. Then, like most MS-DOS programs of that time, it came with excellent documentation, so it was very easy and fast to learn how to rewrite its event handling functions. With Emacs, both learning how to just edit texts with it, and then how to modify it according to what you prefer, is much steeper and more time consuming than it was with BRIEF.
I'm not sure how that's not "recreating everything". What's still shared if you replace the whole state machine (handling file management, etc) and also replace the whole input handling system? A terminal emulator and a copy of Lua?
There are certainly different possibilities of where the line would be drawn. I am not that familiar with vim internals, but I don't consider file management to be part of vim editing model. And we are only talking about editing model (e.g. b being beginning of a word), not the whole editor (plugin system, ui, lsp and treesitter integration, etc). Though it is true that some parts are more tightly related to "vim". There some cases were Neovim already wants to disconnect itself from vim legacy like allowing to scroll viewport without moving the cursor
By "recreating everything" I was also referring to the ecosystem around the editor
I think the idea's good, but I don't think you could modify Neovim to support it while retaining the ecosystem – not without also modifying much of the ecosystem, at which point it may be easier to do a straight port to a different editor, or construct a compatibility layer (essentially reimplementing enough of Neovim in the new editor that the ecosystem still works).
> And we are only talking about editing model (e.g. b being beginning of a word)
nmap can shadow the built-in commands, replacing them with whatever function you want.
> There some cases were Neovim already wants to disconnect itself from vim legacy like allowing to scroll viewport without moving the cursor
I think you mean beyond where the cursor is so it's hidden, but you can jump the screen so the cursor's current position is at the top/middle/bottom without moving the cursor using zt/zz/zb.
An editor that tries to be layout agnostic is almost certainly going to be a nightmare for people like me who set the layout in the keyboard itself. I downloaded the editor (I'm on Windows at the moment), and tried it with my keyboard set to Dvorak, which was plainly broken. I'm sure there's a way to fix the mapping, but when software thinks it's smarter than you, you end up feeling pretty dumb.
Yeah and using combobulate feels so natural, I just wish it was easier to add support for other languages. In addition to the navigation, and simple edits (e.g. M-k kills node) you have Searching/editing/etc. using tree-sitter queries directly within emacs, possibly replacing other uses of external tree-sitter based tools.
It makes editing supported languages almost as pleasant as editing Lisp.
With how well the integration already work in existing editors, I'm curious how the UX can be improved further by an editor made specifically with AST editing in mind.
That comparison table is strange and sometimes wrong. Neovim for example detects and updates external file changes by default. And the coherence of the keybindings in Ki is "Great" bit vim/helix:
> As you can see, there's no single logical categorization for these keymaps, they are either lowercase-uppercase, normal-alt, left-right bracket, or outright unexplainable.
Word, End, Back, Change Word and even Change Inner (, etc are very logical to me and I feel like I'm talking to the editor when editing. I get that it doesn't make sense when one has learned another way to do it, but it does make total sense you just have to make an effort to try and understand it.
It's like learning and always driving automatic then calling manual "outright unexplainable". You simply learned another way and are conditioned into believing that's the one true way. It shows the creator comes from VSCode (multi-cursor is a useless feature, just use s/search/replace and get used to macros and a whole new world will open).
Re. incoherence of vim's keybindings, I partially agree.
Most of the times, shift means "bigger", and only in a few places it means "invert".
Examples of "bigger" are all the motions involving words vs. WORDS, where WORDS are a broader interpretation of words; "V" is like "v" but by lines, thus in larger chunks; "C", "D", "Y" do the same as their unshifted counterparts, but extended to the end of line; etc.
Examples of shift meaning "invert" are fewer, and all sound irrational to me:
With x/X and o/O, the shift inverts the direction left/right or above/below, but we don't have for example i/I to insert to the left/right of the cursor, and H is not the opposite of h, J is not the opposite of j, etc.
With n/N, shift inverts next and previous. Why not n/p? I know, I know: p was already taken for paste, but still...
Finally, t/T/f/F are completely mixed up. In my mental model, t/f sound like to/from and I'd rather use them to move forward/backward till (not including) a char, while T/F would be the "bigger" variants which include the destination char.
I think you've touched on it, but I'm going try to take it one step further into explicitness.
Just over a year ago I decided to switch to Neovim. The reason for switching was personal; I was struggling with what I'll call "clutter" in other tools and I wanted a tool that would reinforce, at least lightly, a mode of working that promoted focus on what I was working on, while making it easy to reference other files without loading up my editor with tabs and other visual clutter (buttons/menus) I don't care about most of the time.
I took the advice I seemed to bump into repeatedly: try out vim mode in my current editor before making the plunge.
I really struggled at first. It felt wildly foreign. All the shortcuts were nowhere near to the world I was familiar with.
As I was about to give up, I ran into some advice that was along the lines of "stop trying to memorize shortcuts and start thinking in terms of what you want to achieve" (words and motions in vim-speak).
Your example of [C]hange [I]nner is a great one; that one in particular was life changing. Sure there are some words and motions that do require memorization, but so many others just flow naturally. And once you start thinking in actions, it's easy to see how they can layer on top of each other in really elegant ways.
I'm not even here trying to tout vim-like editors, I'd wager there are many editors that have some semblance of this kind of interaction, but rather to reiterate there's a shift from a PoV of function vs. goal.
Again, I don't think this is "the right way" but rather one of many perspectives that works in context with the phenomenology of me.
I had an experience like this when I switched to a Keyboardio split keyboard. It was impossible to use at first. Then I do some deliberate practice with a typing tutor app and now I love it so much I own multiple.
Is there a good tutorial for some of these advanced text editing features?
In particular I’d like to get platform independent shortcuts / key bindings. I use both windows and MacOS daily, and it throws off my muscle memory for shortcuts like “go to beginning of line”
I still think their point about search and replace still stand though. I make most my edits with regex in neovim nowadays and I feel this is the superior paradigm: You don't even need to get to some specific location in order to edit it. I also almost only use search to move around the file and I can even reuse the searches for substitutions. It makes most vim motions and commands almost useless for me nowadays.
I also feel like macros are a more clunky and error prone way to do what substitutions can do. Almost never use them.
> I was a Neovim macro user until I figured out how insane that was compared to multi-cursor after using Helix.
Multi-cursor was the first plugin I installed when I moved from VSCode to Vim because I was used to hitting Ctrl+d to select all words and then replacing. Does Helix do something different?
1) First I reach for <C-v> for visual block selection if everything is neatly aligned.
2) Next choice is %s/search/replace(/c if I need confirm).
3) Macros, and I love it everytime I get to use them. I just record the movements, copy what I need to copy, paste it where I need to paste it, and it's repeatable for every line or block where the *formatting* matches. And this is the important part, the words don't matter. I still feel like a wizard using them.
As far as I understand multi-cursor option 3 is a no-go without macros if the words don't match. But macros don't care as long as the movements translate to the same edits. How does Helix multi-cursor work that make macros insane?
Sorry for derailing a bit, the search and replace using a query make sense for purly textual (non-syntactic) editing, but if you want to apply consistent syntactic modifications across multiple locations in the same file, you will need both multi-cursor and syntax node selection/navigation/modification.
It's hard to explain unless you actually try Ki, because it is a paradigm shift
Or you could use grn for LSP rename[1] in Neovim and it will rename all references across the project. VSCode and other IDEs have a similar feature, but compared to multi-cursor that's cheating.
I'll give you that using the AST to select the references is an interesting addition to multi-cursor, but I still don't see how they would be useful compared to my current workflow.
That comparison table is misleading because it mixes implementation details with user-facing behavior, and for example Neovim uses inotify or platform-specific filesystem watchers to surface external changes so marking that row as 'no' is incorrect without platform and config qualifiers.
Keybinding coherence is an engineering tradeoff, and if Ki wants consistent behavior it should expose orthogonal primitives like operators and motions plus AST-aware textobjects using Tree-sitter style node names, then provide discoverability tools such as a which-key overlay and explicit node prefixes so power users get composability while newcomers have a fighting chance.
I think the challenge with AST editing is discoverability. Like, I know what I want to select, I can see it there on the screen, but I don't know its name.
I've been dreaming of writing a plugin that surrounded the cursor in differently colored scopes. So instead of "next function" I'd be thinking "next blue" (blue being the color that functions are currently painted in).
I think there's still value in generic AST-level operations. Like expand selection and shrink selection. Or select around the current node vs select inside (whatever that means for the current node type).
In Ki you don't have to know the name of the syntax node, you can just press `d m`, and the editor will show the labels of all the syntax node visible that you can jump to.
This stuff reminds me of intentional programming (Charles Simonyi). That never really panned out but the premise with intentional programming was that programming is about AST transformations that are correct (sort of like refactoring) but all the way down to the machine code level. You get these increasingly more complex languages that are defined in terms of building blocks that are ASTs.
Refactoring of course is based on the same notion that manipulating code correctly at the AST level is a lot easier than messing around with text files. That came out of the smalltalk world originally which also had very fancy IDEs already in the nineties where everything (including the IDE) was something you could mess with. Refactorings were defined as syntactically correct transformation of one AST into another (rename, extract class, move function to parent class, etc.).
Charles Simony had a solution for what you describe as well. Just serialize stuff to text if somebody wants to "edit" and then parse it back when they are done. What happens to the text in between of course is the programmer's problem.
> Refactorings were defined as syntactically correct transformation of one AST into another
Refactorings preserved behavior.
Maybe that isn't what we wanted:
"A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns.
Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary.
A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer.
A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation.
Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period.
If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules.
The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36."
from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002
> Just serialize stuff to text if somebody wants to "edit" and then parse it back when they are done. What happens to the text in between of course is the programmer's problem.
I think that's working against the way that a lot of people write code.
You need things like syntax highlighting and code completion even while typing something like "my_var = " on a line. A normal parser doesn't work here, and bailing out to make it the programmer's problem leaves the programmer with a lot of work
I don't like the positional keybindings. There is no real difference from regular keybindings configured with profiles for Qwerty, Dvorak, etc. In practice, it just means presenting them drawn into keyboards in a way that is hard to quickly search or use in the terminal and anywhere really. Where's the "line" key? Search the drawing! (Oh... it is there in th top left corner, but which exact key?) I prefer a list. So, really, it is not even that big of a "special thing" except the presentation.
I just want to ask a practical question to people who are familiar with Vim and have already tried Ki, what are your thoughts on it? Is it easy to get used to Ki's keybindings?
I always liked Lisps for how easy it is to edit them with proper editor support such as Paredit in EMacs. Always wanted something like that for more syntactically dense languages.
If it’s all tree-sitter based though I think it ought to be possible to do this in EMacs too.
The problem with editors is that each one has a huge cognitive load for minimal benefit. I can't imagine learning this one just to be able to move blocks around easier.
At this point you can just let your IDE's AI refactor any code by just telling it what you want to do.
True, but muscle memory and a couple keyboard shortcuts (or heck, even using the mouse to select and drag the block) is always going to be faster then describing the changes you want and reviewing the output, at least for simple stuff.
/* A source code editor that operates directly on the AST, and affords different visual representations on top of it, was the topic of my (unfinished) PhD thesis about 30 years ago. */
I’m wondering who else recalls Charles Simonyi’s Intentional Software efforts at similar concepts? Not completely novel even then (1995) but mature for the time.
> Note that the Ki keybindings cannot be simply implemented in Vim/Helix via key- remapping, due to the lack of the concept of Selection Mode, and implementing that requires major architectural changes in the core.
Not by default, and it's not a neat and tidy solution, but you can rebind keys as a result of inputing other keys in neovim via a callback (and edit the statusbar ofc to display said selection mode)
This, alongside the use of a variable means you can, in fact, build your own selection modes!
Likewise, nvim does have a seperate select and visual mode.
This is primarily designed as an option to replicate the more traditional shift+movement selection and type to replace functionality of something like vscode or notepad, however it can be exploited as an additional layer for storing keybinds in a pinch
Unfortunately however, there's some shortcomings of neovim which make its control scheme impossible (at least not without some workarounds)
One of the biggest is ki's momentary layers feature.
Ki uses the kitty keyboard protocol to detect when a key is being held and when it is released in order to allow for unique actions when multiple keys are pressed simultaneously, and this functionality is a big part of what makes it so ergonomic.
For example, tapping c will copy the currently highlighted text, but pressing c and k simultaneously will duplicate the currently selected text to a new line below the current line.
While I'm currently chipping away at a pull request to introduce similar functionality to neovim itself, in the meantime, I'm needing to work around the issue by making a lot of concessions.
> vim's modal editing is powerful but the movements are text-based.
Eh, v + showmatch / % gets you very far in languages that use ()[]{}<> (angle brackets you have to add to matchpairs). It gets you nowhere in Python though, not without plugins.
In python you can use {} to jump paragraphs. It's not great since it's based on blank lines and you'll probably need a few for a whole indented block, but it's built in and nicer than going line by line.
The "First-class syntactic selection" reminds me of my most used shortcut(s) in Jetbrains IDEs: the Expand / Shrink Selection.
https://www.jetbrains.com/help/idea/working-with-source-code...It really changed my perspective on interacting with the 'text' of a file.
VS Code, Zed, etc. have similar operations, but in my experience they expand and shrink too coarsely.
Mine are:
Cmd+Shift+V - Stacked clipboard, you can start typing to search or hit a number to choose what to paste (keeps everything you've copied/cut inside jetbrains for a while)
Cmd+Shift+E - Recent locations, you can start typing to search - shows little buffers of where you've been recently
Cmd+Shift+A - Action tab of the command palette - fuzzy search for any command (really the only shortcut you need, other than maybe Shift+Shift for main command palette shortcut)
--- Through the Action bar...
Local History / Local History of Selection - you can start typing to search quite far back the history of all changes of the current file or selection - you can also right click a folder or the project and do the same. Much finer grained than git.
The general concept of being able to search for something and edit directly in the buffer of the search results.
Hero! I had not done my homework/have not been aware, but these all look fantastic! The stacked clipboard is something I periodically mentally complain about (Why is clipboard on every OS/tool I've used single item?)
I will add one that are possibly more well-known:
Windows has had a pretty usable stacking clipboard for a while! You just have to activate it. Since you can pin thing into it it’s also quite useful as a rough and ready way to type special characters you use frequently.
Wow. Looks to not even be a PowerToys feature. Win + V
I use shift+shift and type for all three of these.
Although i do use cmd+shift+r for global replace
> Why is clipboard on every OS/tool I've used single item?
The wonderful Flycut (https://apps.apple.com/us/app/flycut-clipboard-manager/id442...) fixes this on macOS.
Mathematica is the earliest thing I am aware of with this feature where it was Alt+. to expand selection in their notebook interface starting in the early 90s. But the thing I miss most that I still can't shake the muscle memory of after almost a decade of not using much Mathematica, is that single/double/triple/n-click scaled this way as well. So double-click selected a whole word (as in all editors), triple-click selected all the comma-separated multiple args of a function, 4-click for f(a,r,g,s), and so on.
Ctrl-Space in TeXmacs, where the document is an actual tree (http://texmacs.org)
Also available as `incremental selection` in Neovim via tree-sitter.
yep and most of these actions can be implemented with simple mappings based on https://github.com/nvim-treesitter/nvim-treesitter-textobjec...
I work on AST based revision control. I have a stack of ideas on how to achieve the same Ctrl+W effect with commits/diffs/cherry-picks. All still in flux. If you have some thoughts to share, please do.
[1]: https://github.com/gritzko/librdx/tree/master/be#readme
I use it constantly in helix too. The vscode one is meh. I think I saw a discussion in github once about switching to tree-sitter, which would improve AST-related actions. I don't think it went anywhere though.
I love AST aware editing. I think it's one reason it's always been so nice to edit lisps. Stuff that is complicated to describe in javascript (and doesn't have LSP support) p much requires a whole AST parser, but in lisp it's just a simple list operation. When I go back typescript after a weekend of clojure, I reeaally miss slurp! and other paredit commands
JetBrains also experimented with AST-based editing: https://www.jetbrains.com/help/mps/fast-track-to-mps.html#st...
An overview video: https://www.youtube.com/watch?v=XGm_khXZl44
I tried it, but it just was too clumsy. Sometimes refactoring/editing needs to go through phases where the AST is invalid, and MPS makes that just too clumsy.
But with AI this might be a different story.
I agree, top feature. Combined with things like "extract method" makes mundane refactorings super fast.
Yes, Java IDEs have had these since sometime in the 2000s.
And I am sure that people have been complaining about the hand gymnastics you have to do to press those shortcuts since around that time as well
It’s still some time in the 2000s and will be for the next 974 years
Not according to common usage: https://en.wikipedia.org/wiki/2000s
To me, it feels like Zed and VsCode perform most operations in a general way on the text; they don't seem to (in Python and Rust at least) have an understanding of the code structure in the way JB does. (And based on some digging on Ki the way it does as well?) So, I would bet they are using that text-based model, which would be hit/miss here.
One problem is I got so used to Ctrl-W that I use it in other applications and usually wind up inadvertently closing the tab.
I had this issue too, so I remapped Ctrl-W/Shift-Ctrl-W to Ctrl-\/Shift-Ctrl-\ . (Also git operations became two-key sequences, starting with Ctrl-G and that damn Ctrl-K stopped being the shortcut for commit.)
This has got to be my favourite feature of IntelliJ, along with the dumb context actions menu
Cool feature. what's the key for it in visual studio code?
Expand Selection: Alt+Shift+→ (Windows/Linux) or Option+Shift+→ (Mac) Shrink Selection: Alt+Shift+← (Windows/Linux) or Option+Shift+← (Mac)
Ty! I have been missing out. Adding this to the repertoire.
Ah yes.
The shortcut I use the most in Jetbrains IDEs. Also the one I miss the most in VSCode (whatever is present there just doesn't seem to work right).
Also the shortcut that has caused me to close so many browser tabs inadvertently...
Many years ago, I created an editor operating on syntax trees that I think is more "hard-core" than this - that is, only tree-oriented operations are done. There is no parsing of text, since entering plain text, rather than a tree, is impossible. Hence, there can be no syntactically invalid programs.
The challenge is getting this to be a useable way of entering programs. I think I made progress on this, but the feasibility varies with the programming language.
I can't run it any more, since the display hardware it assumed is no longer available, but you can read about it at https://ucalgary.scholaris.ca/items/da8b823b-c344-4ffb-aa37-...
> The challenge is getting this to be a useable way of entering programs.
Well exactly.
When the path between Program A and Program B can only be valid programs, you are going to end up with either a much longer, less intuitive path, or deleting everything and starting again. It can also be quite possible to invent structures which are valid but have no valid path to creating them.
Well, I think most common transformations work reasonably well. One usually doesn't want to do things completely contrary to the AST, such as convert "while a<b" to "whilea = b".
And it's certainly possible to create any valid AST in the editor I describe. The set of valid trees is extended to those with "holes" in places, which one fills in when entering a program, and it's always possible to do this.
The challenge is one of finding an intuitive user interface, not whether it's possible at all. One issue is that infix notation is unnatural for entering trees (prefix is more natural).
no syntax error editing seems like https://scratch.mit.edu/
> It can also be quite possible to invent structures which are valid but have no valid path to creating them.
I'm curious if you have an example of such a structure?
Pedantically: if, for every valid tree, there exists a bidirectional path to the empty root node, there's always at least one path between all given pairs of valid trees ... albeit one that no developer would ever take.
You are quite right to call this out - and quite right in general. With AST verification alone your statement holds strongly.
I was remembering a professor I had who was very into formal verification and had an IDE that would only accept valid programs - however his validity checks were more than just tree based, they involved passing a type check. Which now you've called me on it, is quite definitely beyond valid AST editing.
The base case: if you have a structure with a mandated cyclical reference that doesn't accept a second (e.g. nil) type, you cannot construct it without creating an intermediate invalid program. This doesn't tend to show up if the cyclical reference is all the same type (A1 -> A2 -> A1) because you can often cheat and self reference, but it does if it is different types (A1 -> B1 -> A1). You can't construct an A without a B, which cannot be constructed without an A.
But that's still not a problem you cry! And quite right, you can edit the type itself to remove and re-add the reference.
That is until the type is built into the language, or a library.
OP clarified, but I understood that as a valid tree without a path to empty node. I don't have an example, but with grammar weird enough it sounds plausible. Especially if some form of type checking is involved, as OP mentioned.
One idea would be for it to work like code completion. Once you start writing a structure the rest is auto-suggested so it does not break the tree.
Pantograph[0] seems to be a more recent attempt to implement the same idea. It is still not a general editor but generalizing it to ranges of tree selections looks promising
[0]: https://pantographeditor.github.io/Pantograph/
There are more examples in https://github.com/yairchu/awesome-structure-editors
I actually used a language and editor like this in a previous large company.
It was an experimental language that they wanted to replace their current application level language and was built ontop of JetBrains MPS https://www.jetbrains.com/mps/ which has that feature among others.
My personal opinion after working with that lang is all of this is that its theoretically interesting. But a dead end in practice along with visual coding etc.
The universality, simplicity, and legibility of text as interface for both humans and machines is too hard to beat. I think LLMs is just the most recent example of this.
Things that don't work in practice:
- You need a special editor, its heavy and slow, you lose all the ecosystem around them.
- You can't just cat or inspect the raw file, you can't see it in terminal at all.
- You need a new version control system, review system, and people need to learn it.
- You can't use any existing tools for working with code, you are essentially starting from scratch, and lose all the benefits of all the work everyone else is doing around tooling, dev saas etc.
- humans don't think in trees, they don't think in syntactically correct programs. Its actually absurdly frustrating writing a program in only syntactically correct edits. 99% of the time you are coding, your work is probably syntactically and semantically incorrect.
The tooling that is able to either make the right tradeoff around strictness or allow the users to make the that tradeoff is what ends up being used in practice. A good example of this typescript with gradual typing, python with type annotations, etc.
I think these editors just fall in a bit too far right on the strictness scale.
These are good points, but are mostly pragmatic, resulting from the rest of the system not being designed around a tree representation. Of course, that can be decisive in practice...
A more fundamental issue (which I mention in the document) is that there is a discordance between the 2D textual display and the underlying tree. I think this becomes more apparent when input is not only keystrokes, but also mouse positioning.
Of course, these disadvantages do not necessarily outweigh the advantages of editing in terms of trees!
What was the issue with MPS? MPS also looks like "custom widgets in normal text-based java" i.e. mixed paradigms and not "ground up ast-based editing".
I'm not sure if I can agree with your other points. I can't think of the last time I cat'd or maniuplated rust/java/go outside a heavy and slow editor. Many languages have more or less "one" editor with all the features that most people use, and often use it for change review too. And it seems weird to characterize needing new tools as a "dead end"... all languages need new tools and won't have them at the start.
Plenty of people use go, rust, and java which are not gradually typed.
Nothing wrong with MPS. I think it was just the language itself was design around ast editing, and projectional editor.
I was really excited about the concept and read some papers on it at the time. Then I ran a real world application of it, and writing code in it was just a bad experience.
I think I didn't explain the gradual typing properly. I mean the broader concept of moving developers from one way of doing things to another.
When the vast majority of people and tools are working a certain way. Introducing a whole new thing that requires them to work a whole different way is a hard sell.
So having a way for people to move gradually from point A to point B at their own pace and pick up benefits along the way is really nice.
One example of this is Typescript with gradual typing. Swapping form js to typescript is fairly easy. You can still use your old tools and over time you can start to adapt new features into your code, and over time everyone gets used to this new stricter/better approach.
I think ast based code like this essentially requires you to throw out everything all at once, not just editor, but literally everyone's workflow today and rebuild from the ground up.
I just don't think it will ever be more than a niche thing unless it can loosen up on strictness and play nice with existing tooling.
You laid out the problems nicely, but these are all fixable problems.
If you want to relive it then simh¹ with mame² might be an option. There appears to be some support for VT11³, along with docs⁴ to use as a starting point.
No, I'm not claiming to have read all one hundred pages already. However, from what I have read I'd love to see a functional demo.
¹ https://simh.trailing-edge.com/
² https://www.mamedev.org/
³ https://github.com/simh/simh/blob/master/PDP11/pdp11_vt.c
⁴ https://wiki.mamedev.org/index.php/MAME_and_SIMH
Now you're out of the realm of Ki, but what you're talking about is still being worked on in the modern era, by me! I'm building BABLR which is a modern follow-up to your idea, built on top of a powerful, generic system of parsers which has gaps/holes but no error recovery, so that it works with trees which may be incomplete but which must not be invalid.
The hard part is that we need to be able to talk about these structures. Even just here on this forum we need to be able to communicate precisely about them. I often use · as a typesetting symbol so that I can easily write and read expressions like 2 + · which you would read as "two plus gap". The · symbol is only for typesetting as I say thought because you it's not safe to assume that any one character is reserved for our use in every programming language. Instead we wrap the parts that aren't syntactic in quotes and we use <//> as the symbol for a gap so that it looks more like this:
The * there is a flag on the node, it means this node a leaf of the tree -- a token.We can parse 2 + · into a proper tree now:
And yes, BABLR can really parse this. If we've piqued your curiosity, our Discord server is currently the hub of our community and we'd love to see you. https://discord.gg/NfMNyYN6cXWow! I stumbled onto your paper a while ago when I was looking into structural editing and thinking of a masters in CS, exploring editor interfaces / feedback loops.
(Maybe your paper is famous and it’s not wild that I read it, but it was wild to see after so many years)
I never took that path, spent time in tech industry confused why people didn’t seem interested in structural editing and better editing tools.
Out of curiosity, how do you think LLMs and genai affect the value of structural editors and similar tooling?
Part of me wants to stay disciplined— of course it’s valuable to work efficiently and work on the AST and with a repl. The other part of me gets paid to work on essentially a punch card system (building dev, ship to prod and see what happens)
I feel so illiterate when it comes to AST editing sometimes. I understand what an AST is from a computer science perspective. But I've never worked on a huge software refactor before that required direct AST textobject editing. Maybe an indication of my skill level...
The extent of my usage is having nice textobjects to easily interact with arglists and functions which aren't native to (neo)vim. Very cute and nice to just write "daf" somewhere in a function and just have it "just delete". Or hook it up with basic macros: search for regex, "daf".
I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).
As for LSP's, I do use go to definition and rename all references, which is nice. But the huge structural refactoring part I have never really done. I don't really use many LSP features besides those two either...
Basically, I gotta up my editor game.
> I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).
This is actually what's nice about tools like ast-grep. The pattern language reads almost like the code itself so you can see the transformation right in front of you (at least for small-scale cases) and reason about it. TypeScript examples:
The $X, $A etc. are metavariables that match any AST node and if the same metavariable appears twice (e.g. $X = $X ?? $Y), it requires both occurrences to bind to the same code so `x = x ?? y` will match but `x = y ?? z` won't. You can do way more sophisticated stuff via yaml rules but those are less visually intuitive.Sadly coding agents are still pretty bad at writing ast-grep patterns probably due to sparse training data. Hopefully that improves. The tool itself is solid!
While at it, https://github.com/semgrep/semgrep was around for several years, too.
Came here to recommend ast-grep too!
If your editor of choice supports an extension (vscode does for example) it's a very easy on-ramp for a better search/replace than regex offers. It's syntax-aware so you don't need care about whitespace, indentation etc. Very easy to dip your toes in where a regex would get complex fast, or require multiple passes.
I converted a codebase from commonjs to esm trivially with a few commands right after first installing it. Super useful.
I hope LLMs eventually start operating at this level rather than raw text. And likewise for them to leverage the language server to take advantage of built in refactorings etc
I don't know which editors support this but there's also, for lack of a better word, context aware grep. For example, search and replace foo with bar but only inside strings, or only inside comments, or only variables, or only methods, not class names (maybe ast-grep does this).
TBH, it's actually not as hard as you think, most of the time, what I do is just select the whole syntax node and delete it, copy it, or replace it, and only 20% of the time would actually require deliberate understanding of how the AST is structured in the current language I'm coding in.
Maybe, or maybe not. I think there are many different kinds of work all lumped together under the "software engineering" umbrella, and tools which are indispensable for some people can seem unimpressive or unnecessary to others. We're not all spending our time in the same way, working on the same kinds of tasks.
I don't find myself doing huge structural refactoring often enough that it would be worth my while to learn a specialized tool for doing it; I'm a quick typist, and it's easy enough to just blat through the changes by hand on the rare occasion such a thing is necessary. But I don't work on giant line-of-business apps or sprawling web services, and I can see someone who did a lot of that sort of work having a different take on the matter.
I feel the same way. Learning Elixir and getting into macro writing was really helpful for me. Lisp is also the same way though for me the syntax of Elixir resonated a lot better for me. Many paths to the destination and all that, but figured I'd share in case it's helpful to someone
In my classification of editors:
1. Orthodox. Mostly focused on looks and integrations.
2. Modal, Vim improvement. Focus on keeping basic Vim keybindings with minor improvements.
3. Modal, rethinking Vim approach.
Ki falls into the third category which I constantly monitor.
4. All of the above.
Which is Emacs.
Right, but we are discussing editors here.
Hold on. Operating systems and long-running esoteric history are always on topic here, right?
(This is spoken with something close to affection; I look fondly on my former Emacs days. I'm probably more likely than not to enjoy the company of a human who has at least tried Emacs and had some thoughtful reaction to it. I may not use it now, but the ideas live on in how I think and what I expect. Editors that push the limits, whatever those limits are, are part of the old school ethos I love about programming.)
Evil mode is a perfectly reasonable text editor.
I know, I use Emacs+Evil. I'm going to give Ki a try and, in case of success, to think about writing Ki binding for Emacs.
emacs already has structural editing packages like combobulate
Hey, that one is interesting, somehow I have missed it. Thank you!
This is the first time I've heard about Emacs trying to look nice.
"Emacs is an editor" is as true as "Toyota Hilux is a gattling gun carrier"
>_<
Vim is Emacs applied to Vi
Nah. NeoVim a tiny little bit, maybe.
Not even close.
How is it not?
Scripting language? Check!
Custom commands? Check!
Windows management? Check!
Build tools integration and error-based navigation? Check!
File manager? Check!
I think the "Scripting language? Check!" is missing an important distinctions between Emacs and most other extensible editors. The proportion of the code editor features that are written in the configuration/extension language matters
Looking at the Emacs and NeoVim codebases in GitHub: Emacs is 74.6% ELisp or CommonLisp and NeoVim is 71.9% Lua or VimScript. Never mind, NeoVim is close!
But, can you at runtime modify the NeoVim core functions? Honest question because I've only used vim and VimScript was definately limited compared to elisp. If so, then I say NeoVim and Emacs are both highly-extensible editors.
None of those capture the essence of emacs, which is that it is a scripting language repl where the vast majority of functionality is implemented in that language and can be inspected, debugged and modified at runtime.
Emacs is a general purpose programming/computing environment that ships with a text editor and some other nice features. Vim is a text editor and some other nice features that ships with a scripting mechanism. Not the same thing.
Operating system? Check or not? :D
Vim has stopped before that particular threshold. But if ever Neovim get a GUI version, I believe the community will soon have its own Gnus, Eww, and EMMS.
Vim improvement
So Ed Visual Mode Improved Improved!
I am working on a modal code editor project that you might find interesting then. It also operates on an AST directly, which is represented as UI nodes which closely resemble normal text layout. Email in profile if you’d like to give it a try and possibly give early feedback (still in early development).
Exactly the same for me. So far, there have been many challengers but the same champion (subjective, I know).
I made the vscode integration for this. I feel bad that I haven’t contributed much since, it’s a really cool project. IMO it’s important to try to innovate in the foundational tools of our craft (editors, languages, tooling, OS, etc) which Ki does.
This one?
https://marketplace.visualstudio.com/items?itemName=ki-edito...
Yeah that’s it. Bundles the editor binary with the extension so you don’t have to install it separately.
Thank you for that! I find it really daunting to try a new editor, but having a vs code extension as an entry point really helps me and increases the chances of trying it out soon!
I didn't fully understand until I saw the examples
> First-class syntactic modification
> Notice the comma between the current and the next node is also deleted. > Notice how comma is added automatically.
This is awesome! And I bet it arguably requires less logic to do so as well. Cool stuff.
Now I'm wondering how much effort it would be to get a ki integration (or at least an AST-first rewrite) in Zed
I wrote the vscode extension for Ki. It would almost certainly be usable with Zed since it talks with the editor over a websocket. Check out the code in the Ki repo.
I can't wait to try it. Love that it's keyboard layout agnostic. A lot of other good sounding ideas in the docs. Especially inspiration from Emacs as everything being an editable buffer. There's always some massive tradeoffs between editors though. Guess I'll have to see.
It's a shame that editors are build around the editing model, so much time has been wasted by recreating everything. Neovim is a perfect editor in many aspects but editing model is not one of them, in my opinion. Helix or even Ki would not be needed if Neovim allowed to completely replace the vim part of it with plugins
Can you elaborate on how this might look, both from an abstract perspective, and (if you've thought about it) from an API perspective?
I didn't really spent time thinking about it, because from reading their issue tracker, I very much doubt neovim would consider something like that. Partially because of how much of it is already vim specific and it would be very difficult to abstract that away, but mostly because of ideology.
You can already to a lot in Neovim with plugins, including recreating Helix, Ki, whatever. But the problem is that the vim state machine is always present underneath it, resulting in unwanted behaviors or impossibility to implement some things. I would envision it to pretty much allow to listen on raw keyboard and mouse events. However cursed it sounds, it should be possible to fully implement vscode or any other editing model in neovim
> I would envision it to pretty much allow to listen on raw keyboard and mouse events.
This is pretty much how the BRIEF editor for programmers worked, which was used in MS-DOS around 1990. In my opinion this was one of the best programs that I have ever encountered and I have never encountered again some of the good UI features that existed in the BRIEF text editor and in the XTree file manager, both for MS-DOS.
BRIEF had a tiling-window user interface with optional menus and it came with a default behavior, but it could be customized extremely. For each kind of key press or mouse event you could bind a function to be executed, which replaced the default behavior. For instance, the default behavior for pressing most keys was to invoke a function "self_insert", which inserted in the current text buffer and the current cursor position the character corresponding to the pressed key, but it was easy to change the behavior to anything else, e.g. to do auto-completion or inserting some program template.
The functions bound to events were written in a custom language similar to LISP, so in this respect BRIEF resembled Emacs. A great number of text manipulation functions were provided, so it was easy to describe complex actions in the event handling functions.
This was what I liked most at BRIEF, that by rewriting all the event handling functions it was very easy to convert it in an editor that no longer resembled the default BRIEF editor at all, but which behaved in whatever way you believed that an editor for program sources should behave.
So the text editor provided an extensive library of functions for handling text, text buffers, windows and files and a LISP-like language for gluing all these into a text editor. It also had a default implementation of a text editor, to be able to use it immediately, but that could be replaced partially or totally, when you had more experience and you knew what kind of behavior you want.
I assume that Emacs can be customized in a similar manner with BRIEF, but when I looked at Emacs it required a much greater initial effort than BRIEF, to reach a stage where you are able to modify it.
The default editing mode of BRIEF was very intuitive, at least for someone familiar with the IBM PC keyboard, so you could use it easily out of the box, before learning to customize it. Then, like most MS-DOS programs of that time, it came with excellent documentation, so it was very easy and fast to learn how to rewrite its event handling functions. With Emacs, both learning how to just edit texts with it, and then how to modify it according to what you prefer, is much steeper and more time consuming than it was with BRIEF.
I'm not sure how that's not "recreating everything". What's still shared if you replace the whole state machine (handling file management, etc) and also replace the whole input handling system? A terminal emulator and a copy of Lua?
There are certainly different possibilities of where the line would be drawn. I am not that familiar with vim internals, but I don't consider file management to be part of vim editing model. And we are only talking about editing model (e.g. b being beginning of a word), not the whole editor (plugin system, ui, lsp and treesitter integration, etc). Though it is true that some parts are more tightly related to "vim". There some cases were Neovim already wants to disconnect itself from vim legacy like allowing to scroll viewport without moving the cursor
By "recreating everything" I was also referring to the ecosystem around the editor
But doesn't the ecosystem around the editor depend on the editor being, at its core, Vim? https://neovim.io/doc/user/lua-guide/#lua-guide-api describes "three different layers": basic examples like https://neovim.io/doc/user/lua/#watch-file call into the Nvim layer.
I think the idea's good, but I don't think you could modify Neovim to support it while retaining the ecosystem – not without also modifying much of the ecosystem, at which point it may be easier to do a straight port to a different editor, or construct a compatibility layer (essentially reimplementing enough of Neovim in the new editor that the ecosystem still works).
Add that to a long list of side projects I would like to do "at some point"
> And we are only talking about editing model (e.g. b being beginning of a word)
nmap can shadow the built-in commands, replacing them with whatever function you want.
> There some cases were Neovim already wants to disconnect itself from vim legacy like allowing to scroll viewport without moving the cursor
I think you mean beyond where the cursor is so it's hidden, but you can jump the screen so the cursor's current position is at the top/middle/bottom without moving the cursor using zt/zz/zb.
An editor that tries to be layout agnostic is almost certainly going to be a nightmare for people like me who set the layout in the keyboard itself. I downloaded the editor (I'm on Windows at the moment), and tried it with my keyboard set to Dvorak, which was plainly broken. I'm sure there's a way to fix the mapping, but when software thinks it's smarter than you, you end up feeling pretty dumb.
Have you used `*` to pick the keyboard layout?
As always emacs has it already
https://github.com/mickeynp/combobulate
Yeah and using combobulate feels so natural, I just wish it was easier to add support for other languages. In addition to the navigation, and simple edits (e.g. M-k kills node) you have Searching/editing/etc. using tree-sitter queries directly within emacs, possibly replacing other uses of external tree-sitter based tools.
It makes editing supported languages almost as pleasant as editing Lisp.
With how well the integration already work in existing editors, I'm curious how the UX can be improved further by an editor made specifically with AST editing in mind.
Vim-like (terminal and VSCode extension) that prioritizes syntax-based navigation.
Comparison to Vim and Helix: https://ki-editor.org/docs/comparison#user-content-fn-1
That comparison table is strange and sometimes wrong. Neovim for example detects and updates external file changes by default. And the coherence of the keybindings in Ki is "Great" bit vim/helix:
> As you can see, there's no single logical categorization for these keymaps, they are either lowercase-uppercase, normal-alt, left-right bracket, or outright unexplainable.
Word, End, Back, Change Word and even Change Inner (, etc are very logical to me and I feel like I'm talking to the editor when editing. I get that it doesn't make sense when one has learned another way to do it, but it does make total sense you just have to make an effort to try and understand it.
It's like learning and always driving automatic then calling manual "outright unexplainable". You simply learned another way and are conditioned into believing that's the one true way. It shows the creator comes from VSCode (multi-cursor is a useless feature, just use s/search/replace and get used to macros and a whole new world will open).
Re. incoherence of vim's keybindings, I partially agree.
Most of the times, shift means "bigger", and only in a few places it means "invert".
Examples of "bigger" are all the motions involving words vs. WORDS, where WORDS are a broader interpretation of words; "V" is like "v" but by lines, thus in larger chunks; "C", "D", "Y" do the same as their unshifted counterparts, but extended to the end of line; etc.
Examples of shift meaning "invert" are fewer, and all sound irrational to me:
With x/X and o/O, the shift inverts the direction left/right or above/below, but we don't have for example i/I to insert to the left/right of the cursor, and H is not the opposite of h, J is not the opposite of j, etc.
With n/N, shift inverts next and previous. Why not n/p? I know, I know: p was already taken for paste, but still...
Finally, t/T/f/F are completely mixed up. In my mental model, t/f sound like to/from and I'd rather use them to move forward/backward till (not including) a char, while T/F would be the "bigger" variants which include the destination char.
i/I is because we already have a, and A is a super-a in the same way I is a super-i.
I think you've touched on it, but I'm going try to take it one step further into explicitness.
Just over a year ago I decided to switch to Neovim. The reason for switching was personal; I was struggling with what I'll call "clutter" in other tools and I wanted a tool that would reinforce, at least lightly, a mode of working that promoted focus on what I was working on, while making it easy to reference other files without loading up my editor with tabs and other visual clutter (buttons/menus) I don't care about most of the time.
I took the advice I seemed to bump into repeatedly: try out vim mode in my current editor before making the plunge.
I really struggled at first. It felt wildly foreign. All the shortcuts were nowhere near to the world I was familiar with.
As I was about to give up, I ran into some advice that was along the lines of "stop trying to memorize shortcuts and start thinking in terms of what you want to achieve" (words and motions in vim-speak).
Your example of [C]hange [I]nner is a great one; that one in particular was life changing. Sure there are some words and motions that do require memorization, but so many others just flow naturally. And once you start thinking in actions, it's easy to see how they can layer on top of each other in really elegant ways.
I'm not even here trying to tout vim-like editors, I'd wager there are many editors that have some semblance of this kind of interaction, but rather to reiterate there's a shift from a PoV of function vs. goal.
Again, I don't think this is "the right way" but rather one of many perspectives that works in context with the phenomenology of me.
I had an experience like this when I switched to a Keyboardio split keyboard. It was impossible to use at first. Then I do some deliberate practice with a typing tutor app and now I love it so much I own multiple.
Is there a good tutorial for some of these advanced text editing features?
In particular I’d like to get platform independent shortcuts / key bindings. I use both windows and MacOS daily, and it throws off my muscle memory for shortcuts like “go to beginning of line”
> It shows the creator comes from VSCode
Hey, one of the creators here, I actually daily drove Neovim for two years, before switching to Helix for a while, then finally Ki.
> multi-cursor is a useless feature
I was a Neovim macro user until I figured out how insane that was compared to multi-cursor after using Helix.
I still think their point about search and replace still stand though. I make most my edits with regex in neovim nowadays and I feel this is the superior paradigm: You don't even need to get to some specific location in order to edit it. I also almost only use search to move around the file and I can even reuse the searches for substitutions. It makes most vim motions and commands almost useless for me nowadays.
I also feel like macros are a more clunky and error prone way to do what substitutions can do. Almost never use them.
Yeah, macros are definitely helpful at times, but not as much as one might think.
I think the same goes for multi-cursor, though.
> I was a Neovim macro user until I figured out how insane that was compared to multi-cursor after using Helix.
Multi-cursor was the first plugin I installed when I moved from VSCode to Vim because I was used to hitting Ctrl+d to select all words and then replacing. Does Helix do something different?
1) First I reach for <C-v> for visual block selection if everything is neatly aligned.
2) Next choice is %s/search/replace(/c if I need confirm).
3) Macros, and I love it everytime I get to use them. I just record the movements, copy what I need to copy, paste it where I need to paste it, and it's repeatable for every line or block where the *formatting* matches. And this is the important part, the words don't matter. I still feel like a wizard using them.
As far as I understand multi-cursor option 3 is a no-go without macros if the words don't match. But macros don't care as long as the movements translate to the same edits. How does Helix multi-cursor work that make macros insane?
Sorry for derailing a bit, the search and replace using a query make sense for purly textual (non-syntactic) editing, but if you want to apply consistent syntactic modifications across multiple locations in the same file, you will need both multi-cursor and syntax node selection/navigation/modification.
It's hard to explain unless you actually try Ki, because it is a paradigm shift
Or you could use grn for LSP rename[1] in Neovim and it will rename all references across the project. VSCode and other IDEs have a similar feature, but compared to multi-cursor that's cheating.
I'll give you that using the AST to select the references is an interesting addition to multi-cursor, but I still don't see how they would be useful compared to my current workflow.
[1]: https://neovim.io/doc/user/lsp/#vim.lsp.buf.rename()
That comparison table is misleading because it mixes implementation details with user-facing behavior, and for example Neovim uses inotify or platform-specific filesystem watchers to surface external changes so marking that row as 'no' is incorrect without platform and config qualifiers.
Keybinding coherence is an engineering tradeoff, and if Ki wants consistent behavior it should expose orthogonal primitives like operators and motions plus AST-aware textobjects using Tree-sitter style node names, then provide discoverability tools such as a which-key overlay and explicit node prefixes so power users get composability while newcomers have a fighting chance.
I'll wait till an Emacs package is available
https://github.com/mickeynp/combobulate
[dead]
I think the challenge with AST editing is discoverability. Like, I know what I want to select, I can see it there on the screen, but I don't know its name.
I've been dreaming of writing a plugin that surrounded the cursor in differently colored scopes. So instead of "next function" I'd be thinking "next blue" (blue being the color that functions are currently painted in).
I think there's still value in generic AST-level operations. Like expand selection and shrink selection. Or select around the current node vs select inside (whatever that means for the current node type).
In Ki you don't have to know the name of the syntax node, you can just press `d m`, and the editor will show the labels of all the syntax node visible that you can jump to.
Oh neat, I'm gonna give it a try!
This is like slurp, barf, splice, raise, kill, cut, yank for lisps.
How does this work in the many cases where in-progress edits result in an unparseable structure?
You just complete to a parsable structure.
This stuff reminds me of intentional programming (Charles Simonyi). That never really panned out but the premise with intentional programming was that programming is about AST transformations that are correct (sort of like refactoring) but all the way down to the machine code level. You get these increasingly more complex languages that are defined in terms of building blocks that are ASTs.
Refactoring of course is based on the same notion that manipulating code correctly at the AST level is a lot easier than messing around with text files. That came out of the smalltalk world originally which also had very fancy IDEs already in the nineties where everything (including the IDE) was something you could mess with. Refactorings were defined as syntactically correct transformation of one AST into another (rename, extract class, move function to parent class, etc.).
Charles Simony had a solution for what you describe as well. Just serialize stuff to text if somebody wants to "edit" and then parse it back when they are done. What happens to the text in between of course is the programmer's problem.
> Refactorings were defined as syntactically correct transformation of one AST into another
Refactorings preserved behavior.
Maybe that isn't what we wanted:
"A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns.
Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary.
A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer.
A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation.
Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period.
If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules.
The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36."
from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002
https://dl.acm.org/doi/10.1145/604251.604258
> Just serialize stuff to text if somebody wants to "edit" and then parse it back when they are done. What happens to the text in between of course is the programmer's problem.
I think that's working against the way that a lot of people write code.
You need things like syntax highlighting and code completion even while typing something like "my_var = " on a line. A normal parser doesn't work here, and bailing out to make it the programmer's problem leaves the programmer with a lot of work
I assume it uses an error tolerant parser, same as most language servers.
Ex: https://github.com/microsoft/tolerant-php-parser/blob/main/d...
I don't like the positional keybindings. There is no real difference from regular keybindings configured with profiles for Qwerty, Dvorak, etc. In practice, it just means presenting them drawn into keyboards in a way that is hard to quickly search or use in the terminal and anywhere really. Where's the "line" key? Search the drawing! (Oh... it is there in th top left corner, but which exact key?) I prefer a list. So, really, it is not even that big of a "special thing" except the presentation.
Have you actually tried it?
A great addition to the helix workflow, I knew something was missing from the movement -> action approach. This completes the trifecta.
There was some Show HN some weeks ago on VCS/diff that operates on AST instead of lines/chars, anyone remembers its name?
There was Weave which just uses git for VCS and tree-sitter for conflicts. Gritzko is on-thread with his Beagle system, and I'm here too for BABLR ; )
Diffsitter and/or weave?
There is difftastic
I just want to ask a practical question to people who are familiar with Vim and have already tried Ki, what are your thoughts on it? Is it easy to get used to Ki's keybindings?
I always liked Lisps for how easy it is to edit them with proper editor support such as Paredit in EMacs. Always wanted something like that for more syntactically dense languages.
If it’s all tree-sitter based though I think it ought to be possible to do this in EMacs too.
The problem with editors is that each one has a huge cognitive load for minimal benefit. I can't imagine learning this one just to be able to move blocks around easier.
At this point you can just let your IDE's AI refactor any code by just telling it what you want to do.
True, but muscle memory and a couple keyboard shortcuts (or heck, even using the mouse to select and drag the block) is always going to be faster then describing the changes you want and reviewing the output, at least for simple stuff.
First impression. Muscle memory is a problem :(
Vim's j moves down.
Ki's j in line mode moves up...
Cannot figure how to create new file
Please feel free to ask for help in https://ki-editor.zulipchat.com/join/zzhagqzl6wyzpqfeqxcsrki... :)
I like this editor, but I have less code to type nowadays.
I don't know what to feel
/* A source code editor that operates directly on the AST, and affords different visual representations on top of it, was the topic of my (unfinished) PhD thesis about 30 years ago. */
What can you tell us about it? What worked and what didn't? What was hard?
I’m wondering who else recalls Charles Simonyi’s Intentional Software efforts at similar concepts? Not completely novel even then (1995) but mature for the time.
Ok ok all talk of the giant heavy java written jet brains IDE which I’m sure is great. What about all the experiences of using Ki and similar editors?
I misread this a little bit, as KI is the German equivalent of AI (Artificial Intelligence = Künstliche Intelligenz).
Have you seen any of the lisp tree editing modes for eMacs or vim?
Why not a vim plugin
We have one underway in https://codeberg.org/alicealysia/ki-bindings.nvim
> Note that the Ki keybindings cannot be simply implemented in Vim/Helix via key- remapping, due to the lack of the concept of Selection Mode, and implementing that requires major architectural changes in the core.
Not by default, and it's not a neat and tidy solution, but you can rebind keys as a result of inputing other keys in neovim via a callback (and edit the statusbar ofc to display said selection mode)
This, alongside the use of a variable means you can, in fact, build your own selection modes! Likewise, nvim does have a seperate select and visual mode.
This is primarily designed as an option to replicate the more traditional shift+movement selection and type to replace functionality of something like vscode or notepad, however it can be exploited as an additional layer for storing keybinds in a pinch
It's funny you mention that. I am currently working on a neovim plugin which implements the ki-editor keymap.
https://codeberg.org/alicealysia/ki-bindings.nvim
Unfortunately however, there's some shortcomings of neovim which make its control scheme impossible (at least not without some workarounds)
One of the biggest is ki's momentary layers feature.
Ki uses the kitty keyboard protocol to detect when a key is being held and when it is released in order to allow for unique actions when multiple keys are pressed simultaneously, and this functionality is a big part of what makes it so ergonomic.
For example, tapping c will copy the currently highlighted text, but pressing c and k simultaneously will duplicate the currently selected text to a new line below the current line.
While I'm currently chipping away at a pull request to introduce similar functionality to neovim itself, in the meantime, I'm needing to work around the issue by making a lot of concessions.
I highly appreciate the Nix-first approach
Does it support C++?
[dead]
> vim's modal editing is powerful but the movements are text-based.
Eh, v + showmatch / % gets you very far in languages that use ()[]{}<> (angle brackets you have to add to matchpairs). It gets you nowhere in Python though, not without plugins.
In python you can use {} to jump paragraphs. It's not great since it's based on blank lines and you'll probably need a few for a whole indented block, but it's built in and nicer than going line by line.
[dead]