Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
llvm-project
The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
-
Let's give them the benefit of the doubt, and let next actions speak for themselves.
I've created an issue in their repo, so they are fully aware of my comment and they have the opportunity to amend this issue :)
https://github.com/microsoft/vscode-wasm/issues/62
> WASI stands for WebAssembly System Interface. It's an API designed by the Wasmtime project that provides access to several operating-system-like features, including files and filesystems, Berkeley sockets, clocks, and random numbers, that we'll be proposing for standardization.
Read more here - https://github.com/bytecodealliance/wasmtime/blob/main/docs/...
Getting a FS going on the web has been such a brutal slog for me. There's some pretty decent existing projects lole lightning-fs but it feels like the web APIs have been changing & changing & I keep wanting newer.
Very cool to see this nicely tackled. Still poking around but seems integrative with wasi, which hopefully grows closer & closer with good fast web APIs. There's been a ton of criticism I think justly so that neither the wasi nor the web fs teams have spent much time trying to figure out how to work together, & so there's real peril possible too, but I think wasi is a much better much less hacked out model & is probably the "right" real target. Having build in object capability sandbox security puts it lightyears beyond what the web spec folk have been up to, and the rest of their fs model is also much much better built out.
Alas it just means that running on the web the target so going to be primarily in-memory for a long time because the web spec folk seem so uninterested in doing a good actual filesystems impl. Origin Private File System seems to be the one place there's interest in being fast at all, and they seem overwhelmingly concerned about making sqlite in emscripted fast & not a ton else. What we would do for real wasi on the web!
Already a very solid start building out coreutils. There's a bunch of similar ish efforts out there already, building JS impls of core utils. For example, posix-tools, https://github.com/shama/posix-tools
>Using compilers for highlighting basically doesn't happen because they are too slow.
This is not completely accurate, as clangd does support this: https://github.com/llvm/llvm-project/blob/main/clang-tools-e...
Getting a syntax tree from clangd is faster than you might think because it just needs to hook into the lexing/parsing portion of the compiler, which is actually pretty fast and much, much faster than actually compiling code. And generating fixes and warnings using clang-tidy (which can be built as a standalone binary, but is typically just linked into clangd) requires parsing and lexing anyway, so the parse information is basically "free" since it's work clangd needs to do regardless. And at least for C/C++ you can't really do accurate parsing without the compiler anyway due to the fact that preprocessor macros can do nearly anything. Even if your preprocessor macros are generally doing sane things and could be parsed without the compiler, most people want to see things like #if/#else/#endif branches highlighted so they can see which code paths the compiler is actually going to select when it compiles code, and again that requires actually running a full preprocessor with the same flags as the code will use when it is compiled.
That being said, I believe that most editors do a hybrid approach to lexing/parsing because getting syntax information entirely from clangd is too much to ask. So basically the editor will do a best-effort parse in real time of your code using some built-in syntax highlighting engine (possibly tree-sitter, possibly something else) and then once clangd sends syntax information they'll update the document as necessary.
Not true of most compilers.
Can you design compilers that can do this? Sure, i was at IBM when we did visualage, which could do this in other ways, though not optimally.
Is it common? Not a chance.
Your claim that increasingly more modern languages have semantic highlighting in real time is simply false - most cannot real time semantic highlight even a 100k file on every keystroke. There are a very small number which can, and it's mostly by luck - they fall down on larger files because they have no incrementality. Meanwhile, this is trivial with tree-sitter for all languages because of it's optimality. If you want to see it in action - turn on semantic highlighting for vscode and type fast - most of the time you will lose syntax highlighting because things can't keep up. see, e.g., https://github.com/microsoft/vscode-extension-samples/issues...
Again, yes, you could make a compiler for every language which supports a mode that does what tree-sitter does. and people could carefully implement parsers/lexers in each of their favorite compilers and languages that support optimal incremental parsing in them. and then pay the cost of integrating 50 language specific ways of transforming these to work with the editor, basically reinventing what tree-sitter already did right. (As per above, the current semantic token and highlighting support does not resolve this).
You seem to really otherwise be complaining that you believe it does always generate correct parsers. Like I said, that seems totally orthogonal to anything about the speed issue, and if that's your real concern, have at it.