compiler
otp
Our great sponsors
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
compiler
-
A copy-and-patch JIT compiler for CPython
Wow! Thank you for your hard work. I use python for all experimental work so this would speed up my scripting work, such as processing data from API calls or filesystem.
I wrote a simple toy JIT for a Javascript-like language. It might be useful for others to learn from because it's so simply written and not complicated. I do lazy patching of callsites, I haven't got anywhere near as advanced as tracing or copy-and-patching. Much of the code I wrote for this JIT was written in Python and ported to C. The Java Virtual Machine has a template interpreter which is interesting to research.
I haven't got around to encoding amd64 x86_64 instructions as bitmasks yet, so I've hardcoded it.
-
Building a Programming Language in Twenty-Four Hours
https://github.com/samsquire/compiler
It's a toy and incomplete but I've worked on compiling MOV and ADD instructions.
-
Let's make a Teeny Tiny compiler
I used this page to work out the Mod/RM format for opcodes.
https://www.cs.uaf.edu/2016/fall/cs301/lecture/09_28_machine...
I have a barebones toy compiler here that compiles a simple mathematical expression to assembly:
https://replit.com/@Chronological/Compiler3
I have the beginnings of a toy JIT compiler here which generates some machine code for MOVs and function calls but I haven't implemented much else...
http://github.com/samsquire/compiler
For code how I generate an "add" instruction:
https://github.com/samsquire/compiler/blob/main/jitcompiler....
I use a case statement to match on the registers and literally insert opcode bytes into a malloced array.
On thing you can do it implement a JIT compiler.
Here's Martin Jacob's code to execute arbitrary memory:
https://gist.github.com/martinjacobd/3ee56f3c7b7ce621034ec3e...
Since your C program is already in memory, you have access to the C standard library and don't have to worry about linking or object formats :-) but you'll have to worry about parameter passing and FFI.
My JIT compiler based on this idea is here https://github.com/samsquire/compiler but it is incomplete.
- How to get started?
- Notes on my incomplete JIT compiler
-
Erlang: More Optimizations in the Compiler and JIT
This is interesting, thank you.
I really should learn from BEAM and the OTP and learn Erlang. I get the feeling it's super robust and reliable and low maintenance. I wrote a userspace multithreaded scheduler which distributes N lightweight threads to M kernel threads.
https://github.com/samsquire/preemptible-thread
I recently wrote a JIT compiler and got lazy compilation of machine code working and I'm nowhere near beginning optimisation
https://github.com/samsquire/compiler
How do you write robust software, that doesn't crash when something unexpected goes on?
I looked at sozo https://github.com/sozu-proxy/sozu
and I'm thinking how to create something that just stays up and running regardless.
- Is it possible to optimize this bytecode interpreter more?
- How do you create a correct AST with interaction between method call and function call?
- How do you parse function calls?
otp
-
Perfect Elixir: Environment Setup
I’m on MacOS and erlang.org, elixir-lang.org, and postgresql.org all suggest installation via Homebrew, which is a very popular package manager for MacOS.
- Epoll: The API that powers the modern internet (2022)
-
Open Sourcing Erlfuzz
- a massive speedup of a common static analyzer for Erlang (https://github.com/erlang/otp/pull/5997)
-
Erlang: More Optimizations in the Compiler and JIT
It looks more like some of the JIT improvements made it profitable to manually unroll some loops in the base64 module: https://github.com/erlang/otp/commit/a03cf1601605dee767cd9d5...
-
Mixing sync and async views in the same application
https://github.com/erlang/otp as far as I know. It's somewhat confusing and I honestly couldn't say exactly where the BEAM VM or OTP or ERTS (Erlang Runtime System) start and end. I've never dug into it. I just install Elixir and sometimes Erlang through the ASDF tool, which does all the compiling for me.
- Erlang: The coding language that finance forgot
-
In Defense of Linked Lists
is Erlang and JDK general enough for you? Oh, both use linked lists,
https://github.com/erlang/otp/blob/master/lib/stdlib/src/que...
- [New] How do you verify program correctness in Elixir?
-
Reaching 100M MQTT Connections with EMQX 5.0
In our initial tests with Mria, without going into too many details, the replication mechanism basically involved logging all transactions to a "phantom" Mnesia table, which was subscribed to by replicant nodes. This effectively generated a bit of network overhead between the core nodes because each transaction was essentially "duplicated". In our OTP fork, we added a new Mnesia module that allows us to capture all committed transaction logs more easily, removing the need for the "duplicate" writes and reducing network usage significantly, and allowing the cluster to sustain higher connection / transaction rates. While stressing the cluster further after those optimizations, we found new bottlenecks that prompted further performance tunings[4][5][6].
-
EMQX 5.0 Released: 100M+ IoT Connections Per Cluster
The latest version adopts a new Mria extension for Erlang’s Mnesia database that increases horizontal scalability by defining two different node types: core nodes and replicant nodes. This new architecture allows EMQX 5.0 to better adapt to increasing demand in IoT networks. The latest performance testing shows it can easily support 100M connections with a single cluster—a 10-fold increase over previous versions—making it the world’s most scalable open-source MQTT broker.
What are some alternatives?
protoactor-go - Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
Elixir - Elixir is a dynamic, functional language for building scalable and maintainable applications
calypso - Calypso is a mostly imperative language with some functional influences that is focused on flexibility and simplicity.
scryer-prolog - A modern Prolog implementation written mostly in Rust.
caramel - :candy: a functional language for building type-safe, scalable, and maintainable applications
cdk-emqx-cluster
emqtt-bench - Lightweight MQTT benchmark tool written in Erlang
pwntools - CTF framework and exploit development library
emqx - The most scalable open-source MQTT broker for IoT, IIoT, and connected vehicles
aws-cdk - The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
glibc - Unofficial mirror of sourceware glibc repository. Updated daily.
gradient - Gradient is a static typechecker for Elixir