Pyston VS jax

Just for reference,

* Nuitka[0] "is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, and 3.11."

* Pypy[1] "is a replacement for CPython" with builtin optimizations such as on the fly JIT compiles.

* Cython[2] "is an optimising static compiler for both the Python programming language and the extended Cython programming language... makes writing C extensions for Python as easy as Python itself."

* Numba[3] "is an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code."

* Pyston[4] "is a performance-optimizing JIT for Python, and is drop-in compatible with ... CPython 3.8.12"

[0] https://github.com/Nuitka/Nuitka

[1] https://www.pypy.org/

[2] https://cython.org/

[3] https://numba.pydata.org/

[4] https://github.com/pyston/pyston

and if you need more speed you can try https://www.pypy.org/ or https://github.com/tonybaloney/Pyjion or https://www.pyston.org/

What efforts have been done come with limitations. PyPy is mostly compatible. Pyston seems mostly compatible but offers only modest speedups. IronPython and Jython run on the .NET and Java runtimes, respectively. They’re JITed as a consequence of that, but that also means they’re stuck in those environments and don’t work with CPython modules that use native code.

https://github.com/pyston/pyston/blob/69b190003f14dfd2f6d276...

Seems easier to use the C functions to do this, rather than rely on system commands.

Releases · pyston/pyston

root@julia-75444d5c79-686cf:/# curl -LO [https://github.com/pyston/pyston/releases/download/pyston\_2.3.2/PystonConda-1.1-Linux-x86\_64.sh](https://github.com/pyston/pyston/releases/download/pyston_2.3.2/PystonConda-1.1-Linux-x86_64.sh) % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 670 100 670 0 0 8072 0 --:--:-- --:--:-- --:--:-- 7976 100 88.2M 100 88.2M 0 0 89.3M 0 --:--:-- --:--:-- --:--:-- 89.3M root@julia-75444d5c79-686cf:/# chmod +x PystonConda-1.1-Linux-x86_64.sh root@julia-75444d5c79-686cf:/# ./PystonConda-1.1-Linux-x86_64.sh Welcome to PystonConda 1.1 In order to continue the installation process, please review the license agreement. Please, press ENTER to continue >>> PystonConda installer code uses BSD-3-Clause license as stated below. Binary packages that come with it have their own licensing terms and by installing PystonConda you agree to the licensing terms of individual packages as well. They include different OSI-approved licenses including the GNU General Public License and can be found in pkgs//info/licenses folders. ============================================================================= Copyright (c) 2021, Anaconda, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Anaconda, Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL ANACONDA, INC BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Do you accept the license terms? [yes|no] [no] >>> yes PystonConda will now be installed into this location: /root/pystonconda - Press ENTER to confirm the location - Press CTRL-C to abort the installation - Or specify a different location below [/root/pystonconda] >>> PREFIX=/root/pystonconda Unpacking payload ... Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /root/pystonconda added / updated specs: - _libgcc_mutex==0.1=conda_forge - _openmp_mutex==4.5=1_gnu - brotlipy==0.7.0=py38h79d3a15_1003 - bzip2==1.0.8=h7f98852_4 - ca-certificates==2021.10.8=ha878542_0 - certifi==2021.10.8=py38hc2d5299_1 - cffi==1.15.0=py38h9a12ab7_0 - charset-normalizer==2.0.11=pyhd8ed1ab_0 - colorama==0.4.4=pyh9f0ad1d_0 - conda-package-handling==1.7.3=py38h79d3a15_1 - conda==4.11.0=py38h4c12d10_0 - cryptography==36.0.0=py38ha252339_0 - freetype==2.10.4=h0708190_1 - idna==3.3=pyhd8ed1ab_0 - jbig==2.1=h7f98852_2003 - jpeg==9e=h7f98852_0 - lerc==3.0=h9c3ff4c_0 - libdeflate==1.8=h7f98852_0 - libffi==3.4.2=h7f98852_5 - libgcc-ng==11.2.0=h1d223b6_12 - libgomp==11.2.0=h1d223b6_12 - libpng==1.6.37=h21135ba_2 - libstdcxx-ng==11.2.0=he4da1e4_12 - libtiff==4.3.0=h6f004c6_2 - libwebp-base==1.2.2=h7f98852_1 - libzlib==1.2.11=h36c2ea0_1013 - lz4-c==1.9.3=h9c3ff4c_1 - ncurses==6.2=h58526e2_4 - openssl==1.1.1l=h7f98852_0 - pip==22.0.3=pyhd8ed1ab_0 - pycosat==0.6.3=py38h79d3a15_1009 - pycparser==2.21=pyhd8ed1ab_0 - pyopenssl==22.0.0=pyhd8ed1ab_0 - pysocks==1.7.1=py38h4c12d10_4 - pyston2.3==2.3.2=0_23_pyston - pyston==2.3.2=3 - python==3.8.12=3_23_pyston - python_abi==3.8=1_23_pyston - readline==8.1=h46c0cb4_0 - requests==2.27.1=pyhd8ed1ab_0 - ruamel_yaml==0.15.80=py38h79d3a15_1006 - setuptools==60.7.0=py38hc2d5299_0 - six==1.16.0=pyh6c4a22f_0 - sqlite==3.37.0=h9cd32fc_0 - tk==8.6.11=h27826a3_1 - tqdm==4.62.3=pyhd8ed1ab_0 - tzdata==2021e=he74cb21_0 - urllib3==1.26.8=pyhd8ed1ab_1 - wheel==0.37.1=pyhd8ed1ab_0 - xz==5.2.5=h516909a_1 - yaml==0.2.5=h7f98852_2 - zlib==1.2.11=h36c2ea0_1013 - zstd==1.5.2=ha95c52a_0 The following NEW packages will be INSTALLED: _libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-1_gnu brotlipy pyston/linux-64::brotlipy-0.7.0-py38h79d3a15_1003 bzip2 conda-forge/linux-64::bzip2-1.0.8-h7f98852_4 ca-certificates conda-forge/linux-64::ca-certificates-2021.10.8-ha878542_0 certifi pyston/linux-64::certifi-2021.10.8-py38hc2d5299_1 cffi pyston/linux-64::cffi-1.15.0-py38h9a12ab7_0 charset-normalizer conda-forge/noarch::charset-normalizer-2.0.11-pyhd8ed1ab_0 colorama conda-forge/noarch::colorama-0.4.4-pyh9f0ad1d_0 conda pyston/linux-64::conda-4.11.0-py38h4c12d10_0 conda-package-han~ pyston/linux-64::conda-package-handling-1.7.3-py38h79d3a15_1 cryptography pyston/linux-64::cryptography-36.0.0-py38ha252339_0 freetype conda-forge/linux-64::freetype-2.10.4-h0708190_1 idna conda-forge/noarch::idna-3.3-pyhd8ed1ab_0 jbig conda-forge/linux-64::jbig-2.1-h7f98852_2003 jpeg conda-forge/linux-64::jpeg-9e-h7f98852_0 lerc conda-forge/linux-64::lerc-3.0-h9c3ff4c_0 libdeflate conda-forge/linux-64::libdeflate-1.8-h7f98852_0 libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5 libgcc-ng conda-forge/linux-64::libgcc-ng-11.2.0-h1d223b6_12 libgomp conda-forge/linux-64::libgomp-11.2.0-h1d223b6_12 libpng conda-forge/linux-64::libpng-1.6.37-h21135ba_2 libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-11.2.0-he4da1e4_12 libtiff conda-forge/linux-64::libtiff-4.3.0-h6f004c6_2 libwebp-base conda-forge/linux-64::libwebp-base-1.2.2-h7f98852_1 libzlib conda-forge/linux-64::libzlib-1.2.11-h36c2ea0_1013 lz4-c conda-forge/linux-64::lz4-c-1.9.3-h9c3ff4c_1 ncurses conda-forge/linux-64::ncurses-6.2-h58526e2_4 openssl conda-forge/linux-64::openssl-1.1.1l-h7f98852_0 pip conda-forge/noarch::pip-22.0.3-pyhd8ed1ab_0 pycosat pyston/linux-64::pycosat-0.6.3-py38h79d3a15_1009 pycparser conda-forge/noarch::pycparser-2.21-pyhd8ed1ab_0 pyopenssl conda-forge/noarch::pyopenssl-22.0.0-pyhd8ed1ab_0 pysocks pyston/linux-64::pysocks-1.7.1-py38h4c12d10_4 pyston pyston/noarch::pyston-2.3.2-3 pyston2.3 pyston/linux-64::pyston2.3-2.3.2-0_23_pyston python pyston/linux-64::python-3.8.12-3_23_pyston python_abi pyston/linux-64::python_abi-3.8-1_23_pyston readline conda-forge/linux-64::readline-8.1-h46c0cb4_0 requests conda-forge/noarch::requests-2.27.1-pyhd8ed1ab_0 ruamel_yaml pyston/linux-64::ruamel_yaml-0.15.80-py38h79d3a15_1006 setuptools pyston/linux-64::setuptools-60.7.0-py38hc2d5299_0 six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0 sqlite conda-forge/linux-64::sqlite-3.37.0-h9cd32fc_0 tk conda-forge/linux-64::tk-8.6.11-h27826a3_1 tqdm conda-forge/noarch::tqdm-4.62.3-pyhd8ed1ab_0 tzdata conda-forge/noarch::tzdata-2021e-he74cb21_0 urllib3 conda-forge/noarch::urllib3-1.26.8-pyhd8ed1ab_1 wheel conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0 xz conda-forge/linux-64::xz-5.2.5-h516909a_1 yaml conda-forge/linux-64::yaml-0.2.5-h7f98852_2 zlib conda-forge/linux-64::zlib-1.2.11-h36c2ea0_1013 zstd conda-forge/linux-64::zstd-1.5.2-ha95c52a_0 Preparing transaction: done Executing transaction: done installation finished. Do you wish the installer to initialize PystonConda by running conda init? [yes|no] [no] >>> yes no change /root/pystonconda/condabin/conda no change /root/pystonconda/bin/conda no change /root/pystonconda/bin/conda-env no change /root/pystonconda/bin/activate no change /root/pystonconda/bin/deactivate no change /root/pystonconda/etc/profile.d/conda.sh no change /root/pystonconda/etc/fish/conf.d/conda.fish no change /root/pystonconda/shell/condabin/Conda.psm1 no change /root/pystonconda/shell/condabin/conda-hook.ps1 no change /root/pystonconda/lib/python3.8/site-packages/xontrib/conda.xsh no change /root/pystonconda/etc/profile.d/conda.csh modified /root/.bashrc ==> For changes to take effect, close and re-open your current shell. <== If you'd prefer that conda's base environment not be activated on startup, set the auto_activate_base parameter to false: conda config --set auto_activate_base false Thank you for installing PystonConda!

Honestly, even that seems trivial? By my reading of https://github.com/pyston/pyston#installing-packages , the only impact is that when you install (compiled) libraries they need to be recompiled, just like if you use Alpine (which is also ABI-incompatible because it uses musl libc), which is a little bit of pain at build/packaging time but doesn't actually break anything (i.e. there are no libraries that you can't use, just libraries with an extra compile step) and doesn't affect runtime behavior at all.

The dual numbers exist just as surely as the real numbers and have been used well over 100 years

https://en.m.wikipedia.org/wiki/Dual_number

Pytorch has had them for many years.

https://pytorch.org/docs/stable/generated/torch.autograd.for...

JAX implements them and uses them exactly as stated in this thread.

https://github.com/google/jax/discussions/10157#discussionco...

As you so eloquently stated, "you shouldn't be proclaiming things you don't actually know on a public forum," and doubly so when your claimed "corrections" are so demonstrably and totally incorrect.

On your last point, as long as you jit the topmost level, it doesn't matter whether or not you have inner jitted functions. The end result should be the same.

Source: https://github.com/google/jax/discussions/5199#discussioncom...

The design of MLX is inspired by frameworks like NumPy, PyTorch, Jax, and ArrayFire.

I'm still not totally sure what the issue is. Jax uses program transformations to compile programs to run on a variety of hardware, for example, using XLA for TPUs. It can also run cuda ops for Nvidia gpus without issue: https://jax.readthedocs.io/en/latest/installation.html

There is also support for custom cpp and cuda ops if that's what is needed: https://jax.readthedocs.io/en/latest/Custom_Operation_for_GP...

I haven't worked with float4, but can imagine that new numerical types would require some special handling. But I assume that's the case for any ml environment.

But really you probably mean fixed point 4bit integer types? Looks like that has had at least some work done in Jax: https://github.com/google/jax/issues/8566

>

Are they even comparing apples to apples to claim that they see these improvements over NumPy?

> While the code complexity and length are roughly the same, the MatX version shows a 2100x over the Numpy version, and over 4x faster than the CuPy version on the same GPU.

NumPy doesn't use GPU by default unless you use something like Jax [1] to compile NumPy code to run on GPUs. I think more honest comparison will mainly compare MatX running on same CPU like NumPy as focus the GPU comparison against CuPy.

[1] https://github.com/google/jax

Actually that never changed. The README has always had an example of differentiating through native Python control flow:

https://github.com/google/jax/commit/948a8db0adf233f333f3e5f...

The constraints on control flow expressions come from jax.jit (because Python control flow can't be staged out) and jax.vmap (because we can't take multiple branches of Python control flow, which we might need to do for different batch elements). But autodiff of Python-native control flow works fine!

For a similar "benchmark" (also Mandelbrot) but took place in Jax repo discussion: https://github.com/google/jax/discussions/11078#discussionco...

2. https://github.com/fantasyland/fantasy-land (A bit heavy on jargon)

Note there is a python version of Ramda available on pypi and there’s a lot of FP tidbits inside JAX:

3. https://pypi.org/project/ramda/ (Worth making your own version if you want to learn, though)

4. For nested data, JAX tree_util is epic: https://jax.readthedocs.io/en/latest/jax.tree_util.html and also their curry implementation is funny: https://github.com/google/jax/blob/4ac2bdc2b1d71ec0010412a32...

Anyway don’t put FP on a pedestal, main thing is to focus on the core principles of avoiding external mutation and making helper functions. Doesn’t always work because some languages like Rust don’t have legit support for currying (afaik in 2023 August), but in those cases you can hack it with builder methods to an extent.

Finally, if you want to understand the middle of the midwit meme, check out this wiki article and connect the free monoid to the Kleene star (0 or more copies of your pattern) and Kleene plus (1 or more copies of your pattern). Those are also in regex so it can help you remember the regex symbols. https://en.wikipedia.org/wiki/Free_monoid?wprov=sfti1

The simplest example might be {0}^* in which case

0: “” // because we use *

Hello! I'm trying to learn JAX over the next couple of weeks. Ideally, I want to be comfortable with using it for projects after about 3 weeks to a month, although I understand that may not be realistic. I currently have experience with PyTorch and TensorFlow. How should I go about learning JAX? Is there a specific YouTube tutorial or online course I should use, or should I just use the tutorial on https://jax.readthedocs.io/? Any information, advice, or experience you can share would be much appreciated!