OpenBLAS VS PurefunctionPipelineDataflow

Sure they do.

https://github.com/OpenMathLib/OpenBLAS

Plenty of assembly in that project but no mention of it in the README.

To take NumPy as an example, NumPy is a single monolithic library, where all of its components, outside of optional third-party dependencies such as OpenBLAS, form a single, indivisible unit. One cannot simply install NumPy routines for array manipulation without installing all of NumPy. If you are deploying an application which only needs NumPy's ndarray object and a couple of its manipulation routines, installing and bundling all of NumPy means including a considerable amount of "dead code". In web development parlance, we'd say that NumPy is not "tree shakeable". For a normal NumPy installation, this implies at least 30MB of disk space, and at least 15MB of disk space for a customized build which excludes all debug statements. For SciPy, those numbers can balloon to 130MB and 50MB, respectively. Needless to say, shipping a 15MB library in a web application for just a few functions is a non-starter, especially for developers needing to deploy web applications to devices with poor network connectivity or memory constraints.

The Fortran implementation is just a reference implementation. The goal of reference BLAS [0] is to provide relatively simple and easy to understand implementations which demonstrate the interface and are intended to give correct results to test against. Perhaps an exceptional Fortran compiler which doesn't yet exist could generate code which rivals hand (or automatically) tuned optimized BLAS libraries like OpenBLAS [1], MKL [2], ATLAS [3], and those based on BLIS [4], but in practice this is not observed.

Justine observed that the threading model for LLaMA makes it impractical to integrate one of these optimized BLAS libraries, so she wrote her own hand-tuned implementations following the same principles they use.

[0] https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra...

[1] https://github.com/OpenMathLib/OpenBLAS

[2] https://www.intel.com/content/www/us/en/developer/tools/onea...

[3] https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Alg...

[4]https://en.wikipedia.org/wiki/BLIS_(software)

Yeah. I'm going to be helping to work on expanding CI for OpenBlas and have been diving into this stuff lately. See the discussion in this closed OpenBlas issue gh-1968 [0] for instance. OpenBlas's Skylake kernels do rely on intrinsics [1] for compilers that support them, but there's a wide range of architectures to support, and when hand-tuned assembly kernels work better, that's what are used. For example, [2].

[0] https://github.com/xianyi/OpenBLAS/issues/1968

[1] https://github.com/xianyi/OpenBLAS/blob/develop/kernel/x86_6...

[2] https://github.com/xianyi/OpenBLAS/blob/23693f09a26ffd8b60eb...

We'll have to wait until part 2 to see what they are actually proposing, but they are trying to solve a real problem. To get a sense of things check out the handwritten assembly kernels in OpenBlas [0]. Note the level of granularity. There are micro-optimized implementations for specific chipsets.

If progress in ML will be aided by a proliferation of hyper-specialized hardware, then there really is a scalability issue around developing optimized matmul routines for each specialized chip. To be able to develop a custom ASIC for a particular application and then easily generate the necessary matrix libraries without having to write hand-crafted assembly for each specific case seems like it could be very powerful.

[0] https://github.com/xianyi/OpenBLAS/tree/develop/kernel

libraries mkl_rt not found in ['C:\python\lib', 'C:\', 'C:\python\libs'] ``` Install this and try again. Might need to reboot, never know with Windows https://www.openblas.net/

There isn't any fortran code in the repo there itself but numpy itself can be linked with several numeric libraries. If you look through the wheels for numpy available on pypi, all the latest ones are packaged with OpenBLAS which uses Fortran quite a bit: https://github.com/xianyi/OpenBLAS

Sure - write functions in another language, export C bindings, and then call those functions from Python. An example is NumPy - a lot of its linear algebra functions are implemented in C and Fortran.

Implement relational data model and programming based on hash-map (NoSQL)

The Math-based Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model

I criticized "AMD Infinity Fabric Architecture" at the end of my article "Prediction: Intel will use "RISC-V plus x86 compatibility layer" or "RISC-V plus x86 heterogeneous computing architecture" to develop a new generation of "warehouse/workshop model" CPU".

What's more, you're being somewhat liberal with your "invention" dates here anyway. I'm sure you realize that anyone can review the commit history to see when content was added to the repo. As of Nov 2020, the day Apple announced a fully operational and tested, ready-to-ship silicon package the repo was a just series of bullet points listing out well-known concepts of functional programming sprinkled with some religious analogies and inspirational quotes. The farther you go back in the repo commit history, the less content is there.