Optimization.jl VS fpm

Interesting response. I develop the Julia SciML organization https://sciml.ai/ and we'd be more than happy to work with you to get wrappers for PRIMA into Optimization.jl's general interface (https://docs.sciml.ai/Optimization/stable/). Please get in touch and we can figure out how to set this all up. I personally would be curious to try this out and do some benchmarks against nlopt methods.

There are also Optimization and Nonconvex , which seem like umbrella packages and I am not sure what methods to use inside these packages. Any help on these?

> but would you say most packages follow or enforce SemVer?

The package ecosystem pretty much requires SemVer. If you just say `PackageX = "1"` inside of a Project.toml [compat], then it will assume SemVer, i.e. any version 1.x is non-breaking an thus allowed, but not version 2. Some (but very few) packages do `PackageX = ">=1"`, so you could say Julia doesn't force SemVar (because a package can say that it explicitly believes it's compatible with all future versions), but of course that's nonsense and there will always be some bad actors around. So then:

> Would enforcing a stricter dependency graph fix some of the foot guns of using packages or would that limit composability of packages too much?

That's not the issue. As above, the dependency graphs are very strict. The issue is always at the periphery (for any package ecosystem really). In Julia, one thing that can amplify it is the fact that Requires.jl, the hacky conditional dependency system that is very not recommended for many reasons, cannot specify version requirements on conditional dependencies. I find this to be the root cause of most issues in the "flow" of the package development ecosystem. Most packages are okay, but then oh, I don't want to depend on CUDA for this feature, so a little bit of Requires.jl here, and oh let me do a small hack for OffSetArrays. And now these little hacky features on the edge are both less tested and not well versioned.

Thankfully there's a better way to do it by using multi-package repositories with subpackages. For example, https://github.com/SciML/GalacticOptim.jl is a global interface for lots of different optimization libraries, and you can see all of the different subpackages here https://github.com/SciML/GalacticOptim.jl/tree/master/lib. This lets there be a GalacticOptim and then a GalacticBBO package, each with versioning, but with tests being different while allowing easy co-development of the parts. Very few packages in the Julia ecosystem actually use this (I only know of one other package in Julia making use of this) because the tooling only recently was able to support it, but this is how a lot of packages should be going.

The upside too is that Requires.jl optional dependency handling is by far and away the main source of loading time issues in Julia (because it blocks precompilation in many ways). So it's really killing two birds with one stone: decreasing package load times by about 99% (that's not even a joke, it's the huge majority of the time for most packages which are not StaticArrays.jl) while making version dependencies stricter. And now you know what I'm doing this week and what the next blog post will be on haha.

Make sure you have the latest fpm binary installed somewhere so that your $PATH can see it: curl -o ~/.local/bin/fpm -L https://github.com/fortran-lang/fpm/releases/download/v0.8.2/fpm-0.8.2-linux-x86_64 && chmod 0755 ~/.local/bin/fpm

Hopefully, the SciPy community can stay open-minded about modern Fortran libraries.

Modern Fortran is quite different from Fortran 77, while being as powerful, if not more.

In addition, there has been a significant community effort on improving and modernising the legacy packages, the ecosystem, and the language itself.

With projects like LFortran (https://lfortran.org/), fpm (https://github.com/fortran-lang/fpm), and stdlib (https://github.com/fortran-lang/stdlib), I believe that Fortran will enjoy prosperity again.

Anyway, first release of Fortran Package Manager was in November 2020: https://github.com/fortran-lang/fpm/releases/tag/v0.1.0 - more recently than I expected.

There is actually a Fortran Package Manager that will hopefully make things easier in the future. It's quite new, so it might not be entirely mature yet.

Author here, so I'm biased toward Fortran, though I've been enjoying learning Rust as well. I think there are a few reasons.

First, Rust's multidimensional arrays are either limited and/or difficult to use. Fast, flexible, and ergonomic multidimensional arrays and arithmetic are essential for HPC. They are possible with Rust, but my two favorite Rust books not mentioning them suggests to me that they're not the focus of the language. This may or may not change in the future.

Second, Rust may be too complex to learn for scientists who aren't paid to write software but to do research. Fortran is opposite--multidimensional whole-array arithmetic looks like you would write it as math on a whiteboard. While scientists can sure learn to program Rust effectively, I think most scientists don't think like Rust, but they do think like Fortran. For somebody not familiar with Fortran but familiar with Python, I'd say Fortran very much feels like NumPy.

Third, such ecosystem would be built in Rust from scratch. In Fortran, most of the value is already there, but needs to be made more accessible with better and more modern tooling. For example, Fortran's fpm (https://github.com/fortran-lang/fpm) is largely modeled after Rust's Cargo because we recognize the importance of good user experience when it comes to building and packaging software. With the recent Fortran-lang efforts, we study many programming language ecosystems and communities (e.g. Python, Julia, Rust, etc.) to find what could work best for modern Fortran tooling.

I recently started learning Fortran for a lark. It reminds me a lot of R, in some respects. It's clearly a very, very good language for doing the parts of one's job that are very math-centric. But it's equally underwhelming as a general purpose programming language.

Largely, I think, due to gaps in the library ecosystem. But there are other challenges. You can see from the install instructions on the linked page, for example, that Fortran still lacks a package manager.

What's interesting, though, is that that's changing. There are currently serious efforts to give it a "standard" library (https://github.com/fortran-lang/stdlib) and package manager (https://github.com/fortran-lang/fpm).

And I've been watching the new LFortran compiler (https://lfortran.org) with extreme interest.

FPM has entered the chat https://github.com/fortran-lang/fpm