SES-shim VS pip

I used E in the 90s: http://erights.org/

I haven't kept up with newer systems but I've heard of https://github.com/endojs/endo and just came across http://reports-archive.adm.cs.cmu.edu/anon/home/anon/isr2017... (which says "in the style of the E programming language" -- that's as far as I've read) while looking that up.

WebAssembly was designed to follow the same capability security principles. CHERI too as someone else just brought up.

There are other potential solutions I haven’t explored close enough (like Endo and SES), or completely omitted as they’re based on an imperfect blacklist-based approach to security (like sandboxed WebWorkers). However, the mentioned 4 solutions are the top contenders, at least in my mind.

I don't know why you are being silently downvoted, as I think it is worth talking about the potential of using static analysis to improve things.

One promising approach is Endo[0] which "uses LavaMoat to automatically generate reviewable policies that determine what capabilities will be distributed to third party dependencies."

[0] https://github.com/endojs/endo

Agoric moved forward and Realms gave way to SES

https://github.com/endojs/endo/tree/master/packages/ses

And Endo is a set of tools (being) built around it to make it more practical for particular usecases

Yea you could restrict the app by whitelisting only the network services and folders that it will use and that's pretty valuable though at least on Linux could already easily be achieved otherwise. It's good that Deno makes it easy but let's be honest, most people will just pass -A.

I'd love to see a permissions system on a library basis. It would ask the first time a dependency is added and when a new permission is requested after an update. Javascript doesn't make that easy though by being so dynamic. SES could maybe help: https://github.com/endojs/endo/blob/master/packages/ses/READ...

I was poking around on the internet a bit earlier and I found this project. It looks pretty cool, and I figured perhaps a few of y'all might find it cool too!

I have no idea if it actually sandboxes networking by default. This other project, endo[0], seems to add some of that functionality.

Regardless of the maturity though, it makes me excited to see this type of work getting done now!

(What made me want to research it was this[1] thread from the other day.)

0: https://github.com/endojs/endo

1: https://news.ycombinator.com/item?id=30215212

Fortunately the problem could become more tractable if something like SES / Endo takes off:

"Endo protects program integrity both in-process and in distributed systems. SES protects local integrity, defending an application against supply chain attacks: hacks that enter through upgrades to third-party dependencies. Endo does this by encouraging the Principle of Least Authority. ... Endo uses LavaMoat to automatically generate reviewable policies that determine what capabilities will be distributed to third party dependencies."

https://github.com/endojs/endo

Yeah. JavaScript is probably the closest to being there (with things like SES[0], LavaMoat[1], etc.) but we're not quite there yet. It's just shocking that this sort of thing is as seemingly obscure as it is; it's like the whole industry has collectively thrown up their hands and said code execution is unavoidably radioactively dangerous. (While simultaneously using package managers that... well.) But it doesn't have to be!

[0] https://github.com/Agoric/ses-shim

[1] https://github.com/LavaMoat/LavaMoat

Whenever you are working on a Python project that has external dependencies installed with pip, it is strongly recommended to first create a virtual environment.

Unfortunately that feature is easy to break: https://github.com/pypa/pip/issues/9644

I think I did my part https://github.com/pypa/pip/issues/6409

$ sudo pip install y$ sudo pip install yaml WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip. Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue. To avoid this problem you can invoke Python with '-m pip' instead of running pip directly. ERROR: Could not find a version that satisfies the requirement yaml (from versions: none) ERROR: No matching distribution found for yaml

What are you implying will happen?

Using the build-in tools, you can save the exact versions of dependencies (i.e. a lock file) using "pip freeze >dependencies.txt". This should give you the exact same set of packages in two years' time.

If you want to be even more sure, you can also store hashes in the lock file. This has to be generated by a separate tools at the moment [1][2] but can be consumed by the built-in tools [3], so "pip install -r requirements.txt" is still all you need in two years' time.

[1] https://github.com/pypa/pip/issues/4732

[2] https://pip-tools.readthedocs.io/en/latest/#using-hashes

[3] https://pip.pypa.io/en/stable/topics/secure-installs/#hash-c...

Here’s the issue, https://github.com/pypa/pip/issues/11664. I think the idea would be to have some file/json description of environment that could be passed to pip to allow it to fully cross compile. They are open to supporting it just needs contributor to be found to implement it and go through review/discussion.

To be fair the only alternative is fixing Python, and even then you still would have to wait a good 5 years at least for all the old Python versions to dwindle.

It doesn't look like the fixing effort is progressing very quickly: https://github.com/pypa/pip/issues/8606

To their credit, at least they didn't close it "works as intended" which I imagine a lot of projects would.

Another good benchmark to trying to resolve apache-airflow[all]==1.10.13 using the state of PyPi on 2020-12-02, I give instructions here on how to reproduce that workflow: https://github.com/pypa/pip/issues/11836. Including a benchmark how how many extra packages your resolver should visit.