Is Python really this slow?

• Numba

  124 9,404 9.9 Python

  NumPy aware dynamic Python compiler using LLVM

  

For HPC you probably want to avoid OO. As for Python speed it should be fine. You can use numba https://numba.pydata.org/ to speed up your non-standard functions.

• h5py

  5 1,996 8.6 Python

  HDF5 for Python -- The h5py package is a Pythonic interface to the HDF5 binary data format.

  

If possible, try to monitor your memory usage during execution and if you see that you are consistently exceeding ~50% (my own rule of thumb, though you may want to discuss this with others as well) of what's available. If you are consistently using most of the available memory, then it's likely worth taking a moment to evaluate whether you can operate on subsets of the data from start to finish, and leave the rest of the data on disk until you are almost ready to use it. Tools like h5py are very helpful in these kinds of situations.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project