hyperparameter VS pytorch-lightning

the full tutorial: https://github.com/reiase/hyperparameter/tree/master/examples/optuna

When I was working on my own configuration framework (HyperParameter, previous post), I suddenly realize that what I want is not another configuration framework with some fancy API. All I want is to change my ML experiments without modifying the code and get rid of the configuration handling codes. The right way of configuration is not writing configurable code and wasting time on different frameworks. The best solution is a tool that makes your code configurable.

github: https://github.com/reiase/hyperparameter

HyperParameter is a configuration and parameter management library for Python. HyperParameter provides the following features:

you can take a look at https://github.com/reiase/hyperparameter, a scoped, thread-safe config object that is lightweight enough. There is no need to modify too much code:

I'm developing a Hyperparameter tuning toolbox for my machine learning projects. It maps keyword arguments to hyper-parameters, for example:

I'm developing [a toolbox for managing hyper-parameters](https://github.com/reiase/hyperparameter) in my data science and machine learning projects. It provides object-style API for nested dict( which is very common for config files):

https://see.stanford.edu/Course/CS229 https://lightning.ai/ https://www.youtube.com/watch?v=00s9ireCnCw&t=57s https://towardsdatascience.com/

I've been wondering on what is the recommended method of saving logits/activations using PyTorch Lightning. I've looked at Callbacks, Loggers and ModelHooks but none of the use-cases seem to be for this kind of activity (even if I were to create my own custom variants of each utility). The ModelCheckpoint Callback in its utility makes me feel like custom Callbacks would be the way to go but I'm not quite sure. This closed GitHub issue does address my issue to some extent.

Our codebase for the diffusion models builds heavily on OpenAI's ADM codebase , lucidrains, Stable Diffusion, Lightning and Hugging Face. Thanks for open-sourcing!

For lightweight use, pytorch-lightning is too heavy, and its source code will be very difficult for beginners to read, at least for me.

For the A100 GPU, theoretical performance is the same for FP16/BF16 and both rely on the same number of bits, meaning memory should be the same. However since it's quite newly added to PyTorch, performance seems to still be dependent on underlying operators used (pytorch lightning debugging in progress here).