nni VS optuna

For a complete tutorial, navigate to this Jupyter Notebook: https://github.com/microsoft/nni/blob/master/examples/notebooks/tabular_data_classification_in_AML.ipynb

optuna, hyperopt, nni, plenty of less-known tools too.

microsoft / nni

Optuna often works fine in this context (even with the memory errors or, with some tuning, with the non-halting runs): https://github.com/optuna/optuna

I didn’t even know WandB did hyperparameter optimization, I figured it was a neural network visualizer based on 2 minute papers. Didn’t seem like many alternatives out there to Optuna with TPE + persistence in conditional continuous & discrete spaces.

Anyway, it’s doable to make a multi objective decide_to_prune function with Optuna, here’s an example https://github.com/optuna/optuna/issues/3450#issuecomment-19...

The network configuration process is usually not particularly scientific and mostly relies on empirical observation. For some cases, tools like Optuna can be used to automatically find the optimal parameters. In others, on others, you can look for modern studies which explore the effect of this parameter on performance, such as this study (2022), but these are typically very specific to one particular architecture.

Keras Tuner, Optuna : https://github.com/optuna/optuna ?

I have used OpenTuner, but I don't think it is maintained anymore. I hear tell that Optuna is what to use now, but have not used it myself. https://optuna.org Optuna - A hyperparameter optimization framework

Research the type of model to get an idea of what hyper parameters to use. I recommend using a hyper parameter optimization library like Optuna to get the best configuration