MTR
The official implementation of the paper "Rethinking Data Augmentation for Tabular Data in Deep Learning" (by somaonishi)
tabular-dl-pretrain-objectives
Revisiting Pretrarining Objectives for Tabular Deep Learning (by puhsu)
MTR | tabular-dl-pretrain-objectives | |
---|---|---|
1 | 1 | |
9 | 58 | |
- | - | |
4.9 | 10.0 | |
7 months ago | over 1 year ago | |
Python | Python | |
MIT License | - |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
MTR
Posts with mentions or reviews of MTR.
We have used some of these posts to build our list of alternatives
and similar projects.
-
Rethinking Data Augmentation for Tabular Data in Deep Learning
Tabular data is the most widely used data format in machine learning (ML). While tree-based methods outperform DL-based methods in supervised learning, recent literature reports that self-supervised learning with Transformer-based models outperforms tree-based methods. In the existing literature on self-supervised learning for tabular data, contrastive learning is the predominant method. In contrastive learning, data augmentation is important to generate different views. However, data augmentation for tabular data has been difficult due to the unique structure and high complexity of tabular data. In addition, three main components are proposed together in existing methods: model structure, self-supervised learning methods, and data augmentation. Therefore, previous works have compared the performance without comprehensively considering these components, and it is not clear how each component affects the actual performance. In this study, we focus on data augmentation to address these issues. We propose a novel data augmentation method, $\textbf{M}$ask $\textbf{T}$oken $\textbf{R}$eplacement ($\texttt{MTR}$), which replaces the mask token with a portion of each tokenized column; $\texttt{MTR}$ takes advantage of the properties of Transformer, which is becoming the predominant DL-based architecture for tabular data, to perform data augmentation for each column embedding. Through experiments with 13 diverse public datasets in both supervised and self-supervised learning scenarios, we show that $\texttt{MTR}$ achieves competitive performance against existing data augmentation methods and improves model performance. In addition, we discuss specific scenarios in which $\texttt{MTR}$ is most effective and identify the scope of its application. The code is available at https://github.com/somaonishi/MTR/.
tabular-dl-pretrain-objectives
Posts with mentions or reviews of tabular-dl-pretrain-objectives.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-01-03.
-
[D] Influential papers round-up 2022. What are your favorites?
Found relevant code at https://github.com/puhsu/tabular-dl-pretrain-objectives + all code implementations here
What are some alternatives?
When comparing MTR and tabular-dl-pretrain-objectives you can also consider the following projects:
Papers-in-100-Lines-of-Code - Implementation of papers in 100 lines of code.
Swin-Transformer - This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".
rtdl - Research on Tabular Deep Learning (Python package & papers) [Moved to: https://github.com/Yura52/rtdl]
tabular-dl-tabr - The implementation of "TabR: Unlocking the Power of Retrieval-Augmented Tabular Deep Learning"
rtdl - Research on Tabular Deep Learning [Moved to: https://github.com/yandex-research/rtdl]