CSVLint
csv_log_cleaner
| CSVLint | csv_log_cleaner | |
|---|---|---|
| 44 | 2 | |
| 134 | 2 | |
| - | - | |
| 7.6 | 6.3 | |
| about 1 month ago | about 2 months ago | |
| C# | Rust | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
CSVLint
-
A question for the pro's, am I misusing SQL?
Also, a little self-promotion here, I've created the CSV Lint plug-in for Notepad++ to work with CSV text data files. It can reformat, validate and sort csv files, as well as convert csv to different formats including SQL. Meaning it can take a csv data file and generate INSERT INTO statements, including CREATE TABLE with the corresponding column datatypes and everything.
-
Looking for a CSV editor that doesn't modify the data like Excel does
I've created the CSV Lint plug-in for Notepad++ which can do all kinds of validation and transformations on a CSV file, processing it just as a text-file. Although it's only on Windows.
-
Best Way to Import a CSV From Into PostrgeSQL
fyi Notepad++ has a CSV Lint plug-in which can convert a csv file into an SQL INSERT VALUES script, including a CREATE TABLE statement with the appropriate column datatypes (based on the content of the csv data)
-
How to Import Data (XLSX, CSV, etc) into pgadmin
Maybe you could use Notepad++ with the CSV Lint plug-in to convert a csv file to an SQL INSERT VALUES script, including a CREATE TABLE statement.
-
Problem importing CSV file
You could try opening the file as a plain text file in notepad, or maybe using Notepad++ and the CSV Lint plug-in
-
Best language/tool to work with CSV files?
I just want to mention I've created a CSV Lint plug-in for Notepad++, maybe not exactly what you're looking for but it can generate initial Python scripts based on csv files, so might be useful.
-
CSV Lint plug-in for Notepad++ to view csv files, validate and convert to SQL insert script
The CSV Lint plug-in for Notepad++ was updated recently, it's available in the latest release of Notepad++ (v8.5.3). It is a useful plug-in for anyone working with csv datasets. I have created the plugin and had posted about it before, and this latest update has some more improvements and bugfixes.
-
Just joined a company in a sunset industry, data is in Excel. I want to migrate from Excel to PostgreSQL. I have zero knowledge in SQL, but i have some experience in programming using MatLab. Is this possible? I am thinking of Jose Portilla's course on Udemy as starting point.
If you just want to create tables you could experiment a bit with Notepad++ and the CSV Lint plug-in (Disclaimer: I'm the author of this plugin).
-
How do you guys handle pandas and its sh*tty data type inference
There's also the CSV Lint plug-in for Notepad++ which can detect datatypes, and then you can do CSV Lint > Generate metadata > Python script. Although idk it might not work correctly for all datetime datatypes.
-
Data manipulation tools
idk if it counts as an ETL tool, but with the CSV Lint plug-in for Notepad++ you can quickly check a csv file for errors, validate a dataset or get a column summary report.
csv_log_cleaner
-
How do you guys handle pandas and its sh*tty data type inference
Sounds like it could be more of a data cleansing problem you're facing than a data inference one. Even a single non-numerical value in a million rows of numbers will necessarily mess up type inference for the whole column. I work with a lot of CSVs and that's one of the issues we have to spend a huge amount of time dealing with. I even ended up writing this open source tool to handle the cleansing: https://github.com/ambidextrous/csv_log_cleaner
-
Hey Rustaceans! Got a question? Ask here! (39/2022)!
Hi. I'm new to Rust. I've written up a little opensource tool to clean CSV files as a practical learning exercise that will help me with my job: https://github.com/ambidextrous/csv_cleaner Where would be a good place to post it for code review?
What are some alternatives?
OpenRefine - OpenRefine is a free, open source power tool for working with messy data and improving it
unescape-rs - "Unescapes" strings with escape sequences written with literal characters and converts it into a properly escaped one.
datasetmultitool - CSV lint tool to validate csv files. It is a helper utility to process csv textfiles and check for data errors. It can check text width, validate and reformat date and datetime values, change point or comma decimal separator, remove thousand separator and change column order.
doku - fn(Code) -> Docs
CsvQuery - Plugin for Notepad++ that treats CSV files as (read only) SQL tables
mimalloc - mimalloc is a compact general purpose allocator with excellent performance.
Customer-Analysis-Tableau - This repository contains the data source and the tableau workbook used in my YouTube video: https://www.youtube.com/watch?v=_qReGTOrKTk
dtype_diet - Tries to shrink your Pandas column dtypes with no data loss so you have more spare RAM
NppPluginLexerExample - Notepad++ Plug-in Lexer and Folder example using the C# template
Peroxide - Rust numeric library with R, MATLAB & Python syntax
Pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
esp8266-hal - A experimental hardware abstraction layer for the esp8266 written in Rust.