-
DataPowerTools
Bridging the gap between IEnumerable and IDataReader for dealing with unstructured and loosely-structured data, plus fast ETL + SQL Bulk Copy.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
I created a lib https://github.com/nh43de/DataPowerTools for this purpose (basically a lot of extensions around IDataReader), but I haven't had the time to really maintain or document it properly. Basically, there are readers: ExcelReader, CsvReader, etc. that implement IDataReader, a standard ADO.NET interface, and then extension methods can be called on IDataReader to add/modify/transform/lookup columns. Once the (possibly transformed) IDataReader is obtained it can then be used to write the data somewhere else, or analyzed further e.g. to create a table to fit the transformed data. Currently supported destinations are Excel, Sql Server, SqlLite, Csv, or any destination that takes in an IDataReader. It's highly performant and in production in a lot of places.