til VS tsv-utils

> Kudos to the TIL's author for trailblazing this idea based on TCL. It will be very beneficial and handy for scripting commands and shell like behaviors.

Thanks! I love the concept of "scripting" (that is a bit different from simply a "dynamic language"). I'm quite aware it's just "yet another programming language" but if I can dream of something is that it serves as some kind of incentive for people to develop more libraries in D.

I mean, if you just want to create a Til module that allows you to serve some Web pages using HTTP/2, it shouldn't be that difficult and, at the same time, it could be the end goal itself: just creating a useful module, not something like "it's a crucial part, besides other five, of this big project X I'm working on" (I believe this kind of situation almost always ends with "nah, I'll just use instead").

> Just wondering is this type based TCL like language similar to Little?

No, it's not. I first heard about Little a couple months ago and it's a very interesting project. But I don't plan, right now, to include any kind of builtin Tcl compatibility layer in Til (although users are free to create its own implementations, of course).

> [2] Will it eventually support compilation similar to Emacs Lisp? [3]

I created the language much more as a tool to learn how to create languages than anything else, but now it's kind of mature enough, I'll confess my dream is to implement JIT compilation, following the steps of LuaJIT (that is an AWESOME project IMHO).

> Personally I'd love to have superset language in D for data science.

That would be nice. Having a autowrap-like way of exposing D code to Til would be even nicer. (https://github.com/atilaneves/autowrap)

> It should be also easily embeddable and support prototyping like Lua.

I believe embedding it is already in a very tolerable state. If you look into the "interpreter" code you'll see it is only 82 lines (actual 69 LOC).

(https://github.com/til-lang/til/blob/master/interpreter/sour...)

And it has a lot of debugging code. Loading a string, parsing it as a "SubProgram" and running it is kind of trivial.

Now, about the prototyping part, I never thought about it, actually...

> On top of that it should have excellent support for array, ndarray and dataframe like R [4].

It's very easy to create new types in Til and they support both "operate" (to apply, you know, operators, like +, -, /, etc) and "extract (to index things or extract information in general from values). I believe it wouldn't be difficult to create a nice module for using these things.

> Since it is based on D, then it can fulfill the requirements for both type A and B data scientists [5].

Maybe. But, I don't know... isn't data scientists all over the world happy and satisfied with Python, already?

If you need just csv/tsv parsing, you can also take a look at https://github.com/eBay/tsv-utils

You might want to look at tsv-utils, or a similar project: https://github.com/eBay/tsv-utils

For the SQL part, but maybe a lot heavier, you can use one of the projects listed on this page: https://github.com/multiprocessio/dsq (No longer maintained, but has links to lots of other projects)

TSV is most often a better format than CSV. Localization, in particular, is a nightmare with CSV.

For anything down and dirty, what's wrong with -F'"'? For anything fancy there are plenty of things like the below.

eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.

includes csv to tsv: https://github.com/eBay/tsv-utils

HT: https://simonwillison.net/

As an example, eBay's tsv-utils took full advantage of the GC and performed better than existing programs that had been hand-optimized in C etc.

tsv-utils - Command line csv data manipulation toolkit. D

In the majority of cases you'll be better off with Tab Separated Values over Comma Separated Values. More info here.

May well be overkill for your needs, but I'm a fan of tsv-utils It's fast and enormously flexible, and seems to me a "best of breed" toolset for data mining CSV files (that is what it was written for). https://github.com/eBay/tsv-utils