the-pile VS opendyslexic

[2] https://github.com/EleutherAI/the-pile/issues/56

I came so close to getting my dataset DebateSum (https://huggingface.co/datasets/Hellisotherpeople/DebateSum) into the pile, but they decided at the last minute not to add it: https://github.com/EleutherAI/the-pile/issues/56

I'm still a tiny bit salty about that.

Anyone want to check if the book in question is in ThePile dataset?:

https://github.com/EleutherAI/the-pile/blob/master/the_pile/...

All of it, it’s quite diverse. Especially the commoncrawl bit, https://github.com/EleutherAI/the-pile.

For example, before this I didn't realize one of the sources of data that the pile uses is a massive number of emails gathered during the Enron lawsuits. Weird, but cool I guess.

NovelAI's Krake and Euterpe, and the rest, are finetuned versions of existing models. The original models were trained on a mass of text. Krake is a finetune of Neo-X 20b, which was trained on The Pile. NovelAI's finetunes involve further training but on various works of fiction rather than more text trawled from the internet. The statistical rules in the existing models are thus shifted in a (slightly) new direction. Modules refine those statistical rules, or weights, just a little bit more.

Literally The Pile.

The parameters are taken from large datasets like The Pile.

We use the GPT-Neo as our base model which trained on The Pile and you can see it's contents in their github repo: https://github.com/EleutherAI/the-pile

I changed the font on my kindle to open dyslexic & it made a huge difference in my ability to focus on what I’m reading. You’ll know pretty quickly if it will work for you.

Too bad the article is not open-access, as I would expect from JetBrains.

Extra cognitive load slows everyone. It's just that the effect is measurably distinct in people with executive function (distractability) issues, with respect to speed. The distinction between debugging and coding is not really active vs monotonous but driven by your own ideas vs chasing (a problem). The study isn't realistic, but it's designed to get a measurable result (and to showcase the "efficiency tracking" plugin).

Anecdotally, everyone adjusts their IDE, or accommodates what can't (easily) be changed. Too bad that wisdom is lost and hard to share.

I think the solution here is more configurable UI's, with the configuration being automated/scriptable so that once you've established your preferences, you can replicate them through upgrades, etc.

The most configurable IDE of course is Eclipse (which is in decline because no one gets paid directly to write for it, and it's cheaper to publish a language server for your new language than build an IDE). You can arrange views as you like, change menu and toolbar visibility, change key bindings, and of course add whatever plugins/features you need. You can save view configurations as a workspace and save various preferences. But because components come from everywhere, support for configuration capture varies.

People share their dotfiles for shell and vi/emacs configuration, but not their IDE configurations. It's too bad, because then there would be a configuration population to analyze when raising UI issues.

ADHD and ASD are a broad spectrum. It may help to join the tribe because it validates our experience, but then we can fail to recognize our brain's specific biases. Worse, anyone over 7 has been getting good at compensating, which hides the issue, and our culture of excellence/competition/success == good (therefore failure bad) further obscures with shame, defeat, and self-sabotage. Legal requirements for accommodation help set a global floor, but may also work as a local ceiling by supplanting ordinary fellow-feeling.

For reading fatigue, consider a dyslexia font, e.g., https://opendyslexic.org.

GitHub link Website link Source license SIL-OFL I've come across this font that aims to make reading easier for people with dyslexia and I've never seen it implemented before. I feel like it would be really cool if it got implemented into more things :)

I would really like to see OpenDyslexic as a font option in boosts

https://opendyslexic.org This is where it was made I think you can download an add on here to change all your fonts, as well as ‘most’ applications use open dyslexia or dyslexia open as the font name