long-range-arena
hn-search
Our great sponsors
long-range-arena | hn-search | |
---|---|---|
6 | 1,619 | |
682 | 524 | |
2.9% | 1.5% | |
0.0 | 2.9 | |
4 months ago | 6 months ago | |
Python | TypeScript | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
long-range-arena
-
The Secret Sauce behind 100K context window in LLMs: all tricks in one place
https://github.com/google-research/long-range-arena
-
[R] The Annotated S4: Efficiently Modeling Long Sequences with Structured State Spaces
The Structured State Space for Sequence Modeling (S4) architecture is a new approach to very long-range sequence modeling tasks for vision, language, and audio, showing a capacity to capture dependencies over tens of thousands of steps. Especially impressive are the model’s results on the challenging Long Range Arena benchmark, showing an ability to reason over sequences of up to 16,000+ elements with high accuracy.
-
[D] Is there a repo on which many light-weight self-attention mechanism are introduced?
1.1 Long Range Arena: A Benchmark for Efficient Transformers. From authors of above, they proposed a benchmark for modeling long range interactions. It also inlcudes a repository
- [R] Google’s H-Transformer-1D: Fast One-Dimensional Hierarchical Attention With Linear Complexity for Long Sequence Processing
- [2107.11906] H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences
-
[R][D] Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. Zhou et al. AAAI21 Best Paper. ProbSparse self-attention reduces complexity to O(nlogn), generative style decoder to obtainsequence output in one step, and self-attention distilling for further reducing memory
I think the paper is written in a clear style and I like that the authors included many experiments, including hyperparameter effects, ablations and extensive baseline comparisons. One thing I would have liked is them comparing their Informer to more efficient transformers (they compared only against logtrans and reformer) using the LRA (https://github.com/google-research/long-range-arena) benchmark.
hn-search
-
Gary Killdall, creator of CP/M, wrote Pixar's original 3D renderer [pdf]
The submitted title was "Gary Killdall, creator of CP/M, wrote Pixar's original 3D renderer".
Submitters: If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
(From https://news.ycombinator.com/newsguidelines.html: "Please use the original title, unless it is misleading or linkbait; don't editorialize.")
-
Nearsightedness is at epidemic levels – and the problem begins in childhood
Vision therapy for myopia helps some people, but not everyone, likely due to genetic and neuroplasticity differences, https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Nevertheless, many of the principles are useful for children whose eyes and brains are still developing.
-
Tesla driver arrested for homicide after running over motorcyclist on Autopilot
I'm a huge Tesla skeptic, but Tesla and Musk are lightning rods for tabloid-style garbage that doesn't belong on HN, so it doesn't surprise me that we often see negative Tesla content flagged to death. Meanwhile we also see plenty of content that hits the front page and stays there [0].
Do you have examples of professional, interesting Tesla content that got flagged?
[0] More than half of the past year's most popular Tesla articles were negative: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...
-
The Man Who Killed Google Search
It's April 23rd, 2024, and I am still looking for a good, reliable, honest and simple search engine.
All I want to do is search.
No AI.
No ads.
No shopping.
Please don't "Answer my question." I enjoy doing my own original research, thanks.
I'm entirely willing - wanting even - to pay for it.
Currently Kagi has my $, but I'm saddened and frustrated that they're not even focused on Search, they're focused on AI[1] and t-shirts.
Amazingly, in 2024, there is still a market opportunity for a good search engine.
It can't really just be me, can it?
[1]: https://hn.algolia.com/?query=%22kagi%22+%22ai%22
-
Ask HN: Is Hacker News under attack from spam bots?
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
For historical purposes
-
Tesla Recalls All Cybertrucks for Faulty Accelerator Pedals
Most likely because there have been oodles of low-quality stories on these topics. We turned the flags off on this one since it maybe rises above the noise (see https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... for past explanations on how we approach that).
-
Show HN: What Are You Working On?
Hey HN,
I'm sure you've seen the monthly "Ask HN: What Are You Working On?" headlines on [Hacker News](https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...).
Honestly, it's my favorite topic because it's packed with insights about what other hackers are up to.
I wondered what it would be like if instead of just a headline, there was a whole website where hackers could post daily updates, and where we could follow the hackers we're interested in for their latest updates. And so, this web site was born.
I hope it gets used frequently so we can all benefit from it together. I look forward to hearing your thoughts.
Let me know what you think!
-
Not Apply to YC
I don't know what one thing you're referring to, but it's a core principle of HN to try to avoid repetition, and especially the repetition+indignation combo, which is the commonest and most tedious thing on the internet.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
-
Nand to Tetris: Building a Modern Computer System from First Principles
Happy 10,000 day to you
https://news.ycombinator.com/from?site=nand2tetris.org
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
-
Moxie: I'm no longer involved at Signal
not sure. I searched comments: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=fal...
Most recent are more culture wars stuff but some earlier ones appear to suggesting a degree of alignment with the USA government.
What are some alternatives?
performer-pytorch - An implementation of Performer, a linear attention-based transformer, in Pytorch
duckduckgo-locales - Translation files for <a href="https://duckduckgo.com"> </a>
attention-is-all-you-need-pytorch - A PyTorch implementation of the Transformer model in "Attention is All You Need".
v - Simple, fast, safe, compiled language for developing maintainable software. Compiles itself in <1s with zero library dependencies. Supports automatic C => V translation. https://vlang.io
HJxB - Continuous-Time/State/Action Fitted Value Iteration via Hamilton-Jacobi-Bellman (HJB)
parser - 📜 Extract meaningful content from the chaos of a web page
jax-resnet - Implementations and checkpoints for ResNet, Wide ResNet, ResNeXt, ResNet-D, and ResNeSt in JAX (Flax).
readability - A standalone version of the readability lib
tldr-transformers - The "tl;dr" on a few notable transformer papers (pre-2022).
yq - Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
elegy - A High Level API for Deep Learning in JAX
milkdown - 🍼 Plugin driven WYSIWYG markdown editor framework.