Ask HN: Have you seen anything original produced by generative AI?

• WeightWatcher

  4 1,393 9.1 Python

  The WeightWatcher tool for predicting the accuracy of Deep Neural Networks

  

These models are pretty much always extrapolating [0]

Whether the extrapolation is crude/low-rank or astute/high-rank is a question of memorization vs generalization. That gets into the question of whether or not the model is over-fitted or under-fitted. There are certain heuristics borrowed from high dimensional statistical physics that can be used to guess how good the test performance of a model will be on a typical task without even knowing what the test data is [1].

Originality for me means finding better answers to sub-tasks, and then combining those answers together in a better way. This is the nirvana of cross-entropy minimization - the emergence of capability results from gaining the ability to amass a wider range of skills, improving upon them, and percolating those improvements towards multiply the leverage of other skills.

How long such a thing can keep improving with current tech, who knows, but you should really think critically about whether that sounds just like interpolation through the corpus.

[0] Learning in High Dimension Always Amounts to Extrapolation - https://arxiv.org/abs/2110.09485

[1] https://github.com/CalculatedContent/WeightWatcher

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project