Douglas Hofstadter changes his mind on Deep Learning and AI risk

• nanoGPT

  69 31,914 5.4 Python

  The simplest, fastest repository for training/finetuning medium-sized GPTs.

Just being a part of any auto-regressive system does not contradict his statement.

Go look at the GPT training code, here is the exact line: https://github.com/karpathy/nanoGPT/blob/master/train.py#L12...

The model is only trained to predict the next token. The training regime is purely next-token prediction. There is no loopiness whatsoever here, strange or ordinary.

Just because you take that feedforward neural network and wrap it in a loop to feed it its own output does not change the architecture of the neural net itself. The neural network was trained in one direction and runs in one direction. Hofstadter is surprised that such an architecture yields something that looks like intelligence.

He specifically used the correct term "feedforward" to constrast with recurrent neural networks, which GPT is not: https://en.wikipedia.org/wiki/Feedforward_neural_network

• AutoGPT

  180 161,405 9.9 JavaScript

  AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

  

> but LLMs can’t do anything, really

Do you think https://github.com/Significant-Gravitas/Auto-GPT et al will become more performant as models improve?

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project