[D] Anyone else witnessing a panic inside NLP orgs of big tech companies?

• Killed by Google

  2,302 2,348 7.0 TypeScript

  Part guillotine, part graveyard for Google's doomed apps, services, and hardware.

Google 2023 is the company who killed RSS, removed "don't be evil", slapped adverts all over YouTube while asking you to pay for it, produces garbage SEO-spam-filled ad-ridden search results which ignore what you searched for, turned Android into creepy stalker advert OS, shuts people's paid-for accounts without warning and no human support available, has an also-ran cloud service behind even Microsoft's, and is generally known for being unable to innovate, unable to release products, unwilling to support things long term - https://killedbygoogle.com/ and https://gcemetery.co/

• mup

  12 1,169 3.8 Jupyter Notebook

  maximal update parametrization (µP)

  

Well, but it isn't like this kind of research is new. Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (2022) tuned hyperparameters in 40M model, transferred it to 6.7B model, and beat OpenAI's 6.7B run. It is likely what OpenAI did is perfecting this kind of research. I note that four authors of that paper (Igor Babuschkin, Szymon Sidor, David Farhi, Jakub Pachocki) are credited for pretraining optimization & architecture at https://openai.com/contributions/gpt-4.

• SurveyJS

  surveyjs.io featured

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

Suggest a related project