llama3 VS lapce

and deceptive if not inaccurate. Meta's Model Cards specifically call out that they were trained on publicly available datasets and NOT any Meta user data.

For example: https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

I was curious so I tried to answer this question

---

Training Llama 3 models emitted 2290 tons CO2e (https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md), and took 7.7 million GPU hours. Those GPU hours are for H100s, which consume 700W. So the conversion is approximately 2290 / (7.7e6 * 3600 * 700 / 1e9) ~= 0.12 tons CO2e per GPU-gigajoule.

A100s (what Huggingface offers) consume 400W (https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...) and cost $2.21/hour (in e.g. CoreWeave https://www.coreweave.com/gpu-cloud-pricing). So $10 million in H100s buys you ($10e6 / $2.21/h * 3600s/h) * 400W ~= 6515 Gigajoules in GPU-hours.

So Huggingface's offering will emit ~781 tons CO2e. Less if they've inflated the value of the compute they provide, which they have an incentive to do, but let's round to 800 tons.

---

According to https://www.carbonindependent.org/22.html, one Boeing-737-400 flying for 926km emits (3.61 tons fuel/flight * 3.15(g CO2e / g fuel)) = 11.37 tons CO2e .

So $10million in compute is like ~72 Boeing-737-400 international flights.

> It takes years to become competent at the math needed for AI

(Assuming that "AI" refers to large language models)

The best open source LLM fits in less than 300 lines of code and consists mostly of matrix multiplications. https://github.com/meta-llama/llama3/blob/main/llama/model.p...

Anyone with a basic grasp of linear algebra can probably learn to understand it in a week.

From the readme [0]:

> All models support sequence length up to 8192 tokens, but we pre-allocate the cache according to max_seq_len and max_batch_size values. So set those according to your hardware.

[0] https://github.com/meta-llama/llama3/tree/14aab0428d3ec3a959...

Now, let's start building the next part of the chatbot. In this part, we will be using the LLM from Ollama and integrating it with the chatbot. More particularly, we will be using the Llama-3 model. Llama-3 is Meta's latest and most advanced open-source large language model (LLM). It is the successor to the previous Llama 2 model and represents a significant improvement in performance across a variety of benchmarks and tasks. Llama 3 comes in two main versions - an 8 billion parameter model and a 70 billion parameter model. Llama 3 supports longer context lengths of up to 8,000 tokens.

Apparently, currently based on width: https://github.com/lapce/lapce/commit/87e0fc06f1862d9124d3fe...

The list of available Linux packages seems to be here:

https://github.com/lapce/lapce/blob/master/docs/installing-w...

As a Neovim afficionado - I think you lose some credibility recommending it as an alternative to VSCode and Sublime. They're different beasts. I imagine a lot of people would be immediately turned off if they were expecting a VSCode/Sublime-like editing experience.

I'd put Lapce in that spot: https://lapce.dev/