DeepSeek-V3 VS openai-python

DeepSeek API integration is one of the most popular topics recently, mostly because of the results of its commendable benchmarks and cheaper costs. So, there’s a lot to gain from exploring how you can integrate DeepSeek APIs into your websites. Read on below to get started!

What distinguishes DeepSeek-V3 is its training efficiency—completed using only 2.664M H800 GPU hours on 14.8 trillion tokens, making it remarkably cost-effective for its size. Technical specifications are available on the GitHub page for DeepSeek-V3.

DeepSeek, known for its high-performance AI models like R1 and V3, has been a game-changer in the AI landscape. However, recent reports have highlighted issues with API instability, affecting developers and users who rely on these services. Understanding the root causes of this instability is essential for addressing and mitigating these issues.

It is not FOSS. The LLM industry has repurposed "open source" to mean "you can run the model yourself." They've released the model, but it does not meet the 'four freedoms' standard: https://github.com/deepseek-ai/DeepSeek-V3/blob/main/LICENSE...

Typically, training parts of an AI model usually meant updating the whole thing, even if some parts didn't contribute anything, which lead to a massive waste of resources. To solve this, they introduced an Auxiliary-Loss-Free (ALS) Load Balancing. The ALS Load Balancing works by introducing a bias factor to prevent overloading one chip, while under-utilizing another (Source). This resulted in only 5% of the model's parameters being trained per-token, and around 91% cheaper cost to train than GPT 4 (GPT 4 costed $63 million to train (Source) and V3 costed $5.576 million to train. (Source))

According to the official paper, DeepSeek took only $5.6 mln to train with impressive results. This is a remarkable achievement for a large language model (LLM). In comparison, OpenAI's CEO Sam Altman admitted that training OpenAI GPT-4 took over $100 mln, not saying how much more. Some AI specialists assume that the estimation of the DeepSeek training expense is underreported. Nevertheless, the hidden gem is not how much it cost to train but how drastically it improved runtime requirements.

> If the magic values are some kind of microcode or firmware, or something else that is executed in some way, then no, it is not really open source.

To my understanding, the contents of a .safetensors file is purely numerical weights - used by the model defined in MIT-licensed code[0] and described in a technical report[1]. The weights are arguably only really "executed" to the same extent kernel weights of a gaussian blur filter would be, though there is a large difference in scale and effect.

[0]: https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inferen...

[1]: https://arxiv.org/html/2412.19437v1

The github issue showed in the livestream is getting lots of traction: https://github.com/openai/openai-python/issues/2472

It was (attempted to be) solved by a human before, yet not merged...

Issue https://github.com/openai/openai-python/issues/2472 they worked and promised to submit the PR after the show is still open.

Just saying.

Check the official migration guide for updates.

Okay, I know Tesla's extremely high P/E ratio is because it's worth is not just tied to cars, and so xAI priced at $20B more than Anthropic does not necessarily mean xAI's AI products are that much better than Anthropic's (e.g. presumably xAI's worth is tied to synergies with Tesla FSD, Optimus, and maybe even Neurolink)...but what products does xAI actually offer, other than Grok being an add-on for premium X subscriptions?

Not only does the Grok API not have access to Grok 3, which was released more than a month ago, it doesn't even have it's own SDK? [0]

> Some of Grok users might have migrated from other LLM providers. xAI API is designed to be compatible with both OpenAI and Anthropic SDKs, except certain capabilities not offered by respective SDK. If you can use either SDKs, we recommend using OpenAI SDK for better stability.

(every code example has a call for `from openai import OpenAI`)

How would using Grok be viable for any enterprise? And if Grok's API is designed to be drop-in replacement for OpenAI's, how are they not able to just use Grok to whip up their own SDK variant based on OpenAI's open-sourced SDK [1] and API spec?

[0] https://docs.x.ai/docs/guides/migration

[1] https://github.com/openai/openai-python

If you want to get an idea for the changes, here's a giant commit where they updated ALL of the Python library examples in one go from the old chat completions to the new resources APIs: https://github.com/openai/openai-python/commit/2954945ecc185...

The API itself is pretty straightforward. You can use it with the OpenAI package on NPM or PIP, or make an HTTP Request. Note for this demo I will be using NodeJS. I will be working in an empty folder with an index.js file, and a package.json file.

To interact with the OpenAI API, you will install the openai package:

Python was chosen for its versatile libraries, particularly linkedin_jobs_scraper and openai. These packages streamlined the scraping and processing of job data.