fetch-event-source VS openai-node

* `text/event-stream` relies on a modified version of [Azure's fetch-event-source](https://github.com/Azure/fetch-event-source). They handle errors and chunking better than I was. New version has been tested out to 4 billion fragments rendered on a single page with no errors.

The EventSource Web API for handling SSEs (built into most modern browsers) unfortunately supports only GET requests, which was quite limiting when a POST request with larger body JSON data was required. As an alternative, you can use the Fetch API or a ready library like Microsoft’s Fetch Event Source.

Aforementioned limitations are taken care with a specialised SSE client (inspired by the awesome @microsoft/fetch-event-source) and two separate connection modes: the HTTP/1 safe "single connection mode" that uses a single SSE connection for receiving events with separate HTTP requests dictating the behaviour, and the HTTP/2+ "distinct connections mode" that uses distinct SSE connections for each GraphQL operation, accommodating the parameters in the request itself.

We try to keep it to a minimum, especially in JS (though we have some nice improvements coming soon when we deprecate node-fetch in favor of built-in fetch). The package sizes aren't tiny because we include thorough types and sourcemaps, but the bundle sizes are fairly tidy.

Here's an example of a typical RESTful endpoint (Lithic's `client.cards.create()`:

https://github.com/lithic-com/lithic-node/blob/36d4a6a70597e...

Here are some example repos produced by Stainless:

1. https://github.com/openai/openai-node

Have you seen/tried the `.runTools()` helper?

Docs: https://github.com/openai/openai-node?tab=readme-ov-file#aut...

Example: https://github.com/openai/openai-node/blob/bb4bce30ff1bfb06d...

(if what you're fundamentally trying to do is really just get JSON out, then I can see how json_mode is still easier).

And so, I impulsively upgraded to the latest version of openai (I guess not anymore) without the fear of getting cut by cutting edge 😝 and got it working for some random text

UPDATE 2023/11/15: I used fetch and custom streams because at the time of writing, the openai module on NPM did not properly support streaming responses. That issue has been fixed, and I think a better solution would be to use that module and pipe their data through a TransformStream to send to the client. That version is not reflected here.

You may notice the JavaScript package available on NPM called openai. We will not be using this, as it doesn’t quite support some things we’ll want to do, that fetch can.

openai - For using the GPT-3.5-turbo model to answer the questions

We have a bit of context about this in the readme: https://github.com/lastmile-ai/aiconfig#what-problem-it-solv.... The main issue with keeping it in code is that it tangles application code with prompts and model-specific logic.

That makes it hard to evaluate the genAI parts of the application, and also iterating on the prompts is not as straightforward as opening up a playground.

Having the config be the source of truth let's you connect it to your application code (and still source controlled), lets you evaluate the config as the AI artifact, and also lets you open the config in a playground to edit and iterate.

For example, compare how much simpler openai function calling becomes with storing the stuff as a config: https://github.com/lastmile-ai/aiconfig/blob/main/cookbooks/... vs using vanilla openai directly (https://github.com/openai/openai-node/blob/v4/examples/funct...)

In your preferred serverless environment, make sure you install the OpenAI API Library and Vercel AI library to get started.

A package I help maintain was released about a month ago, and we've had bug reports related to this: https://github.com/openai/openai-node/issues/304#issuecommen...

We simply chose to mark Jest 27 as unsupported in this case, but it did cost some debugging time.