openai-node VS SlothAI

Okay, now we've confirmed the API endpoint is working, let's connect it to OpenAI first, install the OpenAI package, navigate to the route directory, and run the command below in our terminal

For my setup, I am using the node version of the openai sdk.

We try to keep it to a minimum, especially in JS (though we have some nice improvements coming soon when we deprecate node-fetch in favor of built-in fetch). The package sizes aren't tiny because we include thorough types and sourcemaps, but the bundle sizes are fairly tidy.

Here's an example of a typical RESTful endpoint (Lithic's `client.cards.create()`:

https://github.com/lithic-com/lithic-node/blob/36d4a6a70597e...

Here are some example repos produced by Stainless:

1. https://github.com/openai/openai-node

Have you seen/tried the `.runTools()` helper?

Docs: https://github.com/openai/openai-node?tab=readme-ov-file#aut...

Example: https://github.com/openai/openai-node/blob/bb4bce30ff1bfb06d...

(if what you're fundamentally trying to do is really just get JSON out, then I can see how json_mode is still easier).

And so, I impulsively upgraded to the latest version of openai (I guess not anymore) without the fear of getting cut by cutting edge 😝 and got it working for some random text

UPDATE 2023/11/15: I used fetch and custom streams because at the time of writing, the openai module on NPM did not properly support streaming responses. That issue has been fixed, and I think a better solution would be to use that module and pipe their data through a TransformStream to send to the client. That version is not reflected here.

You may notice the JavaScript package available on NPM called openai. We will not be using this, as it doesn’t quite support some things we’ll want to do, that fetch can.

openai - For using the GPT-3.5-turbo model to answer the questions

We have a bit of context about this in the readme: https://github.com/lastmile-ai/aiconfig#what-problem-it-solv.... The main issue with keeping it in code is that it tangles application code with prompts and model-specific logic.

That makes it hard to evaluate the genAI parts of the application, and also iterating on the prompts is not as straightforward as opening up a playground.

Having the config be the source of truth let's you connect it to your application code (and still source controlled), lets you evaluate the config as the AI artifact, and also lets you open the config in a playground to edit and iterate.

For example, compare how much simpler openai function calling becomes with storing the stuff as a config: https://github.com/lastmile-ai/aiconfig/blob/main/cookbooks/... vs using vanilla openai directly (https://github.com/openai/openai-node/blob/v4/examples/funct...)

In your preferred serverless environment, make sure you install the OpenAI API Library and Vercel AI library to get started.

I built https://mitta.us during Covid. I use it every day and am now working on AI pipelines for the documents saved to it over at https://mitta.ai. It's not a traditional crawler, and only saves what you tell it to save. There's also a Chrome extension to save pages quickly while browsing, and it allows you to remap search in chrome to the console, which then forwards you where you need to go based on the content. Still very much a WIP, but getting there!

On https://mitta.us, I snapshot the whole page into a PNG using a headless browser. That PNG can then be sent into a pipeline to have the text extracted by Google's Document AI. That text is sent off to OpenAI to embed it, and stored in Solr, where it can be used later to search for it. The main issue is that I can't pick up links with it, but most of the time that really doesn't matter as long as I can see the page like it was rendered and have the text searchable.

I love this! Been working on something similar that uses containers and Jinja2 templates: https://github.com/kordless/SlothAI. A demo is running at https://mitta.ai. No docs yet and the auth system is getting an upgrade.