Our great sponsors
-
-
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
-
-
-
Puts Debuggerer
Ruby library for improved puts debugging, automatically displaying bonus useful information such as source line number and source code.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
The /embed endpoint is the engine that fuels the process of downloading and generating embeddings for GitHub repositories. A simpler and straightforward approach was embraced for fetching a repository; instead of pursuing individual requests, Repo-Query taps into the GitHub API's /archive service. This streamlined the process by condensing repository downloads into a single, efficient request per repository. Without the need for iterating through individual file retrieval requests, reminiscent of Langchain's GitHub document loader. The download time for the (https://github.com/open-sauced/app) repository was now down to 5 seconds(50 Mbps) for me.
Repo-Query, a REST service that indexes public repositories and provides insightful answers to user queries, all within your browser through the OpenSaucedAI browser extension.
In perhaps a day or two, the prototype of Repo-Query was hacked together. Leaning on the abstractions provided by modern web development, the initial version of the service was woven together using Typescript and Langchain. The sandbox for this initial experiment was the repository gh-answering-proto, and the outputs of the semantic-search were close to what a human would find when asked to find a code chunk relevant to a query. This accuracy produced some impressive results.
Repo-Query, a REST service that indexes public repositories and provides insightful answers to user queries, all within your browser through the OpenSaucedAI browser extension.
At the time of the prototype's development, the Langchain GitHub loader sent one request per file to fetch the repository sequentially, leading to prolonged download times. In our case about 2 minutes for the insights.opensauced.pizza repository. This issue was later resolved in hwchase17/langchainjs#2224, enabling parallel requests for faster retrieval.
In the quest for more efficient solutions, the ONNX runtime emerged as a beacon of performance. The decision to transition from Typescript to Rust was an unconventional yet pivotal one. Driven by Rust's robust parallel processing capabilities using Rayon and seamless integration with ONNX through the ort crate, Repo-Query unlocked a realm of unparalleled efficiency. The result? A transformation from sluggish processing to, I have to say it, blazing-fast performance.
pub async fn fetch_repo_files(repository: &Repository) -> Result> { let Repository { owner, name, branch, } = repository; let url = format!("https://github.com/{owner}/{name}/archive/{branch}.zip"); let response = reqwest::get(url).await?.bytes().await?; ... }
In the quest for more efficient solutions, the ONNX runtime emerged as a beacon of performance. The decision to transition from Typescript to Rust was an unconventional yet pivotal one. Driven by Rust's robust parallel processing capabilities using Rayon and seamless integration with ONNX through the ort crate, Repo-Query unlocked a realm of unparalleled efficiency. The result? A transformation from sluggish processing to, I have to say it, blazing-fast performance.