webextension-polyfill VS spider

Porting this extension to Firefox should be relatively straightforward using the webextension polyfill: https://github.com/mozilla/webextension-polyfill

Firefox maintain a library for unified extension API https://github.com/mozilla/webextension-polyfill

Their type definition for HAR request isn't exported https://github.com/DefinitelyTyped/DefinitelyTyped/blob/mast...

best is to use https://github.com/mozilla/webextension-polyfill

The Browser Extension Working Group at W3.org proposes the web standards based on the Chrome extension manifest, which supports all web browsers. Based on that proposal, Mozilla has released the Browser Extension Polyfill library that supports the modern promise pattern instead of callback. Therefore, if you import this polyfill library, theoretically, your Chrome extension quickly turns into the browser extension that runs on multiple browser engines.

And the biggest tip that i received late. Use Typescript type by Mozilla to make your development much easier(autocomplete, inline docs etc): https://github.com/mozilla/webextension-polyfill

I've written extensions before and Firefox has a very good polyfill [0] that makes it quite easy to write extensions for all browsers. It does get a bit trickier if you also want to incorporate TypeScript [1] or React however.

[0] https://github.com/mozilla/webextension-polyfill

[1] https://github.com/Lusito/webextension-polyfill-ts

I used WebExtension polyfill[0] when adapting my FF addon to Chrome and admittedly all the intricate differences between APIs still costed me half a day of work.

I managed to have it done with only a few places where I branch on navigator.vendor, but If I wanted to ship different versions to AMO and CWS, I'd make use of something like DefinePlugin[1] for webpack to include/exclude code based on build target.

[0] https://github.com/mozilla/webextension-polyfill/

[1] https://github.com/webpack/docs/wiki/list-of-plugins#definep...

Initially I created wrapper functions to convert Chrome functions that require callback to return promise instead. The better approach, as I found out later, is probably to use webextension-polyfill from Mozilla and its types.

I mean, the browser apis are close (and Mozilla still has much better documentation) but there are a LOT of edges cases where behavior diverges.

Frankly - I'm a little peeved that Optional permissions in Firefox are STILL broken - The prompt can only be triggered in response to a user action, and Firefox blows the fuck up if you put a promise anywhere in between the user click and the call to the api. Which is hugely ironic, since Mozilla is the one pushing to move all the webext APIs to be promise based (and provides a nice helpful library for Chrome/Edge/Safari support: https://github.com/mozilla/webextension-polyfill) which... doesn't work on their platform. Doubly ironic, since the result is that most FF extensions just ask for more permissions up front, which is exactly the opposite of what you'd want in the "secure/private" world Mozilla claims they're pushing towards.

Hey all - I'm currently working on a self-service, natural language BI tool that aims to go beyond the base "text to sql" of current tools. I've got the bones built, but I'm struggling to develop a suite of test questions (ideally with complex metrics like "What is our profitability" or abstract concepts like "How are our sales doing"). Does anyone know of any lists of questions (and ideally answers on the quantitative questions) for the MS adventureworks database, or any other complex (30+ table) public test databases? I've looked at Spider, but most of the datasets are too small to simulate real-world business datasets, and the questions are more "can you write fancy SQL" and less "can you answer a vague stakeholder question on unknown data".

Hi HN community. We are excited to open source Dataherald’s natural-language-to-SQL engine today (https://github.com/Dataherald/dataherald). This engine allows you to set up an API from your structured database that can answer questions in plain English.

GPT-4 class LLMs have gotten remarkably good at writing SQL. However, out-of-the-box LLMs and existing frameworks would not work with our own structured data at a necessary quality level. For example, given the question “what was the average rent in Los Angeles in May 2023?” a reasonable human would either assume the question is about Los Angeles, CA or would confirm the state with the question asker in a follow up. However, an LLM translates this to:

select price from rent_prices where city=”Los Angeles” AND month=”05” AND year=”2023”

This pulls data for Los Angeles, CA and Los Angeles, TX without getting columns to differentiate between the two. You can read more about the challenges of enterprise-level text-to-SQL in this blog post I wrote on the topic: https://medium.com/dataherald/why-enterprise-natural-languag...

Dataherald comes with “batteries-included.” It has best-in-class implementations of core components, including, but not limited to: a state of the art NL-to-SQL agent, an LLM-based SQL-accuracy evaluator. The architecture is modular, allowing these components to be easily replaced. It’s easy to set up and use with major data warehouses.

There is a “Context Store” where information (NL2SQL examples, schemas and table descriptions) is used for the LLM prompts to make the engine get better with usage. And we even made it fast!

This version allows you to easily connect to PG, Databricks, BigQuery or Snowflake and set up an API for semantic interactions with your structured data. You can then add business and data context that are used for few-shot prompting by the engine.

The NL-to-SQL agent in this open source release was developed by our own Mohammadreza Pourreza, whose DIN-SQL algorithm is currently top of the Spider (https://yale-lily.github.io/spider) and Bird (https://bird-bench.github.io/) NL 2 SQL benchmarks. This agent has outperformed the Langchain SQLAgent anywhere from 12%-250%.5x (depending on the provided context) in our own internal benchmarking while being only ~15s slower on average.

Needless to say, this is an early release and the codebase is under swift development. We would love for you to try it out and give us your feedback! And if you are interested in contributing, we’d love to hear from you!

This is an active field of research. You might want to look at the main challenge dataset for it : https://yale-lily.github.io/spider. It would be interesting to use ChatGPT's model as a pre-processor and then feed its output in to a more finetuned model like PICARD.

We tried using OpenAI/Davinci for SQL query authoring, but it quickly became obvious that we are still really far from something the business could find value in. The state of the art as described below is nowhere near where we would need it to be:

https://yale-lily.github.io/spider

https://arxiv.org/abs/2109.05093

https://github.com/ElementAI/picard

To be clear, we haven't tried this on actual source code (i.e. procedural concerns), so I feel like this is a slightly different battle.

The biggest challenge I see is that the queries we would need the most assistance with are the same ones that are the rarest to come by in terms of training data. They are also incredibly specific in the edge cases, many time requiring subjective evaluation criteria to produce an acceptable outcome (i.e. recursive query vs 5k lines of unrolled garbage).

I've been looking for databases with real-world schema and faker data (eg 10,000 entries of fake users) to test my natural langaugae to SQL generative model, as well as the efficiency of the generated queries

The cloest thing I can find is annotated dataset like Spider (https://yale-lily.github.io/spider) but after digging more into it, it's not as real-world-ish as I've hoped for.

Are there any SaSS, paid services, etc, where I can have access databases with complex real-world(-ish) schemas (populated with real-world-ish data)?

Thanks!

There are projects out there that do this.

Possibly relevant: https://yale-lily.github.io/spider

I briefly worked on a startup to commercialize this tech, but we decided it wasn't accurate enough to be useful. It was very cool when it actually worked. If you can only produce what you want half the time on simple queries, that doesn't seem very useful to me though.

There is very less description available in the Spider dataset research paper. Which says that these files are used for value specific queries. Does that means gold.sql files should only contain queries with value check (for example: SELECT * FROM table WHERE student_name = 'Student_A'). If that's the case there are many instances in gold files without actual values from the dataset (for example: SELECT COUNT(*) FROM table). Thanks.