libpostal
duckling
Our great sponsors
libpostal | duckling | |
---|---|---|
5 | 13 | |
3,951 | 4,019 | |
0.9% | 0.7% | |
5.9 | 0.0 | |
3 months ago | 2 months ago | |
C | Haskell | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
libpostal
-
Install Python Libraries Using Command Prompt
@echo off REM Check if MSYS2 and MinGW are installed where msys2 2>nul >nul if %errorlevel% equ 0 ( echo MSYS2 is already installed. Use --force to reinstall. ) else ( REM Install MSYS2 and MinGW choco install msys2 refreshenv ) REM Check if MSYS2 packages are updated pacman -Qu 2>nul >nul if %errorlevel% equ 0 ( echo MSYS2 packages are already updated. Use --force to reinstall. ) else ( REM Update MSYS2 packages pacman -Syu ) REM Check if build dependencies are installed pacman -Q autoconf automake curl git make libtool gcc mingw-w64-x86_64-gcc 2>nul >nul if %errorlevel% equ 0 ( echo Build dependencies are already installed. Use --force to reinstall. ) else ( REM Install build dependencies pacman -S autoconf automake curl git make libtool gcc mingw-w64-x86_64-gcc ) REM Check if libpostal is cloned if exist libpostal ( echo libpostal repository is already cloned. Use --force to reinstall. ) else ( REM Clone libpostal repository git clone https://github.com/openvenues/libpostal ) cd libpostal REM Check if libpostal is built and installed if exist C:/Program Files/libpostal/bin/libpostal.dll ( echo libpostal is already built and installed. Use --force to reinstall. ) else ( REM Build and install libpostal cp -rf windows/* ./ ./bootstrap.sh ./configure --datadir=C:/libpostal make -j4 make install ) REM Check if libpostal is added to PATH environment variable setx /m PATH "%PATH%;C:\Program Files\libpostal\bin" 2>nul >nul if %errorlevel% equ 0 ( echo libpostal is already added to PATH environment variable. Use --force to reinstall. ) else ( REM Add libpostal to PATH environment variable setx PATH "%PATH%;C:\Program Files\libpostal\bin" ) REM Test libpostal installation libpostal "100 S Broad St, Philadelphia, PA" pause
-
Transforming free-form geospatial directions into addresses - SOTA?
I know of https://github.com/openvenues/libpostal which handles typos and omissions in addresses, but I am looking into a more fuzzy description of a location.
-
[P] Better ways to clean lots of text?
use an address parser library like libpostal https://github.com/openvenues/libpostal
-
complete stack for an analysis team
Also, what OS(s) does IT support for clients and servers? I think Libpostal doesn't officially support Windows, but you can build it to target that. Seems difficult and/or unreliable though: https://github.com/openvenues/libpostal/issues/219
-
Automating a Web Scraper
You can feed libpostal sequence of string until it gives good results. A lot of miss, some hits, score the hits. https://github.com/openvenues/libpostal
duckling
-
Experimental library for scraping websites using OpenAI's GPT API
For the reasons others have said I don't see it replacing 'traditional' scraping soon. But I am looking forward to it replacing current methods of extracting data from the scraped content.
I've been using Duckling [0] for extracting fuzzy dates and times from text. It does a good job but I needed a custom build with extra rules to make that into a great job. And that's just for dates, 1 of 13 dimensions supported. Being able to use an AI that handles them with better accuracy will be fantastic.
Does a specialised model trained to extract times and dates already exist? It's entity tagging but a specialised form (especially when dealing with historical documents where you may need Gregorian and Julian calendars).
[0] https://github.com/facebook/duckling
-
Automatisiert Kalendereinträge erstellen aus Mails mit Formatlosen Datumsangaben
Ah, sorry: https://github.com/facebook/duckling
-
Transforming free-form geospatial directions into addresses - SOTA?
To understand what relative distance and direction is indicated from the reference point, I'd look into something like Facebook & Wit.AI's Duckling, and a custom classifier to identify if it's on the reference point ("corner of"), or some distance from ("200 meters southwest"). If you can parse out a distance and direction, then it's all logic to plot the point.
-
Programming languages endorsed for server-side use at Meta
It also powers the backend of Wit.ai which FB owns. Wit's open-source entity parser, duckling, is written entirely in Haskell. https://github.com/facebook/duckling
- Data Cleaning using Machine Learning?
-
Unsplash chatbot for Discord, Pt. 2: more ways to bring pictures to Discord
Our RandomPicForLater intent will have one slot called reminderTime and will be of type @duckling.time. Duckling is a library that extracts entities from text, and it is one of the tools used in JAICP for this purpose. Entity types in Duckling are called dimensions and there's a number of them built in, among them is Time which suits us perfectly since we need to ask users when they want us to schedule a post for and then parse a text input into a datetime object.
-
Dependencies difference between cabal and stack
I'm working on a pretty interesting project right now and I'm having different results depending on the build tool used: with cabal, the test suite fails but it passes with stack.
-
Running Duckling on Windows
Try downloading the v0.2.0.0 release, extracting it somewhere, opening that location in powershell, and running these commands:
-
[ANN] Duckling v0.2.0.0 released
Duckling (https://github.com/facebook/duckling) is a library for parsing text into structured data.
-
Extract name:value relationships from plain text
If you really want high precision, Duckling is a good project to check out https://github.com/facebook/duckling
What are some alternatives?
usaddress - :us: a python library for parsing unstructured United States address strings into address components
spaCy - 💫 Industrial-strength Natural Language Processing (NLP) in Python
neuralcoref - ✨Fast Coreference Resolution in spaCy with Neural Networks
ctparse - Parse natural language time expressions in python
splink - Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
Giveme5W1H - Extraction of the journalistic five W and one H questions (5W1H) from news articles: who did what, when, where, why, and how?
rmlint - Extremely fast tool to remove duplicates and other lint from your filesystem
syntaxdot - Neural syntax annotator, supporting sequence labeling, lemmatization, and dependency parsing.
jdupes - A powerful duplicate file finder and an enhanced fork of 'fdupes'.
Kornia - Geometric Computer Vision Library for Spatial AI
kvdo - A kernel module which provide a pool of deduplicated and/or compressed block storage.
BLINK - Entity Linker solution