grobid
Deep Java Library (DJL)
grobid | Deep Java Library (DJL) | |
---|---|---|
11 | 13 | |
3,075 | 3,853 | |
- | 1.6% | |
9.2 | 9.5 | |
3 days ago | 2 days ago | |
Java | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
grobid
- Show HN: Open-source Rule-based PDF parser for RAG
- How to ingest image based PDFs into private GPT model?
- 🥪 Best Sites For ebooks, articles, research papers etc..🥪
- Grobid – ML software for extracting information from scholarly documents
-
How to create a web app that turns academic papers into text documents
Interesting concept. Grobid tries to do the same https://github.com/kermitt2/grobid
-
Extract research paper`s references
I would suggest using grobid - a pipeline for extracting scientific PDFs into a common XML format which can be easily parsed. Grobid has quite a nice mature REST API that I've used in some of my own projects. It parses references and matches them to their DOI using the CrossRef API with a reported 95% F1 score. This should make your job pretty simple as far as I can tell - all you'd need to do is run your papers through grobid and then build a citation graph by comparing document DOIs.
- Free/open-source alternatives to Connected Papers...?
-
Seeking Advice: How to extract Abstract from scientific journals (.pdfs) 10k+.
Just use science-parse or GROBID. They have been designed for that exact reason.
-
Project to rebuild papers with plaintext markup languages
- I ended up using Grobid, which converts the PDF to a very detailed XML format. The format is not a word processing format though, but a format specifically for representing scientific documents. I don't know, if it would, for example, contain tags about bold or italicized text. The tool is working really well, but since you probably cannot use the output XML format directly, it will need some postprocessing, which would be relatively simple with XML parsing libraries.
-
[D] What pdf parser do you use for paragraph parsing for huggingface models
A few years ago I evaluated a few open source tools. In the end focused on GROBID. As usual, it depends on the type of document whether it works well for your use-case. There is some focus on it being "fast" (if that is a concern).
Deep Java Library (DJL)
-
Is deeplearning4j a good choice?
It seems to have been picked up by Eclipse and there is also Oracle Labs' Tribuo and Deep Java Library. All seem active, but I don't know much about any of them. I agree it's probably best to follow the community and use a more popular tool like PyTorch.
-
Just want to vent a bit
Although it may be a bit more work, you can do both machine learning and AI in Java. If you are doing deep learning, you can use DeepJavaLibrary (I do work on this one at Amazon). If you are looking for other ML algorithms, I have seen Smile, Tribuo, or some around Spark.
-
Best way to combine Python and Java?
Image preprocessing I know less about, but tokenization is something I've dealt with a bunch. There are a few options, either push the tokenizer into the ONNX model and use MS's ONNX Runtime extensions (we've used this when working with sentencepiece tokenizers), port the tokenizer entirely to Java (we did this for BERT), or use a sentencepiece or HF tokenizers wrapper directly (e.g. Amazon's DJL did this - HF, sentencepiece).
-
Anybody here using Java for machine learning?
https://djl.ai/ seems very promising. I've played around with it quite a bit, not in real production though. It's a very well documented (https://d2l.djl.ai/) and active project, with Amazon working on it.
- Good document classification library in Java
-
2021-09 - Plans & Hopes for Clojure Data Science
Here is link number 1 - Previous text "DJL"
-
[D] Java vs Python for Machine learning
To give a contrasting perspective, I think the Java ecosystem is much better suited for many data science tasks, and has a growing and well-maintained set of libraries for general purpose machine learning. I won't list them all, but TF-Java, DJL et al. have implementations of many modern architectures and there are a number of excellent libraries (CoreNLP, Lucene et al.) for working with text.
- Does Java has similar project like this one in C#? (ml, data)
-
If it gets better w age, will java become compatible for machine learning and data science?
I think DJL also use use it for their tutorials - https://docs.djl.ai/jupyter/tutorial/01_create_your_first_network.html.
-
Machine learning on JVM
AWS Deep Learning more deep learning.
What are some alternatives?
Parsr - Transforms PDF, Documents and Images into Enriched Structured Data
Deeplearning4j - Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.
CERMINE - Content ExtRactor and MINEr
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
Smile - Statistical Machine Intelligence & Learning Engine
mediapipe - Cross-platform, customizable ML solutions for live and streaming media.
science-parse - Science Parse parses scientific papers (in PDF form) and returns them in structured form.
Tribuo - Tribuo - A Java machine learning library
datahub - The Metadata Platform for your Data Stack
CoreNLP - CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
Apache Flink - Apache Flink