paperetl
rdm
Our great sponsors
| paperetl | rdm | |
|---|---|---|
| 12 | 2 | |
| 316 | 105 | |
| 7.9% | 0.0% | |
| 6.3 | 0.0 | |
| 5 months ago | over 1 year ago | |
| Python | Python | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
paperetl
- Show HN: Open-source Rule-based PDF parser for RAG
-
Oracle of Zotero: LLM QA of Your Research Library
Nice project!
I've spent quite a lot of time in the medical/scientific literature space. With regards to LLMs, specifically RAG, how the data is chunked is quite important. With that, I have a couple projects that might be beneficial additions.
paperetl (https://github.com/neuml/paperetl) - supports parsing arXiv, PubMed and integrates with GROBID to handle parsing metadata and text from arbitrary papers.
paperai (https://github.com/neuml/paperai) - builds embeddings databases of medical/scientific papers. Supports LLM prompting, semantic workflows and vector search. Built with txtai (https://github.com/neuml/txtai).
While arbitrary chunking/splitting can work, I've found that integrating parsing that has knowledge of medical/scientific paper structure increases the overall accuracy and experience of downstream applications.
-
[P] Parse research papers into structured data
paperai | paperetl
- Parse research papers into a structured dataset
- ETL for medical and scientific papers
- Show HN: ETL for Medical and Scientific Papers
-
Seeking Advice: How to extract Abstract from scientific journals (.pdfs) 10k+.
paperai and paperetl are a set of projects to consider for this task.
- paperetl: ETL processes for medical and scientific papers
rdm
-
Ask HN: How do you keep track of software requirements and test them?
Been working as a consultant and engineer on FDA regulated software for about 8 years now. I have seen strategies from startups to huge companies.
I have seen requirements captured in markdown files, spreadsheets, ticket management systems like Redmine, Pivotal, Jira, GitLab, Azure Devops, GitHub Issues, and home grown systems.
If I had to start a new medical device from scratch today, I would use Notion + https://github.com/innolitics/rdm to capture user needs, requirements, risks, and test cases. Let me know if there is interest and I can make some Notion templates public. I think the ability to easily edit relations without having to use IDs is nice. And the API makes it possible to dump it all to yaml, version control and generate documentation for e-signature when you need it. Add on top of that an easy place to author documentation, non-software engineer interoperability, discoverable SOPs, granular permissions, and I think you have a winning combination.
-
How We Develop AI for 510(k)-Cleared Devices
We've had somewhat of a hard time applying agile to IEC 62304, although it's not impossible.
This document has some useful tips:
AAMI TIR45:2012 (R2018) Guidance On The Use Of AGILE Practices In The Development Of Medical Device Software
Also, we have an open source offering that includes an IEC62304 compliant software plan. You can check this out here:
https://github.com/innolitics/rdm/blob/master/rdm/init_files...
What are some alternatives?
SciencePlots - Matplotlib styles for scientific plotting
strictdoc - Software for technical documentation and requirements management.
tika-python - Tika-Python is a Python binding to the Apache Tika™ REST services allowing Tika to be called natively in the Python community.
MkDocs - Project documentation with Markdown.
ciscoconfparse - Parse, Audit, Query, Build, and Modify Cisco IOS-style configurations.
wtfpython - What the f*ck Python? 😱
paperai - 📄 🤖 Semantic search and workflows for medical/scientific papers
doorstop - Requirements management using version control.
dagster - An orchestration platform for the development, production, and observation of data assets.
professional-programming - A collection of learning resources for curious software engineers
science-parse - Science Parse parses scientific papers (in PDF form) and returns them in structured form.