JSON Machine
etl
| JSON Machine | etl | |
|---|---|---|
| 4 | 1 | |
| 1,007 | 339 | |
| - | 1.2% | |
| 6.9 | 9.3 | |
| 5 months ago | 4 days ago | |
| PHP | PHP | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
JSON Machine
-
Handling big API request (2.5gb) without timeout
To handle this problem i've used "https://github.com/halaxa/json-machine" to decrease the memory usage and "https://github.com/spatie/async" to decrease the take taken for the "update process" to complete
-
Lazy JSON
Under the hood, it uses JSON Machine as a lexer and parser.
Seems like the issue is still open ! https://github.com/halaxa/json-machine/issues/36
-
Is any one actually using PHP generators in their projects?
Yes. We used it heavily when processing large amounts of data from huge API responses. I like how the whole processing chain can be built on top of it, for example using nikic/iter. Generators are also the heart of a low-memory JSON parsing library halaxa/json-machine, which I maintain, and many other similar projects.
etl
-
Is any one actually using PHP generators in their projects?
We are dealing with so many imports/syncs with external systems that half of the project is currently based on Generators/Iterators. It's all nicely packed inside of https://github.com/flow-php/etl in order to provide common interface and unified developer experience. This ETL is nothing more than an abstraction yielding Rows through the whole pipeline. It helps us to deal with many popular file/api formats, like JSON, CSV, XML We have following use cases for generators/itsrators:
What are some alternatives?
msgpack.php - A pure PHP implementation of the MessagePack serialization format / msgpack.org[PHP]
UniExtract2 - Universal Extractor 2 is a tool to extract files from any type of archive or installer.
Laminas Serializer - Serialize and deserialize PHP structures to a variety of representations
Tukio - A complete and robust implementation of the PSR-14 EventDispatcher specification.
php-caching-generator - A rewindable PHP Generator class that caches its generated values.
collection - A (memory) friendly, easy, lazy and modular collection class.
collection - The Museum of Modern Art (MoMA) collection data
Iter - Iteration primitives using generators
aphiria - A simple, extensible REST API framework for PHP
lazy-json - 🐼 Framework-agnostic package to load JSON of any dimension and from any source into Laravel lazy collections recursively.
Fregata - Fregata - a PHP database migrator