meilisearch-js VS traefik

Thanks! I removed the URLs and now the searchable attributes are only title, description and some author fields!

> Just a detail, if you're making a `du -sh` on your computer, the size on the disk will stay unchanged because we are doing soft deletion ;). Don't worry. It will be physically deleted after a while if you need it in the future.

Ah I was just wildy undershooting the size I gave the PVC! I just gave it much more and it's fine -- right now it's resting around 19Gi of usage, which is actually a bit of a problem considering the data set was only like 4GB or something like that originally. That said, disk is really not an issue so I'll just throw more at it, maybe leave it at 32GB and call it a day (it's around 1.6MM documents out of ~2MM), so shouldn't be too much more.

> If you kept the default configuration of Meilisearch, the maximum size of the HTTP payload is 100Mb (for security). You change it here -> https://docs.meilisearch.com/learn/configuration/instance_op...

Thanks for this, I'll keep this in mind -- so I could actually pass off HUGE chunks to Meilisearch.

It seems like the larger the chunk the more efficient? There didn't seem to be much of a change in how much time it took to work through a chunk of documents, more just that having lots of smaller chunks would go slower. I started off with 10k in a batch, then went to 1k then back to 5k, maybe I should go to 100k docs in a batch and see the performance.

There's a blog post waiting to be written in here...

> addDocumentsInBatches() is just an helper to send your big json array into multiple parts, not absolutely sure you'll need it. (Code -> https://github.com/meilisearch/meilisearch-js/blob/807a6d827...)

Thanks! Was this something someone requested? Is there a tangible benefit (were there some customers that didn't want to split up the payloads themselves)? Because it seems like unnecessary cruft in the API otherwise.

sunspot: looks active, but I would question the idea of using Solr as your search engine. Have you considered https://www.meilisearch.com it a bit like Algolia but open source.

You could use Meilisearch: https://www.meilisearch.com/

COMPANY: Meilisearch, here is our website and Github repository.

Meilisearch - Provides search for the main BookStack static site/docs/blog.

Meilisearch is an open source search engine that adds highly performant search engines to any website or app, including ecommerce stores.

Medusa allows you to integrate any search engine of your choice into the platform. It already integrates with search systems like Meilisearch or Algolia to provide an intuitive search experience for the customers.

I would return a question to you. How much of a database do you need? Wouldn't it be more efficient to implement something like Elastic or https://www.meilisearch.com to index your metadata and quickly return to the users?

Not so long ago I was working on a system based on Keystone.js CMS. But there it was used much more sophisticated way than just as basic headless CMS. I was easily able to extend it with search engine (Rust-based Meilisearch) and connect to external APIs.

Only semi-related, but I've recently started using https://www.meilisearch.com/. It's relatively limited, but works great for small use cases. It's also pretty easy to operate. I'm hoping as it continues to grow it will support more features and use cases. I don't think the creators intend to address the same depth of complex features in ElasticSearch (and the like), but that's a desirable attribute in my opinion.

Pronounced "traffic", Traefik is a modern HTTP reverse proxy and load balancer aimed at making deploying microservices easier. It integrates with your existing infrastructure components such as Docker, Kubernetes, and others, and configures itself automatically and dynamically. The latest version adds lots of new options and enhancements such as adding healthcheck options, support for custom headers, and more. Read the migration guide on how to update to the latest version which is now required due to breaking changes.

> I think etcd is basically a k8s only project now

I hate etcd with the best of them, but etcd is used in a lot more places than just kubernetes:

https://github.com/apache/apisix/blob/master/docs/en/latest/...

https://github.com/traefik/traefik#:~:text=Etcd,

https://github.com/zalando/patroni#patroni-a-template-for-po...

https://github.com/purpleidea/mgmt/tree/0.0.26/etcd (this one shows up on HN quite a bit)

https://github.com/sorintlab/stolon#features

It's actually one of the major reasons I wouldn't touch those projects

However, it's very unlikely that .NET developers will directly expose their Kestrel-based web apps to the internet. Typically, we use other popular web servers like Nginx, Traefik, and Caddy to act as a reverse-proxy in front of Kestrel for various reasons:

Not as good though. Case in point: https://github.com/traefik/traefik/issues/5472#issuecomment-... (that's just from this morning)

I'm speak objectively here. Of course, any built-in auto HTTPS that works (more or less) is better than none. Traefik uses an ACME library that was originally written for Caddy. After the original author left that project, Traefik team started maintaining it. Caddy's users' requirements exceeded what the library was capable of, but unfortunately there was friction in getting it to achieve our requirements. So I ended up writing a new ACME client library in Go and, together with upgrades in CertMagic (Caddy's auto-TLS lib), Caddy has the more flexible, robust, and capable auto-HTTPS functionality.

That is to say, not all auto-HTTPS functionalities are the same.

We'll use Traefik, an open source cloud native gateway that can plug into a Kubernetes cluster. It has the concept of "middleware" that can process API requests before passing them through to a backend. We can configuring a rate limit for all of our API endpoints by matching on the request path:

I did the same question here and here

Sadly there is currently no way of doing so. https://github.com/traefik/traefik/issues/6999