huggingface_hub VS NLTK

Something to think about:

https://github.com/huggingface/huggingface_hub

Yes! We have a well documented Python library (https://github.com/huggingface/huggingface_hub) and public endpoints (https://huggingface.co/docs/hub/api#endpoints-table) you can use to retrieve information about the models and potentially build UIs with specific use cases in mind

i think if you search for pixel art here there are some models worth checking out: https://huggingface.co/

I received another error saying OSError: We couldn't connect to 'https://huggingface.co' to load this model, couldn't find it in the cached files and it looks like ./vae is not the path to a directory containing a file named diffusion_pytorch_model.bin

I plan to release it on https://huggingface.co/, where all this cool AI stuff is available for free for everyone that wishes to try it.

For this I used mainly the chainner application with models from here but I also used the google colab automatic1111 stable diffusion webui (for example for Lanczos) and also spaces fromhuggingface like this one or then from the replicate.com website super resolution collection.

you will now need to create a huggingface account ( https://huggingface.co/) if you haven't already. When you have, go here and accept the terms, https://huggingface.co/runwayml/stable-diffusion-v1-5. When you have done both, click on your profile icon and go to settings. Click access tokens and then create token, name it whatever you want, select "write". When you are finished with all this, then you can run the next cell which is the hugging face cell. It will ask for a token, you copy and paste what you just created.

alternatively, could we not simply split by common characters such as newlines and periods, to split it within sentences? it would be fragile with special handling required for numbers with decimal points and probably various other edge cases, though.

there are also Python libraries meant for natural language parsing[0] that could do that task for us. I even see examples on stack overflow[1] that simply split text into sentences.

[0]: https://www.nltk.org/

Check out https://www.nltk.org/ and work through it, it'll give you a foundational understanding of how all this works, but very basically it's just a fancy auto-complete.

NLTK Documentation

If this is True, then you need also Natural Language Toolkit to process the words.

In that case, you should definitely have a look at Python's nltk library which stands for Natural Language Toolkit. They have a rich corpus collection for all kinds of specialized things like grammars, taggers, chunkers, etc.

If not, start with Python's Natural Language Toolkit.

Try https://www.nltk.org it runs you through the basics. The book is here

In terms of programming languages, Python is a great first programming language. the learnpython subreddit has lots of good recommendations for resources to get started. Once you're comfortable with the language, NLTK would be a good place to start, and the docs have heaps of examples. Check it out https://www.nltk.org/

The most popular library to do this is NLTK though I believe you can use some of the popular AI API services today as well. Bloomberg launched one.