I Made Stable Diffusion XL Smarter by Finetuning It on Bad AI-Generated Images

• diffusers

  266 22,543 9.9 Python

  🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

  

Merging LoRAs is essentially taking a weighted average of the LoRA adapter weights. It's more common in other UIs.

diffusers is working on a PR for it: https://github.com/huggingface/diffusers/pull/4473

• stable-diffusion-webui

  2,808 129,299 9.9 Python

  Stable Diffusion web UI

I'm pretty sure that it's just serially summing the network weights, which results in an accumulated offset to the self-attention layers of the transformer. It's not doing any kind of analysis of multiple networks prior to application to make them "play nice" together; it's just looping and summing.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob...

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• safetensors

  31 2,426 8.4 Python

  Simple, safe way to store and distribute tensors

  

Thank you for note on this. I had not heard there were already trojan horse malware being slipped into tensor files as python scripts. Apparently torch pickle uses eval on the tensor file with no filter.

Heard surprisingly little commentary on this topic. The full explanation of how Safetensors are "Safe" can be found from the developer at: https://github.com/huggingface/safetensors/discussions/111

• enact

  2 48 8.9 Python

  A framework for generative software.

  

You may be interested in the open source framework we're developing at https://github.com/agentic-ai/enact

It's still early, but the core insight is that a lot of these generative AI flows (whether text, image, single models, model chains, etc) will need to be fit via some form of feedback signal, so it makes sense to build some fundamental infrastructure to support that. One of the early demos (not currently live, but I plan on bringing it back soon) was precisely the type of flow you're talking about, although we used 'prompt refinement' as a cheap proxy for tuning the actual model weights.

Roughly, we aim to build out core python-level infra that makes it easy to write flows in mostly native python and then allows you track executions of your generative flows, including executions of 'human components' such as raters. We also support time travel / rewind / replay, automatic gradio UIs, fastAPI (the latter two very experimental atm).

Medium term we want to make it easy to take any generative flow, wrap it in a 'human rating' flow, auto-deploy as an API or gradio UI and then fit using a number of techniques, e.g., RLHF, finetuning, A/B testing of generative subcomponents, etc, so stay tuned.

At the moment, we're focused on getting the 'bones' right, but between the quickstart (https://github.com/agentic-ai/enact/blob/main/examples/quick...) and our readme (https://github.com/agentic-ai/enact/tree/main#why-enact) you get a decent idea of where we're headed.

We're looking for people to kick the tires / contribute, so if this sounds interesting, please check it out.

Suggest a related project