xformers
diffusers
xformers | diffusers | |
---|---|---|
46 | 266 | |
7,631 | 22,646 | |
3.1% | 2.8% | |
9.3 | 9.9 | |
2 days ago | 2 days ago | |
Python | Python | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
xformers
-
Animediff error
(venv) G:\A1111\Animediff\animatediff-cli-prompt-travel>animatediff generate -c config/prompts/01-ToonYou.json -W 256 -H 384 -L 128 -C 16 WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 2.1.0+cu121 with CUDA 1201 (you have 2.0.1+cpu) Python 3.10.11 (you have 3.10.6) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source? warn( 15:07:25 INFO Using generation config: config\prompts\01-ToonYou.json cli.py:291 โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\src\animatediff\cli.py:292 in generate โ โ โ โ 289 โ โ โ 290 โ config_path = config_path.absolute() โ โ 291 โ logger.info(f"Using generation config: {path_from_cwd(config_path)}") โ โ โฑ 292 โ model_config: ModelConfig = get_model_config(config_path) โ โ 293 โ is_v2 = is_v2_motion_module(data_dir.joinpath(model_config.motion_module)) โ โ 294 โ infer_config: InferenceConfig = get_infer_config(is_v2) โ โ 295 โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\src\animatediff\settings.py:134 in โ โ get_model_config โ โ โ โ 131 โ โ 132 โ โ 133 def get_model_config(config_path: Path) -> ModelConfig: โ โ โฑ 134 โ settings = ModelConfig(json_config_path=config_path) โ โ 135 โ return settings โ โ 136 โ โ โ โ in pydantic.env_settings.BaseSettings.__init__:40 โ โ โ โ in pydantic.main.BaseModel.__init__:341 โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ ValidationError: 1 validation error for ModelConfig prompt extra fields not permitted (type=value_error.extra) (venv) G:\A1111\Animediff\animatediff-cli-prompt-travel>animatediff generate -c config/prompts/prompt1.json -W 256 -H 38 4 -L 128 -C 16 WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 2.1.0+cu121 with CUDA 1201 (you have 2.0.1+cpu) Python 3.10.11 (you have 3.10.6) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source? warn( 15:08:30 INFO Using generation config: config\prompts\prompt1.json cli.py:291 15:08:35 INFO is_v2=True util.py:361 INFO Using base model: runwayml\stable-diffusion-v1-5 cli.py:309 INFO Will save outputs to ./output\2023-10-30T15-08-35-epicrealism-epicrealism_naturalsinrc1vae cli.py:317 INFO Checking motion module... generate.py:331 INFO Loading tokenizer... generate.py:345 INFO Loading text encoder... generate.py:347 15:08:38 INFO Loading VAE... generate.py:349 INFO Loading UNet... generate.py:351 15:08:59 INFO Loaded 453.20928M-parameter motion module unet.py:578 15:09:00 INFO Using scheduler "euler_a" (EulerAncestralDiscreteScheduler) generate.py:363 INFO Loading weights from generate.py:368 G:\A1111\Animediff\animatediff-cli-prompt-travel\data\models\sd\epicrealism_naturalSin RC1VAE.safetensors โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connection.py:17 โ โ 4 in _new_conn โ โ โ โ 171 โ โ โ extra_kw["socket_options"] = self.socket_options โ โ 172 โ โ โ โ 173 โ โ try: โ โ โฑ 174 โ โ โ conn = connection.create_connection( โ โ 175 โ โ โ โ (self._dns_host, self.port), self.timeout, **extra_kw โ โ 176 โ โ โ ) โ โ 177 โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\util\connection. โ โ py:95 in create_connection โ โ โ โ 92 โ โ โ โ sock = None โ โ 93 โ โ โ 94 โ if err is not None: โ โ โฑ 95 โ โ raise err โ โ 96 โ โ โ 97 โ raise socket.error("getaddrinfo returns an empty list") โ โ 98 โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\util\connection. โ โ py:85 in create_connection โ โ โ โ 82 โ โ โ โ sock.settimeout(timeout) โ โ 83 โ โ โ if source_address: โ โ 84 โ โ โ โ sock.bind(source_address) โ โ โฑ 85 โ โ โ sock.connect(sa) โ โ 86 โ โ โ return sock โ โ 87 โ โ โ โ 88 โ โ except socket.error as e: โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond During handling of the above exception, another exception occurred: โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connectionpool.p โ โ y:703 in urlopen โ โ โ โ 700 โ โ โ โ self._prepare_proxy(conn) โ โ 701 โ โ โ โ โ 702 โ โ โ # Make the request on the httplib connection object. โ โ โฑ 703 โ โ โ httplib_response = self._make_request( โ โ 704 โ โ โ โ conn, โ โ 705 โ โ โ โ method, โ โ 706 โ โ โ โ url, โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connectionpool.p โ โ y:386 in _make_request โ โ โ โ 383 โ โ โ โ 384 โ โ # Trigger any extra validation we need to do. โ โ 385 โ โ try: โ โ โฑ 386 โ โ โ self._validate_conn(conn) โ โ 387 โ โ except (SocketTimeout, BaseSSLError) as e: โ โ 388 โ โ โ # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout. โ โ 389 โ โ โ self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connectionpool.p โ โ y:1042 in _validate_conn โ โ โ โ 1039 โ โ โ โ 1040 โ โ # Force connect early to allow us to validate the connection. โ โ 1041 โ โ if not getattr(conn, "sock", None): # AppEngine might not have `.sock` โ โ โฑ 1042 โ โ โ conn.connect() โ โ 1043 โ โ โ โ 1044 โ โ if not conn.is_verified: โ โ 1045 โ โ โ warnings.warn( โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connection.py:35 โ โ 8 in connect โ โ โ โ 355 โ โ โ 356 โ def connect(self): โ โ 357 โ โ # Add certificate verification โ โ โฑ 358 โ โ self.sock = conn = self._new_conn() โ โ 359 โ โ hostname = self.host โ โ 360 โ โ tls_in_tls = False โ โ 361 โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connection.py:17 โ โ 9 in _new_conn โ โ โ โ 176 โ โ โ ) โ โ 177 โ โ โ โ 178 โ โ except SocketTimeout: โ โ โฑ 179 โ โ โ raise ConnectTimeoutError( โ โ 180 โ โ โ โ self, โ โ 181 โ โ โ โ "Connection to %s timed out. (connect timeout=%s)" โ โ 182 โ โ โ โ % (self.host, self.timeout), โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ ConnectTimeoutError: (, 'Connection to raw.githubusercontent.com timed out. (connect timeout=None)') During handling of the above exception, another exception occurred: โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\requests\adapters.py:489 โ โ in send โ โ โ โ 486 โ โ โ โ 487 โ โ try: โ โ 488 โ โ โ if not chunked: โ โ โฑ 489 โ โ โ โ resp = conn.urlopen( โ โ 490 โ โ โ โ โ method=request.method, โ โ 491 โ โ โ โ โ url=url, โ โ 492 โ โ โ โ โ body=request.body, โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\connectionpool.p โ โ y:787 in urlopen โ โ โ โ 784 โ โ โ elif isinstance(e, (SocketError, HTTPException)): โ โ 785 โ โ โ โ e = ProtocolError("Connection aborted.", e) โ โ 786 โ โ โ โ โ โฑ 787 โ โ โ retries = retries.increment( โ โ 788 โ โ โ โ method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] โ โ 789 โ โ โ ) โ โ 790 โ โ โ retries.sleep() โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\urllib3\util\retry.py:59 โ โ 2 in increment โ โ โ โ 589 โ โ ) โ โ 590 โ โ โ โ 591 โ โ if new_retry.is_exhausted(): โ โ โฑ 592 โ โ โ raise MaxRetryError(_pool, url, error or ResponseError(cause)) โ โ 593 โ โ โ โ 594 โ โ log.debug("Incremented Retry for (url='%s'): %r", url, new_retry) โ โ 595 โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /CompVis/stable-diffusion/main/configs/stable-diffusion/v1-inference.yaml (Caused by ConnectTimeoutError(, 'Connection to raw.githubusercontent.com timed out. (connect timeout=None)')) During handling of the above exception, another exception occurred: โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\src\animatediff\cli.py:326 in generate โ โ โ โ 323 โ global g_pipeline โ โ 324 โ global last_model_path โ โ 325 โ if g_pipeline is None or last_model_path != model_config.path.resolve(): โ โ โฑ 326 โ โ g_pipeline = create_pipeline( โ โ 327 โ โ โ base_model=base_model_path, โ โ 328 โ โ โ model_config=model_config, โ โ 329 โ โ โ infer_config=infer_config, โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\src\animatediff\generate.py:371 in โ โ create_pipeline โ โ โ โ 368 โ โ logger.info(f"Loading weights from {model_path}") โ โ 369 โ โ if model_path.is_file(): โ โ 370 โ โ โ logger.debug("Loading from single checkpoint file") โ โ โฑ 371 โ โ โ unet_state_dict, tenc_state_dict, vae_state_dict = get_checkpoint_weights(mo โ โ 372 โ โ elif model_path.is_dir(): โ โ 373 โ โ โ logger.debug("Loading from Diffusers model directory") โ โ 374 โ โ โ temp_pipeline = StableDiffusionPipeline.from_pretrained(model_path) โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\src\animatediff\utils\model.py:139 in โ โ get_checkpoint_weights โ โ โ โ 136 โ โ 137 def get_checkpoint_weights(checkpoint: Path): โ โ 138 โ temp_pipeline: StableDiffusionPipeline โ โ โฑ 139 โ temp_pipeline, _ = checkpoint_to_pipeline(checkpoint, save=False) โ โ 140 โ unet_state_dict = temp_pipeline.unet.state_dict() โ โ 141 โ tenc_state_dict = temp_pipeline.text_encoder.state_dict() โ โ 142 โ vae_state_dict = temp_pipeline.vae.state_dict() โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\src\animatediff\utils\model.py:124 in โ โ checkpoint_to_pipeline โ โ โ โ 121 โ if target_dir is None: โ โ 122 โ โ target_dir = pipeline_dir.joinpath(checkpoint.stem) โ โ 123 โ โ โ โฑ 124 โ pipeline = StableDiffusionPipeline.from_single_file( โ โ 125 โ โ pretrained_model_link_or_path=str(checkpoint.absolute()), โ โ 126 โ โ local_files_only=True, โ โ 127 โ โ load_safety_checker=False, โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\diffusers\loaders.py:147 โ โ 1 in from_single_file โ โ โ โ 1468 โ โ โ โ force_download=force_download, โ โ 1469 โ โ โ ) โ โ 1470 โ โ โ โ โฑ 1471 โ โ pipe = download_from_original_stable_diffusion_ckpt( โ โ 1472 โ โ โ pretrained_model_link_or_path, โ โ 1473 โ โ โ pipeline_class=cls, โ โ 1474 โ โ โ model_type=model_type, โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\diffusers\pipelines\stab โ โ le_diffusion\convert_from_ckpt.py:1234 in download_from_original_stable_diffusion_ckpt โ โ โ โ 1231 โ โ โ # only refiner xl has embedder and one text embedders โ โ 1232 โ โ โ config_url = "https://raw.githubusercontent.com/Stability-AI/generative-mode โ โ 1233 โ โ โ โ โฑ 1234 โ โ original_config_file = BytesIO(requests.get(config_url).content) โ โ 1235 โ โ โ 1236 โ original_config = OmegaConf.load(original_config_file) โ โ 1237 โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\requests\api.py:73 in โ โ get โ โ โ โ 70 โ :rtype: requests.Response โ โ 71 โ """ โ โ 72 โ โ โ โฑ 73 โ return request("get", url, params=params, **kwargs) โ โ 74 โ โ 75 โ โ 76 def options(url, **kwargs): โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\requests\api.py:59 in โ โ request โ โ โ โ 56 โ # avoid leaving sockets open which can trigger a ResourceWarning in some โ โ 57 โ # cases, and look like a memory leak in others. โ โ 58 โ with sessions.Session() as session: โ โ โฑ 59 โ โ return session.request(method=method, url=url, **kwargs) โ โ 60 โ โ 61 โ โ 62 def get(url, params=None, **kwargs): โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\requests\sessions.py:587 โ โ in request โ โ โ โ 584 โ โ โ "allow_redirects": allow_redirects, โ โ 585 โ โ } โ โ 586 โ โ send_kwargs.update(settings) โ โ โฑ 587 โ โ resp = self.send(prep, **send_kwargs) โ โ 588 โ โ โ โ 589 โ โ return resp โ โ 590 โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\requests\sessions.py:701 โ โ in send โ โ โ โ 698 โ โ start = preferred_clock() โ โ 699 โ โ โ โ 700 โ โ # Send the request โ โ โฑ 701 โ โ r = adapter.send(request, **kwargs) โ โ 702 โ โ โ โ 703 โ โ # Total elapsed time of the request (approximately) โ โ 704 โ โ elapsed = preferred_clock() - start โ โ โ โ G:\A1111\Animediff\animatediff-cli-prompt-travel\venv\lib\site-packages\requests\adapters.py:553 โ โ in send โ โ โ โ 550 โ โ โ if isinstance(e.reason, ConnectTimeoutError): โ โ 551 โ โ โ โ # TODO: Remove this in 3.0.0: see #2811 โ โ 552 โ โ โ โ if not isinstance(e.reason, NewConnectionError): โ โ โฑ 553 โ โ โ โ โ raise ConnectTimeout(e, request=request) โ โ 554 โ โ โ โ โ 555 โ โ โ if isinstance(e.reason, ResponseError): โ โ 556 โ โ โ โ raise RetryError(e, request=request) โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ ConnectTimeout: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /CompVis/stable-diffusion/main/configs/stable-diffusion/v1-inference.yaml (Caused by ConnectTimeoutError(, 'Connection to raw.githubusercontent.com timed out. (connect timeout=None)')) (venv) G:\A1111\Animediff\animatediff-cli-prompt-travel>
-
Colab | Errors when installing x-formers
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. fastai 2.7.12 requires torch<2.1,>=1.7, but you have torch 2.1.0+cu118 which is incompatible. torchaudio 2.0.2+cu118 requires torch==2.0.1, but you have torch 2.1.0+cu118 which is incompatible. torchdata 0.6.1 requires torch==2.0.1, but you have torch 2.1.0+cu118 which is incompatible. torchtext 0.15.2 requires torch==2.0.1, but you have torch 2.1.0+cu118 which is incompatible. torchvision 0.15.2+cu118 requires torch==2.0.1, but you have torch 2.1.0+cu118 which is incompatible. WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 2.1.0+cu121 with CUDA 1201 (you have 2.1.0+cu118) Python 3.10.13 (you have 3.10.12) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details xformers version: 0.0.22.post3
-
FlashAttention-2, 2x faster than FlashAttention
This enables V1. V2 is still yet to be integrated into xformers. The team replied saying it should happen this week.
See the relevant Github issue here: https://github.com/facebookresearch/xformers/issues/795
-
Xformers issue
My Xformers doesnt work, any help see code. info ( Exception training model: 'Refer to https://github.com/facebookresearch/xformers for more information on how to install xformers'. ) or
-
Having xformer troubles
ModuleNotFoundError: Refer to https://github.com/facebookresearch/xformers for more
-
Question: these 4 crappy picture have been generated with the same seed and settings. Why they keep coming mildly different?
Xformers is a module that that can be used with Stable Diffusion. It decreases the memory required to generate an image as well as speeding things up. It works very well but there are two problems with Xformers:
-
Stuck trying to update xformers
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 1.13.1+cu117 with CUDA 1107 (you have 2.0.1+cu118) Python 3.10.9 (you have 3.10.7) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details ================================================================================= You are running xformers 0.0.16rc425. The program is tested to work with xformers 0.0.17. To reinstall the desired version, run with commandline flag --reinstall-xformers. Use --skip-version-check commandline argument to disable this check. =================================================================================
-
Question about updating Xformers for A1111
# Your version of xformers is 0.0.16rc425. # xformers >= 0.0.17.dev is required to be available on the Dreambooth tab. # Torch 1 wheels of xformers >= 0.0.17.dev are no longer available on PyPI, # but you can manually download them by going to: https://github.com/facebookresearch/xformers/actions # Click on the most recent action tagged with a release (middle column). # Select a download based on your environment. # Unzip your download # Activate your venv and install the wheel: (from A1111 project root) cd venv/Scripts activate pip install {REPLACE WITH PATH TO YOUR UNZIPPED .whl file} # Then restart your project.
-
Is there a Pygmalion roadmap?
Further reading/resources: RedPajama: https://www.together.xyz/blog/redpajama xFormers: https://github.com/facebookresearch/xformers Flash Attention: https://arxiv.org/abs/2205.14135 Sparsity [NEW!]: https://arxiv.org/abs/2304.07613
- Slow/short replies?
diffusers
- StableDiffusionSafetyChecker
- ๐งจ diffusers 0.24.0 is out with Kandinsky 3.0, IP Adapters, and others
-
What am I missing here? wheres the RND coming from?
I'm missing something about the random factor, from the sample code from https://github.com/huggingface/diffusers/blob/main/README.md
-
T2IAdapter+ControlNet at the same time
Hey people, I noticed that combining these two methods in a single forward pass increases the controllability of the generation quite a bit. I was kind of puzzled that sometimes ControlNet yielded better results than T2IAdapter for some cases, and sometimes it was the other way around, so I decided to test both at the same time, and results were quite nice. Some visuals and more motivation here: https://github.com/huggingface/diffusers/issues/5847 And it was already merged here: https://github.com/huggingface/diffusers/pull/5869
-
Won't you benchmark me?
Open Parti Prompts: The better way to evaluate diffusion models (repo)
-
kohya_ss error. How do I solve this?
You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
- Making a ControlNet inpaint for sdxl
-
Stable Diffusion Gets a Major Boost with RTX Acceleration
For developers, TensorRT support also exists for the diffusers library via community pipelines. [1] It's limited, but if you're only supporting a subset of features, it can help.
In general, these insane speed boosts comes at the cost of bleeding edge features.
[1] https://github.com/huggingface/diffusers/blob/28e8d1f6ec82a6...
-
Mysterious weights when training UNET
I was training sdxl UNET base model, with the diffusers library, which was going great until around step 210k when the weights suddenly turned back to their original values and stayed that way. I also tried with the ema version, which didn't change at all. I also looked at the tensor's weight values directly which confirmed my suspicions.
-
I Made Stable Diffusion XL Smarter by Finetuning It on Bad AI-Generated Images
Merging LoRAs is essentially taking a weighted average of the LoRA adapter weights. It's more common in other UIs.
diffusers is working on a PR for it: https://github.com/huggingface/diffusers/pull/4473
What are some alternatives?
flash-attention - Fast and memory-efficient exact attention
stable-diffusion-webui - Stable Diffusion web UI
stable-diffusion - A latent text-to-image diffusion model
SHARK - SHARK - High Performance Machine Learning Distribution
lora - Using Low-rank adaptation to quickly fine-tune diffusion models.
Dreambooth-Stable-Diffusion - Implementation of Dreambooth (https://arxiv.org/abs/2208.12242) with Stable Diffusion
invisible-watermark - python library for invisible image watermark (blind image watermark)
diffusers - ๐ค Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch
automatic - SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
InvokeAI - InvokeAI is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, supports terminal use through a CLI, and serves as the foundation for multiple commercial products.
Dreambooth-Stable-Diffusion - Implementation of Dreambooth (https://arxiv.org/abs/2208.12242) by way of Textual Inversion (https://arxiv.org/abs/2208.01618) for Stable Diffusion (https://arxiv.org/abs/2112.10752). Tweaks focused on training faces, objects, and styles.