KoboldAI-Client VS mesh-transformer-jax

You can use KoboldAI to run a LLM locally. There are hundreds / thousands of models on hugging face. Some uncensored ones are Pygmalion AI (chatbot), Erebus (story writing AI), or Vicuna (general purpose).

GPT-J 6B is a pretty weak and outdated model. Nerys 13B would probably give you better replies but they lean more towards SFW stuff. Erebus was their best model for erotic roleplay but they removed it as it went against Google's TOS. You can check out their documentation here.

If you do have that kind of hardware, the next step would be looking for what model to run. I came across Kobold's models. Their main github page is here: https://github.com/KoboldAI/KoboldAI-Client

Then there are graphical user interfaces like text-generation-webui and gpt4all for general purpose chat. There are also KoboldAI and SillyTavern, they have focus more on storytelling and roleplay and have tools to improve that.

And finally the folks from the KoboldAi do some interesting stuff with Pseudocode and Soft-Prompts that might also be relevant.

Click this link. ---> https://github.com/KoboldAI/KoboldAI-Client/tree/main

Do you believe the problem could be that my KoboldAI is outdated? I did download the one from henk717 at https://github.com/KoboldAI/KoboldAI-Client but it was a little while ago.

Found TPU at: grpc://10.18.240.10:8470 Now we will need your Google Drive to store settings and saves, you must login with the same account you used for Colab. Drive already mounted at /content/drive/; to attempt to forcibly remount, call drive.mount("/content/drive/", force_remount=True). --2023-02-22 17:46:35-- https://koboldai.org/ckds Resolving koboldai.org (koboldai.org)... 104.21.21.176, 172.67.199.170, 2606:4700:3036::ac43:c7aa, ... Connecting to koboldai.org (koboldai.org)|104.21.21.176|:443... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh [following] --2023-02-22 17:46:35-- https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6675 (6.5K) [text/plain] Saving to: ‘STDOUT’ - 100%[===================>] 6.52K --.-KB/s in 0s 2023-02-22 17:46:35 (85.5 MB/s) - written to stdout [6675/6675] mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/stories/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/models/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/settings/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/softprompts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/userscripts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/presets/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/themes/’: File exists mkdir: cannot create directory ‘/content/KoboldAI-Client’: File exists Reinitialized existing Git repository in /content/KoboldAI-Client/.git/ Fetching origin From https://github.com/koboldai/KoboldAI-Client * [new branch] main -> origin/main Already on 'main' HEAD is now at c27faf5 Updated Silence Audio - GPU cp: cannot stat 'softprompts/*': No such file or directory cp: cannot stat 'presets/*': No such file or directory cp: cannot stat 'themes/*': No such file or directory rm: cannot remove 'stories': Is a directory rm: cannot remove 'userscripts': Is a directory rm: cannot remove 'models': Is a directory ln: failed to create symbolic link 'settings/settings': File exists Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting git+https://github.com/VE-FORBRYDERNE/mesh-transformer-jax@ck (from -r requirements_mtj.txt (line 11)) Cloning https://github.com/VE-FORBRYDERNE/mesh-transformer-jax (to revision ck) to /tmp/pip-req-build-q85v5kg0 Running command git clone --filter=blob:none --quiet https://github.com/VE-FORBRYDERNE/mesh-transformer-jax /tmp/pip-req-build-q85v5kg0 Running command git checkout -b ck --track origin/ck Switched to a new branch 'ck' Branch 'ck' set up to track remote branch 'ck' from 'origin'. Resolved https://github.com/VE-FORBRYDERNE/mesh-transformer-jax to commit 89577e3d49aef94179a138260f6fbd6ca2964327 Preparing metadata (setup.py) ... done Requirement already satisfied: torch<1.13,>=1.9 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 1)) (1.12.1) Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 2)) (1.22.4) Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 3)) (4.64.1) Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 4)) (2.25.1) Requirement already satisfied: dm-haiku==0.0.5 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 5)) (0.0.5) Requirement already satisfied: jax==0.2.21 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 6)) (0.2.21) Requirement already satisfied: jaxlib<=0.3.7,>=0.1.69 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 7)) (0.3.7) Requirement already satisfied: transformers==4.24.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 8)) (4.24.0) Requirement already satisfied: huggingface_hub>=0.10.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 9)) (0.12.1) Requirement already satisfied: progressbar2 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 10)) (3.38.0) Requirement already satisfied: flask in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 12)) (1.1.4) Requirement already satisfied: Flask-SocketIO in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 13)) (5.3.2) Requirement already satisfied: flask-cloudflared==0.0.10 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 14)) (0.0.10) Requirement already satisfied: flask-ngrok in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 15)) (0.0.25) Requirement already satisfied: eventlet in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 16)) (0.33.3) Requirement already satisfied: dnspython==2.2.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 17)) (2.2.1) Requirement already satisfied: lupa==1.10 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 18)) (1.10) Requirement already satisfied: markdown in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 19)) (3.4.1) Requirement already satisfied: bleach==4.1.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 20)) (4.1.0) Requirement already satisfied: flask-session in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 21)) (0.4.0) Requirement already satisfied: marshmallow>=3.13 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 22)) (3.19.0) Requirement already satisfied: apispec-webframeworks in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 23)) (0.5.2) Requirement already satisfied: loguru in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 24)) (0.6.0) Requirement already satisfied: jmp>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.0.4) Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.8.10) Requirement already satisfied: absl-py>=0.7.1 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (1.4.0) Requirement already satisfied: opt-einsum in /usr/local/lib/python3.8/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 6)) (3.3.0) Requirement already satisfied: scipy>=1.2.1 in /usr/local/lib/python3.8/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 6)) (1.7.3) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (2022.6.2) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (0.13.2) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (6.0) Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (3.9.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (23.0) Requirement already satisfied: webencodings in /usr/local/lib/python3.8/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 20)) (0.5.1) Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.8/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 20)) (1.15.0) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch<1.13,>=1.9->-r requirements_mtj.txt (line 1)) (4.5.0) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (2.10) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (1.24.3) Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (4.0.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (2022.12.7) Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.8/dist-packages (from jaxlib<=0.3.7,>=0.1.69->-r requirements_mtj.txt (line 7)) (2.0.7) Requirement already satisfied: python-utils>=2.3.0 in /usr/local/lib/python3.8/dist-packages (from progressbar2->-r requirements_mtj.txt (line 10)) (3.5.2) Requirement already satisfied: click<8.0,>=5.1 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (7.1.2) Requirement already satisfied: itsdangerous<2.0,>=0.24 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (1.1.0) Requirement already satisfied: Werkzeug<2.0,>=0.15 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (1.0.1) Requirement already satisfied: Jinja2<3.0,>=2.10.1 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (2.11.3) Requirement already satisfied: python-socketio>=5.0.2 in /usr/local/lib/python3.8/dist-packages (from Flask-SocketIO->-r requirements_mtj.txt (line 13)) (5.7.2) Requirement already satisfied: greenlet>=0.3 in /usr/local/lib/python3.8/dist-packages (from eventlet->-r requirements_mtj.txt (line 16)) (2.0.2) Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.8/dist-packages (from markdown->-r requirements_mtj.txt (line 19)) (6.0.0) Requirement already satisfied: cachelib in /usr/local/lib/python3.8/dist-packages (from flask-session->-r requirements_mtj.txt (line 21)) (0.10.2) Requirement already satisfied: apispec[yaml]>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from apispec-webframeworks->-r requirements_mtj.txt (line 23)) (6.0.2) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata>=4.4->markdown->-r requirements_mtj.txt (line 19)) (3.13.0) Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.8/dist-packages (from Jinja2<3.0,>=2.10.1->flask->-r requirements_mtj.txt (line 12)) (2.0.1) Requirement already satisfied: python-engineio>=4.3.0 in /usr/local/lib/python3.8/dist-packages (from python-socketio>=5.0.2->Flask-SocketIO->-r requirements_mtj.txt (line 13)) (4.3.4) Requirement already satisfied: bidict>=0.21.0 in /usr/local/lib/python3.8/dist-packages (from python-socketio>=5.0.2->Flask-SocketIO->-r requirements_mtj.txt (line 13)) (0.22.1) Reading package lists... Done Building dependency tree Reading state information... Done netbase is already the newest version (6.1). aria2 is already the newest version (1.35.0-1build1). The following package was automatically installed and is no longer required: libnvidia-common-510 Use 'sudo apt autoremove' to remove it. 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded. /tools/node/bin/lt -> /tools/node/lib/node_modules/localtunnel/bin/lt.js + [email protected] updated 1 package in 0.712s Launching KoboldAI with the following options : python3 aiserver.py --model KoboldAI/GPT-NeoX-20B-Erebus --colab 2023-02-22 17:46:50.342363: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-22 17:46:51.588645: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 2023-02-22 17:46:51.588777: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 2023-02-22 17:46:51.588800: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. MESSAGE | Welcome to KoboldAI! MESSAGE | You have selected the following Model: KoboldAI/GPT-NeoX-20B-Erebus INIT | Starting | Flask INIT | OK | Flask INIT | Starting | Webserver Initializing Mesh Transformer JAX, please wait... Connecting to your Colab instance's TPU Elapsed Time: 0:00:02 [ █ ] 2023-02-22 17:47:28.996684: E external/org_tensorflow/tensorflow/compiler/xla/python/tpu_driver/grpc_tpu_driver.cc:1070] Failed to open the gRPC driver: 4: Deadline Exceeded: WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.) Traceback (most recent call last): File "aiserver.py", line 10214, in load_model(initial_load=True) File "aiserver.py", line 2806, in load_model tpu_mtj_backend.load_model(vars.custmodpth, hf_checkpoint=vars.model not in ("TPUMeshTransformerGPTJ", "TPUMeshTransformerGPTNeoX") and vars.use_colab_tpu, **vars.modelconfig) File "/content/KoboldAI-Client/tpu_mtj_backend.py", line 1194, in load_model devices = np.array(jax.devices()[:cores_per_replica]).reshape(mesh_shape) File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 314, in devices return get_backend(backend).devices() File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 258, in get_backend return _get_backend_uncached(platform) File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 248, in _get_backend_uncached raise RuntimeError(f"Requested backend {platform}, but it failed " RuntimeError: Requested backend tpu_driver, but it failed to initialize: DEADLINE_EXCEEDED: Failed to connect to remote server at address: grpc://10.18.240.10:8470. Error from gRPC: Deadline Exceeded. Details:

--2022-05-01 02:14:36-- https://henk.tech/ckds Resolving henk.tech (henk.tech)... 172.67.159.108, 104.21.57.53, 2606:4700:3033::ac43:9f6c, ... Connecting to henk.tech (henk.tech)|172.67.159.108|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://henk.tech/ckds/ [following] --2022-05-01 02:14:36-- https://henk.tech/ckds/ Reusing existing connection to henk.tech:443. HTTP request sent, awaiting response... 302 Found Location: https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh [following] --2022-05-01 02:14:36-- https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.111.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6089 (5.9K) [text/plain] Saving to: ‘STDOUT’ - 100%[===================>] 5.95K --.-KB/s in 0.001s 2022-05-01 02:14:36 (11.2 MB/s) - written to stdout [6089/6089] mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/stories/’: File exists Google Drive folders created. --2022-05-01 02:14:37-- https://henk.tech/ckds Resolving henk.tech (henk.tech)... 172.67.159.108, 104.21.57.53, 2606:4700:3033::ac43:9f6c, ... Connecting to henk.tech (henk.tech)|172.67.159.108|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://henk.tech/ckds/ [following] --2022-05-01 02:14:37-- https://henk.tech/ckds/ Reusing existing connection to henk.tech:443. HTTP request sent, awaiting response... 302 Found Location: https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh [following] --2022-05-01 02:14:37-- https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.111.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6089 (5.9K) [text/plain] Saving to: ‘STDOUT’ - 100%[===================>] 5.95K --.-KB/s in 0s 2022-05-01 02:14:37 (51.9 MB/s) - written to stdout [6089/6089] mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/stories/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/models/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/settings/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/softprompts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/userscripts/’: File exists Initialized empty Git repository in /content/KoboldAI-Client/.git/ fatal: No such remote: origin Fetching origin remote: Enumerating objects: 3007, done. remote: Counting objects: 100% (1170/1170), done. remote: Compressing objects: 100% (157/157), done. remote: Total 3007 (delta 1069), reused 1072 (delta 1011), pack-reused 1837 Receiving objects: 100% (3007/3007), 4.86 MiB | 17.52 MiB/s, done. Resolving deltas: 100% (1895/1895), done. From https://github.com/koboldai/KoboldAI-Client * [new branch] main -> origin/main Branch 'main' set up to track remote branch 'main' from 'origin'. Switched to a new branch 'main' HEAD is now at 06fc4c6 Add TPU warning cp: cannot stat 'softprompts/*': No such file or directory rm: cannot remove 'stories': Is a directory rm: cannot remove 'userscripts': Is a directory rm: cannot remove 'softprompts': No such file or directory rm: cannot remove 'models': No such file or directory Collecting git+https://github.com/VE-FORBRYDERNE/mesh-transformer-jax@ck (from -r requirements_mtj.txt (line 10)) Cloning https://github.com/VE-FORBRYDERNE/mesh-transformer-jax (to revision ck) to /tmp/pip-req-build-r0qwj6w0 Running command git clone -q https://github.com/VE-FORBRYDERNE/mesh-transformer-jax /tmp/pip-req-build-r0qwj6w0 Running command git checkout -b ck --track origin/ck Switched to a new branch 'ck' Branch 'ck' set up to track remote branch 'ck' from 'origin'. Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from -r requirements_mtj.txt (line 1)) (1.21.6) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from -r requirements_mtj.txt (line 2)) (4.64.0) Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from -r requirements_mtj.txt (line 3)) (2.23.0) Collecting optax<=0.0.9,>=0.0.5 Downloading optax-0.0.9-py3-none-any.whl (118 kB) |████████████████████████████████| 118 kB 5.4 MB/s Collecting dm-haiku==0.0.5 Downloading dm_haiku-0.0.5-py3-none-any.whl (287 kB) |████████████████████████████████| 287 kB 54.9 MB/s Collecting ray[default] Downloading ray-1.12.0-cp37-cp37m-manylinux2014_x86_64.whl (53.2 MB) |████████████████████████████████| 53.2 MB 211 kB/s Collecting jax==0.2.21 Downloading jax-0.2.21.tar.gz (756 kB) |████████████████████████████████| 756 kB 25.4 MB/s Collecting transformers Downloading transformers-4.18.0-py3-none-any.whl (4.0 MB) |████████████████████████████████| 4.0 MB 39.7 MB/s Requirement already satisfied: progressbar2 in /usr/local/lib/python3.7/dist-packages (from -r requirements_mtj.txt (line 9)) (3.38.0) Requirement already satisfied: flask in /usr/local/lib/python3.7/dist-packages (from -r requirements_mtj.txt (line 11)) (1.1.4) Collecting Flask-SocketIO Downloading Flask_SocketIO-5.1.2-py3-none-any.whl (17 kB) Collecting flask-cloudflared>=0.0.5 Downloading flask_cloudflared-0.0.8-py3-none-any.whl (5.9 kB) Collecting flask-ngrok Downloading flask_ngrok-0.0.25-py3-none-any.whl (3.1 kB) Collecting eventlet Downloading eventlet-0.33.0-py2.py3-none-any.whl (226 kB) |████████████████████████████████| 226 kB 56.9 MB/s Collecting lupa==1.10 Downloading lupa-1.10-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (308 kB) |████████████████████████████████| 308 kB 55.5 MB/s Requirement already satisfied: markdown in /usr/local/lib/python3.7/dist-packages (from -r requirements_mtj.txt (line 17)) (3.3.6) Collecting bleach==4.1.0 Downloading bleach-4.1.0-py2.py3-none-any.whl (157 kB) |████████████████████████████████| 157 kB 61.0 MB/s Requirement already satisfied: absl-py>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (1.0.0) Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.8.9) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (4.2.0) Collecting jmp>=0.0.2 Downloading jmp-0.0.2-py3-none-any.whl (16 kB) Requirement already satisfied: opt_einsum in /usr/local/lib/python3.7/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 7)) (3.3.0) Requirement already satisfied: scipy>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 7)) (1.4.1) Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 18)) (21.3) Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 18)) (1.15.0) Requirement already satisfied: webencodings in /usr/local/lib/python3.7/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 18)) (0.5.1) Requirement already satisfied: jaxlib>=0.1.37 in /usr/local/lib/python3.7/dist-packages (from optax<=0.0.9,>=0.0.5->-r requirements_mtj.txt (line 4)) (0.3.2+cuda11.cudnn805) Collecting chex>=0.0.4 Downloading chex-0.1.3-py3-none-any.whl (72 kB) |████████████████████████████████| 72 kB 568 kB/s Requirement already satisfied: Jinja2<3.0,>=2.10.1 in /usr/local/lib/python3.7/dist-packages (from flask->-r requirements_mtj.txt (line 11)) (2.11.3) Requirement already satisfied: click<8.0,>=5.1 in /usr/local/lib/python3.7/dist-packages (from flask->-r requirements_mtj.txt (line 11)) (7.1.2) Requirement already satisfied: Werkzeug<2.0,>=0.15 in /usr/local/lib/python3.7/dist-packages (from flask->-r requirements_mtj.txt (line 11)) (1.0.1) Requirement already satisfied: itsdangerous<2.0,>=0.24 in /usr/local/lib/python3.7/dist-packages (from flask->-r requirements_mtj.txt (line 11)) (1.1.0) Requirement already satisfied: dm-tree>=0.1.5 in /usr/local/lib/python3.7/dist-packages (from chex>=0.0.4->optax<=0.0.9,>=0.0.5->-r requirements_mtj.txt (line 4)) (0.1.7) Requirement already satisfied: toolz>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from chex>=0.0.4->optax<=0.0.9,>=0.0.5->-r requirements_mtj.txt (line 4)) (0.11.2) Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.7/dist-packages (from jaxlib>=0.1.37->optax<=0.0.9,>=0.0.5->-r requirements_mtj.txt (line 4)) (2.0) Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2<3.0,>=2.10.1->flask->-r requirements_mtj.txt (line 11)) (2.0.1) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements_mtj.txt (line 3)) (2.10) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements_mtj.txt (line 3)) (2021.10.8) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements_mtj.txt (line 3)) (1.24.3) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements_mtj.txt (line 3)) (3.0.4) Collecting pyyaml>=5.1 Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB) |████████████████████████████████| 596 kB 52.7 MB/s Collecting huggingface-hub<1.0,>=0.1.0 Downloading huggingface_hub-0.5.1-py3-none-any.whl (77 kB) |████████████████████████████████| 77 kB 6.6 MB/s Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers->-r requirements_mtj.txt (line 8)) (3.6.0) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers->-r requirements_mtj.txt (line 8)) (2019.12.20) Collecting tokenizers!=0.11.3,<0.13,>=0.11.1 Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB) |████████████████████████████████| 6.6 MB 15.0 MB/s Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers->-r requirements_mtj.txt (line 8)) (4.11.3) Collecting sacremoses Downloading sacremoses-0.0.49-py3-none-any.whl (895 kB) |████████████████████████████████| 895 kB 56.0 MB/s Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->bleach==4.1.0->-r requirements_mtj.txt (line 18)) (3.0.8) Requirement already satisfied: python-utils>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from progressbar2->-r requirements_mtj.txt (line 9)) (3.1.0) Collecting python-socketio>=5.0.2 Downloading python_socketio-5.6.0-py3-none-any.whl (56 kB) |████████████████████████████████| 56 kB 4.5 MB/s Collecting bidict>=0.21.0 Downloading bidict-0.22.0-py3-none-any.whl (36 kB) Collecting python-engineio>=4.3.0 Downloading python_engineio-4.3.2-py3-none-any.whl (52 kB) |████████████████████████████████| 52 kB 1.5 MB/s Requirement already satisfied: greenlet>=0.3 in /usr/local/lib/python3.7/dist-packages (from eventlet->-r requirements_mtj.txt (line 15)) (1.1.2) Collecting dnspython>=1.15.0 Downloading dnspython-2.2.1-py3-none-any.whl (269 kB) |████████████████████████████████| 269 kB 55.9 MB/s Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers->-r requirements_mtj.txt (line 8)) (3.8.0) Collecting grpcio<=1.43.0,>=1.28.1 Downloading grpcio-1.43.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.1 MB) |████████████████████████████████| 4.1 MB 35.8 MB/s Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ray[default]->-r requirements_mtj.txt (line 6)) (1.0.3) Collecting aiosignal Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB) Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/dist-packages (from ray[default]->-r requirements_mtj.txt (line 6)) (4.3.3) Requirement already satisfied: protobuf>=3.15.3 in /usr/local/lib/python3.7/dist-packages (from ray[default]->-r requirements_mtj.txt (line 6)) (3.17.3) Collecting frozenlist Downloading frozenlist-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (144 kB) |████████████████████████████████| 144 kB 57.4 MB/s Requirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (from ray[default]->-r requirements_mtj.txt (line 6)) (21.4.0) Collecting virtualenv Downloading virtualenv-20.14.1-py2.py3-none-any.whl (8.8 MB) |████████████████████████████████| 8.8 MB 48.1 MB/s Collecting colorful Downloading colorful-0.5.4-py2.py3-none-any.whl (201 kB) |████████████████████████████████| 201 kB 58.2 MB/s Collecting opencensus Downloading opencensus-0.9.0-py2.py3-none-any.whl (128 kB) |████████████████████████████████| 128 kB 56.8 MB/s Collecting prometheus-client<0.14.0,>=0.7.1 Downloading prometheus_client-0.13.1-py3-none-any.whl (57 kB) |████████████████████████████████| 57 kB 5.3 MB/s Collecting aiohttp>=3.7 Downloading aiohttp-3.8.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB) |████████████████████████████████| 1.1 MB 48.5 MB/s Requirement already satisfied: smart-open in /usr/local/lib/python3.7/dist-packages (from ray[default]->-r requirements_mtj.txt (line 6)) (6.0.0) Collecting gpustat>=1.0.0b1 Downloading gpustat-1.0.0b1.tar.gz (82 kB) |████████████████████████████████| 82 kB 262 kB/s Collecting py-spy>=0.2.0 Downloading py_spy-0.3.11-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (3.0 MB) |████████████████████████████████| 3.0 MB 50.9 MB/s Collecting aiohttp-cors Downloading aiohttp_cors-0.7.0-py3-none-any.whl (27 kB) Collecting asynctest==0.13.0 Downloading asynctest-0.13.0-py3-none-any.whl (26 kB) Collecting multidict<7.0,>=4.5 Downloading multidict-6.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (94 kB) |████████████████████████████████| 94 kB 3.6 MB/s Collecting yarl<2.0,>=1.0 Downloading yarl-1.7.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (271 kB) |████████████████████████████████| 271 kB 40.6 MB/s Collecting async-timeout<5.0,>=4.0.0a3 Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[default]->-r requirements_mtj.txt (line 6)) (2.0.12) Requirement already satisfied: nvidia-ml-py3>=7.352.0 in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[default]->-r requirements_mtj.txt (line 6)) (7.352.0) Requirement already satisfied: psutil in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[default]->-r requirements_mtj.txt (line 6)) (5.4.8) Collecting blessed>=1.17.1 Downloading blessed-1.19.1-py2.py3-none-any.whl (58 kB) |████████████████████████████████| 58 kB 6.1 MB/s Requirement already satisfied: wcwidth>=0.1.4 in /usr/local/lib/python3.7/dist-packages (from blessed>=1.17.1->gpustat>=1.0.0b1->ray[default]->-r requirements_mtj.txt (line 6)) (0.2.5) Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray[default]->-r requirements_mtj.txt (line 6)) (5.7.1) Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray[default]->-r requirements_mtj.txt (line 6)) (0.18.1) Collecting opencensus-context>=0.1.2 Downloading opencensus_context-0.1.2-py2.py3-none-any.whl (4.4 kB) Requirement already satisfied: google-api-core<3.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (1.31.5) Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (2022.1) Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (57.4.0) Requirement already satisfied: google-auth<2.0dev,>=1.25.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (1.35.0) Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (1.56.0) Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (4.2.4) Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (4.8) Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (0.2.8) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default]->-r requirements_mtj.txt (line 6)) (0.4.8) Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers->-r requirements_mtj.txt (line 8)) (1.1.0) Collecting distlib<1,>=0.3.1 Downloading distlib-0.3.4-py2.py3-none-any.whl (461 kB) |████████████████████████████████| 461 kB 66.3 MB/s Collecting platformdirs<3,>=2 Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB) Building wheels for collected packages: mesh-transformer, jax, gpustat Building wheel for mesh-transformer (setup.py) ... done Created wheel for mesh-transformer: filename=mesh_transformer-0.0.0+ck.380.gafeb7e2-py3-none-any.whl size=31449 sha256=78633adf35f63f13390c1955d7b4fe8403b56de8844137db5c4dc8f624fe5b62 Stored in directory: /tmp/pip-ephem-wheel-cache-o54_kb70/wheels/a9/30/27/f63733b84b078350350b83689599c277f545bde522627063fe Building wheel for jax (setup.py) ... done Created wheel for jax: filename=jax-0.2.21-py3-none-any.whl size=869303 sha256=3caf5fde6219e80d8c6dd1c57a3437e7cc3052101844b924c10c4864e53f14e3 Stored in directory: /root/.cache/pip/wheels/5c/69/0d/3784dd6d281be0837d8cef1db0c8b37d108c8bff727b961178 Building wheel for gpustat (setup.py) ... done Created wheel for gpustat: filename=gpustat-1.0.0b1-py3-none-any.whl size=15979 sha256=83dc07bcf02de43127871b300520977ac2b9fe1bd7003f0dfbf4ef4e7ef0146a Stored in directory: /root/.cache/pip/wheels/1a/16/e2/3e2437fba4c4b6a97a97bd96fce5d14e66cff5c4966fb1cc8c Successfully built mesh-transformer jax gpustat Installing collected packages: multidict, frozenlist, yarl, platformdirs, distlib, asynctest, async-timeout, aiosignal, virtualenv, pyyaml, python-engineio, opencensus-context, jax, grpcio, blessed, bidict, aiohttp, tokenizers, sacremoses, ray, python-socketio, py-spy, prometheus-client, opencensus, jmp, huggingface-hub, gpustat, dnspython, colorful, chex, aiohttp-cors, transformers, optax, mesh-transformer, lupa, Flask-SocketIO, flask-ngrok, flask-cloudflared, eventlet, dm-haiku, bleach Attempting uninstall: pyyaml Found existing installation: PyYAML 3.13 Uninstalling PyYAML-3.13: Successfully uninstalled PyYAML-3.13 Attempting uninstall: jax Found existing installation: jax 0.3.4 Uninstalling jax-0.3.4: Successfully uninstalled jax-0.3.4 Attempting uninstall: grpcio Found existing installation: grpcio 1.44.0 Uninstalling grpcio-1.44.0: Successfully uninstalled grpcio-1.44.0 Attempting uninstall: prometheus-client Found existing installation: prometheus-client 0.14.1 Uninstalling prometheus-client-0.14.1: Successfully uninstalled prometheus-client-0.14.1 Attempting uninstall: bleach Found existing installation: bleach 5.0.0 Uninstalling bleach-5.0.0: Successfully uninstalled bleach-5.0.0 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. tensorflow 2.8.0 requires tf-estimator-nightly==2.8.0.dev2021122109, which is not installed. Successfully installed Flask-SocketIO-5.1.2 aiohttp-3.8.1 aiohttp-cors-0.7.0 aiosignal-1.2.0 async-timeout-4.0.2 asynctest-0.13.0 bidict-0.22.0 bleach-4.1.0 blessed-1.19.1 chex-0.1.3 colorful-0.5.4 distlib-0.3.4 dm-haiku-0.0.5 dnspython-2.2.1 eventlet-0.33.0 flask-cloudflared-0.0.8 flask-ngrok-0.0.25 frozenlist-1.3.0 gpustat-1.0.0b1 grpcio-1.43.0 huggingface-hub-0.5.1 jax-0.2.21 jmp-0.0.2 lupa-1.10 mesh-transformer-0.0.0+ck.380.gafeb7e2 multidict-6.0.2 opencensus-0.9.0 opencensus-context-0.1.2 optax-0.0.9 platformdirs-2.5.2 prometheus-client-0.13.1 py-spy-0.3.11 python-engineio-4.3.2 python-socketio-5.6.0 pyyaml-6.0 ray-1.12.0 sacremoses-0.0.49 tokenizers-0.12.1 transformers-4.18.0 virtualenv-20.14.1 yarl-1.7.2 Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libnvidia-common-460 nsight-compute-2020.2.0 Use 'sudo apt autoremove' to remove them. The following NEW packages will be installed: netbase 0 upgraded, 1 newly installed, 0 to remove and 42 not upgraded. Need to get 12.7 kB of archives. After this operation, 45.1 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 netbase all 5.4 [12.7 kB] Fetched 12.7 kB in 0s (98.0 kB/s) debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.) debconf: falling back to frontend: Readline debconf: unable to initialize frontend: Readline debconf: (This frontend requires a controlling tty.) debconf: falling back to frontend: Teletype dpkg-preconfigure: unable to re-open stdin: Selecting previously unselected package netbase. (Reading database ... 155202 files and directories currently installed.) Preparing to unpack .../archives/netbase_5.4_all.deb ... Unpacking netbase (5.4) ... Setting up netbase (5.4) ... /tools/node/bin/lt -> /tools/node/lib/node_modules/localtunnel/bin/lt.js + [email protected] added 22 packages from 22 contributors in 2.305s Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libnvidia-common-460 nsight-compute-2020.2.0 Use 'apt autoremove' to remove them. The following additional packages will be installed: libc-ares2 The following NEW packages will be installed: aria2 libc-ares2 0 upgraded, 2 newly installed, 0 to remove and 42 not upgraded. Need to get 1,274 kB of archives. After this operation, 4,912 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libc-ares2 amd64 1.14.0-1ubuntu0.1 [37.5 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 aria2 amd64 1.33.1-1 [1,236 kB] Fetched 1,274 kB in 0s (3,249 kB/s) Selecting previously unselected package libc-ares2:amd64. (Reading database ... 155209 files and directories currently installed.) Preparing to unpack .../libc-ares2_1.14.0-1ubuntu0.1_amd64.deb ... Unpacking libc-ares2:amd64 (1.14.0-1ubuntu0.1) ... Selecting previously unselected package aria2. Preparing to unpack .../aria2_1.33.1-1_amd64.deb ... Unpacking aria2 (1.33.1-1) ... Setting up libc-ares2:amd64 (1.14.0-1ubuntu0.1) ... Setting up aria2 (1.33.1-1) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for libc-bin (2.27-3ubuntu1.3) ... /sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 158 100 158 0 0 410 0 --:--:-- --:--:-- --:--:-- 410 05/01 02:15:37 [NOTICE] Downloading 1 item(s) 05/01 02:15:38 [ERROR] CUID#9 - Download aborted. URI=https://cdn-2.ralf.wtf/KoboldAI/gpt-j-6b-lit-jax.7z Exception: [AbstractCommand.cc:351] errorCode=3 URI=https://cdn-2.ralf.wtf/KoboldAI/gpt-j-6b-lit-jax.7z -> [HttpSkipResponseCommand.cc:219] errorCode=3 Resource not found *** Download Progress Summary as of Sun May 1 02:16:38 2022 *** = [#2b4d5e 2.4GiB/7.9GiB(30%) CN:2 DL:41MiB ETA:2m17s] FILE: /content/gpt-j-6b-lit-jax.7z - *** Download Progress Summary as of Sun May 1 02:17:38 2022 *** = [#2b4d5e 5.0GiB/7.9GiB(63%) CN:2 DL:40MiB ETA:1m12s] FILE: /content/gpt-j-6b-lit-jax.7z - *** Download Progress Summary as of Sun May 1 02:18:39 2022 *** = [#2b4d5e 7.5GiB/7.9GiB(95%) CN:2 DL:32MiB ETA:11s] FILE: /content/gpt-j-6b-lit-jax.7z - 05/01 02:18:53 [NOTICE] Download complete: /content/gpt-j-6b-lit-jax.7z Download Results: gid |stat|avg speed |path/URI ======+====+===========+======================================================= 2b4d5e|OK | 41MiB/s|/content/gpt-j-6b-lit-jax.7z Status Legend: (OK):download completed. 7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs Intel(R) Xeon(R) CPU @ 2.20GHz (406F0),ASM,AES-NI) Scanning the drive for archives: 1 file, 8526031491 bytes (8132 MiB) Extracting archive: /content/gpt-j-6b-lit-jax.7z -- Path = /content/gpt-j-6b-lit-jax.7z Type = 7z Physical Size = 8526031491 Headers Size = 1561 Method = LZMA2:24 Solid = + Blocks = 6 Everything is Ok Folders: 9 Files: 128 Size: 12107378304 Compressed: 8526031491 Launching KoboldAI with the following options : python3 aiserver.py --model TPUMeshTransformerGPTJ --path /content/drive/MyDrive/KoboldAI/models/gpt-j-6b-lit-jax --colab /usr/local/lib/python3.7/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: . """) Welcome to KoboldAI! You have selected the following Model: TPUMeshTransformerGPTJ You have selected the following path for your Model : /content/drive/MyDrive/KoboldAI/models/gpt-j-6b-lit-jax Initializing Flask... OK! Initializing Mesh Transformer JAX, please wait... Connecting to your Colab instance's TPU Elapsed Time: 0:00:17 [ █ ] 2022-05-01 02:32:08.372060: E external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected Downloading: 100% 0.99M/0.99M [00:00<00:00, 5.98MB/s] Downloading: 100% 446k/446k [00:00<00:00, 3.20MB/s] Downloading: 100% 1.29M/1.29M [00:00<00:00, 7.08MB/s] Downloading: 100% 665/665 [00:00<00:00, 428kB/s] /usr/local/lib/python3.7/dist-packages/jax/experimental/maps.py:499: UserWarning: xmap is an experimental feature and probably has bugs! warn("xmap is an experimental feature and probably has bugs!") 8 TPU cores will be used to run the model. Please wait as we initialize the transformer neural network necessary to run the model. Elapsed Time: 0:02:54 ▗ This model has 6 053 381 344 parameters. Please wait while we load the model's tensors into the TPU memory. ⡀ Time: 0:03:51 287/287 100% [███████████████████████████████████████████] Finished loading the model!