decided not to launch, not sure why, this is the log

• KoboldAI-Client

  185 3,369 6.3 Python

Found TPU at: grpc://10.18.240.10:8470 Now we will need your Google Drive to store settings and saves, you must login with the same account you used for Colab. Drive already mounted at /content/drive/; to attempt to forcibly remount, call drive.mount("/content/drive/", force_remount=True). --2023-02-22 17:46:35-- https://koboldai.org/ckds Resolving koboldai.org (koboldai.org)... 104.21.21.176, 172.67.199.170, 2606:4700:3036::ac43:c7aa, ... Connecting to koboldai.org (koboldai.org)|104.21.21.176|:443... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh [following] --2023-02-22 17:46:35-- https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6675 (6.5K) [text/plain] Saving to: ‘STDOUT’ - 100%[===================>] 6.52K --.-KB/s in 0s 2023-02-22 17:46:35 (85.5 MB/s) - written to stdout [6675/6675] mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/stories/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/models/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/settings/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/softprompts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/userscripts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/presets/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/themes/’: File exists mkdir: cannot create directory ‘/content/KoboldAI-Client’: File exists Reinitialized existing Git repository in /content/KoboldAI-Client/.git/ Fetching origin From https://github.com/koboldai/KoboldAI-Client * [new branch] main -> origin/main Already on 'main' HEAD is now at c27faf5 Updated Silence Audio - GPU cp: cannot stat 'softprompts/*': No such file or directory cp: cannot stat 'presets/*': No such file or directory cp: cannot stat 'themes/*': No such file or directory rm: cannot remove 'stories': Is a directory rm: cannot remove 'userscripts': Is a directory rm: cannot remove 'models': Is a directory ln: failed to create symbolic link 'settings/settings': File exists Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting git+https://github.com/VE-FORBRYDERNE/mesh-transformer-jax@ck (from -r requirements_mtj.txt (line 11)) Cloning https://github.com/VE-FORBRYDERNE/mesh-transformer-jax (to revision ck) to /tmp/pip-req-build-q85v5kg0 Running command git clone --filter=blob:none --quiet https://github.com/VE-FORBRYDERNE/mesh-transformer-jax /tmp/pip-req-build-q85v5kg0 Running command git checkout -b ck --track origin/ck Switched to a new branch 'ck' Branch 'ck' set up to track remote branch 'ck' from 'origin'. Resolved https://github.com/VE-FORBRYDERNE/mesh-transformer-jax to commit 89577e3d49aef94179a138260f6fbd6ca2964327 Preparing metadata (setup.py) ... done Requirement already satisfied: torch<1.13,>=1.9 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 1)) (1.12.1) Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 2)) (1.22.4) Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 3)) (4.64.1) Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 4)) (2.25.1) Requirement already satisfied: dm-haiku==0.0.5 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 5)) (0.0.5) Requirement already satisfied: jax==0.2.21 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 6)) (0.2.21) Requirement already satisfied: jaxlib<=0.3.7,>=0.1.69 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 7)) (0.3.7) Requirement already satisfied: transformers==4.24.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 8)) (4.24.0) Requirement already satisfied: huggingface_hub>=0.10.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 9)) (0.12.1) Requirement already satisfied: progressbar2 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 10)) (3.38.0) Requirement already satisfied: flask in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 12)) (1.1.4) Requirement already satisfied: Flask-SocketIO in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 13)) (5.3.2) Requirement already satisfied: flask-cloudflared==0.0.10 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 14)) (0.0.10) Requirement already satisfied: flask-ngrok in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 15)) (0.0.25) Requirement already satisfied: eventlet in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 16)) (0.33.3) Requirement already satisfied: dnspython==2.2.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 17)) (2.2.1) Requirement already satisfied: lupa==1.10 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 18)) (1.10) Requirement already satisfied: markdown in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 19)) (3.4.1) Requirement already satisfied: bleach==4.1.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 20)) (4.1.0) Requirement already satisfied: flask-session in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 21)) (0.4.0) Requirement already satisfied: marshmallow>=3.13 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 22)) (3.19.0) Requirement already satisfied: apispec-webframeworks in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 23)) (0.5.2) Requirement already satisfied: loguru in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 24)) (0.6.0) Requirement already satisfied: jmp>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.0.4) Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.8.10) Requirement already satisfied: absl-py>=0.7.1 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (1.4.0) Requirement already satisfied: opt-einsum in /usr/local/lib/python3.8/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 6)) (3.3.0) Requirement already satisfied: scipy>=1.2.1 in /usr/local/lib/python3.8/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 6)) (1.7.3) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (2022.6.2) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (0.13.2) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (6.0) Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (3.9.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (23.0) Requirement already satisfied: webencodings in /usr/local/lib/python3.8/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 20)) (0.5.1) Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.8/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 20)) (1.15.0) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch<1.13,>=1.9->-r requirements_mtj.txt (line 1)) (4.5.0) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (2.10) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (1.24.3) Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (4.0.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (2022.12.7) Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.8/dist-packages (from jaxlib<=0.3.7,>=0.1.69->-r requirements_mtj.txt (line 7)) (2.0.7) Requirement already satisfied: python-utils>=2.3.0 in /usr/local/lib/python3.8/dist-packages (from progressbar2->-r requirements_mtj.txt (line 10)) (3.5.2) Requirement already satisfied: click<8.0,>=5.1 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (7.1.2) Requirement already satisfied: itsdangerous<2.0,>=0.24 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (1.1.0) Requirement already satisfied: Werkzeug<2.0,>=0.15 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (1.0.1) Requirement already satisfied: Jinja2<3.0,>=2.10.1 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (2.11.3) Requirement already satisfied: python-socketio>=5.0.2 in /usr/local/lib/python3.8/dist-packages (from Flask-SocketIO->-r requirements_mtj.txt (line 13)) (5.7.2) Requirement already satisfied: greenlet>=0.3 in /usr/local/lib/python3.8/dist-packages (from eventlet->-r requirements_mtj.txt (line 16)) (2.0.2) Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.8/dist-packages (from markdown->-r requirements_mtj.txt (line 19)) (6.0.0) Requirement already satisfied: cachelib in /usr/local/lib/python3.8/dist-packages (from flask-session->-r requirements_mtj.txt (line 21)) (0.10.2) Requirement already satisfied: apispec[yaml]>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from apispec-webframeworks->-r requirements_mtj.txt (line 23)) (6.0.2) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata>=4.4->markdown->-r requirements_mtj.txt (line 19)) (3.13.0) Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.8/dist-packages (from Jinja2<3.0,>=2.10.1->flask->-r requirements_mtj.txt (line 12)) (2.0.1) Requirement already satisfied: python-engineio>=4.3.0 in /usr/local/lib/python3.8/dist-packages (from python-socketio>=5.0.2->Flask-SocketIO->-r requirements_mtj.txt (line 13)) (4.3.4) Requirement already satisfied: bidict>=0.21.0 in /usr/local/lib/python3.8/dist-packages (from python-socketio>=5.0.2->Flask-SocketIO->-r requirements_mtj.txt (line 13)) (0.22.1) Reading package lists... Done Building dependency tree Reading state information... Done netbase is already the newest version (6.1). aria2 is already the newest version (1.35.0-1build1). The following package was automatically installed and is no longer required: libnvidia-common-510 Use 'sudo apt autoremove' to remove it. 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded. /tools/node/bin/lt -> /tools/node/lib/node_modules/localtunnel/bin/lt.js + [email protected] updated 1 package in 0.712s Launching KoboldAI with the following options : python3 aiserver.py --model KoboldAI/GPT-NeoX-20B-Erebus --colab 2023-02-22 17:46:50.342363: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-22 17:46:51.588645: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 2023-02-22 17:46:51.588777: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 2023-02-22 17:46:51.588800: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. MESSAGE | Welcome to KoboldAI! MESSAGE | You have selected the following Model: KoboldAI/GPT-NeoX-20B-Erebus INIT | Starting | Flask INIT | OK | Flask INIT | Starting | Webserver Initializing Mesh Transformer JAX, please wait... Connecting to your Colab instance's TPU Elapsed Time: 0:00:02 [ █ ] 2023-02-22 17:47:28.996684: E external/org_tensorflow/tensorflow/compiler/xla/python/tpu_driver/grpc_tpu_driver.cc:1070] Failed to open the gRPC driver: 4: Deadline Exceeded: WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.) Traceback (most recent call last): File "aiserver.py", line 10214, in load_model(initial_load=True) File "aiserver.py", line 2806, in load_model tpu_mtj_backend.load_model(vars.custmodpth, hf_checkpoint=vars.model not in ("TPUMeshTransformerGPTJ", "TPUMeshTransformerGPTNeoX") and vars.use_colab_tpu, **vars.modelconfig) File "/content/KoboldAI-Client/tpu_mtj_backend.py", line 1194, in load_model devices = np.array(jax.devices()[:cores_per_replica]).reshape(mesh_shape) File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 314, in devices return get_backend(backend).devices() File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 258, in get_backend return _get_backend_uncached(platform) File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 248, in _get_backend_uncached raise RuntimeError(f"Requested backend {platform}, but it failed " RuntimeError: Requested backend tpu_driver, but it failed to initialize: DEADLINE_EXCEEDED: Failed to connect to remote server at address: grpc://10.18.240.10:8470. Error from gRPC: Deadline Exceeded. Details:

• mesh-transformer-jax

  2 22 0.0 Python

  Fork of kingoflolz/mesh-transformer-jax with memory usage optimizations and support for GPT-Neo, GPT-NeoX, BLOOM, OPT and fairseq dense LM. Primarily used by KoboldAI and mtj-softtuner. (by VE-FORBRYDERNE)

  

Found TPU at: grpc://10.18.240.10:8470 Now we will need your Google Drive to store settings and saves, you must login with the same account you used for Colab. Drive already mounted at /content/drive/; to attempt to forcibly remount, call drive.mount("/content/drive/", force_remount=True). --2023-02-22 17:46:35-- https://koboldai.org/ckds Resolving koboldai.org (koboldai.org)... 104.21.21.176, 172.67.199.170, 2606:4700:3036::ac43:c7aa, ... Connecting to koboldai.org (koboldai.org)|104.21.21.176|:443... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh [following] --2023-02-22 17:46:35-- https://raw.githubusercontent.com/henk717/KoboldAI/united/colabkobold.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6675 (6.5K) [text/plain] Saving to: ‘STDOUT’ - 100%[===================>] 6.52K --.-KB/s in 0s 2023-02-22 17:46:35 (85.5 MB/s) - written to stdout [6675/6675] mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/stories/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/models/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/settings/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/softprompts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/userscripts/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/presets/’: File exists mkdir: cannot create directory ‘/content/drive/MyDrive/KoboldAI/themes/’: File exists mkdir: cannot create directory ‘/content/KoboldAI-Client’: File exists Reinitialized existing Git repository in /content/KoboldAI-Client/.git/ Fetching origin From https://github.com/koboldai/KoboldAI-Client * [new branch] main -> origin/main Already on 'main' HEAD is now at c27faf5 Updated Silence Audio - GPU cp: cannot stat 'softprompts/*': No such file or directory cp: cannot stat 'presets/*': No such file or directory cp: cannot stat 'themes/*': No such file or directory rm: cannot remove 'stories': Is a directory rm: cannot remove 'userscripts': Is a directory rm: cannot remove 'models': Is a directory ln: failed to create symbolic link 'settings/settings': File exists Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting git+https://github.com/VE-FORBRYDERNE/mesh-transformer-jax@ck (from -r requirements_mtj.txt (line 11)) Cloning https://github.com/VE-FORBRYDERNE/mesh-transformer-jax (to revision ck) to /tmp/pip-req-build-q85v5kg0 Running command git clone --filter=blob:none --quiet https://github.com/VE-FORBRYDERNE/mesh-transformer-jax /tmp/pip-req-build-q85v5kg0 Running command git checkout -b ck --track origin/ck Switched to a new branch 'ck' Branch 'ck' set up to track remote branch 'ck' from 'origin'. Resolved https://github.com/VE-FORBRYDERNE/mesh-transformer-jax to commit 89577e3d49aef94179a138260f6fbd6ca2964327 Preparing metadata (setup.py) ... done Requirement already satisfied: torch<1.13,>=1.9 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 1)) (1.12.1) Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 2)) (1.22.4) Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 3)) (4.64.1) Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 4)) (2.25.1) Requirement already satisfied: dm-haiku==0.0.5 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 5)) (0.0.5) Requirement already satisfied: jax==0.2.21 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 6)) (0.2.21) Requirement already satisfied: jaxlib<=0.3.7,>=0.1.69 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 7)) (0.3.7) Requirement already satisfied: transformers==4.24.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 8)) (4.24.0) Requirement already satisfied: huggingface_hub>=0.10.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 9)) (0.12.1) Requirement already satisfied: progressbar2 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 10)) (3.38.0) Requirement already satisfied: flask in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 12)) (1.1.4) Requirement already satisfied: Flask-SocketIO in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 13)) (5.3.2) Requirement already satisfied: flask-cloudflared==0.0.10 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 14)) (0.0.10) Requirement already satisfied: flask-ngrok in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 15)) (0.0.25) Requirement already satisfied: eventlet in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 16)) (0.33.3) Requirement already satisfied: dnspython==2.2.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 17)) (2.2.1) Requirement already satisfied: lupa==1.10 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 18)) (1.10) Requirement already satisfied: markdown in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 19)) (3.4.1) Requirement already satisfied: bleach==4.1.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 20)) (4.1.0) Requirement already satisfied: flask-session in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 21)) (0.4.0) Requirement already satisfied: marshmallow>=3.13 in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 22)) (3.19.0) Requirement already satisfied: apispec-webframeworks in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 23)) (0.5.2) Requirement already satisfied: loguru in /usr/local/lib/python3.8/dist-packages (from -r requirements_mtj.txt (line 24)) (0.6.0) Requirement already satisfied: jmp>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.0.4) Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (0.8.10) Requirement already satisfied: absl-py>=0.7.1 in /usr/local/lib/python3.8/dist-packages (from dm-haiku==0.0.5->-r requirements_mtj.txt (line 5)) (1.4.0) Requirement already satisfied: opt-einsum in /usr/local/lib/python3.8/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 6)) (3.3.0) Requirement already satisfied: scipy>=1.2.1 in /usr/local/lib/python3.8/dist-packages (from jax==0.2.21->-r requirements_mtj.txt (line 6)) (1.7.3) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (2022.6.2) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (0.13.2) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (6.0) Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (3.9.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers==4.24.0->-r requirements_mtj.txt (line 8)) (23.0) Requirement already satisfied: webencodings in /usr/local/lib/python3.8/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 20)) (0.5.1) Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.8/dist-packages (from bleach==4.1.0->-r requirements_mtj.txt (line 20)) (1.15.0) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch<1.13,>=1.9->-r requirements_mtj.txt (line 1)) (4.5.0) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (2.10) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (1.24.3) Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (4.0.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->-r requirements_mtj.txt (line 4)) (2022.12.7) Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.8/dist-packages (from jaxlib<=0.3.7,>=0.1.69->-r requirements_mtj.txt (line 7)) (2.0.7) Requirement already satisfied: python-utils>=2.3.0 in /usr/local/lib/python3.8/dist-packages (from progressbar2->-r requirements_mtj.txt (line 10)) (3.5.2) Requirement already satisfied: click<8.0,>=5.1 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (7.1.2) Requirement already satisfied: itsdangerous<2.0,>=0.24 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (1.1.0) Requirement already satisfied: Werkzeug<2.0,>=0.15 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (1.0.1) Requirement already satisfied: Jinja2<3.0,>=2.10.1 in /usr/local/lib/python3.8/dist-packages (from flask->-r requirements_mtj.txt (line 12)) (2.11.3) Requirement already satisfied: python-socketio>=5.0.2 in /usr/local/lib/python3.8/dist-packages (from Flask-SocketIO->-r requirements_mtj.txt (line 13)) (5.7.2) Requirement already satisfied: greenlet>=0.3 in /usr/local/lib/python3.8/dist-packages (from eventlet->-r requirements_mtj.txt (line 16)) (2.0.2) Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.8/dist-packages (from markdown->-r requirements_mtj.txt (line 19)) (6.0.0) Requirement already satisfied: cachelib in /usr/local/lib/python3.8/dist-packages (from flask-session->-r requirements_mtj.txt (line 21)) (0.10.2) Requirement already satisfied: apispec[yaml]>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from apispec-webframeworks->-r requirements_mtj.txt (line 23)) (6.0.2) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata>=4.4->markdown->-r requirements_mtj.txt (line 19)) (3.13.0) Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.8/dist-packages (from Jinja2<3.0,>=2.10.1->flask->-r requirements_mtj.txt (line 12)) (2.0.1) Requirement already satisfied: python-engineio>=4.3.0 in /usr/local/lib/python3.8/dist-packages (from python-socketio>=5.0.2->Flask-SocketIO->-r requirements_mtj.txt (line 13)) (4.3.4) Requirement already satisfied: bidict>=0.21.0 in /usr/local/lib/python3.8/dist-packages (from python-socketio>=5.0.2->Flask-SocketIO->-r requirements_mtj.txt (line 13)) (0.22.1) Reading package lists... Done Building dependency tree Reading state information... Done netbase is already the newest version (6.1). aria2 is already the newest version (1.35.0-1build1). The following package was automatically installed and is no longer required: libnvidia-common-510 Use 'sudo apt autoremove' to remove it. 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded. /tools/node/bin/lt -> /tools/node/lib/node_modules/localtunnel/bin/lt.js + [email protected] updated 1 package in 0.712s Launching KoboldAI with the following options : python3 aiserver.py --model KoboldAI/GPT-NeoX-20B-Erebus --colab 2023-02-22 17:46:50.342363: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-22 17:46:51.588645: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 2023-02-22 17:46:51.588777: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 2023-02-22 17:46:51.588800: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. MESSAGE | Welcome to KoboldAI! MESSAGE | You have selected the following Model: KoboldAI/GPT-NeoX-20B-Erebus INIT | Starting | Flask INIT | OK | Flask INIT | Starting | Webserver Initializing Mesh Transformer JAX, please wait... Connecting to your Colab instance's TPU Elapsed Time: 0:00:02 [ █ ] 2023-02-22 17:47:28.996684: E external/org_tensorflow/tensorflow/compiler/xla/python/tpu_driver/grpc_tpu_driver.cc:1070] Failed to open the gRPC driver: 4: Deadline Exceeded: WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.) Traceback (most recent call last): File "aiserver.py", line 10214, in load_model(initial_load=True) File "aiserver.py", line 2806, in load_model tpu_mtj_backend.load_model(vars.custmodpth, hf_checkpoint=vars.model not in ("TPUMeshTransformerGPTJ", "TPUMeshTransformerGPTNeoX") and vars.use_colab_tpu, **vars.modelconfig) File "/content/KoboldAI-Client/tpu_mtj_backend.py", line 1194, in load_model devices = np.array(jax.devices()[:cores_per_replica]).reshape(mesh_shape) File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 314, in devices return get_backend(backend).devices() File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 258, in get_backend return _get_backend_uncached(platform) File "/usr/local/lib/python3.8/dist-packages/jax/_src/lib/xla_bridge.py", line 248, in _get_backend_uncached raise RuntimeError(f"Requested backend {platform}, but it failed " RuntimeError: Requested backend tpu_driver, but it failed to initialize: DEADLINE_EXCEEDED: Failed to connect to remote server at address: grpc://10.18.240.10:8470. Error from gRPC: Deadline Exceeded. Details:

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• colabtools

  19 2,071 8.7 Jupyter Notebook

  Python libraries for Google Colaboratory

  

We are still waiting for Google to fix the TPU driver. You can see the status here : https://github.com/googlecolab/colabtools/issues/3405

Suggest a related project