Single-GPU-Passthrough VS rom-parser

Link to hook scripts: https://github.com/joeknock90/Single-GPU-Passthrough

Yes but that's a different kettle of fish. You then don't need two gpus, although if you do it can be useful. Start here and see how deep the rabbit hole goes https://github.com/joeknock90/Single-GPU-Passthrough

Im assuming you're on a single gpu setup and, as such are trying to do single gpu passthrough with a single nvidia card. so you'll likely have to run some libvirt hooks or smth to run scripts to unbind your gpu once you start your vm. In which case id suggest going through these two guides 1 2 which go over completely passing through your single gpu when you start your vm

Single GPU passthrough: https://github.com/joeknock90/Single-GPU-Passthrough

Out of curiosity, who is Muta, the video dude, or the guide writer, or someone else? From the GitHub it sounded like running the commands from the guide would create a qemu.d/[VM Name]/prepare/begin/start.sh . If I were to simply create those directories myself, and then script as described in the video, I should be good?

Thanks for the info. Some of these I recognize from a single gpu passthrough project on github that i used on my main rig... https://github.com/joeknock90/Single-GPU-Passthrough

NVIDIA VBIOS Step 1: How to extract or obtain your VBIOS Option 1 (preferred): Pull the rom via nvidia's proprietary tool nvflash (https://www.techpowerup.com/download/nvidia-nvflash/). The download is a zipped binary (not source), simply unzip it, chmod +x, and run it: ./nvflash --save original.rom. Note1: This tool does not work when the nvidia proprietary driver is bound to the card. It will work with no driver bound, vfio-pci, or in some cases nouveau (sketchy). Note2: This can also be done with windows both native and virtualized using GPU-z's save vbios button. GPU-z uses nvflash internally Option 2: Download a rom from Techpowerup's database that matches your PCI ID's and Subsystem ID's. Here is an example with a GT 710: $ lspci -nnk (...) 0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208B [GeForce GT 710] [10de:128b] (rev a1) Subsystem: Micro-Star International Co., Ltd. [MSI] GK208B [GeForce GT 710] [1462:8c93] The ID's are 10de:128b and 1462:8c93. Google techpowerup + these numbers and download a rom that matches. Double check that all ID's match. (edited) [10:30 PM]Aiber: NVIDIA VBIOS Step 2: Determining the start of the VBIOS within the ROM file NVIDIA rom files contain extra data (IFR, other firmware, etc.) at the start of the file. We can obtain the vbios portion of the rom by trimming off the unwanted bits. This must be removed before passing it to a VM. Use rom-parser (https://github.com/awilliam/rom-parser) and look for the first entry Valid ROM signature found @XXXXh: $ rom-parser original.rom Valid ROM signature found @600h, PCIR offset 190h PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 128b, class: 030000 PCIR: revision 0, vendor revision: 1 Valid ROM signature found @fc00h, PCIR offset 1ch PCIR: type 3 (EFI), vendor: 10de, device: 128b, class: 030000 PCIR: revision 3, vendor revision: 0 EFI: Signature Valid, Subsystem: Boot, Machine: X64 Last image The header size is the hexadecimal value @600h which is 1536 bytes in decimal. (edited) [10:30 PM]Aiber: NVIDIA VBIOS Step 3: Removing the headers If the value from step 2 is not zero, the extraneous data must be removed, 0x600 bytes in particular this case: $ dd if=original.rom of=vbios.rom bs=$((0x600)) skip=1 You may verify the result by redoing step 2 on the new file to confirm the rom now starts at @0h (the beginning of the file.) NVIDIA VBIOS Step 4: Passing a VBIOS to the VM Libvirt: Under for the GPU video device add Qemu: -device vfio-pci,host=01:00.0,romfile=/path/to/vbios.rom (edited) August 26, 2018

git clone https://github.com/awilliam/rom-parser cd rom-parser make ./rom-parser /usr/share/kvm/vbios.bin

When your PC boots and initializes its NVIDIA gpu as pci devices go. The the vbios rom provided by the GPU gets completely truncated, overwritten and in short, fucked. This isn't a problem though because your host has already initialized the card, you would never need to initialize it again and if you try to do a vbios dump after this you'll notice it does not pass a vbios check with a project such as rom-parser (https://github.com/awilliam/rom-parser) and instead errors early. Now you're about to unbind your nvidia driver on the host and bind vfio-pci and give it to your VM. The TianoCore (OVMF.fd) EFI bios your VM will boot into will do the same thing your host did to initialize the card. But that program stored in your GPU's ROM is completely borked because your host already initialized it. The VM won't be able to draw anything if this happens.

In order to get the vBIOS I had to use NVFLASH, since my GPU model isn't on TechPowerUp. Initially I modded it manually with a hex editor, but after reading some posts I decided to try checking the rom using rom-parser, which told me that it was valid, and patching the rom with NVIDIA-vBIOS-VFIO-Patcher (At first this script was throwing an exception, but I got around it using --disable-footer-strip), after starting the VM with this new patched rom both my monitors turned black and I was forced to shutdown the PC. I also tried patching the only GTX950 Zotac ROM on TechPowerUp and using it but I get no display out.

Have you patched this and verified it with something like rom-parser?

/u/compile_me_world I'd recommend checking your vbios with rom-parser to make sure it terminates correctly, and maybe attempt patching a new one using NVIDIA-vBIOS-VFIO-Patcher. Don't be afraid to grab a dump from TechPowerup, it has a whole catalogue and has been helpful for my 2080Ti, 780Ti, 680 and GT710.

I figured this process would be a lot harder than it was, but that's also what I assumed about updating the bios. But thankfully the #wiki-and-psa channel in the discord pretty much guided me through it. I used chmod +x ./x64/nvflash followed by disabling nvidia's driver modules via this guide. Though trying to start the restart the modules with sudo systemctl start multi-user.target didn't work so I just rebooted. I moved onto step 2, determining the start of the rom with rom-parser and then needed to use dd to remove the header, if you use fish, swap back to bash to execute dd if=vbios.rom of=vbios.fixed.rom bs=((0xHEX)) skip=1 (replacing HEX with whatever your parser output was.)

I remembered, the rom file was extracted following a manual, with a rom-parser from https://github.com/awilliam/rom-parser using git clone.

FWIW, rom-fixes would be an easier means to change the device ID in the ROM, and it'll fix the checksum as well - https://github.com/awilliam/rom-parser

I basically followed this post, I also found that my GPU is not UEFI compatible (GPU-Z and rom-parser confirm this) and I remember reading in a forum somewhere that this might be the cause of code 43. I'm not really sure, but I have ran out of things to try. The manufacturer nor NVIDIA have a newer rom publicly available. So I'm asking you fine folks for some help. Has anyone else encountered this problem or knows something that might be related to this?