Our great sponsors
-
vendor-reset
Linux kernel vendor specific hardware reset module for sequences that are too complex/complicated to land in pci_quirks.c
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
OSX-KVM
Run macOS on QEMU/KVM. With OpenCore + Monterey + Ventura + Sonoma support now! Only commercial (paid) support is available now to avoid spammy issues. No Mac system is required.
-
fedora-acs-override
Using the ACS override patch for Fedora to split identical hardware in the kernel
-
LookingGlass
An extremely low latency KVMFR (KVM FrameRelay) implementation for guests with VGA PCI Passthrough.
Unfortunately AMD cards suffer from a reset bug, still.
The reset bug being that you can pass through the card fine, once. But if you try to pass it through again (or the card experiences an issue and needs to reset), they get caught in some kind of bad state and won’t work until power is removed and restored. Which requires a reboot or a only slightly less disruptive dance with system power states.
For vega and 5000 series gpu’s, there’s https://github.com/gnif/vendor-reset
Incidentally, nvidia gpus are so good at resetting, they’ve probably done so without you noticing. If the screen ever goes black for a fraction of a second and returns in normal usage, it was probably because it reset itself.
The lower 6000 series lower than the 6800’s for example may or may not have the issue. It seems most “reference” cards are fine, but custom vendor cards often but not always have issues. My reference 6700 works fine, but a sapphire 6700 probably won’t.
And the 7000 series is also fucky in a new way somehow. Gnif knows far more about this than me, and has basically thrown up his hands at how AMD doesn’t care. He’s made occasional posts about it on https://forum.level1techs.com/
Gnif is also responsible for Looking glass: https://github.com/gnif/LookingGlass
Note also: it just got an experimental Windows driver!
[1]https://github.com/virtio-win/kvm-guest-drivers-windows/pull...
VirGL is definitely an interesting project, but all one has to do to get GPU passthrough working (from a Linux QEMU host to any guest OS) is: 1.) research a cheap, secondary GPU that is natively supported by the guest OS, 2.) plug such a secondary GPU into a PCIe slot on the host and hook it up to the primary monitor with a secondary cable (D-Sub vs. DVI, etc.), 3.) setup Linux to ignore the secondary GPU at boot and configure a QEMU VM for the GPU passthrough. The whole process takes perhaps one or two hours and as works flawlessly, with no stability issues. (Switching across the two GPU cables can be accomplished in software by using Display Data Channel /DDC/ utilities and switching keyboard/mouse can be accomplished by using evdev /event device/ passthrough.) More information: https://github.com/kholia/OSX-KVM/blob/master/notes.md#gpu-p...
- Code + RPMs - https://github.com/some-natalie/fedora-acs-override
As far as concerns around stability with ACS override, I tend to only enable the override for the specific GPU (or other hardware) that I'm passing through and haven't encountered any stability problems or memory leaks that'd interrupt desktop or light server usage. I also used to run this for a bunch of white-box GPU hardware for a customer at a former job and it worked well for exploratory AI/ML workloads before investing in the big Nvidia DGX boxes. YMMV, of course!
A better way [for me, anyways] is getting GRID drivers running. However, this only works with the 9xx cards up to the 2080.
https://gitlab.com/polloloco/vgpu-proxmox
Unfortunately AMD cards suffer from a reset bug, still.
The reset bug being that you can pass through the card fine, once. But if you try to pass it through again (or the card experiences an issue and needs to reset), they get caught in some kind of bad state and won’t work until power is removed and restored. Which requires a reboot or a only slightly less disruptive dance with system power states.
For vega and 5000 series gpu’s, there’s https://github.com/gnif/vendor-reset
Incidentally, nvidia gpus are so good at resetting, they’ve probably done so without you noticing. If the screen ever goes black for a fraction of a second and returns in normal usage, it was probably because it reset itself.
The lower 6000 series lower than the 6800’s for example may or may not have the issue. It seems most “reference” cards are fine, but custom vendor cards often but not always have issues. My reference 6700 works fine, but a sapphire 6700 probably won’t.
And the 7000 series is also fucky in a new way somehow. Gnif knows far more about this than me, and has basically thrown up his hands at how AMD doesn’t care. He’s made occasional posts about it on https://forum.level1techs.com/
Gnif is also responsible for Looking glass: https://github.com/gnif/LookingGlass