Post

Magik6k
@magik.net

Let’s see how smooth setting up RDMA between two non-server AMD machines goes

Magik6k@magik.net

NVIDIA DIGITS couldn't be announced at a better time. I have a dark fiber that I could use for remote display with Looking Glass over RDMA with lgproxy (4k 120hz hdr, ~60Gbps), and this cute little thing solves all of my client side problems with its hopefully 2x 100Gbps NICs

Magik6k
@magik.net

* The card seems to have Infiniband disabled * To enable I need mellanox drivers * It also seems to see PCIe errors (?) not sure if that's a big issue :shrug: * It's also getting high temperature errors, probably due to being put into a peasant desktop case instead of a real machine

Magik6k
@magik.net

The IB disabled issue wouldn't be half bad, but both of those machines are on Arch Linux, and best packages are either Debs or RPMs. At least there are also from-source versions, but I don't think I have the patience for that I guess let's reboot and see if there are any bios knobs. Also add a fan

0 reposts0 likes

Magik6k
@magik.net

Added a cable between computers, setup OpenSM and now it links up! Benchmarked at ~60Gbps with ib_read_bw. The bottleneck is explainable by.... PCIe on the non-threadripper end of this setup LnkSta: Speed 8GT/s (downgraded), Width x8 (downgraded)