Quite the opposite: games often just run on GL or Vulkan (actually usually Unity or Unreal which will then use GL or Vulkan), and will abstract input with something like libsdl, so they don't have to care about Wayland or X11. But still, some problems that affect games to have to be solved in the X11/Wayland level. Some are solved by the graphics companies (Intel, AMD, etc), some others are either left or picked by companies with a different kind of interest in the graphics world, like Valve.

If only we had more real world money-making products actually relying on the desktop graphics stack to make money, then we'd have a better graphics situation.