The below works to hold back 
xserver-xorg-video-all to stable, so you need to repace stabls/testing with whatever ubuntu uses and change the package name - then it should work for you. 

Package: xserver-xorg-video-all
Pin: release a=stable
Pin-Priority: 1001

Package: *
Pin: release a=testing
Pin-Priority: 60