@phantasus honestly, I do only have very little experience. I have installed an instance on my Raspberry Pi 5 on my home network. Conversion of videos is not very performant there. Anything else is running fine. Once the videos are rendered in the various resolution levels, the most important part in terms of performance is the bandwidth of the network connection and pf course storage.
For a small instance with a few users only who do not upload dozens of videos at once, a relatively small virtual private server should serve you well. For larger instances you may want some didicated, more performant server.