And Signal is open source so, if it did anything weird with private keys, everyone would know
Well, no. At least not by default as you are running a compiled version of it. Someone could inject code you don't know anything about before compilation that for example leaked your keys.
One way to be more confident no one has, would be to have predictable builds that you can recreate and then compare the file fingerprints. But I do not think that is possible, at least on android, as google holds they signature keys to apps.
Not an answer to your question, but don't let some gatekeeper.. well, gate keep. There are many ways to selfhost. Running your own hardware is one, renting a VPS but hosting the services yourself is another one. One is not better than the other.
Just pick what is the best solution for you and your problem.