Follow these instructions to get Steam Remote Play working.

Once Remote Play has been launched, the game will behave as if you there were multiple people playing locally together. 

When you launch Ultimate Chicken Horse, as usual, there will only be one cursor in the main menu (P1). 

Start a lobby (either local or online). When you see the Treehouse, your Steam remote players can join by pressing jump on their controllers. 

Note that with Remote Play, only one player can use the keyboard and mouse. Every keyboard and mouse will be recognized as the same input, so you cannot add a player with them if someone else is already playing with that input method in your Remote Play group. Everyone else will have to use a controller.