Say you have a group named "Test". If you want multiple users to manage it, you could create a user group for each of the users and make them direct managers of those groups. Then make the "Test" group a sub-group to all the groups you just created. Since permissions are inherited, all users will be able to manage the "Test" group.
Alternatively (and preferably in my opinion), you could create a custom action that would allow users to assume the Direct Manager role by updating the Direct Manager field.
I hope this helps.