It would be nice if this ability were in the admin UI, but I don't remember seeing anywhere.
Anyway, one way to do this would be to query the users from the command line via the zmprov command.
First, you need to find out what the zimbraId is of the CoS you are looking for.
Code:
$ zmprov gc default | grep zimbraId
This gives you a big long string value. Now use that string in another zmprov command in order to get the list of users in that CoS.
Code:
$ zmprov sa zimbraCOSId=biglongstring
-Aubrey