Do you have a soldering iron? If so you could use that to heat up the set screw.
Make sure anytime you are working on a potentially problem screw, especially a set screw, that you use a quality wrench to start. If you use a cheap bit and it strips, you're going to have problems. Make sure your...
There are some that have a set screw. But that is not visible in the pic of this one. So I think youreright. It's either a bang it out, or a set screw. Like this one is obviously a set screw. But I have seen fully concentric cups with a slot cut in them to access a set screw locking the pin in.