This is an interesting question and I'd hazard a guess that the majority will say "you've earned it, wear it and don't worry". And I think that is a acceptable response.
However, we are a forum of watch enthusiasts who have already decided that it's fine to spend hundreds/thousands on indulgent, "excessive" luxury items. (I quote excessive because that's what they are generally seen as by joe public.)
It's a tough one, but I'm of the same opinion as you here Iain. There are people working in the NHS who are working so bloody hard to make ends meet atm and it says much about your character that you are conscious of not wanting to risk upsetting your fellow co-workers.
People know Rolex and the fact that they cost a lot. After all, you'd be wearing the brand of all the celebrities, influencers
and social media schmuck that we are bombarded with these days.
Re cars, these can also be seen in a similar light to a degree. I know of down-to-earth people who have "done good", bought a car they'd always wanted, only to find some jealous t**t has run a key down the side or similar.
I guess this also depends on the kind of community you live in. I'm from an area where there is significant poverty alongside blue/white collar and pockets of significant wealth.
But cars are generally a necessity that require a relatively significant outlay for most people. A "cheap" alternative can also run into 4/5 figures for something reliable, and I'd think that most people prefer to have something that gets them to work reliably, otherwise they'd end up with no workplace to travel to. Also, they are not constantly with you throughout the working day, unlike a watch.
Holidays seem to be one of those accepted things. They will no doubt create a bit of envy, and possibly jealousy with some, but I think it'd be a stretch to try and make someone feel guilty about spending their hard-earned getting away from the stresses of day-to-day life.
I've babbled on quite a bit, so will stop now. There isn't a right or wrong answer here. Ultimately, only you can decide if you feel comfortable enough to wear the Explorer to work.
If not, you have some lovely alternatives that are sneaking under the radar.