Gear and trophy swapping is a nightmare in this game and this might fix at least the crafting and refining part of it.
As title says, keep a slot for each armor piece and trophy inside each respective station so you always have them active when using that table.
There 2 ways this could work:
Make them global: you slot them in one and can see and use its benefits regardless of the town you are in. (I like this idea more)
Make them locked to each town: They will only work on the town where you slotted them and if you want to craft or refine on another town you would need to manually move them. (Would make things a little less annoying still)