Nice plan. I’m not sure if the bolt will end up in the same place when you tighten it again though. Not that it matters the jam bolt will still hold it.
I think if it was me I’d just clean it out real good with brake cleaner to be rid of all the oil. Then I’d put that cooler bolt in with Loctite 640 retaining compound (green shit for bearing races and sleeves). That bolt won’t come out then.