Since you are the one merging in I would say they have the right of way… to a point.
First: You want to merge in as soon as possible and not ride the lane out to the end. In my perspective that looks like either someone is trying to be a fuckknob and sneak in at the last minute or just plain does not know how to drive. Either way I would be less inclined to let them in for being stupid on either count.
Second: If someone is going a reasonable speed and close to yours then you should slow down a BIT and tuck in behind them.
Third: If that person OBVIOUSLY speeds up, early on, and you are side by side while you are speeding up to merge in then that person is being a dickhead and you should cut them off
Fourth: You cant expect eveyone else to be looking out for you. Reality is that most drivers on the road suck. Be more aggressive and you will have less aggravation to deal with because you are in control.