If you think that by removing remote batteries, your remote will get unpaired, then it is just a misinterpretation. To unpair remote, there is some procedure which you have to follow. you need to unpair remote in that case when your amazon fire remote stopped working.
Follow up these simple procedure to unpair the remote;

* Press the button Fast Forward, Mute, and Zero simultaneously
* Hold all these keys for 3 to 4 seconds.
* Then Your remote gets unpaired.
If still, your queries remain left then dial Amazon Fire Stick Customer Service Number & get the answer from experts of your queries.