It involves two steps.
First: In case you are already subscribed to push notification for your store, you would need to first unsubscribe yourself. Here's the instruction to: Unsubscribe from Push Notifications
The second step is to clear your browser cache.
An example step by step instruction to clear cache on Chrome is:
Please follow instructions to clear site data for respective browsers accordingly.