It involves two steps.
First: In case you are already subscribed to push notification for your store, you would need to first reset notification permission by clicking on lock icon.
Set the Notification Permission to "Ask (default)"
The second step is to clear your browser cache.
Right click anywhere on the page > Inspect > Application to see the following option:
An example screenshot to clear cache on Chrome is:
Please follow instructions to clear site data for respective browsers accordingly.