You can remove SafeHouse volume passwords from your smartcard at any time using the dialog shown below.
Click on the entry to remove and then click the Delete button.
See How to Manage Stored Passwords.