If I'm not saying stupid things you have to be an administrator and on the local view make a right click Properties and in the "deleted articles" tab you can delete completely.
if the directories are grayed out on the clents workstations, it means that these directories no longer exist on the EPDM server and that the client workstation has not cleared its local cache since the deletion of this directory.
So from the workstation or client workstations you can delete the directories without any risk.
Normally, the @flegendre solution meets your needs.
On the other hand, be careful, I don't know if it happens to you, but sometimes I have files that are grayed out, but not of my own volition. These are files that have not gone through the EPDM, and on which you have to right-click "Add to Vault". If this is your case, you will lose those files too.
a greyed out directory or file can also be a file present in EPDM (and therefore stored in the local view) to which the user connected to EPDM at the moment does not have access (in the case of a workstation shared between several users, some of whom do not have access to all the contents of the Vault).
I just had this case during my EPDM setup: I had imported files into the local view with the admin user. The next day I log in as a normal user and everything appeared grayed out. A quick look at the groups settings box allowed me to see that normal users had no rights on the vault (so it didn't have access to the files entered by the admin the day before in the local view).