The icon is only stuck on the workbench if a lock remains on it.
Apparently Workbench puts a lock on a directory when you open a window for that directory on Workbench. Making a directory the 'current directory' in a shell also locks the directory. I retried my simple procedure of inserting a USB stick, opening it in a Workbench window, removing the USB stick and then closing the window. The USB stick icon remained on my Workbench for over an hour. However, the USB stick I tested is formatted with SFS2. When I tried with a FAT formatted USB stick, the icon disappears after a few seconds. If I select the "Window/Show/All Files" for the FAT USB stick window. I get an "Insert volume" requester for the stuck SFS2 USB flash icon for each file icon that gets displayed. I also noticed that the USB MassStorages window is showing a 'W' (Write) when the USB stick is being read.
EDIT: I tested with an FFS formatted USB stick and it works normally like a FAT formatted USB stick. It looks like the stranded icon issue is an SFS2 issue and not a USB issue. It still should be fixed.