thumbs.db being accessed by Windows Explorer - folder cannot be deleted

I'm able to delete the folder by simply clicking on a different folder, then deleting the desired folder by right-click > Delete.