My obsessive compulsive nature of having everything neat and tidy finally got the better of me and prompt me to delete some tags from a remote Git repository. Strangely, SourceTree and TortoiseGit do not provide an option for this in the GUI.
Thankfully, a quick trip to the command line helped solve the problem and it goes like this:
git tag -d name_of_tag
git push origin :refs/tags/name_of_tag
This should explicitly delete the tag name_of_tag on the remote repository.