Although I have disabled the loopback check on other SharePoint farms, in this case I forgot to do it.
The problem surfaced in the User Profile Application that worked perfectly fine until I added a picture to my profile and tried to perform a sync.
The FIM logs were showing stopped-extension-dll-exception on the MOSS_DELTAIMPORT
If I removed the picture and performed a re-sync, the sync worked successfully.
Checking the Event Logs I had the Event 6801, FIMSynchronizationService with a 401 Unauthorized error:
"System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request) at System.Net.WebClient.DownloadData(Uri address) at Microsoft.Office.Server.UserProfiles.ManagementAgent.ProfileImportExportExtension.Download Pictures(ProfileChangeData[] profiles) at Microsoft.Office.Server.UserProfiles.ManagementAgent.ProfileImportExportExtension.Microso ft.MetadirectoryServices.IMAExtensibleFileImport.GenerateImportFile(String fileName, String conn ectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullIm port, TypeDescriptionCollection types, String& customData) Forefront Identity Manager 4.0.2450.51"
Most blog posts I found regarding a UPA picture error with stopped-extension-dll-exception in FIM had to do with some incorrect PictureURL, mine was fine, and I could browse to it (from a client machine)
If I had tried to browse to it from the server I would have also got the Access Denied with 3 credential prompts.
To fix, disable the Windows Server Loopback check on your Site Collections.
You should be able to confirm this worked by browsing directly to MySite from the server, if no credential prompts are requested you should be fine.
Do the same to any other SharePoint servers in the Farm.