safeguarded dynamic label regression for noisy supervision · safeguarded dynamic label regression...
TRANSCRIPT
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Safeguarded Dynamic Label Regression for
Noisy Supervision
Jiangchao Yao†,‡, Hao Wu†, Ya Zhang†
Ivor W. Tsang‡, Jun Sun††Shanghai Jiao Tong University‡University of Technology Sydney
November 14, 2018
Safeguarded Dynamic Label Regression for Noisy Supervision 1/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Outline
1 Background
2 Literature Review
3 Latent Class-Conditional Noise Model
4 Experiments
5 Conclusion
Safeguarded Dynamic Label Regression for Noisy Supervision 2/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Background
Low Expensive Noisy Data Meets Deep Learning
Inexhaustible social images with annotations on websites.
Fine-grained annotations from crowdsourcing platforms.
Rich medical diagnosis by numerous levels of doctors.
Large amount of unreliable stock labels for revenue.
Learning with Noisy Supervision Brings Robustness
Existing deep learning based methods,
Learning with Noise Transition
Learning with Sample Re-weighting
Learning with Model Regularization
Safeguarded Dynamic Label Regression for Noisy Supervision 3/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Outline
1 Background
2 Literature Review
3 Latent Class-Conditional Noise Model
4 Experiments
5 Conclusion
Safeguarded Dynamic Label Regression for Noisy Supervision 4/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Probabilistic ModelingLearning with Noise Transition
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
z<latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit>
Learning with Noise Transition
lnP(y |x) = ln∑
z
P(y |z , x)︸ ︷︷ ︸Noise transition
P(z |x)︸ ︷︷ ︸Classifier
(1)
Classification Risk:
Ex ,z [− lnP(z |x)] 6= Ex ,y [− lnP(y |x)] if z 6≡ y .
Safeguarded Dynamic Label Regression for Noisy Supervision 5/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Probabilistic ModelingLearning with Noise Transition
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
z<latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit>
Learning with Noise Transition
If it is given the class-conditional noise, i.e.,
P(y |z , x) = P(y |z) = Tzy (given and invertible),
we have the following theorem for parameter estimation,
arg minθ
Ex ,z [− lnP(z |x)] = arg minθ
Ex ,y [− lnP(y |x)] .
Safeguarded Dynamic Label Regression for Noisy Supervision 6/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Probabilistic ModelingLearning with Sample Re-weighting
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
z<latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit>
Learning with Sample Re-weighting
Ex ,z
− lnP(z |x)︸ ︷︷ ︸classifier
= Ex ,y
− Pc(x , z)
Pn(x , y)
∣∣∣∣z=y︸ ︷︷ ︸
weight
lnP(y |x)︸ ︷︷ ︸classifier
= Ex ,y
− β(x , y)︸ ︷︷ ︸weight
lnP(y |x)︸ ︷︷ ︸classifier
(2)
Safeguarded Dynamic Label Regression for Noisy Supervision 7/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Probabilistic ModelingLearning with Sample Re-weighting
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
Learning with Sample Re-weighting
Typically, the weight can be estimated with small clean data,
β(x , y) =Pc(x , y)
Pn(x , y)=
Pc(y |x)
Pn(y |x).
A simpler way is based on the classifier itself motivated by theperceptual consistency, i.e., bootstrapping,
β(x , y) = α + (1− α)P(y |x)
Pn(y |x), where α ∈ [0, 1].
Safeguarded Dynamic Label Regression for Noisy Supervision 8/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Probabilistic ModelingLearning with Model Regularization
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
z<latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit>
s<latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit><latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit><latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit><latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit>
Learning with Model Regularization
lnP(y |x) = ln∑
z
P(y |z , s)︸ ︷︷ ︸Noise transition
P(z |x)︸ ︷︷ ︸Classifier
(3)
Explicitly introduce a noise source to apportion the reasoningfrom z to y , which alleviates the uncertain effect on classifier
Safeguarded Dynamic Label Regression for Noisy Supervision 9/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Probabilistic ModelingLearning with Model Regularization
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
s<latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit><latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit><latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit><latexit sha1_base64="N1eK0lTaQQAFA6yHzcECkl4oWJk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipqQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/epYz3</latexit>
Learning with Model Regularization
Roughly treat z=y and the following conjecture is quite useful,i.e., deep neural networks memorize physic patterns ♠ in order.
Conjecture: simple ♠simple (x ,y)
∣∣ hard(x ,y)−−−−−−−−−−−−−−→
memorizing orderhard ♠
Since most simple clean data belongs to simple (x , y), we candetain memorizing in the early phase by dropout regularization.
Safeguarded Dynamic Label Regression for Noisy Supervision 10/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Outline
1 Background
2 Literature Review
3 Latent Class-Conditional Noise Model
4 Experiments
5 Conclusion
Safeguarded Dynamic Label Regression for Noisy Supervision 11/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Motivation
In this work, we focus on learning with noise transition.
noisy labelsimages classifier network
Wolf
Car
Mouse
Roseestimated transition
forward correction
backward correction
Backward correction
min T−1︸ ︷︷ ︸fixed
∗` (y, P(y|x))
Forward correction
min− ln
T︸︷︷︸fixed
∗P(y|x)
Noise adaptation
min− ln
φ︸︷︷︸tunable
∗P(y|x)
noisy labelsimages classifier network Noise modeling
Wolf
Car
Mouse
Rose
forward correction
Safeguarded Dynamic Label Regression for Noisy Supervision 12/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Motivation
Problems
Forward and backward corrections critically depends onthe accurate estimation of noise transition, which isimpractical via a finite anchor set in real-world scenarios.
Stochastic approximation to EM learning between theclassifier and the noise model via a noise adaptation layeris not rigorous and suffers from instability in tuning.
Solutions
Learning on posterior labels along with learning the noisetransition is more reliable than that on noisy labels.
Gibbs sampling for Bayesian class-conditional noise modelcan reduce computational costs and avoid tweaking issues.
Safeguarded Dynamic Label Regression for Noisy Supervision 13/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Latent Class-Conditional Noise Model
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
z<latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit>
Learning on Posterior Labels
zm ∼ P(zm|xm, ym)︸ ︷︷ ︸Posterior
∝ P(zm|xm)︸ ︷︷ ︸Classifier
P(ym|xm, zm)︸ ︷︷ ︸Noise transition
(4)
Explicitly Decoupled Minimization:
minEx ,z [− lnP(z |x)] and minEx ,z,y [− lnP(y |x , z)] (5)
Classification Risk:
Ex ,z [− lnP(z |x)]← Ex ,z [− lnP(z |x)] if P(x , z)← P(x , z)
Safeguarded Dynamic Label Regression for Noisy Supervision 14/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Latent Class-Conditional Noise ModelDynamic Label Regression
x<latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit><latexit sha1_base64="f2yzimwbR/Dgjzp6tZ360fHRqNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N2IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB5jmM/A==</latexit>
y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>
z<latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit><latexit sha1_base64="HDzXchlsPlmuEyZZ/9zFJ+iVC6I=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae0oWy2k3btZhN2N0IN/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3H1FpHst7M0nQj+hQ8pAzaqzUeOqXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHpwX5935WLQWnHzmGP7A+fwB6UGM/g==</latexit>
N<latexit sha1_base64="FsFohXQjc3OA/BBAi7JXP9TMCN0=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRiydpwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8fppGM0g==</latexit><latexit sha1_base64="FsFohXQjc3OA/BBAi7JXP9TMCN0=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRiydpwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8fppGM0g==</latexit><latexit sha1_base64="FsFohXQjc3OA/BBAi7JXP9TMCN0=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRiydpwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8fppGM0g==</latexit><latexit sha1_base64="FsFohXQjc3OA/BBAi7JXP9TMCN0=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRiydpwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8fppGM0g==</latexit>
�<latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AAAB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8mm2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzzqmiTTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX355lXQvmr7X9O8vG62bMo4qnMApnIMPV9CCO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIjj8=</latexit><latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AAAB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8mm2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzzqmiTTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX355lXQvmr7X9O8vG62bMo4qnMApnIMPV9CCO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIjj8=</latexit><latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AAAB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8mm2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzzqmiTTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX355lXQvmr7X9O8vG62bMo4qnMApnIMPV9CCO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIjj8=</latexit><latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AAAB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8mm2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzzqmiTTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX355lXQvmr7X9O8vG62bMo4qnMApnIMPV9CCO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIjj8=</latexit>
↵<latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit><latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit><latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit><latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit>
K<latexit sha1_base64="NrnJFnQip0mnCQuMQd53f76pYSs=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi+ClBfsBbSib7aRdu9mE3Y1QQn+BFw+KePUnefPfuG1z0NYHA4/3ZpiZFySCa+O6305hbX1jc6u4XdrZ3ds/KB8etXScKoZNFotYdQKqUXCJTcONwE6ikEaBwHYwvp357SdUmsfywUwS9CM6lDzkjBorNe775Ypbdecgq8TLSQVy1Pvlr94gZmmE0jBBte56bmL8jCrDmcBpqZdqTCgb0yF2LZU0Qu1n80On5MwqAxLGypY0ZK7+nshopPUkCmxnRM1IL3sz8T+vm5rw2s+4TFKDki0WhakgJiazr8mAK2RGTCyhTHF7K2EjqigzNpuSDcFbfnmVtC6qnlv1GpeV2k0eRxFO4BTOwYMrqMEd1KEJDBCe4RXenEfnxXl3PhatBSefOYY/cD5/AKIFjM8=</latexit><latexit sha1_base64="NrnJFnQip0mnCQuMQd53f76pYSs=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi+ClBfsBbSib7aRdu9mE3Y1QQn+BFw+KePUnefPfuG1z0NYHA4/3ZpiZFySCa+O6305hbX1jc6u4XdrZ3ds/KB8etXScKoZNFotYdQKqUXCJTcONwE6ikEaBwHYwvp357SdUmsfywUwS9CM6lDzkjBorNe775Ypbdecgq8TLSQVy1Pvlr94gZmmE0jBBte56bmL8jCrDmcBpqZdqTCgb0yF2LZU0Qu1n80On5MwqAxLGypY0ZK7+nshopPUkCmxnRM1IL3sz8T+vm5rw2s+4TFKDki0WhakgJiazr8mAK2RGTCyhTHF7K2EjqigzNpuSDcFbfnmVtC6qnlv1GpeV2k0eRxFO4BTOwYMrqMEd1KEJDBCe4RXenEfnxXl3PhatBSefOYY/cD5/AKIFjM8=</latexit><latexit sha1_base64="NrnJFnQip0mnCQuMQd53f76pYSs=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi+ClBfsBbSib7aRdu9mE3Y1QQn+BFw+KePUnefPfuG1z0NYHA4/3ZpiZFySCa+O6305hbX1jc6u4XdrZ3ds/KB8etXScKoZNFotYdQKqUXCJTcONwE6ikEaBwHYwvp357SdUmsfywUwS9CM6lDzkjBorNe775Ypbdecgq8TLSQVy1Pvlr94gZmmE0jBBte56bmL8jCrDmcBpqZdqTCgb0yF2LZU0Qu1n80On5MwqAxLGypY0ZK7+nshopPUkCmxnRM1IL3sz8T+vm5rw2s+4TFKDki0WhakgJiazr8mAK2RGTCyhTHF7K2EjqigzNpuSDcFbfnmVtC6qnlv1GpeV2k0eRxFO4BTOwYMrqMEd1KEJDBCe4RXenEfnxXl3PhatBSefOYY/cD5/AKIFjM8=</latexit><latexit sha1_base64="NrnJFnQip0mnCQuMQd53f76pYSs=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi+ClBfsBbSib7aRdu9mE3Y1QQn+BFw+KePUnefPfuG1z0NYHA4/3ZpiZFySCa+O6305hbX1jc6u4XdrZ3ds/KB8etXScKoZNFotYdQKqUXCJTcONwE6ikEaBwHYwvp357SdUmsfywUwS9CM6lDzkjBorNe775Ypbdecgq8TLSQVy1Pvlr94gZmmE0jBBte56bmL8jCrDmcBpqZdqTCgb0yF2LZU0Qu1n80On5MwqAxLGypY0ZK7+nshopPUkCmxnRM1IL3sz8T+vm5rw2s+4TFKDki0WhakgJiazr8mAK2RGTCyhTHF7K2EjqigzNpuSDcFbfnmVtC6qnlv1GpeV2k0eRxFO4BTOwYMrqMEd1KEJDBCe4RXenEfnxXl3PhatBSefOYY/cD5/AKIFjM8=</latexit>
Dynamic Label Regression
If it is the latent class-conditional noise (LCCN), i.e.,
P(y |z , x) = P(y |z) = φzy ,
the noise model can degrade to a nonparametric counting nzy .We then simplify the computation with the Gibbs sampling,
zm|Z¬(m) ∼ P(zm|xm)︸ ︷︷ ︸Classifier
αym + n¬zmym∑k ′(αk ′) + n¬zmk ′︸ ︷︷ ︸Noise transition
. (6)
Safeguarded Dynamic Label Regression for Noisy Supervision 15/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Latent Class-Conditional Noise ModelSafeguarded Transition Update
Theorem
Suppose αi is a positive smoothing scalar, Ni is the current samplenumber of the ith category (i=1,. . . ,K ), Mi is the sum of thesample numbers newly allocated into (positive) and removed from(negative) the ith category after a batch of training samples, andMi is its absolute sum of such two cases. Then, for the transitionvector φi of the ith category, its variation via a training batch ischaracterized by the following equation,
∣∣φnewi − φold
i
∣∣ ≤ |ri |+ ri1 + ri
where ri = Mi
Ni +∑K
j=1 αjand ri = Mi
Ni +∑K
j=1 αj. According to the
definition, we have ri > −1, ri ≥ 0 and ri ≥ |ri |.
Safeguarded Dynamic Label Regression for Noisy Supervision 16/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Latent Class-Conditional Noise ModelSafeguarded Dynamic Label Regression
Algorithm 1 Dynamic Label Regression for LCCN
Require: A noisy dataset D = {(xn, yn)}Nn=1, a classifier P(·|x) modeled by DNN fθ,
warming-up steps δ, the running epoch number L and the batch-size M.1: Directly pretrain the classifier fθ on the noisy dataset D.2: Compute the warming-up noise transition matrix φ′.3: for epoch i = 1 to L do4: for batch j = 1 to dN/Me do5: Let step=i×dN/Me+j and hook a batch of samples.6: if step < δ then7: Substitute the transition in Equation (6) with φ′, and sample zn.8: else9: Sample zn with Equation (6) for the batch.
10: end if11: Update the confusion matrix N(·)(·) based on observations {(zn, yn)}.12: Optimize Equation (5) to learn fθ and estimate φ.13: end for14: end for15: Output the classifier fθ and the noise transition φ.
Safeguarded Dynamic Label Regression for Noisy Supervision 17/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Latent Class-Conditional Noise Model
Complexity Analysis
Stochastic training a DNN model involves two steps, the forwardand backward computations. In each mini-batch update, its timecomplexity is O(MΛ), where M is the mini-batch size and Λ is theparameter size. Here, in Algorithm 1, we additionally add asampling operation via Equation (6) whose complexity isO(M + K 2) (K is the class size). Note that, the first term in theRHS of Equation (6) has been computed in the forward procedure.So the extra cost for the sampling is negligible compared toO(MΛ). An optimization for noise modeling is also negligible,which involves the normalization of a confusion matrix only andthe complexity is O(K 2). Since the big-O complexity of eachmini-batch remains the same, our method is scalable to big data.
Safeguarded Dynamic Label Regression for Noisy Supervision 18/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Outline
1 Background
2 Literature Review
3 Latent Class-Conditional Noise Model
4 Experiments
5 Conclusion
Safeguarded Dynamic Label Regression for Noisy Supervision 19/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Experiments
Asymmetric Noise & Wild Noise
(1) Inject asymmetric noise manually
CIFAR-10: trkr−→ atm, brd
r−→ apl , deerr−→ horse, cat
r−→ dog
CIFAR-100: oner−→ the next circularly within the superclass
(2) Totally agnostic wild noiseClothing1M: 14 predefined clothing classes.WebVision17: 1000 classes same to those of ImageNet.
Safeguarded Dynamic Label Regression for Noisy Supervision 20/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Experiments
Experimental Setup
CIFAR-10 and CIFAR-100:
Resnet-32 and data augmentation
mo=0.9, weight-decay=1e-4, batch-size=128
40 (lr=0.5), 80 (lr=0.1), 120 (lr=0.01) epochs
Clothing1M and WebVision17:
Resnet-50 (ImageNet pretrained) and data augmentation
lr=0.01, mo=0.9, weight-decay=1e-3, batch-size=32
10 epochs (lr is divided by 10 after each 5 epochs)
Baselines:CE, Forward, S-adaptation, Bootstrapping, Joint Optimization.
Safeguarded Dynamic Label Regression for Noisy Supervision 21/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
ExperimentsCIFAR-10 and CIFAR-100
Dataset CIFAR-10
# Method \ Noise Ratio 0.1 0.3 0.5 0.7 0.91 CE 90.10 88.12 76.93 59.01 56.852 Bootstrapping 90.73 88.12 76.29 57.04 56.793 Forward 90.86 89.03 82.47 67.11 57.294 S-adaptation 91.02 88.83 86.79 72.74 60.925 LCCN 91.35 89.33 88.41 79.48 64.82
6 CE with the clean data 91.63
Table: The average accuracy(%) over 5 trials on noisy CIFAR-10.
Safeguarded Dynamic Label Regression for Noisy Supervision 22/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
ExperimentsCIFAR-10 and CIFAR-100
Dataset CIFAR-100
# Method \ Noise Ratio 0.1 0.2 0.3 0.4 0.51 CE 66.15 64.31 60.11 51.68 33.372 Bootstrapping 66.48 64.61 63.01 55.27 34.523 Forward 65.43 62.72 61.28 52.64 33.824 S-adaptation 65.52 64.11 62.39 52.74 30.075 LCCN 67.83 67.63 66.86 65.52 33.71
6 CE with the clean data 69.41
Table: The average accuracy(%) over 5 trials on noisy CIFAR-100.
Safeguarded Dynamic Label Regression for Noisy Supervision 23/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
ExperimentsCIFAR-10 and CIFAR-100
0
0.5
1
1.5
2
2.5
0 10000 20000 30000 40000 50000
BCCN
S-adaption
0.55
0.65
0.75
0.85
0.95
0 10000 20000 30000 40000 50000
LCCN
S-adaption
LCCN
S-adaption
Figure: Test accuracy of LCCN andS-adaptation on CIFAR-10 with r=0.5 (left),and the corresponding histogram (right) forthe variation of φ via a batch of samples.
0.5
0.6
0.7
0.8
0.9
1
0 10000 20000 30000 40000 50000
labe
l cor
rect
ion
ratio
horse car
car car
truck truck
car ship
plane car
truck Frog
Figure: The colormap for the confusion matrix on CIFAR-10 with r=0.5. We use the log-scale of eachentry in the confusion matrix for fine-grained visualization. The left three maps are gradually learned byLCCN and the right one is the groudtruth.
Safeguarded Dynamic Label Regression for Noisy Supervision 24/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Experiments
# Method Accuracy
1 CE 68.942 Bootstrapping 69.123 Forward 69.844 S-adaptation 70.365 Joint Optimization 72.236 LCCN 73.07
7 CE with the clean data 75.28
# Method Accuracy@1 Accuracy@5
1 CE 63.11 83.692 Bootstrapping 63.20 83.813 Forward 63.10 83.784 S-adaptation 62.54 81.735 LCCN 63.52 84.27
Table: Results on Clothing1M (top) and WebVision (bottom).
Safeguarded Dynamic Label Regression for Noisy Supervision 25/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Outline
1 Background
2 Literature Review
3 Latent Class-Conditional Noise Model
4 Experiments
5 Conclusion
Safeguarded Dynamic Label Regression for Noisy Supervision 26/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Conclusion
Summary
We present a Latent Class-Conditional Noise model tosolve the issues when training the classifier and modelingthe noise transition together in previous models .
A dynamic label regression method is deduced for LCCN.Theoretical analysis guarantees the safeguarded transitionupdate and introduces the negligible computational cost.
Experiments on CIFAR-10, CIFAR-100 datasets and thereal-world noisy datasets, confirm the superiority of ourmodel compared with some state-of-the-art methods.
More future works based on LCCN can be extended todeal with general multiple sources of noise in practise.
Safeguarded Dynamic Label Regression for Noisy Supervision 27/28
Background Literature Review Latent Class-Conditional Noise Model Experiments Conclusion
Q&AThank you!
Safeguarded Dynamic Label Regression for Noisy Supervision 28/28