cs11-747 neural networks for nlp unsupervised and semi ... · learning features vs. learning...
TRANSCRIPT
![Page 1: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/1.jpg)
CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised
Learning of StructureGraham Neubig
Sitehttps://phontron.com/class/nn4nlp2019/
![Page 2: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/2.jpg)
Supervised, Unsupervised, Semi-supervised
• Most models handled here are supervised learning
• Model P(Y|X), at training time given both
• Sometimes we are interested in unsupervised learning
• Model P(Y|X), at training time given only X
• Or semi-supervised learning
• Model P(Y|X), at training time given both or only X
![Page 3: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/3.jpg)
Learning Features vs. Learning Structure
![Page 4: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/4.jpg)
Learning Features vs. Learning Discrete Structure
• Learning features, e.g. word/sentence embeddings:
this is an example
• Learning discrete structure:
• Why discrete structure? • We may want to model information flow differently • More interpretable than features?
this is an example this is an examplethis is an example this is an example
![Page 5: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/5.jpg)
Unsupervised Feature Learning (Review)
• When learning embeddings, we have an objective and use the intermediate states of this objective
• CBOW
• Skip-gram
• Sentence-level auto-encoder
• Skip-thought vectors
• Variational auto-encoder
![Page 6: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/6.jpg)
How do we Use Learned Features?
• To solve tasks directly (Mikolov et al. 2013)
• And by proxy, knowledge base completion, etc., to be covered in a few classes
• To initialize downstream models
![Page 7: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/7.jpg)
What About Discrete Structure?
• We can cluster words
• We can cluster words in context (POS/NER)
• We can learn structure
![Page 8: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/8.jpg)
What is our Objective?• Basically, a generative model of the data X
• Sometimes factorized P(X|Y)P(Y), a traditional generative model
• Sometimes factorized P(X|Y)P(Y|X), an auto-encoder
• This can be made mathematically correct through variational autoencoder P(X|Y)Q(Y|X)
![Page 9: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/9.jpg)
Clustering Words in Context
![Page 10: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/10.jpg)
A Simple First Attempt• Train word embeddings
• Perform k-means clustering on them
• Implemented in word2vec (-classes option)
• But what if we want single words to appear in different classes (same surface form, different values)
![Page 11: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/11.jpg)
Hidden Markov Models• Factored model of P(X|Y)P(Y) • State→state transition probabilities • State→word emission probabilities
<s> JJ NN NN LRB NN RRB … </s>
Natural Language Processing ( NLP ) …
PE(Natural|JJ) * PE(Language|JJ) * PE(Processing|JJ) * …
PT(JJ|<s>) * PT(NN|JJ) * PT(NN|NN) * PT(NN|LRB) * …
![Page 12: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/12.jpg)
Unsupervised Hidden Markov Models
• Change label states to unlabeled numbers
0 13 17 17 6 12 6 … 0
Natural Language Processing ( NLP ) …
PE(Natural|13) * PE(Language|17) * PE(Processing|17) * …
PT(13|0) * PT(17|13) * PT(17|17) * PT(6|17) * …
• Can be trained with forward-backward algorithm
![Page 13: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/13.jpg)
Hidden Markov Models w/ Gaussian Emissions
• Instead of parameterizing each state with a categorical distribution, we can use a Gaussian (or Gaussian mixture)!
• Long the defacto-standard for speech • Applied to POS tagging by training to emit word embeddings by
Lin et al. (2015)
0 13 17 17 6 12 6 … 0
…
![Page 14: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/14.jpg)
A Simple Approximation: State Clustering (Giles et al. 1992)
• Simply train an RNN according to a standard loss function (e.g. language model)
• Then cluster the hidden states according to k-means, etc.
![Page 15: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/15.jpg)
Featurized Hidden Markov Models (Tran et al. 2016)
• Calculate the transition/emission probabilities with neural networks! • Emission: Calculate representation of each word in vocabulary w/
CNN, dot product with tag representation and softmax to calculate emission prob
• Transition Matrix: Calculate w/ LSTMs (breaks Markov assumption)
![Page 16: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/16.jpg)
Problem: Embeddings May Not be Indicative of Syntax
(He et al. 2018)
noun proper
verb base
cardinal number
verb gerund
adjective
noun singular
noun plural
adverb
verb past tenseverb past participleverb 3rd singular
![Page 17: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/17.jpg)
Learning POS Taggersw/ Latent Embeddings
(He et al. 2018)
z1<latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit>
z2<latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit>
z3<latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit>
x3<latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit>
x2<latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit>
x1<latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit>
e1<latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit>
e2<latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit>
e3<latexit sha1_base64="LgiizHRmhHOKEEbwqVR4tWBS3lw=">AAACDXicZVDLTgIxFL2DL8QXauLGzURi4oKQQRfqjujGJSQiJECwUzrQ0E4n7R0TMuEfjFv9DlfGrd/gxn/wD+wAC4GbNPf03HOb0+NHghv0vG8ns7K6tr6R3cxtbe/s7uX3Dx6MijVldaqE0k2fGCZ4yOrIUbBmpBmRvmANf3ibzhtPTBuuwnscRawjST/kAacELdVs+zJh4+5FN1/wSt6k3GVQnoFC5aj28wgA1W7+t91TNJYsRCqIMa2yF2EnIRo5FWyca8eGRYQOSZ+1LAyJZKaTTPyO3VPL9NxAaXtCdCfs/42ESCMJDqwybWZuljKolDBFq8KBTFv6zORuRtIv+rKYirQJzIIRDK46CQ+jGFlIpz6CWLio3DQbt8c1oyhGFhCquf2KSwdEE4o2wZzNqLyYyDKon5euS17NZnUD08rCMZzAGZThEipwB1WoAwUBL/AKb86z8+58OJ9TacaZ7RzCXDlff9YqntM=</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit>
f�(e)<latexit sha1_base64="8oxgnabmEamXCM6ELWdhWTrrt8c=">AAACHHicZVA9SwNBEJ3z2/gVFSubwyhECOFio3ZBG0sFzwgmhL3NnlncvT1254Rw3G8RW+2tbazEVvBnWNi7d0lh4sAyb9+8Wd6+IBbcoOd9OVPTM7Nz8wuLpaXlldW18vrGlVGJpsynSih9HRDDBI+YjxwFu441IzIQrBXcnebz1j3ThqvoEgcx60hyG/GQU4KW6pa3wm7aDmTajvs8y6o5ZNl+t1zx6l5R7n/QGIFKc/fn5RUAzrvl73ZP0USyCKkgxtw0vBg7KdHIqWBZqZ0YFhN6R27ZjYURkcx00sJ+5u5ZpueGStsToVuwfzdSIo0k2LfKvJmxWc6gUsLUrAr7Mm/5M8XdDGRQC2QtF2kTmgkjGB51Uh7FCbKIDn2EiXBRuXlUbo9rRlEMLCBUc/sVl/aJJhRtoCWbUWMykf/AP6gf170Lm9UJDGsBtmEHqtCAQ2jCGZyDDxRSeIQneHYenDfn3fkYSqec0c4mjJXz+QtiiKWs</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit>
ei ⇠ N (µzi ,⌃zi)<latexit sha1_base64="8CLMr+o0dZvNiJKf88rFwAP//jQ=">AAACQ3icZVBNa9tAEB0laZO4H3GaYy9LQiGlxki5JIUcQnvJKaS0bgyWEKv1yl6yqxW7o4Ir9F9Kfkzptb31P+TS0Guhu7IPiT2wzJs3b5aZl5VSWAzD38Ha+sajx5tb250nT5893+nuvvhsdWUYHzAttRlm1HIpCj5AgZIPS8OpyiS/yq7f+/7VF26s0MUnnJU8UXRSiFwwio5Ku6dxpmrepILEVigSK4pTRmV90Rz6TqyqJq2/pqLpkbb+KCaKLqjXafcg7IdtkFUQLcDB2X785hsAXKbdP/FYs0rxApmk1o6isMSkpgYFk7zpxJXlJWXXdMJHDhZUcZvU7ZUNeeWYMcm1ca9A0rL3J2qqrF/fKX2yD3qeQa2l7TkVTpVP/pu2tjOV9TLV8yJjc7u0COYnSS2KskJesPkeeSUJauIdJWNhOEM5c4AyI9wphE2poQyd7x3nUbTsyCoYHPXf9sMPzqt3MI8teAn7cAgRHMMZnMMlDIDBDfyAn/Ar+B7cBnfB37l0LVjM7MGDCP79BwcMtOc=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit>
zi ⇠<latexit sha1_base64="A0vAn9BPw9kDXHS9rkuB77rpTAE=">AAACDnicZVDLTgIxFO3gC/GFunTTSExcEDJjTNQd0Y1LTEQwMCGd0oGGdjpp75jghI8wbvU7XBm3/oKf4R/YgVkI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKWtSJZRuB8QwwSPWBA6CtWPNiAwEawWjm2zeemLacBXdwzhmviSDiIecErDU43OP467hEvfKFbfmTgsvAy8HFZRXo1f+7fYVTSSLgApiTMdzY/BTooFTwSalbmJYTOiIDFjHwohIZvx0aniCTyzTx6HS9kSAp+z/jZRIIwkMrTJrZm6WMaCUMFWrgqHMWvbM9G7GMqgGspqJtAnNghEIL/2UR3ECLKIzH2EiMCichYP7XDMKYmwBoZrbr2A6JJpQsBGWbEbeYiLLoHlWu6q5d+eV+nUeVhEdoWN0ijx0geroFjVQE1Ek0St6Q+/Oi/PhfDpfM2nByXcO0Vw5339MaJ0J</latexit><latexit sha1_base64="A0vAn9BPw9kDXHS9rkuB77rpTAE=">AAACDnicZVDLTgIxFO3gC/GFunTTSExcEDJjTNQd0Y1LTEQwMCGd0oGGdjpp75jghI8wbvU7XBm3/oKf4R/YgVkI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKWtSJZRuB8QwwSPWBA6CtWPNiAwEawWjm2zeemLacBXdwzhmviSDiIecErDU43OP467hEvfKFbfmTgsvAy8HFZRXo1f+7fYVTSSLgApiTMdzY/BTooFTwSalbmJYTOiIDFjHwohIZvx0aniCTyzTx6HS9kSAp+z/jZRIIwkMrTJrZm6WMaCUMFWrgqHMWvbM9G7GMqgGspqJtAnNghEIL/2UR3ECLKIzH2EiMCichYP7XDMKYmwBoZrbr2A6JJpQsBGWbEbeYiLLoHlWu6q5d+eV+nUeVhEdoWN0ijx0geroFjVQE1Ek0St6Q+/Oi/PhfDpfM2nByXcO0Vw5339MaJ0J</latexit><latexit sha1_base64="A0vAn9BPw9kDXHS9rkuB77rpTAE=">AAACDnicZVDLTgIxFO3gC/GFunTTSExcEDJjTNQd0Y1LTEQwMCGd0oGGdjpp75jghI8wbvU7XBm3/oKf4R/YgVkI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKWtSJZRuB8QwwSPWBA6CtWPNiAwEawWjm2zeemLacBXdwzhmviSDiIecErDU43OP467hEvfKFbfmTgsvAy8HFZRXo1f+7fYVTSSLgApiTMdzY/BTooFTwSalbmJYTOiIDFjHwohIZvx0aniCTyzTx6HS9kSAp+z/jZRIIwkMrTJrZm6WMaCUMFWrgqHMWvbM9G7GMqgGspqJtAnNghEIL/2UR3ECLKIzH2EiMCichYP7XDMKYmwBoZrbr2A6JJpQsBGWbEbeYiLLoHlWu6q5d+eV+nUeVhEdoWN0ijx0geroFjVQE1Ek0St6Q+/Oi/PhfDpfM2nByXcO0Vw5339MaJ0J</latexit>
Markov Structure
xi ⇠<latexit sha1_base64="4XmkaafD4EWeU6wrjePkY8qE6No=">AAACE3icZVDLTgIxFO3gC/GFunTTSExcEDIYE3VHdOMSE0dIGCSd0oGGdjpp7xjJhN8wbvU7XBm3foCf4R/YGVgI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKfOoEkq3A2KY4BHzgINg7VgzIgPBWsHoJpu3npg2XEX3MI5ZV5JBxENOCVjq0Q9k+jzpcewbLnGvXHFrbl54GdRnoIJm1eyVf/2+oolkEVBBjOnU3Ri6KdHAqWCTkp8YFhM6IgPWsTAikplumrue4BPL9HGotD0R4Jz9v5ESaSSBoVVmzczNMgaUEqZqVTCUWcueye9mLINqIKuZSJvQLBiB8LKb8ihOgEV06iNMBAaFs4Rwn2tGQYwtIFRz+xVMh0QTCjbHks2ovpjIMvDOalc19+680riehVVER+gYnaI6ukANdIuayEMUafSK3tC78+J8OJ/O11RacGY7h2iunO8/lwWfXA==</latexit><latexit sha1_base64="4XmkaafD4EWeU6wrjePkY8qE6No=">AAACE3icZVDLTgIxFO3gC/GFunTTSExcEDIYE3VHdOMSE0dIGCSd0oGGdjpp7xjJhN8wbvU7XBm3foCf4R/YGVgI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKfOoEkq3A2KY4BHzgINg7VgzIgPBWsHoJpu3npg2XEX3MI5ZV5JBxENOCVjq0Q9k+jzpcewbLnGvXHFrbl54GdRnoIJm1eyVf/2+oolkEVBBjOnU3Ri6KdHAqWCTkp8YFhM6IgPWsTAikplumrue4BPL9HGotD0R4Jz9v5ESaSSBoVVmzczNMgaUEqZqVTCUWcueye9mLINqIKuZSJvQLBiB8LKb8ihOgEV06iNMBAaFs4Rwn2tGQYwtIFRz+xVMh0QTCjbHks2ovpjIMvDOalc19+680riehVVER+gYnaI6ukANdIuayEMUafSK3tC78+J8OJ/O11RacGY7h2iunO8/lwWfXA==</latexit><latexit sha1_base64="4XmkaafD4EWeU6wrjePkY8qE6No=">AAACE3icZVDLTgIxFO3gC/GFunTTSExcEDIYE3VHdOMSE0dIGCSd0oGGdjpp7xjJhN8wbvU7XBm3foCf4R/YGVgI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKfOoEkq3A2KY4BHzgINg7VgzIgPBWsHoJpu3npg2XEX3MI5ZV5JBxENOCVjq0Q9k+jzpcewbLnGvXHFrbl54GdRnoIJm1eyVf/2+oolkEVBBjOnU3Ri6KdHAqWCTkp8YFhM6IgPWsTAikplumrue4BPL9HGotD0R4Jz9v5ESaSSBoVVmzczNMgaUEqZqVTCUWcueye9mLINqIKuZSJvQLBiB8LKb8ihOgEV06iNMBAaFs4Rwn2tGQYwtIFRz+xVMh0QTCjbHks2ovpjIMvDOalc19+680riehVVER+gYnaI6ukANdIuayEMUafSK3tC78+J8OJ/O11RacGY7h2iunO8/lwWfXA==</latexit>
f�(ei)<latexit sha1_base64="ISFMuiBWVv5rfvJnqq9uM8Osfvo=">AAACHnicZVBNS8NAEN34WetX1YMHL8EiVCglEUG9Fb14rGBsoS1hs920S3ezYXcilJAfI171d3gSr/oz/Adu0h5sO7DM2zdvhpkXxJxpcJwfa2V1bX1js7RV3t7Z3duvHBw+aZkoQj0iuVSdAGvKWUQ9YMBpJ1YUi4DTdjC+y+vtZ6o0k9EjTGLaF3gYsZARDIbyK8ehn/YCkfbiEcuyWg5p5rNzv1J1Gk4R9jJwZ6CKZtHyK7+9gSSJoBEQjrXuuk4M/RQrYITTrNxLNI0xGeMh7RoYYUF1Py0OyOwzwwzsUCrzIrAL9n9HioUWGEZGmSc9V8sZkJLrulHBSOQpH1P89UQE9UDUc5HSoV5YBMLrfsqiOAEakekeYcJtkHZulj1gihLgEwMwUcycYpMRVpiAsbRsPHIXHVkG3kXjpuE8XFabtzOzSugEnaIactEVaqJ71EIeIihDr+gNvVsv1of1aX1NpSvWrOcIzYX1/QfnyqO9</latexit><latexit sha1_base64="ISFMuiBWVv5rfvJnqq9uM8Osfvo=">AAACHnicZVBNS8NAEN34WetX1YMHL8EiVCglEUG9Fb14rGBsoS1hs920S3ezYXcilJAfI171d3gSr/oz/Adu0h5sO7DM2zdvhpkXxJxpcJwfa2V1bX1js7RV3t7Z3duvHBw+aZkoQj0iuVSdAGvKWUQ9YMBpJ1YUi4DTdjC+y+vtZ6o0k9EjTGLaF3gYsZARDIbyK8ehn/YCkfbiEcuyWg5p5rNzv1J1Gk4R9jJwZ6CKZtHyK7+9gSSJoBEQjrXuuk4M/RQrYITTrNxLNI0xGeMh7RoYYUF1Py0OyOwzwwzsUCrzIrAL9n9HioUWGEZGmSc9V8sZkJLrulHBSOQpH1P89UQE9UDUc5HSoV5YBMLrfsqiOAEakekeYcJtkHZulj1gihLgEwMwUcycYpMRVpiAsbRsPHIXHVkG3kXjpuE8XFabtzOzSugEnaIactEVaqJ71EIeIihDr+gNvVsv1of1aX1NpSvWrOcIzYX1/QfnyqO9</latexit><latexit sha1_base64="ISFMuiBWVv5rfvJnqq9uM8Osfvo=">AAACHnicZVBNS8NAEN34WetX1YMHL8EiVCglEUG9Fb14rGBsoS1hs920S3ezYXcilJAfI171d3gSr/oz/Adu0h5sO7DM2zdvhpkXxJxpcJwfa2V1bX1js7RV3t7Z3duvHBw+aZkoQj0iuVSdAGvKWUQ9YMBpJ1YUi4DTdjC+y+vtZ6o0k9EjTGLaF3gYsZARDIbyK8ehn/YCkfbiEcuyWg5p5rNzv1J1Gk4R9jJwZ6CKZtHyK7+9gSSJoBEQjrXuuk4M/RQrYITTrNxLNI0xGeMh7RoYYUF1Py0OyOwzwwzsUCrzIrAL9n9HioUWGEZGmSc9V8sZkJLrulHBSOQpH1P89UQE9UDUc5HSoV5YBMLrfsqiOAEakekeYcJtkHZulj1gihLgEwMwUcycYpMRVpiAsbRsPHIXHVkG3kXjpuE8XFabtzOzSugEnaIactEVaqJ71EIeIihDr+gNvVsv1of1aX1NpSvWrOcIzYX1/QfnyqO9</latexit>
Point mass at
xi = f�(ei)<latexit sha1_base64="270VJLD7gt0vH7AtNf4cVJpk6VA=">AAACKXicZVC7TisxEJ3lTXgFKGksHhJIUbShgVtcKYKGEiQCSCRaeR0vsbDXK3v26kar7fkYRAmUfAMV0PIFFPTYCQWPkaw5c+aMNXPiTAqLYfgcjIyOjU9MTk1XZmbn5heqi0snVueG8RbTUpuzmFouRcpbKFDys8xwqmLJT+PLfd8//ceNFTo9xn7GO4pepCIRjKKjoupqO1bF/zIS5C9JosJX7awnynLTQ+4aW1F1LayHgyC/QeMTrDXX328fAOAwqr61u5rliqfIJLX2vBFm2CmoQcEkLyvt3PKMskt6wc8dTKnitlMMbinJhmO6JNHGvRTJgP06UVBlFcWeU/pkv/U8g1pLW3Mq7Cmf/DeD2vZVXItVzYuMTeyPRTDZ7RQizXLkKRvukeSSoCbeN9IVhjOUfQcoM8KdQliPGsrQuVtxHjV+OvIbtLbrf+rhkfNqD4YxBSuwCpvQgB1owgEcQgsYXMEN3MF9cB08Bk/By1A6EnzOLMO3CF4/AGOkqtY=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit>
NeuralProjector
![Page 18: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/18.jpg)
A Simpler Method: Map Directly to Space of Prior
Example of Markov prior
xf(e;�)
<latexit sha1_base64="FsElmiZYiYnGj5o7UHjArdk6Bk8=">AAACUXicZVBNT9tAEB2blo9AIWmPvViNkIIURU6E1KJeUHvpkUoEEDiK1psxWbEf1u4YiCz/i17pr+LCX+HEOsmBwEirefvmzertS3MpHMXxUxCuffi4vrG51dje+bS712x9PnOmsByH3EhjL1LmUAqNQxIk8SK3yFQq8Ty9+V3Pz2/ROmH0Kc1yHCl2rUUmOCNPXWadBH8m+VQcjJvtuBfPK3oP+kvQhmWdjFvBIJkYXijUxCVz7qof5zQqmSXBJVaNpHCYM37DrvHKQ80UulE5t1xF+56ZRJmx/miK5uzrjZIppxhNvbJubmVWM2SMdF2voqmqW/3M/O5mKu2mqluLrMvcGyOU/RiVQucFoeYLH1khIzJRHU80ERY5yZkHjFvhvxLxKbOMkw9xxUSqqkYj0XjHjVJMT8rkviqTVJX3VbXK44JHz/uU+28zfQ+Gg95RL/572D7+tYx7E77CN+hAH77DMfyBExgCBw3/4AH+B4/BcwhhuJCGwXLnC6xUuP0C/9e00g==</latexit><latexit sha1_base64="FsElmiZYiYnGj5o7UHjArdk6Bk8=">AAACUXicZVBNT9tAEB2blo9AIWmPvViNkIIURU6E1KJeUHvpkUoEEDiK1psxWbEf1u4YiCz/i17pr+LCX+HEOsmBwEirefvmzertS3MpHMXxUxCuffi4vrG51dje+bS712x9PnOmsByH3EhjL1LmUAqNQxIk8SK3yFQq8Ty9+V3Pz2/ROmH0Kc1yHCl2rUUmOCNPXWadBH8m+VQcjJvtuBfPK3oP+kvQhmWdjFvBIJkYXijUxCVz7qof5zQqmSXBJVaNpHCYM37DrvHKQ80UulE5t1xF+56ZRJmx/miK5uzrjZIppxhNvbJubmVWM2SMdF2voqmqW/3M/O5mKu2mqluLrMvcGyOU/RiVQucFoeYLH1khIzJRHU80ERY5yZkHjFvhvxLxKbOMkw9xxUSqqkYj0XjHjVJMT8rkviqTVJX3VbXK44JHz/uU+28zfQ+Gg95RL/572D7+tYx7E77CN+hAH77DMfyBExgCBw3/4AH+B4/BcwhhuJCGwXLnC6xUuP0C/9e00g==</latexit><latexit sha1_base64="FsElmiZYiYnGj5o7UHjArdk6Bk8=">AAACUXicZVBNT9tAEB2blo9AIWmPvViNkIIURU6E1KJeUHvpkUoEEDiK1psxWbEf1u4YiCz/i17pr+LCX+HEOsmBwEirefvmzertS3MpHMXxUxCuffi4vrG51dje+bS712x9PnOmsByH3EhjL1LmUAqNQxIk8SK3yFQq8Ty9+V3Pz2/ROmH0Kc1yHCl2rUUmOCNPXWadBH8m+VQcjJvtuBfPK3oP+kvQhmWdjFvBIJkYXijUxCVz7qof5zQqmSXBJVaNpHCYM37DrvHKQ80UulE5t1xF+56ZRJmx/miK5uzrjZIppxhNvbJubmVWM2SMdF2voqmqW/3M/O5mKu2mqluLrMvcGyOU/RiVQucFoeYLH1khIzJRHU80ERY5yZkHjFvhvxLxKbOMkw9xxUSqqkYj0XjHjVJMT8rkviqTVJX3VbXK44JHz/uU+28zfQ+Gg95RL/572D7+tYx7E77CN+hAH77DMfyBExgCBw3/4AH+B4/BcwhhuJCGwXLnC6xUuP0C/9e00g==</latexit>
?
max log pGHMM(f�(x))<latexit sha1_base64="QrUGXd8IoThzpkp7A2ek+zngVOU=">AAACcnicZVFLaxsxEJa3r9R9OekxF7WmYIMx61BoegvpobkEUqibQGSMVp61RfRYpNl0jdjf0F/Ta/s78kNyr9bZQ50MiPnmmwczn7JCSY9petNJHj1+8vTZzvPui5evXr/p7e798LZ0AqbCKusuMu5BSQNTlKjgonDAdabgPLv60uTPr8F5ac13XBcw03xpZC4Fx0jNe0OmeUUZZcouaTEPDKHC8PXk9LSuB3mMi5WsB6waDue9fjpON0YfgkkL+qS1s/lu54AtrCg1GBSKe385SQucBe5QCgV1l5UeCi6u+BIuIzRcg5+FzU01/RCZBc2ti88g3bD/dwSuvea4ipWN81u5hkFrlR/FKlzpxjVjNrFf62yU6VFT5Hzu7y2C+eEsSFOUCEbc7ZGXiqKljX50IR0IVOsIuHAynkLFijsuMKq8tUSm626XGfgprNbcLAKr6sAyHaq63ubhjofIR5Un9zV9CKYH48/j9NvH/tFxK/cO2SfvyYBMyCdyRE7IGZkSQX6R3+QP+du5TfaTd0n7N0mn7XlLtiwZ/QPP2MC/</latexit><latexit sha1_base64="QrUGXd8IoThzpkp7A2ek+zngVOU=">AAACcnicZVFLaxsxEJa3r9R9OekxF7WmYIMx61BoegvpobkEUqibQGSMVp61RfRYpNl0jdjf0F/Ta/s78kNyr9bZQ50MiPnmmwczn7JCSY9petNJHj1+8vTZzvPui5evXr/p7e798LZ0AqbCKusuMu5BSQNTlKjgonDAdabgPLv60uTPr8F5ac13XBcw03xpZC4Fx0jNe0OmeUUZZcouaTEPDKHC8PXk9LSuB3mMi5WsB6waDue9fjpON0YfgkkL+qS1s/lu54AtrCg1GBSKe385SQucBe5QCgV1l5UeCi6u+BIuIzRcg5+FzU01/RCZBc2ti88g3bD/dwSuvea4ipWN81u5hkFrlR/FKlzpxjVjNrFf62yU6VFT5Hzu7y2C+eEsSFOUCEbc7ZGXiqKljX50IR0IVOsIuHAynkLFijsuMKq8tUSm626XGfgprNbcLAKr6sAyHaq63ubhjofIR5Un9zV9CKYH48/j9NvH/tFxK/cO2SfvyYBMyCdyRE7IGZkSQX6R3+QP+du5TfaTd0n7N0mn7XlLtiwZ/QPP2MC/</latexit><latexit sha1_base64="QrUGXd8IoThzpkp7A2ek+zngVOU=">AAACcnicZVFLaxsxEJa3r9R9OekxF7WmYIMx61BoegvpobkEUqibQGSMVp61RfRYpNl0jdjf0F/Ta/s78kNyr9bZQ50MiPnmmwczn7JCSY9petNJHj1+8vTZzvPui5evXr/p7e798LZ0AqbCKusuMu5BSQNTlKjgonDAdabgPLv60uTPr8F5ac13XBcw03xpZC4Fx0jNe0OmeUUZZcouaTEPDKHC8PXk9LSuB3mMi5WsB6waDue9fjpON0YfgkkL+qS1s/lu54AtrCg1GBSKe385SQucBe5QCgV1l5UeCi6u+BIuIzRcg5+FzU01/RCZBc2ti88g3bD/dwSuvea4ipWN81u5hkFrlR/FKlzpxjVjNrFf62yU6VFT5Hzu7y2C+eEsSFOUCEbc7ZGXiqKljX50IR0IVOsIuHAynkLFijsuMKq8tUSm626XGfgprNbcLAKr6sAyHaq63ubhjofIR5Un9zV9CKYH48/j9NvH/tFxK/cO2SfvyYBMyCdyRE7IGZkSQX6R3+QP+du5TfaTd0n7N0mn7XlLtiwZ/QPP2MC/</latexit>
Information Loss
![Page 19: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/19.jpg)
Normalizing Flow (Rezende and Mohamed 2015)
• Basic idea: a way to map from one probability distribution to another in an invertible manner
ei ⇠ N (µzi ,⌃zi)<latexit sha1_base64="8CLMr+o0dZvNiJKf88rFwAP//jQ=">AAACQ3icZVBNa9tAEB0laZO4H3GaYy9LQiGlxki5JIUcQnvJKaS0bgyWEKv1yl6yqxW7o4Ir9F9Kfkzptb31P+TS0Guhu7IPiT2wzJs3b5aZl5VSWAzD38Ha+sajx5tb250nT5893+nuvvhsdWUYHzAttRlm1HIpCj5AgZIPS8OpyiS/yq7f+/7VF26s0MUnnJU8UXRSiFwwio5Ku6dxpmrepILEVigSK4pTRmV90Rz6TqyqJq2/pqLpkbb+KCaKLqjXafcg7IdtkFUQLcDB2X785hsAXKbdP/FYs0rxApmk1o6isMSkpgYFk7zpxJXlJWXXdMJHDhZUcZvU7ZUNeeWYMcm1ca9A0rL3J2qqrF/fKX2yD3qeQa2l7TkVTpVP/pu2tjOV9TLV8yJjc7u0COYnSS2KskJesPkeeSUJauIdJWNhOEM5c4AyI9wphE2poQyd7x3nUbTsyCoYHPXf9sMPzqt3MI8teAn7cAgRHMMZnMMlDIDBDfyAn/Ar+B7cBnfB37l0LVjM7MGDCP79BwcMtOc=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit>
xi = f�(ei)<latexit sha1_base64="270VJLD7gt0vH7AtNf4cVJpk6VA=">AAACKXicZVC7TisxEJ3lTXgFKGksHhJIUbShgVtcKYKGEiQCSCRaeR0vsbDXK3v26kar7fkYRAmUfAMV0PIFFPTYCQWPkaw5c+aMNXPiTAqLYfgcjIyOjU9MTk1XZmbn5heqi0snVueG8RbTUpuzmFouRcpbKFDys8xwqmLJT+PLfd8//ceNFTo9xn7GO4pepCIRjKKjoupqO1bF/zIS5C9JosJX7awnynLTQ+4aW1F1LayHgyC/QeMTrDXX328fAOAwqr61u5rliqfIJLX2vBFm2CmoQcEkLyvt3PKMskt6wc8dTKnitlMMbinJhmO6JNHGvRTJgP06UVBlFcWeU/pkv/U8g1pLW3Mq7Cmf/DeD2vZVXItVzYuMTeyPRTDZ7RQizXLkKRvukeSSoCbeN9IVhjOUfQcoM8KdQliPGsrQuVtxHjV+OvIbtLbrf+rhkfNqD4YxBSuwCpvQgB1owgEcQgsYXMEN3MF9cB08Bk/By1A6EnzOLMO3CF4/AGOkqtY=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit>
NeuralProjector
xf�1(e;�)
<latexit sha1_base64="0TvUX8MzRC5izm1cmvW9PX4J8iQ=">AAACWXicZVBNSyNBEO2M7hrHdY169NIYBBdimJGFVbyIXjwqGBWcbOjp1JjG/hi6a9QwzD/xqv9J8MfYk+RgtKCp169eNa9fmkvhMIreGsHC4o+fS83lcOXX6u+11vrGlTOF5dDjRhp7kzIHUmjooUAJN7kFplIJ1+n9aT2/fgDrhNGXOM6hr9idFpngDD01aLWy/+VeXO0mcESTfCT+DFrtqBtNin4H8Qy0yazOB+uN/WRoeKFAI5fMuds4yrFfMouCS6jCpHCQM37P7uDWQ80UuH45sV7RHc8MaWasPxrphP28UTLlFMORV9bNzc1qBo2RruNVOFJ1q5+Z3N1YpZ1UdWqRdZn7YgSzg34pdF4gaD71kRWSoqF1THQoLHCUYw8Yt8J/hfIRs4yjD3PORKqqMEw0PHKjFNPDMnmqyiRV5VNVzfMw5cHzPuX4a6bfQW+/e9iNLv62j09mcTfJFtkmuyQm/8gxOSPnpEc4eSDP5IW8Nt6DIGgG4VQaNGY7m2Sugs0PzFK2FA==</latexit><latexit sha1_base64="0TvUX8MzRC5izm1cmvW9PX4J8iQ=">AAACWXicZVBNSyNBEO2M7hrHdY169NIYBBdimJGFVbyIXjwqGBWcbOjp1JjG/hi6a9QwzD/xqv9J8MfYk+RgtKCp169eNa9fmkvhMIreGsHC4o+fS83lcOXX6u+11vrGlTOF5dDjRhp7kzIHUmjooUAJN7kFplIJ1+n9aT2/fgDrhNGXOM6hr9idFpngDD01aLWy/+VeXO0mcESTfCT+DFrtqBtNin4H8Qy0yazOB+uN/WRoeKFAI5fMuds4yrFfMouCS6jCpHCQM37P7uDWQ80UuH45sV7RHc8MaWasPxrphP28UTLlFMORV9bNzc1qBo2RruNVOFJ1q5+Z3N1YpZ1UdWqRdZn7YgSzg34pdF4gaD71kRWSoqF1THQoLHCUYw8Yt8J/hfIRs4yjD3PORKqqMEw0PHKjFNPDMnmqyiRV5VNVzfMw5cHzPuX4a6bfQW+/e9iNLv62j09mcTfJFtkmuyQm/8gxOSPnpEc4eSDP5IW8Nt6DIGgG4VQaNGY7m2Sugs0PzFK2FA==</latexit><latexit sha1_base64="0TvUX8MzRC5izm1cmvW9PX4J8iQ=">AAACWXicZVBNSyNBEO2M7hrHdY169NIYBBdimJGFVbyIXjwqGBWcbOjp1JjG/hi6a9QwzD/xqv9J8MfYk+RgtKCp169eNa9fmkvhMIreGsHC4o+fS83lcOXX6u+11vrGlTOF5dDjRhp7kzIHUmjooUAJN7kFplIJ1+n9aT2/fgDrhNGXOM6hr9idFpngDD01aLWy/+VeXO0mcESTfCT+DFrtqBtNin4H8Qy0yazOB+uN/WRoeKFAI5fMuds4yrFfMouCS6jCpHCQM37P7uDWQ80UuH45sV7RHc8MaWasPxrphP28UTLlFMORV9bNzc1qBo2RruNVOFJ1q5+Z3N1YpZ1UdWqRdZn7YgSzg34pdF4gaD71kRWSoqF1THQoLHCUYw8Yt8J/hfIRs4yjD3PORKqqMEw0PHKjFNPDMnmqyiRV5VNVzfMw5cHzPuX4a6bfQW+/e9iNLv62j09mcTfJFtkmuyQm/8gxOSPnpEc4eSDP5IW8Nt6DIGgG4VQaNGY7m2Sugs0PzFK2FA==</latexit>
• Advantage: optimizing probability can be shown to be possible by following equation
�1<latexit sha1_base64="whzOwvpgac/k7KeUY3/DRCrw/7o=">AAACTnicdVBNbxMxEPUGSkMotIUjF4sIiUOIdrdR096qcuHYSg2p1I0qrzPbWPHHyp4tjVb7I7jCr+qRX8IJtXYSJILESNY8v3ljPb+8lMJhHP+MWk+ebj3bbj/vvNh5+Wp3b//1F2cqy2HEjTT2MmcOpNAwQoESLksLTOUSxvn8U5iPb8E6YfQFLkqYKHajRSE4Q0+NP2ZCF7i43uvG/eOjw3RwSON+HA+TNAkgHQ4OBjTxTKguWdfZ9X6UZlPDKwUauWTOXSVxiZOaWRRcQtPJKgcl43N2A1ceaqbATeql34a+98yUFsb6o5Eu2b83aqacYjjzytDcxiwwaIx0Pa/CmQotPLO8u4XKe7nqBZF1hfvHCBZHk1roskLQfOWjqCRFQ0M2dCoscJQLDxi3wn+F8hmzjKNPcMNErppOJ9PwlRulmJ7W2V1TZ7mq75pmk4cVD573Kf+Jkv4fjNL+cT8+H3RPTtdxt8lb8o58IAkZkhPymZyREeFkTr6R7+RHdB/9in5HDytpK1rvvCEb1Wo/AhvGtlc=</latexit><latexit sha1_base64="whzOwvpgac/k7KeUY3/DRCrw/7o=">AAACTnicdVBNbxMxEPUGSkMotIUjF4sIiUOIdrdR096qcuHYSg2p1I0qrzPbWPHHyp4tjVb7I7jCr+qRX8IJtXYSJILESNY8v3ljPb+8lMJhHP+MWk+ebj3bbj/vvNh5+Wp3b//1F2cqy2HEjTT2MmcOpNAwQoESLksLTOUSxvn8U5iPb8E6YfQFLkqYKHajRSE4Q0+NP2ZCF7i43uvG/eOjw3RwSON+HA+TNAkgHQ4OBjTxTKguWdfZ9X6UZlPDKwUauWTOXSVxiZOaWRRcQtPJKgcl43N2A1ceaqbATeql34a+98yUFsb6o5Eu2b83aqacYjjzytDcxiwwaIx0Pa/CmQotPLO8u4XKe7nqBZF1hfvHCBZHk1roskLQfOWjqCRFQ0M2dCoscJQLDxi3wn+F8hmzjKNPcMNErppOJ9PwlRulmJ7W2V1TZ7mq75pmk4cVD573Kf+Jkv4fjNL+cT8+H3RPTtdxt8lb8o58IAkZkhPymZyREeFkTr6R7+RHdB/9in5HDytpK1rvvCEb1Wo/AhvGtlc=</latexit><latexit sha1_base64="whzOwvpgac/k7KeUY3/DRCrw/7o=">AAACTnicdVBNbxMxEPUGSkMotIUjF4sIiUOIdrdR096qcuHYSg2p1I0qrzPbWPHHyp4tjVb7I7jCr+qRX8IJtXYSJILESNY8v3ljPb+8lMJhHP+MWk+ebj3bbj/vvNh5+Wp3b//1F2cqy2HEjTT2MmcOpNAwQoESLksLTOUSxvn8U5iPb8E6YfQFLkqYKHajRSE4Q0+NP2ZCF7i43uvG/eOjw3RwSON+HA+TNAkgHQ4OBjTxTKguWdfZ9X6UZlPDKwUauWTOXSVxiZOaWRRcQtPJKgcl43N2A1ceaqbATeql34a+98yUFsb6o5Eu2b83aqacYjjzytDcxiwwaIx0Pa/CmQotPLO8u4XKe7nqBZF1hfvHCBZHk1roskLQfOWjqCRFQ0M2dCoscJQLDxi3wn+F8hmzjKNPcMNErppOJ9PwlRulmJ7W2V1TZ7mq75pmk4cVD573Kf+Jkv4fjNL+cT8+H3RPTtdxt8lb8o58IAkZkhPymZyREeFkTr6R7+RHdB/9in5HDytpK1rvvCEb1Wo/AhvGtlc=</latexit>
+X
log���det
@f�1�
@xi
���<latexit sha1_base64="SkZYIwUJm36ifl+bt7GkjNGFpcg=">AAAClHicdZFbixMxFMfTWS9rvXX1wQdfgkUQrGVmLNvdB2HZRRAEWcG6C5s6ZDKZNmwuQ3JmbRnzofw04pt+EzNtRSt4IOTkdy6c809eSeEgjr93op1r12/c3L3VvX3n7r37vb0HH52pLeMTZqSx5zl1XArNJyBA8vPKcqpyyc/yy5M2fnbFrRNGf4BlxaeKzrQoBaMQUNZ7i59j4mpFpJmRYzH7QoAvoCk4eFJayhpSUQuCSlx+al4kPgtgLrz/w8kiE35VmvX68fDwYD8d7eN4GMfjJE1aJx2PXo5wEkhrfbSx02yvk5LCsFpxDUxS5y6SuIJp03ZmkvsuqR2vKLukM34RXE0Vd9NmtbXHTwMpcGlsOBrwiv5d0VDlFIV5yGwvtxVrCRgj3SBkwVy1V9tm9XZLlQ9yNWiTrCvdP4NAeTBthK5q4Jqt5yhricHgVmFcCMsZyGVwKLMirILZnAYtIfzD1hC58t0u0fwzM0pRXTRkEYTNVbPwfpvzNeeBB5V/S4n/70zS4eEwfj/qHx1v5N5Fj9ET9AwlaIyO0Bt0iiaIoa/oG/qBfkaPolfRSfR6nRp1NjUP0ZZF734BeF7PyQ==</latexit><latexit sha1_base64="SkZYIwUJm36ifl+bt7GkjNGFpcg=">AAAClHicdZFbixMxFMfTWS9rvXX1wQdfgkUQrGVmLNvdB2HZRRAEWcG6C5s6ZDKZNmwuQ3JmbRnzofw04pt+EzNtRSt4IOTkdy6c809eSeEgjr93op1r12/c3L3VvX3n7r37vb0HH52pLeMTZqSx5zl1XArNJyBA8vPKcqpyyc/yy5M2fnbFrRNGf4BlxaeKzrQoBaMQUNZ7i59j4mpFpJmRYzH7QoAvoCk4eFJayhpSUQuCSlx+al4kPgtgLrz/w8kiE35VmvX68fDwYD8d7eN4GMfjJE1aJx2PXo5wEkhrfbSx02yvk5LCsFpxDUxS5y6SuIJp03ZmkvsuqR2vKLukM34RXE0Vd9NmtbXHTwMpcGlsOBrwiv5d0VDlFIV5yGwvtxVrCRgj3SBkwVy1V9tm9XZLlQ9yNWiTrCvdP4NAeTBthK5q4Jqt5yhricHgVmFcCMsZyGVwKLMirILZnAYtIfzD1hC58t0u0fwzM0pRXTRkEYTNVbPwfpvzNeeBB5V/S4n/70zS4eEwfj/qHx1v5N5Fj9ET9AwlaIyO0Bt0iiaIoa/oG/qBfkaPolfRSfR6nRp1NjUP0ZZF734BeF7PyQ==</latexit><latexit sha1_base64="SkZYIwUJm36ifl+bt7GkjNGFpcg=">AAAClHicdZFbixMxFMfTWS9rvXX1wQdfgkUQrGVmLNvdB2HZRRAEWcG6C5s6ZDKZNmwuQ3JmbRnzofw04pt+EzNtRSt4IOTkdy6c809eSeEgjr93op1r12/c3L3VvX3n7r37vb0HH52pLeMTZqSx5zl1XArNJyBA8vPKcqpyyc/yy5M2fnbFrRNGf4BlxaeKzrQoBaMQUNZ7i59j4mpFpJmRYzH7QoAvoCk4eFJayhpSUQuCSlx+al4kPgtgLrz/w8kiE35VmvX68fDwYD8d7eN4GMfjJE1aJx2PXo5wEkhrfbSx02yvk5LCsFpxDUxS5y6SuIJp03ZmkvsuqR2vKLukM34RXE0Vd9NmtbXHTwMpcGlsOBrwiv5d0VDlFIV5yGwvtxVrCRgj3SBkwVy1V9tm9XZLlQ9yNWiTrCvdP4NAeTBthK5q4Jqt5yhricHgVmFcCMsZyGVwKLMirILZnAYtIfzD1hC58t0u0fwzM0pRXTRkEYTNVbPwfpvzNeeBB5V/S4n/70zS4eEwfj/qHx1v5N5Fj9ET9AwlaIyO0Bt0iiaIoa/oG/qBfkaPolfRSfR6nRp1NjUP0ZZF734BeF7PyQ==</latexit>
when is not invertiblef<latexit sha1_base64="mEEGp2ktLOyduAzcmi2vdTnECNI=">AAACSHicdVBNbxMxEPUGWkqgpaVHLhYREoco2l2iprlVcOGYSIRE6kaV15ltrPpjZc9CotX+Aq7wq/gF/AxOFTfsJEgEiZGseX7zxnp+eSmFwzj+EbUePDw4fHT0uP3k6fHJs9Oz5x+dqSyHCTfS2FnOHEihYYICJcxKC0zlEqb53bswn34C64TRH3BdwlyxWy0KwRl6alzcnHbi3vDyIu1f0LgXx4MkTQJIB/03fZp4JlSH7Gp0cxal2cLwSoFGLplz10lc4rxmFgWX0LSzykHJ+B27hWsPNVPg5vXGaUNfeWZBC2P90Ug37N8bNVNOMVx6ZWhubxYYNEa6rlfhUoUWntnc3Vrl3Vx1g8i6wv1jBIvLeS10WSFovvVRVJKioSEVuhAWOMq1B4xb4b9C+ZJZxtFnt2ciV027nWn4zI1STC/qbNXUWa7qVdPs87DlwfM+5T9R0v+DSdob9uJxv3P1dhf3EXlBXpLXJCEDckXekxGZEE6AfCFfybfoe/Qzuo9+baWtaLdzTvaq1foN2xqzzg==</latexit><latexit sha1_base64="mEEGp2ktLOyduAzcmi2vdTnECNI=">AAACSHicdVBNbxMxEPUGWkqgpaVHLhYREoco2l2iprlVcOGYSIRE6kaV15ltrPpjZc9CotX+Aq7wq/gF/AxOFTfsJEgEiZGseX7zxnp+eSmFwzj+EbUePDw4fHT0uP3k6fHJs9Oz5x+dqSyHCTfS2FnOHEihYYICJcxKC0zlEqb53bswn34C64TRH3BdwlyxWy0KwRl6alzcnHbi3vDyIu1f0LgXx4MkTQJIB/03fZp4JlSH7Gp0cxal2cLwSoFGLplz10lc4rxmFgWX0LSzykHJ+B27hWsPNVPg5vXGaUNfeWZBC2P90Ug37N8bNVNOMVx6ZWhubxYYNEa6rlfhUoUWntnc3Vrl3Vx1g8i6wv1jBIvLeS10WSFovvVRVJKioSEVuhAWOMq1B4xb4b9C+ZJZxtFnt2ciV027nWn4zI1STC/qbNXUWa7qVdPs87DlwfM+5T9R0v+DSdob9uJxv3P1dhf3EXlBXpLXJCEDckXekxGZEE6AfCFfybfoe/Qzuo9+baWtaLdzTvaq1foN2xqzzg==</latexit><latexit sha1_base64="mEEGp2ktLOyduAzcmi2vdTnECNI=">AAACSHicdVBNbxMxEPUGWkqgpaVHLhYREoco2l2iprlVcOGYSIRE6kaV15ltrPpjZc9CotX+Aq7wq/gF/AxOFTfsJEgEiZGseX7zxnp+eSmFwzj+EbUePDw4fHT0uP3k6fHJs9Oz5x+dqSyHCTfS2FnOHEihYYICJcxKC0zlEqb53bswn34C64TRH3BdwlyxWy0KwRl6alzcnHbi3vDyIu1f0LgXx4MkTQJIB/03fZp4JlSH7Gp0cxal2cLwSoFGLplz10lc4rxmFgWX0LSzykHJ+B27hWsPNVPg5vXGaUNfeWZBC2P90Ug37N8bNVNOMVx6ZWhubxYYNEa6rlfhUoUWntnc3Vrl3Vx1g8i6wv1jBIvLeS10WSFovvVRVJKioSEVuhAWOMq1B4xb4b9C+ZJZxtFnt2ciV027nWn4zI1STC/qbNXUWa7qVdPs87DlwfM+5T9R0v+DSdob9uJxv3P1dhf3EXlBXpLXJCEDckXekxGZEE6AfCFfybfoe/Qzuo9+baWtaLdzTvaq1foN2xqzzg==</latexit>
log p(x) = log pGHMM(f�1
� (x))<latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="jBAzhNAJ26J1i9dTzZ4SO1PLZqY=">AAACOXicZVHLTsMwEHTKq4Ty6JlLBELiUFVpL8ANiQtHkAhUIhVynA216kdkb6BVlB/gylfxCXwFJ8QVp+2BlpWsHc/uWuPZJBfcYhh+eo219Y3Nrea2v9Pyd/f2D1r3VheGQcS00GaQUAuCK4iQo4BBboDKRMBDMr6q6w8vYCzX6g6nOQwlfVY844yio26eDo7DbjiL4D/oLcAxWcRT2+vHqWaFBIVMUGsfe2GOw5Ia5ExA5ceFhZyyMX2GRwcVlWCH5UxnFZw4Jg0ybdxRGMzYvxMllVZSHLnOOtmlWs2g1sJ2XBeOZJ3qZ2Z3O5VJJ5GdusnYzK4Iwex8WHKVFwiKzXVkhQhQB7UnQcoNMBRTBygz3H0lYCNqKEPn3JKIRFa+Hyt4ZVpKqtIynlRlnMhyUlXLPMx5cLwzubdq6X8Q9bsX3fA2JE1ySI7IKemRM3JJrskNiQgjKXkj796H9+V9z3fR8BZLaZOl8H5+AY4YsXk=</latexit><latexit sha1_base64="Ku8E5xMhx2yjUIS3NBNvbWk1Bus=">AAACcnicZVFNbxMxEJ0sXyUUmnLlYlGBUilEmwoJOCAhcaCXSkUitFI3RF5nNrHqj5U9C4ks/xF+DVf4B/wbvEkOpB3J8ps3z9bMm7JW0lOe/+1kd+7eu/9g72H30f7jJwe9w/2v3jZO4FhYZd1lyT0qaXBMkhRe1g65LhVelNcf2/rFd3ReWvOFVjVONJ8bWUnBKVHT3utC2Tmr+8XymL1nm2QaCsIlhU+nZ2cx9quU1wsZv4VXo9gKj6e9o3yYr4PdBqMtOIJtnE8POyfFzIpGoyGhuPdXo7ymSeCOpFAYu0Xjsebims/xKkHDNfpJWI8X2YvEzFhlXTqG2Jr9/0Xg2mtOi6RsL79TaxmyVvlBUtFCt1f7zTr3K10OSj1oRc5X/kYjVL2dBGnqhtCITR9VoxhZ1lrJZtKhILVKgAsn0yhMLLjjgpLhO02UOna7hcEfwmrNzSwUyxiKUodljLs8bnhMfHJ5dNPT22B8Mnw3zD/nsAfP4Dn0YQRv4AOcwjmMQcBP+AW/4U8G2ctssFlH1tnu5SnsRDb8B6n/wos=</latexit><latexit sha1_base64="Ku8E5xMhx2yjUIS3NBNvbWk1Bus=">AAACcnicZVFNbxMxEJ0sXyUUmnLlYlGBUilEmwoJOCAhcaCXSkUitFI3RF5nNrHqj5U9C4ks/xF+DVf4B/wbvEkOpB3J8ps3z9bMm7JW0lOe/+1kd+7eu/9g72H30f7jJwe9w/2v3jZO4FhYZd1lyT0qaXBMkhRe1g65LhVelNcf2/rFd3ReWvOFVjVONJ8bWUnBKVHT3utC2Tmr+8XymL1nm2QaCsIlhU+nZ2cx9quU1wsZv4VXo9gKj6e9o3yYr4PdBqMtOIJtnE8POyfFzIpGoyGhuPdXo7ymSeCOpFAYu0Xjsebims/xKkHDNfpJWI8X2YvEzFhlXTqG2Jr9/0Xg2mtOi6RsL79TaxmyVvlBUtFCt1f7zTr3K10OSj1oRc5X/kYjVL2dBGnqhtCITR9VoxhZ1lrJZtKhILVKgAsn0yhMLLjjgpLhO02UOna7hcEfwmrNzSwUyxiKUodljLs8bnhMfHJ5dNPT22B8Mnw3zD/nsAfP4Dn0YQRv4AOcwjmMQcBP+AW/4U8G2ctssFlH1tnu5SnsRDb8B6n/wos=</latexit><latexit sha1_base64="xChZ9DUVMtsn1KFXp2HSXCyWRic=">AAACfXicZVDbahsxEJW3t9S9Oc1jX0RNiwOu2Q2FtA+B0Dw0L4EU4iSQdY1WnrVFdFmk2cZG6Ef6NXlN/6B/U63thzoZEHPmzBkxc4pKCodp+reVPHr85OmzreftFy9fvX7T2X577kxtOQy5kcZeFsyBFBqGKFDCZWWBqULCRXF91PQvfoF1wugzXFQwUmyqRSk4w0iNO59zaaa06uXzXXpAV8XY5whz9N+PT05C6JWxrmYi/PSfstAId8edbjpIl0EfgmwNumQdp+Pt1l4+MbxWoJFL5txVllY48syi4BJCO68dVIxfsylcRaiZAjfyy/MC/RCZCS2NjU8jXbL/T3imnGI4i8omuY1ew6Ax0vWjCmeqSc03y9otVNEvVL8RWVe6e4tg+WXkha5qBM1Xe5S1pGhoYyWdCAsc5SICxq2Ip1A+Y5ZxjIZvLFGo0G7nGm64UYrpic/nweeF8vMQNnlY8RD56HJ239OHYLg3+DpIf6Tdw29ru7fIO/Ke9EhG9skhOSanZEg4+U1uyR35k5DkY9JPBitp0lrP7JCNSPb/ARdTw3A=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit>
![Page 20: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/20.jpg)
Results of Learning w/ HMM (He et al. 2018)
noun proper
verb base
cardinal number
verb gerund
adjective
noun singular
noun plural
adverb
verb past tenseverb past participleverb 3rd singular
![Page 21: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/21.jpg)
Unsupervised Phrase-structured Composition Functions
![Page 22: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/22.jpg)
Soft vs. Hard Tree Structure• Soft tree structure: use a differentiable gating function
• Hard tree structure: non-differentiable, but allows for more complicated composition methods
x1 x2
x1,2
x3
x2,3
x1,30.2 0.8
Soft
x1 x2 x3
x2,3
x1,3Hard
![Page 23: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/23.jpg)
One Other Paradigm: Weak Supervision
• Supervised: given X,Y to model P(Y|X)
• Unsupervised: given X to model P(Y|X)
• Weakly Supervised: given X and V to model P(Y|X), under assumption that Y and V are correlated
• Note: different from multi-task or transfer learning because we are given no Y
• Note: different from supervised learning with latent variables, because we care about Y, not V
![Page 24: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/24.jpg)
• Can choose whether to use left node, right node, or combination of both
Gated Convolution (Cho et al. 2014)
• Trained using MT loss
![Page 25: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/25.jpg)
Learning with RL (Yogatama et al. 2016)
• Intermediate tree-structured representation for language modeling
• Predict that tree using shift-reduce parsing, sentence representation composed in tree-structured manner
• Reinforcement learning with supervised loss, prediction loss
![Page 26: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/26.jpg)
Learning w/ Layer-wise Reductions (Choi et al. 2017)
• Choose one parent at each layer, reducing size by one
• Train using Gumbel-straighthrough reparameterization trick
• Faster and more effective than RL?
• Williams et al. (2017) find that this gives less trivial trees as well
![Page 27: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/27.jpg)
Difficulties in Learning Latent Structure
(Williams et al. 2018)• Unfortunately, many models learn trivial structure...
• e.g. balanced binary, left/right branching
• Why? One explanation: tension between (untrained) parser, and (untrained) downstream task model
![Page 28: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/28.jpg)
Learning Dependencies
![Page 29: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/29.jpg)
Phrase Structure vs. Dependency Structure
• Previous methods attempt to learn representations of phrases in tree-structured manner
• We might also want to learn dependencies, that tell which words depend on others
![Page 30: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/30.jpg)
Dependency Model w/ Valence (Klein and Manning 2004)
• Basic idea: top-down dependency based language model that generates left and right sides, then stops
I saw a girl with a telescopeROOT• For both the right and left side, calculate whether to
continue generating words, and if yes generatee.g., a slightly simplified view for word “saw”Pd(<cont> | saw, ←, false) * Pw(I | saw, ←, false) *Pd(<stop> | saw, ←, true) * Pd(<cont> | saw, →, false) * Pw(girl | saw, ←, false) * Pd(<cont> | saw, →, true) * Pw(with | saw, ←, true) *Pd(<stop> | saw, ←, true)
![Page 31: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/31.jpg)
Unsupervised Dependency Induction w/ Neural Nets (Jiang et al. 2016)
• Simple: parameterize the decision with neural nets instead of with count-based distributions
• Like DMV, train with EM algorithm
![Page 32: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/32.jpg)
Invertible Projections for DMV (He et al. 2018)
z1<latexit sha1_base64="kbJpa6Sam6jLYh1k9CLbrRqm2Rk=">AAACMnicZVDLSsNAFJ34Nr5a3ekmWAQXpSQiqLuiG5eK1gqmlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOcpF1YvTDcM+eeO5w5UcKZAd9/d2Zm5+YXFpeW3ZXVtfWNSnXzxqhUE9oiiit9G2FDOZO0BQw4vU00xSLitB3dnxXz9gPVhil5DaOEdgTuSxYzgsFSV0/doFup+Q2/LO8/CCaghiZ10a0622FPkVRQCYRjY+4CP4FOhjUwwmnuhqmhCSb3uE/vLJRYUNPJSq+5t2eZnhcrbY8Er2R/b2RYGIFhYJVFM1OzggGluKlbFQxE0YpnyrsZiageiXoh0iY2f4xAfNzJmExSoJKMfcQp90B5RS5ej2lKgI8swEQz+xWPDLDGBGx6UyYikbtuKOkjUUJg2cvCYZ6FkciGee7aNIO/2f0HrYPGScO/PKw1TyexLqEdtIv2UYCOUBOdowvUQgT10TN6Qa/Om/PhfDpfY+mMM9nZQlPlfP8AG16q6w==</latexit><latexit sha1_base64="kbJpa6Sam6jLYh1k9CLbrRqm2Rk=">AAACMnicZVDLSsNAFJ34Nr5a3ekmWAQXpSQiqLuiG5eK1gqmlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOcpF1YvTDcM+eeO5w5UcKZAd9/d2Zm5+YXFpeW3ZXVtfWNSnXzxqhUE9oiiit9G2FDOZO0BQw4vU00xSLitB3dnxXz9gPVhil5DaOEdgTuSxYzgsFSV0/doFup+Q2/LO8/CCaghiZ10a0622FPkVRQCYRjY+4CP4FOhjUwwmnuhqmhCSb3uE/vLJRYUNPJSq+5t2eZnhcrbY8Er2R/b2RYGIFhYJVFM1OzggGluKlbFQxE0YpnyrsZiageiXoh0iY2f4xAfNzJmExSoJKMfcQp90B5RS5ej2lKgI8swEQz+xWPDLDGBGx6UyYikbtuKOkjUUJg2cvCYZ6FkciGee7aNIO/2f0HrYPGScO/PKw1TyexLqEdtIv2UYCOUBOdowvUQgT10TN6Qa/Om/PhfDpfY+mMM9nZQlPlfP8AG16q6w==</latexit><latexit sha1_base64="kbJpa6Sam6jLYh1k9CLbrRqm2Rk=">AAACMnicZVDLSsNAFJ34Nr5a3ekmWAQXpSQiqLuiG5eK1gqmlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOcpF1YvTDcM+eeO5w5UcKZAd9/d2Zm5+YXFpeW3ZXVtfWNSnXzxqhUE9oiiit9G2FDOZO0BQw4vU00xSLitB3dnxXz9gPVhil5DaOEdgTuSxYzgsFSV0/doFup+Q2/LO8/CCaghiZ10a0622FPkVRQCYRjY+4CP4FOhjUwwmnuhqmhCSb3uE/vLJRYUNPJSq+5t2eZnhcrbY8Er2R/b2RYGIFhYJVFM1OzggGluKlbFQxE0YpnyrsZiageiXoh0iY2f4xAfNzJmExSoJKMfcQp90B5RS5ej2lKgI8swEQz+xWPDLDGBGx6UyYikbtuKOkjUUJg2cvCYZ6FkciGee7aNIO/2f0HrYPGScO/PKw1TyexLqEdtIv2UYCOUBOdowvUQgT10TN6Qa/Om/PhfDpfY+mMM9nZQlPlfP8AG16q6w==</latexit>
z2<latexit sha1_base64="k9/8EMjjLfM8r2mBl7kTM6haUuo=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmRVB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzTr1bOqf31cbpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+AR0qquw=</latexit><latexit sha1_base64="k9/8EMjjLfM8r2mBl7kTM6haUuo=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmRVB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzTr1bOqf31cbpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+AR0qquw=</latexit><latexit sha1_base64="k9/8EMjjLfM8r2mBl7kTM6haUuo=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmRVB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzTr1bOqf31cbpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+AR0qquw=</latexit>
?
z3<latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="bAUTGLr9EgIAz6GcyetjDEUesRE=">AAACIXicZVDLTsJAFJ3iCysqbHXTSExcEFLcqDsTNy4xESGhhEynU5gwj2bmViFNf8CtX+LXGBfGX3FaWAjcZHLPnHvu5MwJE84M+P63U9nZ3ds/qB66RzX3+OS0XnsxKtWE9ojiSg9CbChnkvaAAaeDRFMsQk774eyhmPdfqTZMyWdYJHQk8ESymBEMluqO602/7ZflbYPOCjTRqsYN5yyIFEkFlUA4NmbY8RMYZVgDI5zmbpAammAywxM6tFBiQc0oK33m3qVlIi9W2h4JXsn+38iwMALD1CqLZtZmBQNKcdOyKpiKohXPlHezEGErFK1CpE1sNoxAfDvKmExSoJIsfcQp90B5RSZexDQlwBcWYKKZ/YpHplhjAja5NROhyF03kPSNKCGwjLJgnmdBKLJ5nrs2zM5mdNugd92+a/tPPqqic3SBrlAH3aB79Ii6qIcIitA7+nA+nS/nZ5l5xVmF30Br5fz+AWoppzk=</latexit><latexit sha1_base64="cHNML6NFzNaVJV0shKLwGy2tbDA=">AAACJ3icZVDLTsMwENzwJpTnES4RFRKHqkrhANyQuHAEQWmlpqoc12kt/IjsDVCifAJX+BI+BnFC/AhO6YHSlawdz85a44lTwS2G4Yc3N7+wuLS8suqvVdY3Nre2K3dWZ4ayJtVCm3ZMLBNcsSZyFKydGkZkLFgrvr8o560HZizX6hZHKetKMlA84ZSgo26ee8e9rWpYD8cVzILGBFRhUle9bW836muaSaaQCmJtpxGm2M2JQU4FK/wosywl9J4MWMdBRSSz3XzstQgOHNMPEm3cURiM2b8bOZFWEhw6Zdns1KxkUGtha06FQ1m28pnx3Y5kXItlrRQZm9h/RjA57eZcpRkyRX99JJkIUAdlLkGfG0ZRjBwg1HD3lYAOiSEUXXpTJmJZ+H6k2CPVUhLVz6OnIo9imT8Vhe/SbPzPbhY0j+pn9fA6hBXYg304hAacwDlcwhU0gcIAXuAV3rx379P7+o19zpvkvwNT5X3/AMQVqVA=</latexit><latexit sha1_base64="cHNML6NFzNaVJV0shKLwGy2tbDA=">AAACJ3icZVDLTsMwENzwJpTnES4RFRKHqkrhANyQuHAEQWmlpqoc12kt/IjsDVCifAJX+BI+BnFC/AhO6YHSlawdz85a44lTwS2G4Yc3N7+wuLS8suqvVdY3Nre2K3dWZ4ayJtVCm3ZMLBNcsSZyFKydGkZkLFgrvr8o560HZizX6hZHKetKMlA84ZSgo26ee8e9rWpYD8cVzILGBFRhUle9bW836muaSaaQCmJtpxGm2M2JQU4FK/wosywl9J4MWMdBRSSz3XzstQgOHNMPEm3cURiM2b8bOZFWEhw6Zdns1KxkUGtha06FQ1m28pnx3Y5kXItlrRQZm9h/RjA57eZcpRkyRX99JJkIUAdlLkGfG0ZRjBwg1HD3lYAOiSEUXXpTJmJZ+H6k2CPVUhLVz6OnIo9imT8Vhe/SbPzPbhY0j+pn9fA6hBXYg304hAacwDlcwhU0gcIAXuAV3rx379P7+o19zpvkvwNT5X3/AMQVqVA=</latexit><latexit sha1_base64="RLvPvxg5oSd4xWwIdnMbsHCGlB0=">AAACMnicZVDLTgIxFO3gC8e37nQzkZi4IGTQhbojunGJUYSEIaRTOtDYx6S9o+BkPsGtfokfY1wZt36EHWAhcpPmnp57bnN6wpgzA77/4RQWFpeWV4qr7tr6xubW9s7uvVGJJrRBFFe6FWJDOZO0AQw4bcWaYhFy2gwfrvJ585Fqw5S8g1FMOwL3JYsYwWCp2+fuaXe75Ff8cXnzoDoFJTStenfH2Q96iiSCSiAcG9Ou+jF0UqyBEU4zN0gMjTF5wH3atlBiQU0nHXvNvCPL9LxIaXskeGP270aKhREYBlaZNzMzyxlQipuyVcFA5C1/Znw3IxGWQ1HORdpE5p8RiM47KZNxAlSSiY8o4R4oL8/F6zFNCfCRBZhoZr/ikQHWmIBNb8ZEKDLXDSR9IkoILHtpMMzSIBTpMMtcm2b1f3bzoHFSuaj4N36pdjmNtYgO0CE6RlV0hmroGtVRAxHURy/oFb05786n8+V8T6QFZ7qzh2bK+fkFHbaq6Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit>
z4<latexit sha1_base64="qRJO9a4HbiuS4+vcPiAoCJbeVKg=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmUlB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzSPq2dV/7pebpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+ASDCqu4=</latexit><latexit sha1_base64="qRJO9a4HbiuS4+vcPiAoCJbeVKg=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmUlB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzSPq2dV/7pebpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+ASDCqu4=</latexit><latexit sha1_base64="qRJO9a4HbiuS4+vcPiAoCJbeVKg=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmUlB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzSPq2dV/7pebpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+ASDCqu4=</latexit>
z5<latexit sha1_base64="7v8NYCmbh9REcawh1qKc3rzsbb8=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSBAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx1D7rVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwz/drrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fIo6q7w==</latexit><latexit sha1_base64="7v8NYCmbh9REcawh1qKc3rzsbb8=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSBAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx1D7rVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwz/drrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fIo6q7w==</latexit><latexit sha1_base64="7v8NYCmbh9REcawh1qKc3rzsbb8=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSBAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx1D7rVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwz/drrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fIo6q7w==</latexit>
z6<latexit sha1_base64="pGOJAC+5XIxMYgPoGWdA3fbkKOc=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapShHjcKrhwLIICEokqx3FaCz8iewOUKJ/AFb6Ej0GcEFc+AqftgcJK1o5nZ63xRClnBnz/3Zmanpmdm68suItLyyur1bX1K6MyTWiHKK70TYQN5UzSDjDg9CbVFIuI0+vo7rScX99TbZiSlzBIaShwT7KEEQyWunjqHnSrNb/hD8v7D5pjUEPjanfXnM0gViQTVALh2Jjbpp9CmGMNjHBauEFmaIrJHe7RWwslFtSE+dBr4e1YJvYSpe2R4A3Z3xs5FkZg6Ftl2czErGRAKW7qVgV9UbbymeHdDERUj0S9FGmTmD9GIDkKcybTDKgkIx9Jxj1QXpmLFzNNCfCBBZhoZr/ikT7WmIBNb8JEJArXDSR9IEoILOM8eCzyIBL5Y1G4Ns3m3+z+g85e47jhn+/XWifjWCtoC22jXdREh6iFzlAbdRBBPfSMXtCr8+Z8OJ/O10g65Yx3NtBEOd8/JFqq8A==</latexit><latexit sha1_base64="pGOJAC+5XIxMYgPoGWdA3fbkKOc=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapShHjcKrhwLIICEokqx3FaCz8iewOUKJ/AFb6Ej0GcEFc+AqftgcJK1o5nZ63xRClnBnz/3Zmanpmdm68suItLyyur1bX1K6MyTWiHKK70TYQN5UzSDjDg9CbVFIuI0+vo7rScX99TbZiSlzBIaShwT7KEEQyWunjqHnSrNb/hD8v7D5pjUEPjanfXnM0gViQTVALh2Jjbpp9CmGMNjHBauEFmaIrJHe7RWwslFtSE+dBr4e1YJvYSpe2R4A3Z3xs5FkZg6Ftl2czErGRAKW7qVgV9UbbymeHdDERUj0S9FGmTmD9GIDkKcybTDKgkIx9Jxj1QXpmLFzNNCfCBBZhoZr/ikT7WmIBNb8JEJArXDSR9IEoILOM8eCzyIBL5Y1G4Ns3m3+z+g85e47jhn+/XWifjWCtoC22jXdREh6iFzlAbdRBBPfSMXtCr8+Z8OJ/O10g65Yx3NtBEOd8/JFqq8A==</latexit><latexit sha1_base64="pGOJAC+5XIxMYgPoGWdA3fbkKOc=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapShHjcKrhwLIICEokqx3FaCz8iewOUKJ/AFb6Ej0GcEFc+AqftgcJK1o5nZ63xRClnBnz/3Zmanpmdm68suItLyyur1bX1K6MyTWiHKK70TYQN5UzSDjDg9CbVFIuI0+vo7rScX99TbZiSlzBIaShwT7KEEQyWunjqHnSrNb/hD8v7D5pjUEPjanfXnM0gViQTVALh2Jjbpp9CmGMNjHBauEFmaIrJHe7RWwslFtSE+dBr4e1YJvYSpe2R4A3Z3xs5FkZg6Ftl2czErGRAKW7qVgV9UbbymeHdDERUj0S9FGmTmD9GIDkKcybTDKgkIx9Jxj1QXpmLFzNNCfCBBZhoZr/ikT7WmIBNb8JEJArXDSR9IEoILOM8eCzyIBL5Y1G4Ns3m3+z+g85e47jhn+/XWifjWCtoC22jXdREh6iFzlAbdRBBPfSMXtCr8+Z8OJ/O10g65Yx3NtBEOd8/JFqq8A==</latexit>
z7<latexit sha1_base64="UaMLlKfO/9t66m7p0UvzLTfZft8=">AAACMnicZVDLSsNAFJ34rPFZ3ekmWAQXpaQiqLuiG5cVjQomlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOctF1YvTDcM+eeO5w5ccqZAd9/d2Zm5+YXFitL7vLK6tr6RnXz2qhMExoQxZW+jbGhnEkaAANOb1NNsYg5vYnvz8r5zQPVhil5BcOURgL3JEsYwWCpy6fOUWej5jf8UXn/QXMCamhS7U7V2Q67imSCSiAcG3PX9FOIcqyBEU4LN8wMTTG5xz16Z6HEgpooH3ktvD3LdL1EaXskeCP290aOhREY+lZZNjM1KxlQipu6VUFflK18ZnQ3QxHXY1EvRdok5o8RSI6jnMk0AyrJ2EeScQ+UV+bidZmmBPjQAkw0s1/xSB9rTMCmN2UiFoXrhpI+EiUElt08HBR5GIt8UBSuTbP5N7v/IDhonDT8i8Na63QSawXtoF20j5roCLXQOWqjABHUQ8/oBb06b86H8+l8jaUzzmRnC02V8/0DJiaq8Q==</latexit><latexit sha1_base64="UaMLlKfO/9t66m7p0UvzLTfZft8=">AAACMnicZVDLSsNAFJ34rPFZ3ekmWAQXpaQiqLuiG5cVjQomlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOctF1YvTDcM+eeO5w5ccqZAd9/d2Zm5+YXFitL7vLK6tr6RnXz2qhMExoQxZW+jbGhnEkaAANOb1NNsYg5vYnvz8r5zQPVhil5BcOURgL3JEsYwWCpy6fOUWej5jf8UXn/QXMCamhS7U7V2Q67imSCSiAcG3PX9FOIcqyBEU4LN8wMTTG5xz16Z6HEgpooH3ktvD3LdL1EaXskeCP290aOhREY+lZZNjM1KxlQipu6VUFflK18ZnQ3QxHXY1EvRdok5o8RSI6jnMk0AyrJ2EeScQ+UV+bidZmmBPjQAkw0s1/xSB9rTMCmN2UiFoXrhpI+EiUElt08HBR5GIt8UBSuTbP5N7v/IDhonDT8i8Na63QSawXtoF20j5roCLXQOWqjABHUQ8/oBb06b86H8+l8jaUzzmRnC02V8/0DJiaq8Q==</latexit><latexit sha1_base64="UaMLlKfO/9t66m7p0UvzLTfZft8=">AAACMnicZVDLSsNAFJ34rPFZ3ekmWAQXpaQiqLuiG5cVjQomlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOctF1YvTDcM+eeO5w5ccqZAd9/d2Zm5+YXFitL7vLK6tr6RnXz2qhMExoQxZW+jbGhnEkaAANOb1NNsYg5vYnvz8r5zQPVhil5BcOURgL3JEsYwWCpy6fOUWej5jf8UXn/QXMCamhS7U7V2Q67imSCSiAcG3PX9FOIcqyBEU4LN8wMTTG5xz16Z6HEgpooH3ktvD3LdL1EaXskeCP290aOhREY+lZZNjM1KxlQipu6VUFflK18ZnQ3QxHXY1EvRdok5o8RSI6jnMk0AyrJ2EeScQ+UV+bidZmmBPjQAkw0s1/xSB9rTMCmN2UiFoXrhpI+EiUElt08HBR5GIt8UBSuTbP5N7v/IDhonDT8i8Na63QSawXtoF20j5roCLXQOWqjABHUQ8/oBb06b86H8+l8jaUzzmRnC02V8/0DJiaq8Q==</latexit>
x1<latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit>
e2<latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit>
e1<latexit sha1_base64="5hfya15Z2veNhmcIusOWF706ODA=">AAACS3icZVDLSgMxFM3Ud31VXboZLIKLUmZEUHdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6cCm4xCD68ysLi0vLK6lp1fWNza7u2s3tndWYYtJkW2nRiakFwBW3kKKCTGqAyFnAfP16W8/snMJZrdYvjFHqSPiiecEaxpCLoh/1aPWgGk/L/g3AG6mRW1/0d7zgaaJZJUMgEtbYbBin2cmqQMwFFNcospJQ90gfoOqioBNvLJ2YL/9AxAz/Rxh2F/oT9vZFTaSXFoVOWzc7NSga1FrbhVDiUZSufmdztWMaNWDZKkbGJ/WMEk7NezlWaISg29ZFkwkftl8H4A26AoRg7QJnh7is+G1JDGbr45kzEsqhWIwXPTEtJ1SCPRkUexTIfFcU8D1MeHO9SDv9m+h+0j5vnzeDmpN66mMW9SvbJATkiITklLXJFrkmbMDIkL+SVvHnv3qf35X1PpRVvtrNH5qqy9AOQxLSV</latexit><latexit sha1_base64="5hfya15Z2veNhmcIusOWF706ODA=">AAACS3icZVDLSgMxFM3Ud31VXboZLIKLUmZEUHdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6cCm4xCD68ysLi0vLK6lp1fWNza7u2s3tndWYYtJkW2nRiakFwBW3kKKCTGqAyFnAfP16W8/snMJZrdYvjFHqSPiiecEaxpCLoh/1aPWgGk/L/g3AG6mRW1/0d7zgaaJZJUMgEtbYbBin2cmqQMwFFNcospJQ90gfoOqioBNvLJ2YL/9AxAz/Rxh2F/oT9vZFTaSXFoVOWzc7NSga1FrbhVDiUZSufmdztWMaNWDZKkbGJ/WMEk7NezlWaISg29ZFkwkftl8H4A26AoRg7QJnh7is+G1JDGbr45kzEsqhWIwXPTEtJ1SCPRkUexTIfFcU8D1MeHO9SDv9m+h+0j5vnzeDmpN66mMW9SvbJATkiITklLXJFrkmbMDIkL+SVvHnv3qf35X1PpRVvtrNH5qqy9AOQxLSV</latexit><latexit sha1_base64="5hfya15Z2veNhmcIusOWF706ODA=">AAACS3icZVDLSgMxFM3Ud31VXboZLIKLUmZEUHdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6cCm4xCD68ysLi0vLK6lp1fWNza7u2s3tndWYYtJkW2nRiakFwBW3kKKCTGqAyFnAfP16W8/snMJZrdYvjFHqSPiiecEaxpCLoh/1aPWgGk/L/g3AG6mRW1/0d7zgaaJZJUMgEtbYbBin2cmqQMwFFNcospJQ90gfoOqioBNvLJ2YL/9AxAz/Rxh2F/oT9vZFTaSXFoVOWzc7NSga1FrbhVDiUZSufmdztWMaNWDZKkbGJ/WMEk7NezlWaISg29ZFkwkftl8H4A26AoRg7QJnh7is+G1JDGbr45kzEsqhWIwXPTEtJ1SCPRkUexTIfFcU8D1MeHO9SDv9m+h+0j5vnzeDmpN66mMW9SvbJATkiITklLXJFrkmbMDIkL+SVvHnv3qf35X1PpRVvtrNH5qqy9AOQxLSV</latexit>
e3<latexit sha1_base64="Yo9t2xRNhbcucp40+9En1vW30ug=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYqqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfS2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFabl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+UjLSX</latexit><latexit sha1_base64="Yo9t2xRNhbcucp40+9En1vW30ug=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYqqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfS2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFabl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+UjLSX</latexit><latexit sha1_base64="Yo9t2xRNhbcucp40+9En1vW30ug=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYqqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfS2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFabl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+UjLSX</latexit>
e4<latexit sha1_base64="pOqccpiEKJr77/8/ytM6r0hBvEQ=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUyloO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3rVr7Yhb3Ojkgh+SYNMkpaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+WcLSY</latexit><latexit sha1_base64="pOqccpiEKJr77/8/ytM6r0hBvEQ=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUyloO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3rVr7Yhb3Ojkgh+SYNMkpaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+WcLSY</latexit><latexit sha1_base64="pOqccpiEKJr77/8/ytM6r0hBvEQ=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUyloO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3rVr7Yhb3Ojkgh+SYNMkpaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+WcLSY</latexit>
e5<latexit sha1_base64="MjmPb/ACrHEtZbpogr0zpFexMKk=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqaiqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfa2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFSbl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+YVLSZ</latexit><latexit sha1_base64="MjmPb/ACrHEtZbpogr0zpFexMKk=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqaiqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfa2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFSbl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+YVLSZ</latexit><latexit sha1_base64="MjmPb/ACrHEtZbpogr0zpFexMKk=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqaiqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfa2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFSbl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+YVLSZ</latexit>
e6<latexit sha1_base64="MmCjOdrAP44kfWvHfghiTcRDIeQ=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYiPnZFNy4rWCs4pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHthXBPzj03nJwoEdxiEHx6pYXFpeWV8mplbX1jc2t7Z/fO6tQwaDMttLmPqAXBFbSRo4D7xACVkYBO9HRVzDvPYCzX6hZHCXQlfVQ85oxiQYXQO+1tV4N6MC5/HjSmoEqm1erteMdhX7NUgkImqLUPjSDBbkYNciYgr4SphYSyJ/oIDw4qKsF2s7HZ3D90TN+PtXFHoT9m/25kVFpJceCURbMzs4JBrYWtORUOZNGKZ8Z3O5JRLZK1QmRsbP8Zwfi8m3GVpAiKTXzEqfBR+0Uwfp8bYChGDlBmuPuKzwbUUIYuvhkTkcwrlVDBC9NSUtXPwmGehZHMhnk+y8OEB8e7lBv/M50H7eP6RT24Oak2L6dxl8k+OSBHpEHOSJNckxZpE0YG5JW8kXfvw/vyvr2fibTkTXf2yEyVln4Bmji0mg==</latexit><latexit sha1_base64="MmCjOdrAP44kfWvHfghiTcRDIeQ=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYiPnZFNy4rWCs4pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHthXBPzj03nJwoEdxiEHx6pYXFpeWV8mplbX1jc2t7Z/fO6tQwaDMttLmPqAXBFbSRo4D7xACVkYBO9HRVzDvPYCzX6hZHCXQlfVQ85oxiQYXQO+1tV4N6MC5/HjSmoEqm1erteMdhX7NUgkImqLUPjSDBbkYNciYgr4SphYSyJ/oIDw4qKsF2s7HZ3D90TN+PtXFHoT9m/25kVFpJceCURbMzs4JBrYWtORUOZNGKZ8Z3O5JRLZK1QmRsbP8Zwfi8m3GVpAiKTXzEqfBR+0Uwfp8bYChGDlBmuPuKzwbUUIYuvhkTkcwrlVDBC9NSUtXPwmGehZHMhnk+y8OEB8e7lBv/M50H7eP6RT24Oak2L6dxl8k+OSBHpEHOSJNckxZpE0YG5JW8kXfvw/vyvr2fibTkTXf2yEyVln4Bmji0mg==</latexit><latexit sha1_base64="MmCjOdrAP44kfWvHfghiTcRDIeQ=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYiPnZFNy4rWCs4pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHthXBPzj03nJwoEdxiEHx6pYXFpeWV8mplbX1jc2t7Z/fO6tQwaDMttLmPqAXBFbSRo4D7xACVkYBO9HRVzDvPYCzX6hZHCXQlfVQ85oxiQYXQO+1tV4N6MC5/HjSmoEqm1erteMdhX7NUgkImqLUPjSDBbkYNciYgr4SphYSyJ/oIDw4qKsF2s7HZ3D90TN+PtXFHoT9m/25kVFpJceCURbMzs4JBrYWtORUOZNGKZ8Z3O5JRLZK1QmRsbP8Zwfi8m3GVpAiKTXzEqfBR+0Uwfp8bYChGDlBmuPuKzwbUUIYuvhkTkcwrlVDBC9NSUtXPwmGehZHMhnk+y8OEB8e7lBv/M50H7eP6RT24Oak2L6dxl8k+OSBHpEHOSJNckxZpE0YG5JW8kXfvw/vyvr2fibTkTXf2yEyVln4Bmji0mg==</latexit>
e7<latexit sha1_base64="+2fC0KPrqycQirOuKDePs7MeCTY=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFqO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3p7X2xSzudXJADskxaZIWaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+cHLSb</latexit><latexit sha1_base64="+2fC0KPrqycQirOuKDePs7MeCTY=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFqO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3p7X2xSzudXJADskxaZIWaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+cHLSb</latexit><latexit sha1_base64="+2fC0KPrqycQirOuKDePs7MeCTY=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFqO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3p7X2xSzudXJADskxaZIWaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+cHLSb</latexit>
x4<latexit sha1_base64="9frhfHTAh/tVZBbTFF8T6hotkFU=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvdPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCuoK0qw==</latexit><latexit sha1_base64="9frhfHTAh/tVZBbTFF8T6hotkFU=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvdPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCuoK0qw==</latexit><latexit sha1_base64="9frhfHTAh/tVZBbTFF8T6hotkFU=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvdPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCuoK0qw==</latexit>
x5<latexit sha1_base64="Iqq29VCa1Eput5eq9gbubvV+118=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFUXdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6UCG4xCD680sLi0vLK6lp5fWNza7tS3bmzOjUM2kwLbToRtSC4gjZyFNBJDFAZCbiPHi+L+f0TGMu1usVxAj1JHxSPOaNYUOGof9Kv1IJGMCn/P2jOQI3M6rpf9Y7CgWapBIVMUGu7zSDBXkYNciYgL4ephYSyR/oAXQcVlWB72cRs7h84ZuDH2rij0J+wvzcyKq2kOHTKotm5WcGg1sLWnQqHsmjFM5O7HcuoHsl6ITI2tn+MYHzWy7hKUgTFpj7iVPio/SIYf8ANMBRjBygz3H3FZ0NqKEMX35yJSOblcqjgmWkpqRpk4SjPwkhmozyf52HKg+Ndys2/mf4H7aPGeSO4Oa61LmZxr5I9sk8OSZOckha5ItekTRgZkhfySt68d+/T+/K+p9KSN9vZJXNVWvoBvGa0rA==</latexit><latexit sha1_base64="Iqq29VCa1Eput5eq9gbubvV+118=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFUXdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6UCG4xCD680sLi0vLK6lp5fWNza7tS3bmzOjUM2kwLbToRtSC4gjZyFNBJDFAZCbiPHi+L+f0TGMu1usVxAj1JHxSPOaNYUOGof9Kv1IJGMCn/P2jOQI3M6rpf9Y7CgWapBIVMUGu7zSDBXkYNciYgL4ephYSyR/oAXQcVlWB72cRs7h84ZuDH2rij0J+wvzcyKq2kOHTKotm5WcGg1sLWnQqHsmjFM5O7HcuoHsl6ITI2tn+MYHzWy7hKUgTFpj7iVPio/SIYf8ANMBRjBygz3H3FZ0NqKEMX35yJSOblcqjgmWkpqRpk4SjPwkhmozyf52HKg+Ndys2/mf4H7aPGeSO4Oa61LmZxr5I9sk8OSZOckha5ItekTRgZkhfySt68d+/T+/K+p9KSN9vZJXNVWvoBvGa0rA==</latexit><latexit sha1_base64="Iqq29VCa1Eput5eq9gbubvV+118=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFUXdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6UCG4xCD680sLi0vLK6lp5fWNza7tS3bmzOjUM2kwLbToRtSC4gjZyFNBJDFAZCbiPHi+L+f0TGMu1usVxAj1JHxSPOaNYUOGof9Kv1IJGMCn/P2jOQI3M6rpf9Y7CgWapBIVMUGu7zSDBXkYNciYgL4ephYSyR/oAXQcVlWB72cRs7h84ZuDH2rij0J+wvzcyKq2kOHTKotm5WcGg1sLWnQqHsmjFM5O7HcuoHsl6ITI2tn+MYHzWy7hKUgTFpj7iVPio/SIYf8ANMBRjBygz3H3FZ0NqKEMX35yJSOblcqjgmWkpqRpk4SjPwkhmozyf52HKg+Ndys2/mf4H7aPGeSO4Oa61LmZxr5I9sk8OSZOckha5ItekTRgZkhfySt68d+/T+/K+p9KSN9vZJXNVWvoBvGa0rA==</latexit>
x6<latexit sha1_base64="KY5/j4VzeKfzF+AkT011ZxYsr7Q=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFfOyKblwqWC10Ssmkd2wwjyG5oy3D/IJb/Sr3/ocrcWGm7cLqhXBPzj03nJwoEdxiEHx4pYXFpeWV1bXy+sbm1nalunNndWoYtJkW2nQiakFwBW3kKKCTGKAyEnAfPV4W8/snMJZrdYvjBHqSPigec0axoMJR/6RfqQWNYFL+f9CcgRqZ1XW/6h2FA81SCQqZoNZ2m0GCvYwa5ExAXg5TCwllj/QBug4qKsH2sonZ3D9wzMCPtXFHoT9hf29kVFpJceiURbNzs4JBrYWtOxUOZdGKZyZ3O5ZRPZL1QmRsbP8Ywfisl3GVpAiKTX3EqfBR+0Uw/oAbYCjGDlBmuPuKz4bUUIYuvjkTkczL5VDBM9NSUjXIwlGehZHMRnk+z8OUB8e7lJt/M/0P2keN80Zwc1xrXcziXiV7ZJ8ckiY5JS1yRa5JmzAyJC/klbx5796n9+V9T6Ulb7azS+aqtPQDvkq0rQ==</latexit><latexit sha1_base64="KY5/j4VzeKfzF+AkT011ZxYsr7Q=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFfOyKblwqWC10Ssmkd2wwjyG5oy3D/IJb/Sr3/ocrcWGm7cLqhXBPzj03nJwoEdxiEHx4pYXFpeWV1bXy+sbm1nalunNndWoYtJkW2nQiakFwBW3kKKCTGKAyEnAfPV4W8/snMJZrdYvjBHqSPigec0axoMJR/6RfqQWNYFL+f9CcgRqZ1XW/6h2FA81SCQqZoNZ2m0GCvYwa5ExAXg5TCwllj/QBug4qKsH2sonZ3D9wzMCPtXFHoT9hf29kVFpJceiURbNzs4JBrYWtOxUOZdGKZyZ3O5ZRPZL1QmRsbP8Ywfisl3GVpAiKTX3EqfBR+0Uw/oAbYCjGDlBmuPuKz4bUUIYuvjkTkczL5VDBM9NSUjXIwlGehZHMRnk+z8OUB8e7lJt/M/0P2keN80Zwc1xrXcziXiV7ZJ8ckiY5JS1yRa5JmzAyJC/klbx5796n9+V9T6Ulb7azS+aqtPQDvkq0rQ==</latexit><latexit sha1_base64="KY5/j4VzeKfzF+AkT011ZxYsr7Q=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFfOyKblwqWC10Ssmkd2wwjyG5oy3D/IJb/Sr3/ocrcWGm7cLqhXBPzj03nJwoEdxiEHx4pYXFpeWV1bXy+sbm1nalunNndWoYtJkW2nQiakFwBW3kKKCTGKAyEnAfPV4W8/snMJZrdYvjBHqSPigec0axoMJR/6RfqQWNYFL+f9CcgRqZ1XW/6h2FA81SCQqZoNZ2m0GCvYwa5ExAXg5TCwllj/QBug4qKsH2sonZ3D9wzMCPtXFHoT9hf29kVFpJceiURbNzs4JBrYWtOxUOZdGKZyZ3O5ZRPZL1QmRsbP8Ywfisl3GVpAiKTX3EqfBR+0Uw/oAbYCjGDlBmuPuKz4bUUIYuvjkTkczL5VDBM9NSUjXIwlGehZHMRnk+z8OUB8e7lJt/M/0P2keN80Zwc1xrXcziXiV7ZJ8ckiY5JS1yRa5JmzAyJC/klbx5796n9+V9T6Ulb7azS+aqtPQDvkq0rQ==</latexit>
x7<latexit sha1_base64="QuysJyVG4LzFzaADM3QOo12uCc8=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvbPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCwC60rg==</latexit><latexit sha1_base64="QuysJyVG4LzFzaADM3QOo12uCc8=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvbPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCwC60rg==</latexit><latexit sha1_base64="QuysJyVG4LzFzaADM3QOo12uCc8=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvbPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCwC60rg==</latexit>
cat stopped dog inThe a Paris
x3<latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit>
x2<latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit>
![Page 33: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/33.jpg)
Learning Dependency Heads w/ Attention (Kuncoro et al. 2017)
• Given a phrase structure tree, what child is the head word, the most important word in the phrase?
• Idea: create a phrase composition function that uses attention: examine if attention weights follow heads defined by linguistics
![Page 34: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/34.jpg)
Other Examples
![Page 35: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/35.jpg)
Learning about Word Segmentation from Attention (Boito et al. 2017)
• We want to learn word segmentation in an unsegmented language
• Simple idea: we can inspect the attention matrices from a neural MT system to extract words
![Page 36: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/36.jpg)
Learning Segmentations w/ Reconstruction Loss (Elsner and Shain 2017)• Learn segmentations of speech/text that allow for easy re-
construction of the original • Idea: consistent segmentation should result in easier-to-
reconstruct segments • Train segmentation using policy gradient
![Page 37: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/37.jpg)
Learning Language-level Features (Malaviya et al. 2017)
• All previous work learned features of a single sentence
• Can we learn features of the whole language? e.g. Typology: what is the canonical word order, etc.
• A simple method: train a neural MT system on 1017 languages, and extract its representations
![Page 38: CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning Discrete Structure • Learning features, e.g. word/sentence embeddings: this is an example](https://reader035.vdocuments.site/reader035/viewer/2022062605/5fd8af13975bd971936275ad/html5/thumbnails/38.jpg)
Questions?