lesson 3 2-norm and fourier series · •we want to use the fact that the fourier series is...

49
Lesson 3 2-norm and Fourier series

Upload: others

Post on 09-Apr-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Lesson 32-norm and Fourier series

Page 2: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 6IGEPP�JVSQ�PEWX�PIGXYVI�[I�HMWGYWWIH PIEWX�WUYEVIW JSV�KIRIVEP�MRRIV�TVSHYGXW �x, y��

� -R�XLMW�PIGXYVI� [I�[MPP�ZMI[�XLI�*SYVMIV�FEWMW

. . . , � k�, . . . , � �, 1, �, . . . , k�, . . .

EW�ZIGXSVW�MR�E�ZIGXSV�WTEGI

� ;I�[MPP�½RH�XLEX�XLI]�EVI�SVXLSKSREP�JSV�FSXL�E�GSRXMRYSYW�ERH�E HMWGVIXI MRRIVTVSHYGX

� 8LMW�GSRRIGXMSR�[MPP��IZIRXYEPP] �PIEH�XS�JEWX�ERH�EGGYVEXI�GSQTYXEXMSR�SJ�*SYVMIVWIVMIW

Page 3: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 0IXA =

�a1 | · · · | an

FI�E UYEWMQEXVM\ MR�E��KIRIVEP �ZIGXSV�WTEGI V

� 0EWX�PIGXYVI�[I�WLS[IH�XLEX�MJ�XLI�GSPYQRW�SJ A EVI PMRIEVP]�MRHITIRHIRX� XLIR�XLIYRMUYI�GSQTPI\�ZIGXSV c � Cn XLEX QMRMQM^IW

�Ac � b�

JSV�E�KMZIR b MW�TVIGMWIP]

c = K�1

���a1, b�

����an, b�

��

[LIVI�XLI�+VEQ�QEXVM\ K � Cn�n MW�HI½RIH�F]

K =

���a1, a1� · · · �a1, an�

���� � �

����a1, an� · · · �an, an�

��

� -J�[I�GER�IZEPYEXI�XLI�MRRIV�TVSHYGX�RYQIVMGEPP]� [I�GER�GSRWXVYGX K I\TPMGMXP]

3RGI MW�WIX�YT� [I�[MPP�I\TPSVI�XLMW�ETTVSEGL�EW�E RYQIVMGEP�EPKS�VMXLQ ERH�WII�XLEX�MX�MW�MRREGGYVEXI

8S�EGLMIZI�EGGYVEG]� [I�[MPP�HIZIPST�XLI +VEQ��7GLQMHX�EPKSVMXLQ

Page 4: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 0IXA =

�a1 | · · · | an

FI�E UYEWMQEXVM\ MR�E��KIRIVEP �ZIGXSV�WTEGI V

� 0EWX�PIGXYVI�[I�WLS[IH�XLEX�MJ�XLI�GSPYQRW�SJ A EVI PMRIEVP]�MRHITIRHIRX� XLIR�XLIYRMUYI�GSQTPI\�ZIGXSV c � Cn XLEX QMRMQM^IW

�Ac � b�

JSV�E�KMZIR b MW�TVIGMWIP]

c = K�1

���a1, b�

����an, b�

��

[LIVI�XLI�+VEQ�QEXVM\ K � Cn�n MW�HI½RIH�F]

K =

���a1, a1� · · · �a1, an�

���� � �

����a1, an� · · · �an, an�

��

� -J�[I�GER�IZEPYEXI�XLI�MRRIV�TVSHYGX�RYQIVMGEPP]� [I�GER�GSRWXVYGX K I\TPMGMXP]

3RGI MW�WIX�YT� [I�[MPP�I\TPSVI�XLMW�ETTVSEGL�EW�E RYQIVMGEP�EPKS�VMXLQ ERH�WII�XLEX�MX�MW�MRREGGYVEXI

8S�EGLMIZI�EGGYVEG]� [I�[MPP�HIZIPST�XLI +VEQ��7GLQMHX�EPKSVMXLQ

Page 5: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 0IXA =

�a1 | · · · | an

FI�E UYEWMQEXVM\ MR�E��KIRIVEP �ZIGXSV�WTEGI V

� 0EWX�PIGXYVI�[I�WLS[IH�XLEX�MJ�XLI�GSPYQRW�SJ A EVI PMRIEVP]�MRHITIRHIRX� XLIR�XLIYRMUYI�GSQTPI\�ZIGXSV c � Cn XLEX QMRMQM^IW

�Ac � b�

JSV�E�KMZIR b MW�TVIGMWIP]

c = K�1

���a1, b�

����an, b�

��

[LIVI�XLI�+VEQ�QEXVM\ K � Cn�n MW�HI½RIH�F]

K =

���a1, a1� · · · �a1, an�

���� � �

����a1, an� · · · �an, an�

��

� -J�[I�GER�IZEPYEXI�XLI�MRRIV�TVSHYGX�RYQIVMGEPP]� [I�GER�GSRWXVYGX K I\TPMGMXP]

3RGI MW�WIX�YT� [I�[MPP�I\TPSVI�XLMW�ETTVSEGL�EW�E RYQIVMGEP�EPKS�VMXLQ ERH�WII�XLEX�MX�MW�MRREGGYVEXI

8S�EGLMIZI�EGGYVEG]� [I�[MPP�HIZIPST�XLI +VEQ��7GLQMHX�EPKSVMXLQ

Page 6: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Orthonormal vectors and calculating least squares approximations

Page 7: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� % WIX�SJ�RSR^IVS�ZIGXSVW v1, . . . , vn EVI�GEPPIH SVXLSKSREP MJ

�vi, vj� = 0 [LIRIZIV i �= k.

� 8LI]�EVI�GEPPIH SVXLSRSVQEP MJ�XLI]�EVI�SVXLSKSREP�ERH�EPP�ZIGXSVW�EVI�SJ�YRMX�RSVQ�

1 = �vi� � SV�IUYMZEPIRXP]� �vi, vi� = 1.

� 8LI�+VEQ�QEXVM\�SJ�SVXLSRSVQEP�ZIGXSVW�MW�XLI�MHIRXMX]�

K =

���v1, v1� · · · �v1, vn�

���� � �

����vn, v1� · · · �vn, vn�

��

=

��1

� � �1

�� = I

Page 8: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� % WIX�SJ�RSR^IVS�ZIGXSVW v1, . . . , vn EVI�GEPPIH SVXLSKSREP MJ

�vi, vj� = 0 [LIRIZIV i �= k.

� 8LI]�EVI�GEPPIH SVXLSRSVQEP MJ�XLI]�EVI�SVXLSKSREP�ERH�EPP�ZIGXSVW�EVI�SJ�YRMX�RSVQ�

1 = �vi� � SV�IUYMZEPIRXP]� �vi, vi� = 1.

� 8LI�+VEQ�QEXVM\�SJ�SVXLSRSVQEP�ZIGXSVW�MW�XLI�MHIRXMX]�

K =

���v1, v1� · · · �v1, vn�

���� � �

����vn, v1� · · · �vn, vn�

��

=

��1

� � �1

�� = I

Page 9: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

The L2 inner product and norm� ;I�HI½RI�XLI 2 MRRIV�TVSHYGX��SR T �F]

�f, g� =1

2�

� �

��f̄(�)g(�) �

� %WWSGMEXIH�[MXL�XLMW�MRRIV�TVSHYGX�MW�XLI 2 RSVQ�

�f� =

�1

2�

� �

��|f(�)|2 �

� ;LEX�ZIGXSV� WTEGI�HSIW 2 RSVQ�ERH� MRRIV�TVSHYGX� ETTP]� XSS# 'SRZIRMIRXP]IRSYKL� XLI 2 WTEGI�

Page 10: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 0IXAn =

� � n� | . . . | � � | 1 | � | . . . | n��

FI�E UYEWMQEXVM\ [LSWI�GSPYQRW�EVI�MR 2[T]

� ;LEX�W�XLI�PIEWX�WUYEVIW�ETTVS\MQEXMSR�SJ�E�JYRGXMSR f � 2[T] F] An#

� ;I�LEZI�

k�, k��

=1

2�

� �

��� = 1

ERH�JSV k �= j

�k�, j�

�=

1

2�

� �

��

(j�k)� � =(j�k)� � � (j�k)�

2� (j � k)= 0

� -R�SXLIV�[SVHW� XLI�VS[W�SJ An EVI�SVXLSKSREP�

Page 11: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• The Gram matrix is therefore

K =

��

� � n�, � n��

· · ·� � n�, n�

�n�, � n�

�· · ·

�n�, n�

�� =

��1

1

��

Page 12: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• The Gram matrix is therefore

• The least squares approximation of f is thus precisely the Fourier series:

K�1

��

� � n�, f�

�n�, f

�� =1

2�

��

� ��� f(�) n� �

� ��� f(�) � n� �

�� =

��f̂�n

f̂n

��

K =

��

� � n�, � n��

· · ·� � n�, n�

�n�, � n�

�· · ·

�n�, n�

�� =

��1

1

��

Page 13: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Normwise convergence of Fourier series

Page 14: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges in the 2-norm

• In other words:

• This will follow from completeness

�����f(�) �n�

k=�n

f̂keik�

�����2

� 0

Page 15: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 8LI�GSQTPI\�I\TSRIRXMEPW

. . . , � k�, . . . , � �, 1, �, . . . , k�, . . .

JSVQ�E GSQTPIXI�SVXLSRSVQEP�W]WXIQ SJ 2[T]�

-R�SXLIV�[SVHW� JSV�IZIV] f � 2[T] ERH � > 0 XLIVI�I\MWXW�ER n ERH

�n(�) = s�n� n� + · · · sn

n� = Asn

WYGL�XLEX�f � �n�2 < �.

Page 16: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 8LI�GSQTPI\�I\TSRIRXMEPW

. . . , � k�, . . . , � �, 1, �, . . . , k�, . . .

JSVQ�E GSQTPIXI�SVXLSRSVQEP�W]WXIQ SJ 2[T]�

-R�SXLIV�[SVHW� JSV�IZIV] f � 2[T] ERH � > 0 XLIVI�I\MWXW�ER n ERH

�n(�) = s�n� n� + · · · sn

n� = Asn

WYGL�XLEX�f � �n�2 < �.

Page 17: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� %WWYQI�XLEX f MW�GSRXMRYSYW� f � C[T]

� 6IGEPP� *SYVMIV�WIVMIW HS�RSX RIGIWWEVMP]�GSRZIVKI�TSMRX[MWI�JSV�GSRXMRYSYW�JYRG�XMSRW� ,S[IZIV� [I�GER�HI½RI�E� VIPEXIH�GSQFMREXMSR�SJ�GSQTPI\�I\TSRIRXMEPW[LMGL�GSRZIVKIW YRMJSVQP] JSV�EPP�GSRXMRYSYW�JYRGXMSRW�

� ;I�GSRWXVYGX�WYGL�ER �n I\TPMGMXP]�EW�E�WSVX�SJ�EZIVEKI�SZIV�XLI�*SYVMIV�WIVMIW�

�n(�) =n�

k=�n

n + 1 � |k|n + 1

f̂kk� =

1

2�

� �

��f(t)Kn(� � t) t

[LIVI

Kn(�) =n�

k=�n

n + 1 � |k|n + 1

k�

� % WMQTPI�MRXIKVEXMSR�ZEVMEFPI�XVERWJSVQEXMSR�WLS[W�XLEX

�n(�) =1

2�

� �

��f(t)Kn(� � t) t =

1

2�

� �

��f(� � t)Kn(t) t

Page 18: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� %WWYQI�XLEX f MW�GSRXMRYSYW� f � C[T]

� 6IGEPP� *SYVMIV�WIVMIW HS�RSX RIGIWWEVMP]�GSRZIVKI�TSMRX[MWI�JSV�GSRXMRYSYW�JYRG�XMSRW� ,S[IZIV� [I�GER�HI½RI�E� VIPEXIH�GSQFMREXMSR�SJ�GSQTPI\�I\TSRIRXMEPW[LMGL�GSRZIVKIW YRMJSVQP] JSV�EPP�GSRXMRYSYW�JYRGXMSRW�

� ;I�GSRWXVYGX�WYGL�ER �n I\TPMGMXP]�EW�E�WSVX�SJ�EZIVEKI�SZIV�XLI�*SYVMIV�WIVMIW�

�n(�) =n�

k=�n

n + 1 � |k|n + 1

f̂kk� =

1

2�

� �

��f(t)Kn(� � t) t

[LIVI

Kn(�) =n�

k=�n

n + 1 � |k|n + 1

k�

� % WMQTPI�MRXIKVEXMSR�ZEVMEFPI�XVERWJSVQEXMSR�WLS[W�XLEX

�n(�) =1

2�

� �

��f(t)Kn(� � t) t =

1

2�

� �

��f(� � t)Kn(t) t

Page 19: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� %WWYQI�XLEX f MW�GSRXMRYSYW� f � C[T]

� 6IGEPP� *SYVMIV�WIVMIW HS�RSX RIGIWWEVMP]�GSRZIVKI�TSMRX[MWI�JSV�GSRXMRYSYW�JYRG�XMSRW� ,S[IZIV� [I�GER�HI½RI�E� VIPEXIH�GSQFMREXMSR�SJ�GSQTPI\�I\TSRIRXMEPW[LMGL�GSRZIVKIW YRMJSVQP] JSV�EPP�GSRXMRYSYW�JYRGXMSRW�

� ;I�GSRWXVYGX�WYGL�ER �n I\TPMGMXP]�EW�E�WSVX�SJ�EZIVEKI�SZIV�XLI�*SYVMIV�WIVMIW�

�n(�) =n�

k=�n

n + 1 � |k|n + 1

f̂kk� =

1

2�

� �

��f(t)Kn(� � t) t

[LIVI

Kn(�) =n�

k=�n

n + 1 � |k|n + 1

k�

� % WMQTPI�MRXIKVEXMSR�ZEVMEFPI�XVERWJSVQEXMSR�WLS[W�XLEX

�n(�) =1

2�

� �

��f(t)Kn(� � t) t =

1

2�

� �

��f(� � t)Kn(t) t

Page 20: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

-3 -2 -1 1 2 3

5

10

15

20

� Kn(�) MW TSWMXMZI ERH PSGEPM^IH RIEV�^IVS�

n = 5, 10, 20

Page 21: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

-3 -2 -1 1 2 3

5

10

15

20

� Kn(�) MW TSWMXMZI ERH PSGEPM^IH RIEV�^IVS�

n = 5, 10, 20

� -XW�RSVQ�MW ½\IH� 12�

� ��� Kn(�) � = 1

Page 22: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• We thus get

|sn(�) � f(�)| =

����1

2�

� �

��f(� � t)Kn(t) dt � 1

2�

� �

��f(�)Kn(t) dt

����

=1

2�

����� �

��(f(� � t) � f(�))Kn(t) dt

����

� 1

2�

�����

� �

��(f(� � t) � f(�))Kn(t) dt

����� +1

2�

�����

�� �

��+

� �

�(f(� � t) � f(�))Kn(t) dt

�����

Page 23: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• We thus get

|sn(�) � f(�)| =

����1

2�

� �

��f(� � t)Kn(t) dt � 1

2�

� �

��f(�)Kn(t) dt

����

=1

2�

����� �

��(f(� � t) � f(�))Kn(t) dt

����

� 1

2�

�����

� �

��(f(� � t) � f(�))Kn(t) dt

����� +1

2�

�����

�� �

��+

� �

�(f(� � t) � f(�))Kn(t) dt

�����

Page 24: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• We thus get

|sn(�) � f(�)| =

����1

2�

� �

��f(� � t)Kn(t) dt � 1

2�

� �

��f(�)Kn(t) dt

����

=1

2�

����� �

��(f(� � t) � f(�))Kn(t) dt

����

� 1

2�

�����

� �

��(f(� � t) � f(�))Kn(t) dt

����� +1

2�

�����

�� �

��+

� �

�(f(� � t) � f(�))Kn(t) dt

�����

Page 25: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• We thus get

|sn(�) � f(�)| =

����1

2�

� �

��f(� � t)Kn(t) dt � 1

2�

� �

��f(�)Kn(t) dt

����

=1

2�

����� �

��(f(� � t) � f(�))Kn(t) dt

����

� 1

2�

�����

� �

��(f(� � t) � f(�))Kn(t) dt

����� +1

2�

�����

�� �

��+

� �

�(f(� � t) � f(�))Kn(t) dt

�����

Small because f is continuous Small because Kn is isolated

Page 26: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

• We thus get

|sn(�) � f(�)| =

����1

2�

� �

��f(� � t)Kn(t) dt � 1

2�

� �

��f(�)Kn(t) dt

����

=1

2�

����� �

��(f(� � t) � f(�))Kn(t) dt

����

� 1

2�

�����

� �

��(f(� � t) � f(�))Kn(t) dt

����� +1

2�

�����

�� �

��+

� �

�(f(� � t) � f(�))Kn(t) dt

�����

Small because f is continuous Small because Kn is isolated

• Uniform convergence implies convergence in 2-norm

Page 27: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� ;LEX�EFSYX�KIRIVEP 2[T] JYRGXMSRW#

� %R] 2[T] JYRGXMSR�GER�FI�ETTVS\MQEXIH�MR�RSVQ�F]�E GSRXMRYSYW�JYRGXMSR

� 8LYW� [I�½VWX� ETTVS\MQEXI f � 2[T] F]� E� GSRXMRYSYW� JYRGXMSR f̃ � [LMGL�[IETTVS\MQEXI�F] sn(�)

Page 28: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 8LI�*SYVMIV�WIVMIW�GSRZIVKIW�MR�RSVQ�JSV�EPP f � 2[T]�

� ;I�LEZI�WLS[R�XLEX�XLIVI�I\MWXW�ER sn(�) = As WYGL�XLEX

�f � As�2 < �

� 8LI�*SYVMIV�WIVMIW�MW�XLI�YRMUYI�ZIGXSV c XLEX�QMRMQM^IW�XLI�RSVQ� XLIVIJSVI�

�f � Ac�2 � �f � As�2 < �

Page 29: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 8LI�*SYVMIV�WIVMIW�GSRZIVKIW�MR�RSVQ�JSV�EPP f � 2[T]�

� ;I�LEZI�WLS[R�XLEX�XLIVI�I\MWXW�ER sn(�) = As WYGL�XLEX

�f � As�2 < �

� 8LI�*SYVMIV�WIVMIW�MW�XLI�YRMUYI�ZIGXSV c XLEX�QMRMQM^IW�XLI�RSVQ� XLIVIJSVI�

�f � Ac�2 � �f � As�2 < �

Page 30: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 8LI�*SYVMIV�WIVMIW�GSRZIVKIW�TSMRX[MWI�JSV�EPP f � C2[T]�

� ;I�LEZI�WLS[R�MR�XLI�½VWX�PIGXYVI�XLEX�XLI�*SYVMIV�WIVMIW�GSRZIVKIW�YRMJSVQP]�XSE�GSRXMRYSYW�JYRGXMSR f̃

� 8LI�TVIZMSYW�XLISVIQ�WLS[W�XLEX���f � f̃

���2

= 0

� 7MRGI f ERH f̃ EVI�FSXL�GSRXMRYSYW� XLMW�MQTPMIW�XLEX f = f̃

Page 31: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 8LI�*SYVMIV�WIVMIW�GSRZIVKIW�TSMRX[MWI�JSV�EPP f � C2[T]�

� ;I�LEZI�WLS[R�MR�XLI�½VWX�PIGXYVI�XLEX�XLI�*SYVMIV�WIVMIW�GSRZIVKIW�YRMJSVQP]�XSE�GSRXMRYSYW�JYRGXMSR f̃

� 8LI�TVIZMSYW�XLISVIQ�WLS[W�XLEX���f � f̃

���2

= 0

� 7MRGI f ERH f̃ EVI�FSXL�GSRXMRYSYW� XLMW�MQTPMIW�XLEX f = f̃

Page 32: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Discrete inner product

Page 33: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

-3 -2 -1 0 1 2 3

� ;I�LEZI�WLS[R�XLEX�XLI�GSQTPI\�I\TSRIRXMEPW�EVI�SVXLSKSREP�[MXL�VIWTIGX�XS�XLIMRRIV�TVSHYGX

�f, g� =1

2�

� �

��f̄(�)g(�) �

� % VIQEVOEFPI�JEGX�MW�XLEX�XLI]�EVI�EPWS�SVXLSKSREP�[MXL�VIWTIGX�XS�XLI�JSPPS[MRKHMWGVIXI�WIQM�MRRIV�TVSHYGX�

�f, g�m =1

m

m�

j=1

f̄(�j)g(�j) = f(�)�g(�)

[LIVI � = (�1, . . . , �m) EVI�IZIRP]�WTEGIH�TSMRXW�

� =

���,

�1

m� 1

��, . . . ,

�1 � 1

m

��

�=

� 8LMW�SFWIVZEXMSR�JSVQW�XLI�FEWMW�SJ RYQIVMEP�*SYVMIV�WIVMIW

� -R�TEVXMGYPEV� MX�[MPP�JSVQ�XLI�FEWMW�SJ�XLI JEWX�*SYVMIV�XVERWJSVQ� EVKYEFP]�XLI�QSWXMQTSVXERX�RYQIVMGEP�EPKSVMXLQ�MR�I\MWXIRGI

Page 34: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Evenly spaced points on the unit circle

ei✓

z

-1.0 -0.5 0.5 1.0

-1.0

-0.5

0.5

1.0

-3 -2 -1 0 1 2 3

Page 35: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Some identities (shown for even m):

z

m�

j=1

ei�j =m�

j=1

zj = 0-1.0 -0.5 0.5 1.0

-1.0

-0.5

0.5

1.0

Page 36: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

Some identities (shown for even m):

z

m�

j=1

ei2�j =m�

j=1

z2j = 0

-1.0 -0.5 0.5 1.0

-0.5

0.5

Page 37: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

m�

j=1

k�j = (�)km JSV k = . . . , �2m, �m, 0, m, 2m. . .

m�

j=1

k�j = 0 JSV�EPP�SXLIV�MRXIKIV k

Page 38: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 2SXI�XLEXzj = � 2� (j�1)/m = ��j�1

JSV � = 11/m = 2� /m

� 8LIVIJSVI�m�

j=1

zkj = (�)k

m�1�

j=0

�kj

� -J k = �m MW�E�QYPXMTPI�SJ m� XLIR�[I�LEZI

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

Page 39: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 2SXI�XLEXzj = � 2� (j�1)/m = ��j�1

JSV � = 11/m = 2� /m

� 8LIVIJSVI�m�

j=1

zkj = (�)k

m�1�

j=0

�kj

� -J k = �m MW�E�QYPXMTPI�SJ m� XLIR�[I�LEZI

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

Page 40: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 2SXI�XLEXzj = � 2� (j�1)/m = ��j�1

JSV � = 11/m = 2� /m

� 8LIVIJSVI�m�

j=1

zkj = (�)k

m�1�

j=0

�kj

� -J k = �m MW�E�QYPXMTPI�SJ m� XLIR�[I�LEZI

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

Page 41: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 2SXI�XLEXzj = � 2� (j�1)/m = ��j�1

JSV � = 11/m = 2� /m

� 8LIVIJSVI�m�

j=1

zkj = (�)k

m�1�

j=0

�kj

� -J k = �m MW�E�QYPXMTPI�SJ m� XLIR�[I�LEZI

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

� *SV k RSX�E�QYPXMTPI�SJ m� VIGEPP�XLI�KISQIXVMG�WYQ�JSVQYPE�

m�1�

j=0

zj =zm � 1

z � 1

� 8LYWm�1�

j=0

�kj =m�1�

j=0

(�k)j =�km � 1

�k � 1

� &IGEYWI k MW�RSX�E�QYPXMTPI�SJ m� XLI�HIRSQMREXSV�MW�RSR^IVS

� &IGEYWI �km = (�m)k = 1k � XLI�RYQIVEXSV�MW�^IVS

Page 42: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 2SXI�XLEXzj = � 2� (j�1)/m = ��j�1

JSV � = 11/m = 2� /m

� 8LIVIJSVI�m�

j=1

zkj = (�)k

m�1�

j=0

�kj

� -J k = �m MW�E�QYPXMTPI�SJ m� XLIR�[I�LEZI

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

� *SV k RSX�E�QYPXMTPI�SJ m� VIGEPP�XLI�KISQIXVMG�WYQ�JSVQYPE�

m�1�

j=0

zj =zm � 1

z � 1

� 8LYWm�1�

j=0

�kj =m�1�

j=0

(�k)j =�km � 1

�k � 1

� &IGEYWI k MW�RSX�E�QYPXMTPI�SJ m� XLI�HIRSQMREXSV�MW�RSR^IVS

� &IGEYWI �km = (�m)k = 1k � XLI�RYQIVEXSV�MW�^IVS

Page 43: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

� 2SXI�XLEXzj = � 2� (j�1)/m = ��j�1

JSV � = 11/m = 2� /m

� 8LIVIJSVI�m�

j=1

zkj = (�)k

m�1�

j=0

�kj

� -J k = �m MW�E�QYPXMTPI�SJ m� XLIR�[I�LEZI

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

� *SV k RSX�E�QYPXMTPI�SJ m� VIGEPP�XLI�KISQIXVMG�WYQ�JSVQYPE�

m�1�

j=0

zj =zm � 1

z � 1

� 8LYWm�1�

j=0

�kj =m�1�

j=0

(�k)j =�km � 1

�k � 1

� &IGEYWI k MW�RSX�E�QYPXMTPI�SJ m� XLI�HIRSQMREXSV�MW�RSR^IVS

� &IGEYWI �km = (�m)k = 1k � XLI�RYQIVEXSV�MW�^IVS

Page 44: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

��

k�, j��

m= (�1)j�k JSV j � k = . . . , �2m, �m, 0, m, 2m, . . .

�k�, j�

�m

= SXLIV[MWI

� *SPPS[W�JVSQ�TVIGIHMRK�XLISVIQ�

�k�, j�

�m

=1

m

m�

j=1

(j�k)�j

Page 45: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

40

DFT

Page 46: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

41

� ;I�LEZI�WLS[R�XLEX�XLI�GSQTPI\�I\TSRIRXMEPW

� ��, � (�+1)�, . . . , (��1)�, ��

EVI�SVXLSRSVQEP�[MXL�VIWTIGX�XS ��, ��m [LIRIZIV �m < � < � < m

� 8LYW�XLI�PIEWX�WUYEVIW�ETTVS\MQEXMSR�SJ�E�JYRGXMSR f MW

f ���

k=�

�k�, f

�m

k�

� ;LEX�WLSYPH � ERH � FI# *SV�VIEWSRW�[I�HMWGYWW�PEXIV� MX�QEOIW�WIRWI�XS�LEZII\EGXP] m GSIJ½GMIRXW� 3RI�GLSMGI�MW

� = 0, � = m � 1

[LMGL�QEOIW RS�WIRWI �;L]#

� % QSVI�WIRWMFPI�GLSMGI�JSV m SHH�MW

� = �m � 1

2ERH � =

m � 1

2,

*SV m IZIR� = �m

2ERH � =

m

2� 1

Page 47: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

41

� ;I�LEZI�WLS[R�XLEX�XLI�GSQTPI\�I\TSRIRXMEPW

� ��, � (�+1)�, . . . , (��1)�, ��

EVI�SVXLSRSVQEP�[MXL�VIWTIGX�XS ��, ��m [LIRIZIV �m < � < � < m

� 8LYW�XLI�PIEWX�WUYEVIW�ETTVS\MQEXMSR�SJ�E�JYRGXMSR f MW

f ���

k=�

�k�, f

�m

k�

� ;LEX�WLSYPH � ERH � FI# *SV�VIEWSRW�[I�HMWGYWW�PEXIV� MX�QEOIW�WIRWI�XS�LEZII\EGXP] m GSIJ½GMIRXW� 3RI�GLSMGI�MW

� = 0, � = m � 1

[LMGL�QEOIW RS�WIRWI �;L]#

� % QSVI�WIRWMFPI�GLSMGI�JSV m SHH�MW

� = �m � 1

2ERH � =

m � 1

2,

*SV m IZIR� = �m

2ERH � =

m

2� 1

Page 48: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

41

� ;I�LEZI�WLS[R�XLEX�XLI�GSQTPI\�I\TSRIRXMEPW

� ��, � (�+1)�, . . . , (��1)�, ��

EVI�SVXLSRSVQEP�[MXL�VIWTIGX�XS ��, ��m [LIRIZIV �m < � < � < m

� 8LYW�XLI�PIEWX�WUYEVIW�ETTVS\MQEXMSR�SJ�E�JYRGXMSR f MW

f ���

k=�

�k�, f

�m

k�

� ;LEX�WLSYPH � ERH � FI# *SV�VIEWSRW�[I�HMWGYWW�PEXIV� MX�QEOIW�WIRWI�XS�LEZII\EGXP] m GSIJ½GMIRXW� 3RI�GLSMGI�MW

� = 0, � = m � 1

[LMGL�QEOIW RS�WIRWI �;L]#

� % QSVI�WIRWMFPI�GLSMGI�JSV m SHH�MW

� = �m � 1

2ERH � =

m � 1

2,

*SV m IZIR� = �m

2ERH � =

m

2� 1

Page 49: Lesson 3 2-norm and Fourier series · •We want to use the fact that the Fourier series is precisely the least squares approximation by complex exponentials to show that it converges

41

� ;I�LEZI�WLS[R�XLEX�XLI�GSQTPI\�I\TSRIRXMEPW

� ��, � (�+1)�, . . . , (��1)�, ��

EVI�SVXLSRSVQEP�[MXL�VIWTIGX�XS ��, ��m [LIRIZIV �m < � < � < m

� 8LYW�XLI�PIEWX�WUYEVIW�ETTVS\MQEXMSR�SJ�E�JYRGXMSR f MW

f ���

k=�

�k�, f

�m

k�

� ;LEX�WLSYPH � ERH � FI# *SV�VIEWSRW�[I�HMWGYWW�PEXIV� MX�QEOIW�WIRWI�XS�LEZII\EGXP] m GSIJ½GMIRXW� 3RI�GLSMGI�MW

� = 0, � = m � 1

[LMGL�QEOIW RS�WIRWI �;L]#

� % QSVI�WIRWMFPI�GLSMGI�JSV m SHH�MW

� = �m � 1

2ERH � =

m � 1

2,

*SV m IZIR� = �m

2ERH � =

m

2� 1