rfi, lofar & eor - caastro · 2018. 4. 20. · provocative statement most eor global...
TRANSCRIPT
![Page 1: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/1.jpg)
RFI, LOFAR & EoR
André OffringaRSAA, Mt. Stromlo Observatory, Canberra
CAASTRO Global EoR workshop2012-11-21
![Page 2: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/2.jpg)
Outline
● Methods for accurately detecting RFI● The LOFAR radio environment:
– LOFAR’s performance after RFI excision
– Brightness distribution of RFI sources
– Implications for EoR projects
![Page 3: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/3.jpg)
Provocative statement
● Most EoR global experiments mention two problems:
– RFI
– Other
● But, considering that:– None of the recent detection methods are used
– (Almost) no global experiment is yet using a resolution of > ~50.000 channels/100 MHz
– LOFAR showed these make a huge difference
● We can not say that RFI is really a problem
![Page 4: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/4.jpg)
Example of LOFAR data with RFI
Cross-correlations of two stations showing strong RFI
![Page 5: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/5.jpg)
The AOFlagger
Offringa et al., MNRAS (2010), Offringa et al., A&A (2012a,b)
![Page 6: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/6.jpg)
The AOFlagger
“Novel” algorithmic stepscrucial for accuracy
![Page 7: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/7.jpg)
AOFlagger vs other flaggers● Accuracy higher than other flaggers● Fast
AOFlaggerMAD flagger (comparable to Pyflag, Miriad’s flagger, AIPS flagger)
![Page 8: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/8.jpg)
WSRT data example
![Page 9: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/9.jpg)
WSRT data example
![Page 10: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/10.jpg)
Thresholding vs. AOFlaggerMWA 3 min observation with 32 tiles
Image credit: Natasha Hurley-Walker
![Page 11: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/11.jpg)
The LOFAR radio environment
![Page 12: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/12.jpg)
Radio-Frequency Interference
● The (Dutch) radio spectrum is almost entirely allocated to services other than radio astronomy
Dutch allocations:
![Page 13: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/13.jpg)
The LOFAR radio environment
● Analysis of two LOFAR 24-h RFI surveys● One for the LBA, one for the HBA
LOFAR Low-Band Antenna (LBA)
1.81 m
![Page 14: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/14.jpg)
The survey data
Offringa et al., A&A 2012b.
![Page 15: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/15.jpg)
RFI and variance per station
![Page 16: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/16.jpg)
Detected RFI in the LBA
![Page 17: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/17.jpg)
Detected RFI in the HBA
![Page 18: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/18.jpg)
RFI excision results
● LBA (left) clean● HBA (right) some small residuals
![Page 19: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/19.jpg)
RFI excision results
● “Leaked” RFI in HBA due to “smooth” transmitters● These are only smooth at 1 s/1 kHz resolution● A possible 2nd stage flagger at lower resolution
could solve these residuals● Because they are weak, they are currently not
affecting the (imaging) sensitivity
![Page 20: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/20.jpg)
RFI excision conclusions● LOFAR’s environment:
– Fully automated detection, only a few % lost data
– Only small residuals, do not affect image quality
– 2nd stage flagger not yet used
● Why such good results?– LOFAR has very high time/freq resolutions
– Design has accounted for interference
– Unprecedented accuracy of algorithms
● Some transmitters do remain problematic(e.g., DAB, FM, wind turbines)
![Page 21: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/21.jpg)
Analysis of brightness distribution
![Page 22: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/22.jpg)
Total brightness distribution
![Page 23: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/23.jpg)
The left part is well-behaving noise...
![Page 24: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/24.jpg)
First order distribution implications
● Bi-variate: left part (noise) and right part (RFI)● Left part follows Rayleigh very accurately
– Well explained
● Right part follows power-law distribution (~x^-1.6)– Why?
![Page 25: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/25.jpg)
A uniform distribution of RFI sources
(Brightness of a source at distance r)
(Differential nr. sources with amp < S)
How they propagate:
Resulting in a distribution:
I : instantaneous intrinsic strength of sourceg : instrumental gainr : distance of source to receiverS : apparent brightnessc : constant that describes source density
![Page 26: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/26.jpg)
Why don't we see -2 power law?
(Brightness of a source at distance r)
r^2 fall-off assumes free-space propagation
However, there are effects ofdiffraction, refraction and reflection
![Page 27: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/27.jpg)
Why don't we see -2 power law?
(Brightness of a source at distance r)
● There exists a well-established empirical propagation model by Hata (1980) for propagation of communication signals (υ > 150 MHz)
● Implies radiation fall-off faster than r^2
![Page 28: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/28.jpg)
Slope as function of transmitter height
![Page 29: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/29.jpg)
Actual slope found
![Page 30: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/30.jpg)
Total brightness distribution
![Page 31: RFI, LOFAR & EoR - CAASTRO · 2018. 4. 20. · Provocative statement Most EoR global experiments mention two problems: – RFI – Other But, considering that: – None of the recent](https://reader035.vdocuments.site/reader035/viewer/2022070219/61290eb1cb02995d581dbd9b/html5/thumbnails/31.jpg)
Conclusions & best practices for EoR
● Learned a lot about RFI● Significant detection improvement by:
– High time/frequency resolution (~ 1 s / 1 kHz)
– Recent detections algorithms* (not just thresholding)
– Good signal path design (no ADC/amp saturation)
● Easy and cheap to try● It is to be seen if RFI is a problem
– Not (yet) an argument for more expensive alternatives
*AOFlagger code is publicly available at http://aoflagger.sourceforge.net/