B. Methods

Here we introduce the novel ET encoding method to transfer the timing interval through VLC. In ET encoding, the unit first signalizes the occurrence of the event and then refines the relative timing with consecutive LED patterns. In practice, the unit simulates the frame rate of the camera and switches the LED ‘on’ and ‘off’ using this clock (LED clock). When the unit detects an event, it marks the timing position of the event within this LED clock frame. The unit first sends ’on’ signal immediately after the event detection to inform the decoder of the LED clock frame that should be taken When the camera detects this first ‘on’ signal, we can know at the camera side that the event occurred somewhere in the previous frame shot. The unit continuously sends details of the position of the event in the LED clock frame. From these data, we can refine the timing of the event at the camera side. The flash pattern of the LED is determined according to the flow chart shown in Fig.1. A schematic of the relationship between the event timing and the LED clock frame is shown in Fig.2.

Fig. 1. Flow chart for sending the event timing by flashing LED. 1. The sensor unit monitors the sensor input until an event is detected. 2. IWhen the event is detected, the unit switches on the LED in the next sequence. Note that the LED is switching with Fz frequncy to match the frame rate of the camera. 3. Make the decision window whose size is 1/Fz (the same as frame rate) and mark the (place/timing) the event occured in the window. Reset the count i=1. 4. Check that the event occurred in first half of the decision window or last half of it. 5. If ‘yes’, switch on the LED. 5’. If ‘no’, switch off the LED. 6. Increment the count. 7. Check whehter the sequence needs to be stopped. P is the maximum length of the packet. 8. If not, update the decision window. Halve the window and take the marked side of the window. If ‘yes’, return to the beginning and wait for the next event to ocurr.

The transferred event timing can be reconstructed as follows. Let the packet length be P, and we assume the packet data of the n-th event are extracted starting from the en1-th frame as [b(en1), b(en2], …, b(enP)] for simplicity. The timing of the event En can be decoded as

ϕ+

−−= =

+−−

P

ieni

izn benFE

21

1 211 (1).

The constant ϕ represents the phase difference between the LED clock and sampling frame of the camera. One can obtain the interval data by subtracting this event timing from the previous event timing.

C. Time resolution

In our ET encoding method, the nth interval of an event can be obtained as En-En-1. The time resolution of the interval is calculated as

zFT F

Rz )1(1

min2

1−= (2).

For comparison, we also examined an alternative method that directly transfers the time interval between the events. In this method, which we refer to as interval encoding hereafter, the units calculate the interval between events and send it by flashing an LED. The sensor unit has to send the data range from Tmin to Tmax with TminFz bits of data. Thus, the resolution of the intervals sent to the camera is given as

zFT

TTR

min2minmax

2

−= (3).

Fig. 2. Example of the LED flashing pattern. In this example, electrocardiogram was measured and its peak of QRS complex is detected as an event. Fz is 30Hz and the packet legnth is 4. LED switched on immediately as first bit of the packet after the event detection. The precise event timing is marked on the LED clock frame (see the magnified view). The LED pattern from the 2nd bit is determined according to this marked location. In this example, the event located between 1/4 to 3/8, therefore, according to the flow chart in Fig.1, the packet sent by the LED becomes [1, 1, 0, 1].

Fig. 3. Photo of the developed unit. The analog amplifier, micro computer and FPGA were implemented in a single unit. Microcomputer samples the amplified electrocardiogram signals with 500Hz sampling rate and searches heart beat events by detecting QRS complexes. When an event is detected, the unit flashes the LEDs based on the flowchart shown in Fig.1. The unit has 24 LEDs which switches identically.

From (2) and (3), we can obtain a condition for which ET encoding provides better time resolution than interval encoding as follows:

zFTTRR 2minmax21 >−⇔< (4).

When 30 fps is plugged in for Fz, which is the frame rate of cameras commonly used, ET encoding yields an advantage when Tmax-Tmin is larger than 0.06 seconds, a condition which is applicable in most cases.

D. Implementation

The ET encoding was implemented in our HR monitoring system. The units were custom-built (Fig. 3). They sample an electrocardiogram signal with 500 Hz and 10 bits of resolution. The microcomputer installed in the unit searches for the QRS complex in the measured waveform using the algorithm introduced in [14] and [16]. There are 24 LEDs on the unit, but all the LEDs switch simultaneously to strengthen the intensity of the emitted light. The HR ranges from 50 to 204 bpm [17], that is to say from Tmax=1.2 s to Tmin=0.29 s; therefore, the system can utilize 10 bits to send one packet. However, there is no guarantee that the packet boundary is recognized at the camera side. If the packet boundary is mistaken, the system outputs incorrect HRs. Hence, we added five safety bits (LED switched off) to ensure that the packet boundary will be detected by image processing. Accordingly, the packet length that the unit sends in the occurrence of an event was set to 5 bits. The maximum timing resolution (2 ms at 500 Hz) can be achieved with this packet length.

Fig. 4. (A) Waveform of the electrocardiogram. The original output was recorded with 1000Hz but the data shown here is resampled to 500Hz so as to match the unit’s sampling rate. Heart beat timing is searched as peaks of QRS complexes, and the heart rate was calculated from this data. (B) Comparison between the heart rate observed from direct monitoring and the heart rate decoded from the LED flashing patterns captured by the camera. When 5 bits of data are transferred from the unit, almost identical heart rate was obtained.

Electrocardiogram signals were recorded beforehand by an ECG100 (BIOPACK) with 1-kHz sampling and used for simulation input in testing. The simulated ECG was generated by a 1-kHz DA board and fed into the developed unit and monitoring AD board. The output of the unit, observed as LED ‘on/off’, was captured by a Phantom Miro eX4 (Vision Research) camera with a progressive scan mode, 8-bit grayscale, 256x256 resolution, and 30 fps settings. The intensity of the LED and the diaphragm of the camera were set to have a high enough SN ratio to allow us to determine the ‘on/off’ of the LEDs by extracting the LED flashing pattern through simple thresholding. The recorded images and waveform were analyzed with MATLAB (The MathWorks, Inc.) scripts.

III. RESULTS

HRs were calculated using four different methods: HR extracted from original waveform; HR transferred by the interval encoding method introduced in II.C with 5 bit/packet rate; HR reconstructed using event signal (same as using only first signal in ET encoding); and HR decoded from ET encoding at 5 bit/packet. The same bit/packet was used for interval encoding and ET encoding. The results are shown in Fig. 4. Table 1 shows the root mean square error of the heart beat interval calculated for each method in comparison with the directly observed HR.

TABLE I. ROOT MEAN SQURE ERROR OF HEART BEAT INTERVAL

Encoding method

Interval encoding Event signal

only ET encoding

RMS 0.0161 0.0114 0.0108

IV. DISCUSSION

The purpose of this study was to examine efficient interval encoding methods using VLC with a low-frame-rate image sensor. We introduced ET encoding, which sends rough timing information about the event in the first signal and refines its position within the frame using signals after a second signal. Theoretical analyses were conducted in order to compare with the interval encoding. The analyses showed that ET encoding can send interval information with better resolution when condition (3) is satisfied. This condition is applicable in most cases because in many practical situations the interval range is larger than 0.06 s.

We implemented the ET encoding in a custom-made electrocardiogram sensor. The results showed that ET encoding can transfer HRs with higher resolution when the same bit/packet is used (Table 1). It was also shown that the resolution improves as expected if the packet length becomes larger. In Fig. 4(B), we can see that HR is transferred almost perfectly with ET encoding.

Since our main purpose in this study was to test the ET encoding, our image processing framework did not include position tracking of the units; thus, the HR from only one unit was recorded. However, expanding the communication to multiple units or adding position tracking functionality does not interfere with ET encoding. In this study, HR was transferred, but the data transferrable by ET encoding is not limited to HR. Other data, such as burst of spikes of neuron and turning speed of the wheel, can also be transferred by ET encoding.

Furthermore unlike interval encoding, ET encoding has the potential to transfer the absolute timing of the event. With the current implementation, the reconstructed event timing at the camera side may have a maximum error of 1/Fz s because the phase difference between the shutter of camera and the LED flashing [ϕ in (1)] are not synchronized. However, if synchronization between the camera and units can be achieved by using techniques in [13], we may be able to observe the absolute timing of the event in high resolution. This would open up other applications for ET encoding.

References

[1] M. Khalighi and M. Uysal, “Survey on free space optical communication: A communication theory perspective,” Communications SurveysTutorials, IEEE, vol. 16, no. 4, pp. 2231–2258, Fourthquarter 2014.

[2] T. Nagura, T. Yamazato, M. Katayama, T. Yendo, T. Fujii, H. Okada,“Improved decoding methods of Visible Light Communication system for ITS using LED array and high-speed camera,” in Proc. of IEEE 71st. Vehicular Technologyu Conf., May 2010.

[3] H. Uchiyama, M. Yoshino, H. Saito, M. Nakagawa, S. Haruyama, T. Kakehashi, and N. Nagamoto, “Photogrammetric system using visible light communication,” in Industrial Electronics, 2008. IECO.

[4] C. Seagrave and E. Benjamin, “Seeing sound: Sound sensor array with optical outputs,” in Proc. of the AES 114th Convention, 2010.

[5] M. Kurihara, Y. Honji, J. Fujimori, Y. Oikawa, and Y. Yamasaki, “Examination of sound field visualization-method using small visualization devices built with mems-mic and led,” in Proc. of the 2nd. Meeting on Acoustic Imaging. Acoustical Society of Japan, 2009, in Japanese.

[6] V. Pichot, F. Roche, JM. Gaspot , F. Enjolras, A. Antoniadis, P. Minini, F. Costes, T. Busso, JR. Lacour J. C. Barthelemy, “Relation between heart rate variability and training load in middle-distance runners,” Medicine & Science in Sports & Exercise, pp. 1729-1736, 2000.

[7] X. Jouven, J. P. Empana, P. J. Schwartz, M. Desnos, D. Courbon, P. Ducimetière, "Heart-rate profile during exercise as a predictor of sudden death." New England Journal of Medicine, vol. 352. no. 19, pp. 1951-1958, 2005.

[8] W. C. Levy, M. D. Cerqueira, G. D. Harp, K. A. Johannessen, I. B. Abrass, R. S. Schwartz, J. R. Stratton, "Effect of endurance exercise training on heart rate variability at rest in healthy young and older men." The American journal of cardiology, vol. 82, no. 10, pp. 1236-1241, 1998.

[9] P. Renza, A. Veicsteinas. "Heart rate variability and autonomic activity at rest and during exercise in various physiological conditions." European journal of applied physiology, vol.90, pp. 317-325, 2003.

[10] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor networks,” IEEE Communications Magazine, vol. 40, no. 8, pp. 102–114, August 2002.

[11] R. I. Davis, A. Burns, R. J. Bril, J. J. Lukkien, “Controller Area Network (CAN) schedulability analysis: Refuted, revisited and revised,” Real-Time Systems, vol. 35, no. 3, pp. 239-272, 2007.

[12] A Cailean, B. Cagneau, L. Chassagne, S. Topsu, Y. Alayli, M. Dimian, “A robust system for visible light communication,” in Wireless Vehicular Communications (WiVeC), 2013 IEEE 5th International Symposium on, June 2013, pp. 1–5.

[13] G. Pablo Nava, H. D. Nguyen, Y. Kamamoto, T. G. Sato, Y. Shiraki, N. Harada, T. Moriya, “A High Speed Camera-based Approach to Massive Sound Sensing with Optical Wireless Acoustic Sensors,”, IEEE Trans. Computational Imaging, in press.

[14] S. W. Porges, E. A. Byrne, “Research methods for measurement of heart rate and respiration,” Biological Phsychology, vol.34, pp. 93-130 1992.

[15] B-U Köhler, C. Hennig, R. Orglmeiste, “The principle of software QRS detection-Reviewing and comparing algorithms for detecting this important ECG waveform,” IEEE Engineering in Medicine and Biology, pp.42-57, 2002.

[16] J. Pan W. J. Tompkins “A real-time QRS detection algorithm,” IEEE Transactions on Biomedical Engineering, vol. 3, pp. 230-236, 1985.

[17] H. Tanaka, K. D. Monahan, D. R. Seals. "Age-predicted maximal heart rate revisited." Journal of the American College of Cardiology vol. 37. No.1 pp. 153-156, 2001.