a report on my work on the beagleboard xm (angstrom...
TRANSCRIPT
Independent Study
1
A REPORT ON MY WORK ON THE
BEAGLEBOARD XM (Angstrom OS),
WHICH INCLUDES IMAGE PROCESSING,
OPTIMIZATION AND
WORK WITH LEOPARD CAMERA.
Kaushik Kanetkar______
Independent Study
2
Contents:
1. Porting Angstrom to Beagle Xm 3
2. Getting the code to work alongwith ffmpeg on Beagle 6
3. Work on Leopard camera 7
4. My time-lapse example 9
5. Remote debugging using GDB 10
6. Image processing and optimization 11
1.Neon instructions
7. Porting Ubuntu O.S on the Beagleboard 18
Independent Study
3
Porting Angstrom on to Beagle Xm:
Following are the steps to load Angstrom kernel on your Beagleboard Xm:
Preparing the SD card:
Plug in the SD card on your host system and wait for it to mount. You can check
by the command df -h . Look for the line which starts with /dev/sdXX, it will
generally be mounted in /media/
Make sure the size of the SD card is enough.
Now un-mount the SD card using umount /dev/sdXX
For Angstrom to run you will need to have a sd-card with two partitions; one fat partition to hold the boot files and then a ext3 partition with the root file system on it.
Make 2 partitions p1:Fat32 (0x0c) and other p2: ext3 (0x83) partition.
Make p1 bootable and keep around 50-70 Mb space.
To make the two partitions you can either use the GUI based application: Disk Utility. The other way would be to use the script omap-mkcard.sh in the folder ‘ ‘ by running the following commands: chmod +x omap3-mkcard.sh sudo ./omap3-mkcard.sh /dev/sdX sync
Remove the SD card and plug it in after 2-3 seconds. You will find two partitions /media/boot and /media/Angstrom.
The next step would be to generate a build. You can use the rootangstrom.tar.gz in the folder, or else you can generate your custom build at http://narcissus.angstrom-distribution.org/.
Select: BeagleBoard (the original BeagleBoard and BeagleBoard-xM both use the same build); simple; and console only. Now in the “additional packages” section, make sure you select the following: Toolchain
Native u-boot mkimage
All kernel modules
Ffmpeg
Bootloader Files (x-load/u-boot/scripts) in the “Platform specific packages
section”
Independent Study
4
Now click on ‘build me!’ which will generate a “*.tar.gz”. Download this “*.tar.gz”
in your working directory.
There are only three files that are mandatory for boot partition. The following
lines will extract the files from the download build and copy those to the boot
partition on the sd-card.
tar –xvf [your-file].tar.gz
cp boot/MLO-* /media/boot/MLO
cp boot/uImage-* /media/boot/uImage
cp boot/u-boot-*.bin /media/boot/u-boot.bin
sync
Now we have to copy/extract the filesystem to your SD card.
cd /media/Ansgtrom
sudo tar –xvf [path to you download file].tar.gz
sync
Make it safe to remove your SD card.
umount /media/boot
umount /media Angstrom
Remove the SD card and place it inside the card case on your Beagleboard.
Connect the Serial (BeagleBoardxM) to USB(Host PC) converter and configure the minicom connection.
Start minicom on your host machine in configuration mode. As root: # minicom -o -s -w
· A menu of configuration should appear. Use the Down-arrow key to scroll down and select the Serial port setup option, and press Enter.
· Verify that the listed serial port is the same one that is connected to the target board. If it is not, press A, and enter the correct device. This is /dev/ttyUSB0 on most Linux distributions.
· Set the Bps/Par/Bits option by pressing the letter E and using the next menu toset the appropriate values. You press the key that corresponds to the value 115200, and then press Enter.
· Set Hardware flow control to No using the F key.
· Set Software flow control to No using the G key.
Independent Study
5
· Press Enter to return to the main configuration menu, and then press Esc to exit this menu.
Reset the board, and wait for a moment. If you do not see output from the board, press Enter several times until you see the prompt. If you do not see any output from the board, and have verified that the serial terminal connection is setup correctly, contact your board vendor.
When you boot Angstrom for the first time on the beagleboard, it will take some time. Be patient. Once it is done , you will be asked for a beagleboard login: (enter ‘root’)
Run the following command:
ifconfig usb0 192.168.1.10 (gives an ip address to your usb)
Now run the following command on your host:
ifconfig eth0 192.168.1.4 (configures a static ip address)
From your beagleboard see if you can ping your host.
ping 192.168.1.4
If you are able to ping, then you are all set and ready to work with Angstrom on
beagleboard and to transfer files between your beagle and the host system !
Independent Study
6
Timelapse on the Beagleboard :
Build the code_to_capture_frames.c code on the Beagleboard using: arm-angstrom-linux-gnueabi-gcc code_to_capture_frames.c –lm –pthread –lrt
Run the executable ./a.out file
The leopard camera will now capture images and store it in the YUV folder.
The next step is to convert the YUV’s into PPM’s and encode them into a video.
You have two options:
1. On the Beagleboard:
Build the buildvideo.c code using
gcc buildvideo.c –lm
Now run the ./a.out output file and specify the no. of frames you want to
use in forming the video.
./a.out <no.of frames>
2. On your host system (Ubuntu):
Transfer the YUV folder from Beagleboard to the host system using scp.
scp –r yuvs/ hostname@host_ip_address:/home/
Build the buildvideo.c code and run it as above.
Run the video on VLC player.
Independent Study
7
Leopard Camera:
Refer to the following document regarding Video4Linux in Srinivas’ report and the link :
http://v4l2spec.bytesex.org/spec/
I have made a few changes to his code to optimize the use of Leopard camera.
Firstly, I will just go through some of the important ioctl’s for getting images from the
Leopard camera.
ioctl(fd, VIDIOC_REQBUFS, &req);
-This allocates the number of buffers to be allocated.
ioctl(fd, VIDIOC_QUERYBUF, &buffer);
-It queries the status of the buffers. You need to make sure that all the allocated buffers
are queried.
ioctl(fd, VIDIOC_QBUF, &buffer);
-This ioctl call is used the enqueue a buffer in the driver’s incoming queue.
ioctl(fd, VIDIOC_DQBUF, &buffer);
-This ioctl is used to dequeue a filled buffer from the driver’s outgoing queue.
ioctl(fd, VIDIOC_STREAMON, &type);
ioctl(fd, VIDIOC_STREAMOFF, &type);
-The VIDIOC_STREAMON and VIDIOC_STREAMOFF are used to start and stop the
capture.
Note: While using the VIDIOC_STREAMON there is a high chance that it might prompt
an error that the device is busy. So make sure there is a ‘while’ loop such that wait till
the ioctl stops returning -1 (error value), and then only proceed.
Independent Study
8
It takes a lot of time for the VIDIOC_DQBUF, VIDIOC_QBUF calls to execute, especially if
only one buffer is used.
I added more buffers in the queue ( a maximum of 32 can be allocated ).
Adding more buffers brought down the time for capturing and also brought the frame
capturing rate up.
The following was included after all the settings related to MMAP and STREAMON were
done:
{buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buffer.memory = V4L2_MEMORY_MMAP;
buffer.index = j
//make sure this value of ‘j’ is the value of buffer number which you are currently using.
ioctl(fd, VIDIOC_DQBUF, &buffer);
----getting the yuv-----
ioctl(fd, VIDIOC_QBUF, &buffer);
}
With this, I was able to get a better frame rate, by making use of buffers.
Future Scope:
While aiming to achieve a higher frame rate, it gives an error in the VIDIOC_QBUF or
VIDIOC_DQBUF.
If somehow, this error can be avoided by making optimum use of the maximum amount
of buffers available, then, a stable and a better frame rate can be achieved.
*Special packages to click screenshots*
Mplayer:
Angstrom has an application called ‘mplayer’ which can be used on the Beagleboard, to
view and click images based on the frame rate.
I used this application on the console to click images.
Independent Study
9
Time-lapse example :
1. I have created a time-lapse focusing on a digital clock. The frequency is 1Hz.
The ppms folder contains the images clicked by the Leopard camera at every second.
2. The other time-lapse I created was the melting of ice-cubes.
http://www.youtube.com/watch?v=tsxUPD3gP9w&feature=g-upl
Independent Study
10
Remote debugging using GDB:
Get the following cross-compiler tools (arm-none-linux-gnueabi-gcc) from
http://www.codesourcery.com/sgpp/lite/arm/portal/release1600
Using command line:
Now, for example the ip address of my system (host) is 192.168.1.4 and the ip address
of the target (Beagle) is 192.168.1.14
With the given Angstrom kernel, you indeed have gdbserver installed on your target.
To start gdbserver on the target you need to specify a port number which you will listen
and then connect through and the executable to debug. The port can be anything so
long as it’s not a reserved port e.g. 21,80.
I am using the port number 2345.
gdbserver test:2345 a.out
This creates the pid and starts listening on port 2345.
Run the gdb command from the host system:
arm-none-linux-gnueabi-gdb ./a.out
Once gdb starts, run the following:
(gdb) target remote 192.168.1.14:2345
Independent Study
11
Image processing and optimization:
I used an image from the Leopard cam (pic-5.ppm), having a resolution of 640x480.
I applied the sharpen algorithm on it without any optimization or mulit-threading.
This was the result I got using the arm-angstrom-linux-gnueabi-gcc compiler for ARM.
Then, I used some optimizations mentioned in the following paper by Prof.Sam Siewert
http://software.intel.com/en-us/articles/using-intel-streaming-simd-extensions-and-intel-integrated-
performance-primitives-to-accelerate-algorithms/
These are the Intel SIMD instructions for gcc optimization.
It provided the following speed-up.
Generally we would want the processor to speed up more. So ARM does have a number
of instruction sets, one of them being the NEON instruction set.
NEON is included in all Cortex-A8 devices but is optional in Cortex-A9 devices. Lets see
how the NEON provides better optimization.
Independent Study
12
NEON Instructions :
The Advanced SIMD extension (aka NEON or "MPE" Media Processing Engine) is a
combined 64- and 128-bit single instruction multiple data (SIMD) instruction set that
provides standardized acceleration for media and signal processing applications. NEON
can execute MP3 audio decoding on CPUs running at 10 MHz and can run
the GSM adaptive multi-rate (AMR) speech codec at no more than 13 MHz. It features a
comprehensive instruction set, separate register files and independent execution
hardware. NEON supports 8-, 16-, 32- and 64-bit integer and single-precision (32-bit)
floating-point data and operates in SIMD operations for handling audio and video
processing as well as graphics and gaming processing. In NEON, the SIMD supports up to
16 operations at the same time. Devices such as the ARM Cortex-A8 and Cortex-A9
support 128-bit vectors but will execute with just 64 bits at a time, whereas newer
Cortex-A15 devices can execute 128 bits at once.
NEON Instructions perform “Packed SIMD” processing:
Registers are considered as vectors of elements of the same data type
Data types can be: signed/unsigned 8-bit, 16-bit, 32-bit, 64-bit, single prec. float
Instructions perform the same operation in all lanes
As it can be seen, for adding R1 to R2, it performs the single instruction of adding on
multiple data (four lanes) thus providing speedup.
Independent Study
13
Using the NEON optimization:
-- In order to instruct the compiler to produce NEON you should use the following
compile flags:
-mfpu=neon
-- To specify which floating-point ABI to use, permissible values are: `soft', `softfp' and `hard'. Specifying `soft' causes GCC to generate output containing library calls for floating-point operations. `softfp' allows the generation of code using hardware floating-point instructions, but still uses the soft-float calling conventions. `hard' allows generation of floating-point instructions and uses FPU-specific calling conventions. -mfloat-abi=name -- To specify the name of the target ARM processor. GCC uses this name to determine what kind of instructions it can emit when generating assembly code.
-mcpu=cortex-a8
-- If you want the compiler to attempt to vectorize your integer / floating point code for
the NEON you should add: -ftree-vectorize to your flags.
-ftree-vectorize
Using all these alongwith the –O3 optimization, this was the result I got on the same
single threaded sharpen code on the 640x480 image.
The NEON instructions provided a speedup of around 3X!
Independent Study
14
Multi-threading:
Multi – threading does give better results especially when higher amount of processing
is involved. Threading helps in speeding up the process. I have written a code to
modify/process any sized image into any desired number of thread blocks, and get the
necessary speedup.
This was the result, when the image was sharpened using single-threading.
When multiple threads are used to process the image at its different locations, the
following result is observed on the same 640x480 image. (Using 4x3 threads)
Multithreading using 8x6 threads:
Thus, there is a speed up when 3x4 threads are used as compared to 6x8 threads.
Independent Study
15
Following are the results of some image processing done on the images captured by the
Leopard camera.
Some other transformations:
1. Centroid
<->
Thus, the neon provides a huge speedup in performance.
Independent Study
16
2. Boost
<->
The following output shows the speedup in performance using NEON.
Independent Study
17
Image upscaling:
I wrote a code to convert 320x240 images to 640x480 images.
The method I used was nearest neighbor averaging. Now for example, we have to
transform a 2x2 to a 4x4.
The new added pixel will be the average of the neighboring pixels.
Once, the neighboring pixels method was carried out, I applied the averaging filter, to
smoothen out edges (if any)
Similarly, for a conversion from 640x480 to 1280x720, the neighboring pixels
method is used, where the ratio is taken into account.
640/1280 = (1/2)
480/720 = (2/3)
Every new pixel in the up scaled image corresponds to the (1/2)th location in the
original images’ width while, to the (2/3)rd location in its height.
Independent Study
18
THIS IS A TUTORIAL TO HELP YOU
PORT THE
“UBUNTU”
OPERATING SYSTEM
ON THE BEAGLEBOARD xM
Independent Study
19
Porting UBUNTU on the BeagleBoard:
Following are the steps to port another operating system, Ubuntu, on the
BeagleBoard Xm.
Steps:
Make your home directory to store all the files for the Ubuntu operating system.
(root files)
Plug in your micro SD card onto your host laptop system.
Now we need to get the armel:
wget http://ynezz.ibawizard.net/beagleboard/natty/ubuntu-11.04-r1-minimal-
armel.tar.xz
md5sum ubuntu-11.04-r1-minimal-armel.tar.xz
tar –xvf ubuntu-11.04-r1-minimal-armel.tar.xz
Instead of the above commands, you can use the files in my folder ‘Ubuntu-
make’.
Now, enter the following:
cd ubuntu-11.04-r1-minimal-armel.tar.xz
sudo ./setup_sdcard.sh --mmc /dev/sdb --uboot beagle_bx --rootfstype ext3
Independent Study
20
This process includes formatting your SD card, transferring boot files into the
boot partition and the rootfilesystem onto the other partition of the memory
card.
Independent Study
21
When it is done, remove the SD card, and plug it back after 2-3 seconds.
You will see two partitions, as listed below.
Remove the SD card from the card holder and plug it in the Beagleboard and
switch on the power.
Connect the Serial (BeagleBoardxM) to USB(Host PC) converter and configure the minicom connection.
Start minicom on your host machine in configuration mode. As root: #minicom -o -s -w
· A menu of configuration should appear. Use the Down-arrow key to scroll down and select the Serial port setup option, and press Enter.
· Verify that the listed serial port is the same one that is connected to the target board. If it is not, press A, and enter the correct device. This is /dev/ttyUSB0 on most Linux distributions.
Independent Study
22
· Set the Bps/Par/Bits option by pressing the letter E and using the next menu toset the appropriate values. You press the key that corresponds to the value 115200, and then press Enter.
· Set Hardware flow control to No using the F key.
· Set Software flow control to No using the G key.
· Press Enter to return to the main configuration menu, and then press Esc to exit this menu.
Reset the board, and wait for a moment. If you do not see output from the board, press Enter several times until you see the prompt. If you do not see any output from the board, and have verified that the serial terminal connection is setup correctly, contact your board vendor.
You will see something like this when the kernel boots.
Once, it has booted, it will ask for a login, and a password.
The login is ‘ubuntu’, and the password is ‘temppwd’.
Independent Study
23
It is necessary that you update your kernel using the following command.
To get the Ethernet working, enter the following command.
sudo dhclient eth0
You can check your assigned ip-address using ifconfig –a
Independent Study
24
References:
1. Nisheeth Bhatt’s IS
http://ecee.colorado.edu/~ecen5623/ecen/labs/Linux/IS/Report.pdf
2. Srinivas Panchpakesan’s work on the Leopard camera.