COMPUTER SYSTEM

LABORATORYLab14 – Camera

/ 142Lab 14

Experimental Goal•Learn how to use DSP and CMOS sensor to take pictures on PXA270 by using the miniGUI from GPRS driver.

2013/12/24

TIDSP-5502

CMOS Sensor

/ 143Lab 14

Environment•Host System

• Windows XP

•Build System• VirtualBox + Ubuntu 8.04

•Target System• Creator XScale PXA270

•Software• LCD, GPRS and DSP Drivers• JPEG library and GPRS program

•You can download all software from RSWiki CSL Course Software2013/12/24

/ 144Lab 14

Hardware Connection•PXA270•TIDSP-5502•CMOS Sensor

2013/12/24

CMOS

TIDSP-5502

/ 145Lab 14

Introduction to DSP•A digital signal processor (DSP) is a specialized microprocessor with an architecture optimized for the operational needs of digital signal processing.

•Digital signal processing algorithms typically require a large number of mathematical operations to be performed quickly and repeatedly on a series of data samples.

2013/12/24

Reference: wiki info: DSP, en.wikipedia.org/wiki/Digital_signal_processor

/ 146Lab 14

Introduction to CMOS•An active-pixel sensor (APS) is an image sensor consisting of an integrated circuit containing an array of pixel sensors, each pixel containing a photodetector and an active amplifier.

•There are many types of active pixel sensors including the CMOS APS used most commonly in cell phone cameras, web cameras, etc.

•Such an image sensor is produced by a CMOS process (and is hence also known as a CMOS sensor), and has emerged as an alternative to charge-coupled device (CCD) image sensors.

2013/12/24

Reference: wiki info: APS, en.wikipedia.org/wiki/Active_pixel_sensor

/ 147Lab 14

Lab Steps (1/5)•Step 1: Download drivers2.tar.bz2 and refer to Lab12 to add the following drivers to the kernel.• Creator-pxa270 DSP5502• Creator-pxa270 LCD• Creator-pxa270 GPRS• You can build them as modules.

•Step 2: Create related device files if they does not exist.• % mknod /dev/lcd c 120 0• % mknod /dev/gprs c 122 0• % mknod /dev/mtdsp c 123 0

2013/12/24

/ 148Lab 14

Lab Steps (2/5)• Step 3: Download gprs-remote-ctrl.tar.bz2.• Step 4: Modify configuration files and Makefile.

• In gprs-ctrl-build/jpeg-6b/microtime_configure,• Delete “.$HOME/bin/microtime”.• Modify “--enable-shared” to “--enable-static”.• Add the following to the front of the file.

MICROTIME_USR_LOCAL_DIR=<jpeg-6b installation path>mkdir -p $MICROTIME_USR_LOCAL_DIR

mkdir -p $MICROTIME_USR_LOCAL_DIR/bin

mkdir -p $MICROTIME_USR_LOCAL_DIR/include

mkdir -p $MICROTIME_USR_LOCAL_DIR/lib

mkdir -p $MICROTIME_USR_LOCAL_DIR/man

mkdir -p $MICROTIME_USR_LOCAL_DIR/man/man1

2013/12/24

/ 149Lab 14

Lab Steps (3/5)• In gprs-ctrl-build/creator-pxa270-gprs-1.0/microtime_configure,

• Delete “.$HOME/bin/microtime”.• Delete “export LINUXDIR=$MICROTIME_LINUX_DIR”.• Add the following to the front of the file.

export MICROTIME_USR_LOCAL_DIR=<jpeg-6b installation path>export MICROTIME_BUILD_PACKAGE_DIR=<path of gprs-ctrl-build>export LINUXDIR=<path of pxa270/linux>

• In gprs-ctrl-build/creator-pxa270-gprs-1.0/Makefile,• Add a new flags “-static” at the end of “CFLAGS”.

• In gprs-ctrl-build/creator-pxa270-gprs-1.0/creator-pxa270-gprs.c,• Delete the code of initializing UART about 5 lines at line 227.

2013/12/24

/ 1410Lab 14

Lab Steps (4/5)• Step 5: Compile the JPEG library.

• Execute “gprs-ctrl-build/jpeg-6b/microtime_configure”.• Modify the generated Makefile,

where “AR= arm-unknown-linux-gnu-ar rc”, but not “AR= ar rc”.• % make libjpeg.a

• % cp libjpeg.a <jpeg-6b installation path>/lib

• Step 6: Compile the GPRS program.• Execute “gprs-ctrl-build/creator-pxa270-gprs-1.0/microtime_configure”.• Copy “cmosr.out.bin” and “creator-pxa270-gprs” to PXA270.

• Step 7: Connect DSP module and CMOS to PXA270 and execute.• $ ./creator-pxa270-gprs

• If you use drivers as modules, don’t forget to insmod modules before executing.• By pressing keypad 7, the CMOS sensor will take a picture, and store it to PXA270 which is

named “dsp_data.jpg”.

2013/12/24

/ 1411Lab 14

Lab Steps (5/5)•Step 8: Display the image on the LCD at the same time.

• On PXA270, there is a LCD device which can display not only messages but also a 128*128 image with 4 gray levels.

• We take a colored picture by CMOS and save the image in JPEG format.

2013/12/24

/ 1412Lab 14

Hint (1/2)•The LCD driver has already provided an ioctl command to display image on LCD.• You can implement a new function in creator_lib.c, and use this command

in GPRS program.

•You can refer to the example source code, i.e., lcd.c, to see how to set data to a LCD image buffer.

•Since the captured image is colored, i.e., each pixel consists of three components: red(R), green(G) and blue(B), if you want to display on the LCD, please convert it to 4 gray levels.• Any reasonable conversion is acceptable.• E.g., convert RGB color space to YUV color space, then quantize the

luminance Y to 4 gray levels or apply edge detection filter, then convert the edges to black and white, etc.

2013/12/24

/ 1413Lab 14

Hint (2/2)•For more information, please refer to the following sites:

• Grayscale, http://en.wikipedia.org/wiki/Grayscale• RGB, http://en.wikipedia.org/wiki/RGB_color_model• YUV, http://en.wikipedia.org/wiki/YUV• Edge detection, http://en.wikipedia.org/wiki/Edge_detection

•Note that the resolution of image (320*240) is different from LCD (128*128), so you can scale the resolution before you display.• Any reasonable scaling method is acceptable.• E.g., nearest neighbor, http://

tech-algorithm.com/articles/nearest-neighbor-image-scaling/ or bilinear, http://tech-algorithm.com/articles/bilinear-image-scaling/

2013/12/24

/ 1414Lab 14

Lab Requirement•Take pictures by the CMOS.

• Please modify the DSP driver (creator-pxa270-dsp5502.c) and GPRS program (dsp.c) to take 320*240 pictures.

• The default resolution is 160*120.

•Display the image on PC and on the LCD.• You can use tftp to upload your image file.

• $ tftp -p -l <file> <ip address>

2013/12/24