RIT Senior Design Project 10662

D3 Engineering Camera Platform

Design Review October 9, 2009

Time: Friday October 10, 2009 11:30 to 13:00

Location: RIT Campus. Building 9 Room 2030

Project TeamGregory Hintz

Samuel Skalicky

Jeremy Greene

Jared Burdick

Michelle Bard

Anthony Perrone

AdvisorsBob Kremens (RIT)

Philip Bryan (RIT)

Scott Reardon (D3 Engineering)

Kevin Kearney (D3 Engineering)

1 | P a g e

I. Introduction

This project will combine several pieces of technology into a small configurable multi-spectral camera module with integrated spatial orientation sensors. The customer desires to create a small ruggedized signal processing platform that will withstand harsh environments and perform automated image capture. The goal of this project is to combine several existing pieces of hardware into a demonstration unit for the customer’s technology. (Taken from Project Readiness Package)

Figure 1: Electrical System overview

2 | P a g e

II. Power Distribution

FPGA DDR2OEM Board

Flash MEM

SSD INSD3

CamerasConnector

Board

1.2V 1.8V 5V 3.3V3.3V, 5V,

12V15V 3.3V 9-36V

*Power will not be provided on the Camera Link connector. The camera will receive power through a separate cable. Each camera manufacturer will define their own power connector, current, and voltage requirements. (Taken from Camera Link: Specifications of the Camera Link Interface Standard for Digital Cameras and Frame Grabbers)

3 | P a g e

Possible Step Down Power Schematics that will be used. (Taken from Linear Technology)

4 | P a g e

III. Camera Specifications

MT9J003 Camera Sensor

Imaging Array• Optical Format: 1/2.3-inch • Active Array: 3856(H) x 2764(V) (entire sensor)

3664(H) x 2748(V) (4:3 still mode) 3840(H) x 2160(V) (16:9 video mode)

• Imaging Area: 6.119mm(H) x 4.589mm(V)

Speed/Output• Frame Rate: 15 fps (HiSPi serial I/F)

7.5 fps (parallel I/F) • Data Rate: 2.8 Gb/s (HiSPi serial I/F)

80 Mp/s (parallel I/F)• Master Clock: 6–48 MHz• Data Format: 12-bit RAW

Sensitivity• Pixel Size: 1.67μm x 1.67μm• Dynamic Range: 65.2dB• Responsivity: 0.31 V/lux-sec (550nm)

Power• Supply: Analog: 2.4–3.1V (2.8V nominal)

Digital: 1.7–1.9V (1.8V nominal) I/O: 1.7–1.9V (1.8V nominal) or 2.4–3.1V (2.8V nominal) HiSPi Tx: 0.4–0.8V (0.4V or 0.8V nominal)

• Consumption: 638mW @ full resolution

Optics• CRA: 0 degree

Temperature Range• Operating: –30°C to +70°C

Package: 10mm x 10mm 48-pin iLCC

*Taken from Aptina Imaging

This camera was chosen because it takes a 10 Mpix still image as well as having a digital video mode that has a high resolution. It consumes very little power and the two interface options allow for a parallel output and a four-lane serial high speed pixel interface (HiSpi) differential signaling.

5 | P a g e

Protocols to be used D3 Camera Interface

-16-bit parallel output-6 Miscellaneous positions-Two wire I²C bus interface-Several clock and control positions

CameraLink -LVDS to achieve theoretical transmission rate of 1.923Gbps-Not dependent on a particular supply voltage because of low signal voltage swing

GigE-High bandwidth for high-speed, and high resolution cameras-Downward compatible with 10/100 Mhz Ethernet-Operates at a fast frame rate

6 | P a g e

IV. Connector Board

The connector board will be a circuit board that provides mounting for the various input and output ports used by the system. The original specification provided by D3 (see Figure 1) calls for several ports, including two D3 camera ports, two GigE camera ports, two CameraLink (LVDS) camera ports, a data sync port, a serial port (RS232) and a port for power in (9-36 Volts).

Figure 1: Initial block diagram of overall electrical system with expanded view of the connector board.

After initial considerations it was concluded that, rather than a single enclosure system, the logically separable portions of the system would indeed be split, resulting in a more modular design. The Inertial Navigation System, which will now be outside the main enclosure, will thus need a port on the connector board. In addition to the Serial ATA (SATA) Solid State Drive (SSD) depicted as internal in Figure 1, an SATA port will be included on the connector board to facilitate removable external storage. Therefore, two ports will be included in the final design, which are not in the initial.

7 | P a g e

Beyond simply providing a unified location for the data ports, the connector board will feature several Integrated Circuits (IC's), which could otherwise be on the main, FPGA board, but will be on connector board for space considerations and matters of simple convenience. The most noteworthy example will be IC's to convert the GigE and CameraLink interfaces to and from the D3 format, simplifying the FPGA design by requiring it to support only the D3 interface. (See Figure 2).

Depending upon design requirements, some of the ports may not be mounted on the connector board, instead mounting to the enclosure itself, with individual data buses to the FPGA board. A ribbon cable will transfer data from the FPGA board to the connector board and vice-versa.

Figure 2: The latest block diagram of the overall electrical system, with the connector board outlined, including ideas developed during brainstorming.

8 | P a g e

V. Inertial Navigation SystemAn Inertial Navigation System (INS) combines data from a GPS and from a 3-dimensional

directional sensor, the Inertial Measurement Unit (IMU), to provide a holistic notion of ones position. This is an important function to integrate into a camera system such as the one being designed here, so that if a series of overlapping images are taken, the degree of overlap, and any image skewing resultant from variations in pitch, roll and yaw (Figure 3), can be accounted for in post-processing. The customer, D3, recommended that the MiroStrain 3DM (Figure 1) and NovAtel SPAN (Figure 2) INS systems be considered.

The INS will be housed separately from the FPGA and other electrical systems, with the intent that different models – of different brand, expense and quality – can be easily exchanged. A common 9-pin RS232 connector will facilitate data exchange between the primary electrical enclosure and the INS.

The process of selecting which unit to use is still in its early phase. The price will certainly play a large role in deciding, considering the $745.00 price tag of the basic 3DM unit (the 3DM-GX1 unit in Figure 1 costs $1495.00). Additionally, some units come separate the IMU and the GPS, which will potentially have a number of implications upon the enclosure design. The initial INS selection is imminent.

9 | P a g e

Figure 1: NovAtel SPAN-CPT INS.Figure 1: MicroStrain 3DM-GX1 INS.

Figure 1: Graphic depicting roll, pitch and yaw in an airplane.

VI. Electrical Circuits Design

The Goal of this portion of the project is to have a solution to take pictures using cameras in high resolution, save the data onto a storage medium for the duration of the flight, and finally provide current image data of low quality through out the flight. We will be using a myriad of devices already functional and working, and integrate these devices into a single system.

Design

To solve this problem we need to design a system to interface many different devices. The most opportune way without using an off-the-shelf solution is to use some type of programmable logic. Two ideas are immediately apparent, a computer and a FPGA. FPGAs are standard equipment in many industries today and are the best and fastest way to get an idea to market. We will join this revolution to embed our processing in hardware, the faster way, yet still have our software configurability.

The main components in this system are the cameras, the storage medium (aka hard drive), and the processing FPGA. We are receiving a few various devices to integreate, these include: camera imagers, a prefabricated camera to FastEthernet controller board (OEM), and all IMU and GPS sensors. Using these devices we will be able to take a picture at an exact moment in time and know where the camera is located, which direction it is pointed in order to precisely identify of where the picture is being taken. So in effect the FPGA will be a large and complex multiplexor between the camera imagers, the hard drive, and the OEM board. This will also allow future software revisions to include some image processing onboard. A sample diagram of the overall electronic systems is shown on the next page, labeled Figure 1.

10 | P a g e

Figure 3

As you can see from figure 1, the system is broken up into two pieces, the board “to the external world”, ie. Connector Board, and the FPGA board that serves as the communications hub.

Since this design will use a “modular” interface in almost every way possible, we are including extra camera connecters of various variety onto the Connector board, this will allow the extention of the lifetime of this board, and will allow for future designs to be based in the software realm. A more detailed design of the FPGA Board is shown on the next page, labeled Figure 2.

11 | P a g e

Figure 4

As seen above, there are many other requirements that pop up when we decide to use an FPGA, for example, the Flash memory, the DRR ram, various clocks, etc. These items will need to be added on due to the size of the images we will be using and will also allow more freedom for later redesign of the software system.

To dig more into the FPGA, we have decided on the Spartan 6 line of FPGAs from Xilinx. To further narrow down, we have decided to implement the SATA control and the Gigabit Ethernet inside the FPGA, reducing the need of hardware to get the board manufactured more quickly. These requirements for us into the LXT line of Spartan 6 FPGAs. Of this group we have a few choices to consider. Please see the chart at the end of this document, labeled Appendix 1: Xilinx FPGA Selection. Also, we have created a list of potential needs for inputs/outputs to/from the FPGA, shown on the next page, labeled Figure 3.

12 | P a g e

Figure 5

13 | P a g e

Xilinx FPGA Selection

14 | P a g e

VII. Heat Mitigation ConsiderationsNeeds:

The main heat mitigation need of the system is to prevent damage due to improper operating temperature. The electronics must be kept from overheating or overcooling. In addition, any heat produced by the electronic components must not be allowed to interfere with the operation of the system’s cameras. These must be done while maintaining an air and watertight environment

Considerations:• External environment

• Temperature on ground : assume 40 to 70 degrees F (Rochester in May) • Temperature at 30,000 ft (5.7 miles, assuming an aircraft): -66.8 to -36.8

degrees F *• Internal environment

• External temperature plus temperature of heat generated by electronic components (TBD)

* Temperature drops about 3.56 Degrees F/1000 ft (http://en.wikipedia.org/wiki/Lapse_rate)

Approach:

To allow for individual temperature needs of each set of components to be met, the electronics will be thermally isolated from the optics. In addition, conductive heat transfer methods will be utilized inside the chassis to remove any excess heat from the system while still maintaining an air and watertight environment. If necessary, passive convective heat transfer techniques may be

utilized to remove any excess heat from the outside of the chassis.

15 | P a g e

Basic schematic of separate enclosures for components

Possible conductive heat transfer method: all boards mounted to a single conductive backbone mounted in chassis

T1=0oC T2=-36oC

q

Preliminary Conductive Heat transfer analysis:

Temp. range of boards: 0oC < T < 85oCOutside temp : -54oC < T <-36oC

Assume:Temperature at each surface is equal to the temperature of the air it touchesMaterial is Aluminum, K=250 W/MKThickness of chassis walls = .00635m (.25 in)Negligible heat transfer through bottom of enclosure (at interface with optics enclosure) Electronics enclosure is .1524m x .1524m x .1016m (6 in x 6 in x 4 in) with a total exposed surface area of .0852m2 Perform analysis on minimum temperatures for boards and maximum temperature of atmosphere

Schematic:

Analysis:

Heat transfer rate in Watts (q) is the product of exposed surface area in square meters (A), the thickness of the conductive material in meters (s), the temperature difference across the material in degrees Kelvin (dT), and the thermal conductivity of the material in watts per meter Kelvin (k)

q =(k)(A)(dT)(1/s) in this case:

q= (250 w/mK)(.0852m2)(36k)(1/(.00635m))q = 120.755 kW

16 | P a g e

VIII. Mounting to Airframe

Needs:The main needs for the airframe mount are to ensure the imaging system is securely attached to airframe. It should also reduce any vibration transferred to the system by the airframe.

Considerations: Any pre-existing bolt patterns in the aircraft should be taken into consideration. Pre-existing openings in the aircraft (perhaps used to house previous aerial imaging systems) should also be considered. In addition, the mounting should not interfere with other components of imaging system.

Approach:The airplane’s pre-existing bolt pattern should be utilized in vibration damping mount to attach vibration damping mount directly to airframe.

17 | P a g e

IX. Vibrational DampingNeeds

Stabilize Image

Reduce vibrations as much as possible to maximize image clarity

Prevent Hardware Damage/Malfunctioning

Wire connectors could shake loose, adjacent rigid components could damage one another, and the signal-damaging effects of a loose contact could be amplified.

Considerations

Frequencies of Aircraft

While Cruising- The low-amplitude vibrations of an operating aircraft will be present during the entire duration of the flight.

During Turbulence – Some level of turbulence should be accounted for so that when it inevitably shows up, the image will remain clear and hardware will continue undamaged.

Take-off and Landing Accelerations- Not a vibration, per se, but these scenarios will create unbalanced forces on the hardware that will need to be softened.

All of these considerations are variable in nature, depending heavily on the aircraft being used. Aircraft size and stability are key considerations. Analysis should cover a reasonable range of aircraft, possibly spanning UAVs up to larger private planes.

Allowable Vibration in Image

Flights will typically take images at an altitude of about a thousand feet. At this distance, lateral translating motion of the camera will have small effects on image quality while a rotating motion that causes the camera to “sweep” will have drastic effect. From these considerations, allowable camera motion can be estimated.

Component Resonant Frequencies

Most parts are small and rigid, causing resonant frequencies to be higher than will likely be experienced. This category is not to be forgotten, as more detailed data may reveal otherwise, but it is very low on the scale of actual risk of damaging the system.

18 | P a g e

Approach

Mechanically isolate camera chassis from points of mounting: no “Hard” points of contact

Stock damping hardware can easily be replaced to meet needs of a variety of vibration scenarios

19 | P a g e

Rubber Damping MountsOptical Enclosure

Electronics Enclosure

X. Chassis DesignPhase 1: Individual Compartments

20 | P a g e

Phase 2: Assure Component Scale

Above: Dummy solids of major electrical components fit snugly in a 5.5” x 5.5” x 3” space. Below, four customer-specified lenses fit well into an enclosure of similar cross section.

21 | P a g e

Phase 3: Detailed design to allow for realistic thermal, vibrational, and spatial analysis.

- Vibrational dampers mount on center flange

- Interial grooves allow for component mounting to be modular, changeable, and secure

- Stock extruded enclosure reduces build time

- “Stacked” configuration maintains thermal separation at a minimal “footprint”

22 | P a g e