An Embedded System of Real-Time Acquisition and Display of Images

In this paper, we propose a design of an embedded system which based on CPLD (Complex Programmable Logic Device) and only one chip of SRAM (Static RAM) in order to acquire and display the images real-time. The system uses Omni Vision’s CMOS color image sensor OV7725 as the image acquisition chip and a 5.6-inch TFT-LCD (Thin Film Transistor Liquid Crystal Display), 18bit (RGB666) colors with resolution of 640 × 480 pixels, to display the images real-time, and the device of CPLD (EPM7160E) to control the timing, image format conversion, and TFT-LCD drive signal. At the same time the embedded system use one chip of SRAM as a cache memory to save an image quickly. The experimental results show that the system designed, with high practical value but less hardware, is easy to capture and display the images stably.


Introduction
As embedded systems are presses to deliver greater real-time performance in smaller form-factors with less power, designers must increasingly turn to application-specific design solutions.With the development of CMOS sensor, the embedded image-acquisition system develops fast.A LCD is a standard display device for hand-held embedded systems.As a result of it, the LCD is used for the embedded display system more often.
In tradition, there are several methods to drive the TFT-LCD, such as taking the FPGA and Double Access RAM, and even use two chips of SRAM in Ping-pong mechanism.In these ways, the system wastes too much hardware resource.On the other hand, the continuous images and video data usually transfer throughout the system by the modes of VGA (Video Graphics Array) interface and AV (Audio and Video Composite video connector) interface, the system which transfers the image and video by these interface needs the ADC (Analog to Digital Converter) and DAC (Digital to Analog Converter) of image and video data, and the coder and decoder devices, this solution also increase the cost of the whole embedded system.
For the above reasons, we propose a design of an embedded system for image acquisition and display.In this system, we use a CMOS sensor to acquire images, a TFT-LCD to show the images, a CPLD and only one chip of SRAM to accomplish the functions.

The hardware of system
The embedded system uses CPLD as the core unit to control the timing of this whole system, transform the data format of the real-time images, and control the SRAM in order to read and write the data of the real-time images, at last, the CPLD also generates the signals to drive the TFT-LCD in order to show the images on the screen of LCD.
We use the VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) with the software MAX+PLUSⅡ, a development of integrated environment provided by the Altera for FPGA (Field-Programmable Gate Array) and CPLD, to design the functions of the CPLD in the embedded system.
The structure of the embedded system is shown in Figure 1.
In this system, we take the OV7725 as the main CMOS image senor device.The OV7725 Camera Chip sensor is a high performance 1/4 inch, single-chip VGA camera and image processor in a small footprint package.The OV7725 incorporates a 640×480 image array, capable of operating at 60 frames per second in VGA mode with complete user control over image quality, formatting and output data transfer.It also has automatic image control functions including AEC (automatic exposure control), AGC (automatic gain control), AWB (automatic white balance), ABF (automatic band filter), and ABLC (automatic blacked-level calibration).In order to make the OV7725 to work, we should set the register by standard SCCB interface.
We choose the EPM7160E serious CPLD as the main control chip.Which is a product of Altera, there are 3,200 usable gates, 160 macro cells and 104 I/O pins (maximum) in this chip.
In this system, the image we acquired is a constitution of 640×480 pixels with the format of RGB555.A frame image takes 307,200 words by 16bit.In order to save the data of a frame image, we use the IS61LV51216, A production of ISSI, a high-speed, 8M-bit static RAM organized as 525,288 words by 16 bits.
We use a 5.6-inch digital TFT-LCD, provided by the InnoLux, as the device of displaying the images.This TFT-LCD has 18bit colors in depth, and has a resolution of 640×480 pixels.

The settings of OV7725
The OV7725 has an image array capable of operating at up to 60 frames per second (fps) in VGA with complete user control over image quality, formatting and output data transfer.Its image quality controls include color saturation, hue, gamma, sharpness (edge enhancement level auto adjust), and anti-blooming.The OV7725 should be set by standard SCCB interface in order to work.In this system, we should set several registers, and other registers could be set as the default value.In the embedded system, the OV7725 should output the image data with the mode of 640×480 pixels, RGB555, and 60fps.
In order to output the image in the mode of 640×480 pixels, six registers should be set in the table 1.
In order to output the data in the format of RGB555, the register COM7, address in 0x12, should be set as 0x0A.
In order to output 60 frames per second, the 48MHz crystal should be used as the main system input clock for the OV7725.
The sequence of the output image data is from left to right and from up and down, each line has 640 effective pixels, and each frame image has effective 480 lines.The scanning sequence of a frame image is shown in the Figure 2.
In this system, we use the MCU (Micro Controller Unit) to set the registers of OV7725 before it is used to collect the images.Because of the method of setting registers through SCCB bus and most MCU contains I 2 C module rather than SCCB module, we use the MSP430F149, a production of TI, to simulate the SCCB bus in order to set the OV7725.
The VGA frame timing diagram is shown in the Figure 3.Because of one pixels is made up of two bytes in the mode of VGA and RGB555, one HERF signal contains 1280 clocks rather than 640 clocks in the mode of QVGA in RGB555.And the image data output pins, D9-D2, output the 8-bit data.The data format is shown in the table 2.

The TFT-LCD
The TFT-LCD used in the embedded system has a constitution of 640×480, and could show 18bit colors in one pixel.The drive signals of this LCD are made up of VSYNC, HSYNC, DE (or HV), PIXCLK and RGB data in 18bit, R [5:0], G [5:0], and B [5:0].The logical supply power is 3.3V and the power for the back LED is 5V.The timing diagram is shown in the Figure 4.A frame image being displayed in the screen of LCD has 525 lines, and there are default 13 lines before the effective lines and 32 lines after the effective lines which contains 480 lines; At the same time, each line signal has 800 CLKINs, and the effective RGB data contains 640 CLKINs, there are default 144 CLKINs invalid before the effective CLKINs and 16 CLKINs invalid after the effective CLKINs.This LCD could work in the modes of DE or HV.In this system, we use the DE mode; the signal of DE decides the effective CLKINs which require the RGB data in each line.The display method of LCD could be set by the L/R and U/D.L/R decides whether the display works from left to right or right to left, and the U/R decides the LCD display from up to down or down to up.The settings of these two pins could achieve the screen flipping.

The function of CPLD
As it shown in the Figure 1, the CPLD has four modules: Decoder, Write SRAM, Read SRAM, and TFT-LCD Driver.

The Decoder Module
The Decoder module of the CPLD is to acquire and transform the data format of image.In digital image, the value of one color could be made up by values of Red, Green, and Blue.The OV7725 outputs the image data format with RGB555 in two 8bit bytes, as well as the controlling signals of VSYNC, HREF, and PCLK.Firstly, we should acquire the effective RGB data through these controlling signals; secondly, we should transform the data into a 16bit word in order to save and process in the further.The origin data format is shown in table 2. The highest bit in the first byte is not used in the mode of RGB555.
The transformed data format is shown in the table 3.That is, we merge the first byte and the second byte into a word, and the highest bit is not used.

The Write SRAM Module
While receiving the data, CPLD sets the SRAM as the mode of write-only by the control signal of WE, OE, and CE.At the same time, the Wring SRAM module saves the image data into the IS61LV51216 with the method of cumulative.For example, The RGB data of the first pixels, the first line and the first dot, is saved at the address 0X00000, and the second RGB data of the second pixel, the first line and the second dot, is saved at the address 0X00001, and so on.Followed by analogy, the last RGB data of the whole image, the 640th line and the 480th dot, is saved at the address 0X4AFFF.The address and data are shown in the table.4.
These address signals are generated by the count of the clock input.In this system, we use the 48MHz crystal to generate the clock signal for the OV7725 and the CPLD.

The Read SRAM and TFT-LCD Driving Modules
When a frame image saved successfully, the CPLD gives the control signal of WE, OE, and CE, to set the SRAM only to be read and not to be written.The Read SRAM and the TFT-LCD driver modules of the CPLD then read the data by cumulative.Because the TFT-LCD has 18bit colors with RGB666, but the image data saved in the SRAM is RGB555, we set the lowest bit of R[5:0], G[5:0], and B[5:0] as '0'.That is, the R5 sent to the TFT-LCD is the R4 saved in the SRAM.The R(n) sent to the TFT-LCD is the R(n-1) saved in the SRAM.(n=1.2.3.4.5.)And the R0 sent to the TFT-LCD is '0', as well as G[5:0] and B[5:0].With the timing signals of VSYNC, HSYNC, DE, PIXCLK, the image data, made up of the format RGB666, also be sent to the TFT-LCD, the TFT-LCD received these signals and dates to show the image clearly.

The implementation of the embedded system
Throughout the system, the output images data of OV7725 are 60fps, that is to say, the OV7725 outputs 60 images in one second.But in the application of real-time display, according to the human eye's visual effect of temporary stay, as long as 24 frame images in one second the human would feel that the images are changing smoothly and continuous, and the smear, flicker, pause could not be recognized.In order to save the hardware resource without decreasing the effective of the images showing, we use 30 images, one half the output images of the OV7725.The specific process is like this: When the OV7725 outputs the first frame image, the SRAM is set by the WE, CE, and OE to the write-only status, the Decoder and Writing SRAM modules of CPLD transform and save this frame image into the SRAM.When a frame image saving is finished successfully, the CPLD then gives control signals, WE, CE, and OE, to set the SRAM for read-only status.The Read SRAM and TFT-LCD driving modules read the data from SRAM and send the data with the TFT-LCD driving signals, VSYNC, HSYNC, DE, and PIXCLK, to TFT-LCD.In this reading process, the OV7725 is still output the second frame image, but the SRAM is set to be read only, so the system does not acquire this image data, that is, the system discard this frame image and does not save it.When the TFT-LCD of the system finished the showing a frame image, the CPLD then sets the SRAM to be written-only, and then acquires the third frame like the first frame.In the time of writing SRAM, the TFT-LCD does not refresh the image until the SRAM is set to read-only again.
In this method, OV7725 output 60 frames images and we only display one half, 30 frames.And it does not have influenced our visual and later process, such as detecting the moving objects and the process of real-time robotic vision.

Conclusion
Through using this real-time acquisition and display embedded system, we find that the images are displayed continuous, without flicker and lose frame phenomenon.Experiment confirmed that the embedded system of using CMOS to acquisition and the LCD to display the images based on CPLD and one SRAM is possible.In this way, the embedded system decreases the hardware resource, and has a high relevance in a particular field.At the same time, the images are saved in the SRAM, therefore, the embedded system can be expanded with the high performance micro-processor, such as DSP (Digital Signal Process) chip, ARM (Advanced RISC Machines), FPGA and so on, to do the further process well.For example, the license plate recognition of the cars, the target objects detection.

Figure 1 .Figure 4 .
Figure 1.The hardware of the embedded system

Table 1 .
. The Design of Central Control Unit of LCD Controller.2006.6,Microelectronics&Computer. The Registers and Value

Table 2 .
The output data format of OV7725

Table 3 .
The data format saved in one address