在基于Linux的视频监控采集系统中,摄像头采集到的一帧视频图像数据一般都是经过硬件自动压缩成jpeg格式的,然后再保存到摄像头设备的缓冲区.如果要把采集到的jpeg格式显示在本地LCD屏上,由于我们的Linux系统没有移植任何GUI系统,就要考虑以下方面:
1. 将jpeg格式解压缩为位图格式,也就是jpeg解码.
2. 将解码出来的位图格式输出到本地的LCD屏上. 在Linux系统下是通过写入帧缓冲(framebuffer)来实现的.
3. framebuffer相当于为LCD设备提供一个统一的接口,对framebuffer的操控会反映到LCD显示设备上去. 如果配置Linux内核时没有找到支持本地lcd屏这种型号的驱动,那我们要自己写lcd屏驱动,然后选择静态加入内核或以模块的形式加入内核动态加载.
针对以上三点,我们逐一解决:
1. jpeg解码
先了解一下jpeg标准的编码过程:原始的一帧未压缩过的图像可以看成是RGB(红绿蓝)色彩空间上的一组向量集合,但在RGB空间是不利于数据压缩的,因此为了压缩先要把图像映射到利于压缩的YUV空间上(原因是因为人类的眼睛对于亮度差异的敏感度高于色彩变化,而YUV空间的一个基向量Y就是亮度), 这一步叫色彩空间转换.下一步可以在YUV空间上减少U(色调)和V(饱和度)的成分,也就是在亮度信息不减少的情况下移除部分色彩信息,谁叫人的眼睛对亮度的敏感优于对色彩的敏感呢.这一步叫缩减取样.下一步是将图像从色彩空间映射到频率空间,可以采用的变换方法有:离散余弦变换, 傅氏变换, 正弦变换等. 其中应用最广的是离散余弦变换(DCT).这一步是无损的,目的是为了在下一步称之为量化的过程中可以经过四舍五入删除高频量得到压缩后的矩阵.量化之后就是对这个矩阵的编码问题了.针对这个矩阵的分布特点, 采用”Z”字形的顺序扫描编排,然后进行RLE行程编码, 把大量连续重复的数据压缩.最后再用范式Huffman编码.要了解详细的过程,可以查看JPEG标准.
而解码就是以上编码的逆过程了.除非想要自己实现jpeg的编码和解码函数,我们可以不必细究这些过程,而是直接使用别人已经实现的jpeg编码解码库.在Linux平台下, 有libjpeg库, 它是完全用C语言编写的, 依照它的许可协议,可*使用, 不是GPL协议,它可以用于商业目的.
libjpeg的6b版本有个问题,就是解码接口,它只接受文件源.打开源的函数jpeg_stdio_src(j_decompress_ptr cinfo, FILE *infile)要求解码源infile是文件.而我们希望解码的是直接来自映射内存中的数据.要解码内存流的话就要修改libjpeg的源码了,可以参考这里:http://my.unix-center.net/~Simon_fu/?p=565 目前libjpeg的最新版8c已经解决了这个接口不好的问题了,它增加了对内存流解码的支持.通过调用函数
jpeg_mem_src(&cinfo, fdmem, st.st_size);
就可以将保存在内存的jpeg格式数据作为源输入了.因此我们就用libjpeg 8c这个版本来解码.
用到的函数主要有:
- 初始化jpeg解压对象:
/* init jpeg decompress object error handler */ cinfo.err = jpeg_std_error(&jerr); jpeg_create_decompress(&cinfo);
- 绑定jpeg解压对象到输入流:
/* bind jpeg decompress object to infile */ #if READ_FILE // 从jpeg文件读入 jpeg_stdio_src(&cinfo, infile); #elif READ_MEM // 从内存读入jpeg格式 jpeg_mem_src(&cinfo, fdmem, st.st_size); #endif
- 读取jpeg头部信息:
/* read jpeg header */ jpeg_read_header(&cinfo, TRUE);
- 解压过程:
/* decompress process */ jpeg_start_decompress(&cinfo);
调用这个函数之后,可以得到jpeg图像的下面几个参数:
- output_width // 图像的宽度
- output_height // 图像的高度
- output_components // 每个像素占用的字节数
我们采用每扫描一行像素就输出到屏幕的方法的话,根据以上参数可以确定分配一行信息需要的缓冲区:
buffer = (unsigned char *)malloc(cinfo.output_width * cinfo.output_components);
总共需要扫描output_height行.
- 读取一行扫描数据并输出到LCD屏幕:
y = 0; while (cinfo.output_scanline < cinfo.output_height) { jpeg_read_scanlines(&cinfo, &buffer, 1); if (fb_depth == 16) { // 如果显示设备色深是16位 ... } else if (fb_depth == 24) { // 如果显示设备色深是24位 ... } else if (fb_depth == 32) { // 如果显示设备色深是32位 ... } y++; } - 结束jpeg解码:
/* finish decompress, destroy decompress object */ jpeg_finish_decompress(&cinfo); jpeg_destroy_decompress(&cinfo);
- 释放缓冲区:
/* release memory buffer */ free(buffer);
2. 输出位图到LCD屏
通过framebuffer直接写屏的主要步骤有:
- 打开framebuffer设备:
/* open framebuffer device */ fbdev = fb_open("/dev/fb0"); - 获取framebuffer设备参数:
/* get status of framebuffer device */ fb_stat(fbdev, &fb_width, &fb_height, &fb_depth);
- 映射framebuffer设备到共享内存:
screensize = fb_width * fb_height * fb_depth / 8; fbmem = fb_mmap(fbdev, screensize);
- 直接对映射到那片内存进行写操作,LCD屏刷新刷新时就会反应到屏幕上去了.
y = 0; while (cinfo.output_scanline < cinfo.output_height) { jpeg_read_scanlines(&cinfo, &buffer, 1); if (fb_depth == 16) { unsigned short color; for (x = 0; x < cinfo.output_width; x++) { color = RGB888toRGB565(buffer[x * 3], buffer[x * 3 + 1], buffer[x * 3 + 2]); fb_pixel(fbmem, fb_width, fb_height, x, y, color); } } else if (fb_depth == 24) { memcpy((unsigned char *) fbmem + y * fb_width * 3, buffer, cinfo.output_width * cinfo.output_components); } else if (fb_depth == 32) { // memcpy((unsigned char *) fbmem + y * fb_width * 4, // buffer, cinfo.output_width * cinfo.output_components); for (x = 0; x < cinfo.output_width; x++) { *(fbmem + y * fb_width * 4 + x * 4) = (unsigned char) buffer[x * 3 + 2]; *(fbmem + y * fb_width * 4 + x * 4 + 1) = (unsigned char) buffer[x * 3 + 1]; *(fbmem + y * fb_width * 4 + x * 4 + 2) = (unsigned char) buffer[x * 3 + 0]; *(fbmem + y * fb_width * 4 + x * 4 + 3) = (unsigned char) 0; } } y++; // next scanline } - 卸载映射framebuffer的那部分内存:
/* unmap framebuffer's shared memory */ fb_munmap(fbmem, screensize);
- 关闭framebuffer设备:
close(fbdev);
根据以上两点,可以写一个测试程序,在不开X-window图形系统的情况下,将本地的jpeg文件直接显示到屏幕上.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <linux/fb.h>
#include "jpeglib.h"
#include "jerror.h"
#define FB_DEV "/dev/fb0"
#define __fnc__ __FUNCTION__
#define debug 0
#define debug_printf 0
#define BYREAD 0
#define BYMEM 1
/* function deciaration */
void usage(char *msg);
unsigned short RGB888toRGB565(unsigned char red,
unsigned char green, unsigned char blue);
int fb_open(char *fb_device);
int fb_close(int fd);
int fb_stat(int fd, unsigned int *width, unsigned int *height, unsigned int * depth);
void *fb_mmap(int fd, unsigned int screensize);
void *fd_mmap(int fd, unsigned int filesize);
int fb_munmap(void *start, size_t length);
int fb_pixel(void *fbmem, int width, int height,
int x, int y, unsigned short color);
#if(debug)
void draw(unsigned char *fbp,
struct fb_var_screeninfo vinfo,
struct fb_fix_screeninfo finfo);
#endif
/* function implementation */
int main(int argc, char **argv)
{
struct jpeg_decompress_struct cinfo;
struct jpeg_error_mgr jerr;
#if(BYREAD)
FILE *infile;
#endif
int fd;
unsigned char *buffer;
struct stat st;
int fbdev;
char *fb_device;
unsigned char *fbmem;
unsigned char *fdmem;
unsigned int screensize;
unsigned int fb_width;
unsigned int fb_height;
unsigned int fb_depth;
register unsigned int x;
register unsigned int y;
/* check auguments */
if (argc != 2) {
usage("insuffient auguments");
exit(-1);
}
/* open framebuffer device */
if ((fb_device = getenv("FRAMEBUFFER")) == NULL)
fb_device = FB_DEV;
fbdev = fb_open(fb_device);
/* get status of framebuffer device */
fb_stat(fbdev, &fb_width, &fb_height, &fb_depth);
/* map framebuffer device to shared memory */
screensize = fb_width * fb_height * fb_depth / 8;
fbmem = fb_mmap(fbdev, screensize);
#if (BYREAD)
/* open input jpeg file */
if ((infile = fopen(argv[1], "rb")) == NULL) {
fprintf(stderr, "open %s failed\n", argv[1]);
exit(-1);
}
#endif
if ((fd = open(argv[1], O_RDONLY)) < 0) {
perror("open");
exit(-1);
}
if (fstat(fd, &st) < 0) {
perror("fstat");
exit(-1);
}
fdmem = fd_mmap(fd, st.st_size);
/* init jpeg decompress object error handler */
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_decompress(&cinfo);
/* bind jpeg decompress object to infile */
#if (BYREAD)
jpeg_stdio_src(&cinfo, infile);
#endif
#if (BYMEM)
jpeg_mem_src(&cinfo, fdmem, st.st_size);
#endif
/* read jpeg header */
jpeg_read_header(&cinfo, TRUE);
/* decompress process */
jpeg_start_decompress(&cinfo);
if ((cinfo.output_width > fb_width) ||
(cinfo.output_height > fb_height)) {
printf("too large jpeg file, can't display\n");
#if (0)
return -1;
#endif
}
buffer = (unsigned char *) malloc(cinfo.output_width *
cinfo.output_components);
struct fb_fix_screeninfo fb_finfo;
struct fb_var_screeninfo fb_vinfo;
if (ioctl(fbdev, FBIOGET_FSCREENINFO, &fb_finfo)) {
perror(__fnc__);
return -1;
}
if (ioctl(fbdev, FBIOGET_VSCREENINFO, &fb_vinfo)) {
perror(__fnc__);
return -1;
}
#if(debug)
draw(fbmem, fb_vinfo, fb_finfo);
#endif
y = 0;
while (cinfo.output_scanline < cinfo.output_height) {
jpeg_read_scanlines(&cinfo, &buffer, 1);
if (fb_depth == 16) {
unsigned short color;
for (x = 0; x < cinfo.output_width; x++) {
color =
RGB888toRGB565(buffer[x * 3],
buffer[x * 3 + 1], buffer[x * 3 + 2]);
fb_pixel(fbmem, fb_width, fb_height, x, y, color);
}
} else if (fb_depth == 24) {
memcpy((unsigned char *) fbmem + y * fb_width * 3,
buffer, cinfo.output_width * cinfo.output_components);
} else if (fb_depth == 32) {
// memcpy((unsigned char *) fbmem + y * fb_width * 4,
// buffer, cinfo.output_width * cinfo.output_components);
for (x = 0; x < cinfo.output_width; x++) {
* (fbmem + y * fb_width * 4 + x * 4) = (unsigned char) buffer[x * 3 + 2];
* (fbmem + y * fb_width * 4 + x * 4 + 1) = (unsigned char) buffer[x * 3 + 1];
* (fbmem + y * fb_width * 4 + x * 4 + 2) = (unsigned char) buffer[x * 3 + 0];
* (fbmem + y * fb_width * 4 + x * 4 + 3) = (unsigned char) 0;
}
}
y++; // next scanline
}
/* finish decompress, destroy decompress object */
jpeg_finish_decompress(&cinfo);
jpeg_destroy_decompress(&cinfo);
/* release memory buffer */
free(buffer);
#if (BYREAD)
/* close jpeg inputing file */
fclose(infile);
#endif
/* unmap framebuffer's shared memory */
fb_munmap(fbmem, screensize);
#if (BYMEM)
munmap(fdmem, (size_t) st.st_size);
close(fd);
#endif
/* close framebuffer device */
fb_close(fbdev);
return 0;
}
void usage(char *msg)
{
fprintf(stderr, "%s\n", msg);
printf("Usage: fv some-jpeg-file.jpg\n");
}
/* open framebuffer device.
* return positive file descriptor if success,
* else return -1
*/
int fb_open(char *fb_device)
{
int fd;
if ((fd = open(fb_device, O_RDWR)) < 0) {
perror(__fnc__);
return -1;
}
return fd;
}
int fb_close(int fd)
{
return (close(fd));
}
/* get framebuffer's width, height, and depth.
* return 0 if success, else return -1.
*/
int fb_stat(int fd, unsigned int *width, unsigned int *height, unsigned int * depth)
{
struct fb_fix_screeninfo fb_finfo;
struct fb_var_screeninfo fb_vinfo;
if (ioctl(fd, FBIOGET_FSCREENINFO, &fb_finfo)) {
perror(__fnc__);
return -1;
}
if (ioctl(fd, FBIOGET_VSCREENINFO, &fb_vinfo)) {
perror(__fnc__);
return -1;
}
*width = fb_vinfo.xres;
*height = fb_vinfo.yres;
*depth = fb_vinfo.bits_per_pixel;
return 0;
}
/* map shared memory to framebuffer device.
* return maped memory if success
* else return -1, as mmap dose
*/
void *fb_mmap(int fd, unsigned int screensize)
{
caddr_t fbmem;
if ((fbmem = mmap(0, screensize, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0)) == MAP_FAILED) {
perror(__func__);
return (void *) (-1);
}
return fbmem;
}
/* map shared memmory to a opened file */
void *fd_mmap(int fd, unsigned int filesize)
{
caddr_t fdmem;
if ((fdmem = mmap(0, filesize, PROT_READ,
MAP_SHARED, fd, 0)) == MAP_FAILED) {
perror(__func__);
return (void *) (-1);
}
return fdmem;
}
/* unmap map memory for framebuffer device */
int fb_munmap(void *start, size_t length)
{
return (munmap(start, length));
}
/* convert 24bit RGB888 to 16bit RGB565 color format */
unsigned short RGB888toRGB565(unsigned char red,
unsigned char green, unsigned char blue)
{
unsigned short B = (blue >> 3) & 0x001F;
unsigned short G = ((green >> 2) << 5) & 0x07E0;
unsigned short R = ((red >> 3) << 11) & 0xF800;
return (unsigned short) (R | G | B);
}
/* display a pixel on the framebuffer device.
* fbmem is the starting memory of framebuffer,
* width and height are dimension of framebuffer,
* width and height are dimension of framebuffer,
* x and y are the coordinates to display,
* color is the pixel's color value.
* return 0 if success, otherwise return -1.
*/
int fb_pixel(void *fbmem, int width, int height,
int x, int y, unsigned short color)
{
if ((x > width) || (y > height))
return -1;
unsigned short *dst = ((unsigned short *) fbmem + y * width + x);
*dst = color;
return 0;
}
3. LCD驱动
我们用到的是一块东华3.5寸数字屏,型号为WXCAT35-TG3.下面的驱动程序是韦东山老师课堂上现场写的,如下:
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/dma-mapping.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <asm/irq.h>
#include <asm/setup.h>
/* WXCAT35-TG3 */
struct s3c_lcd_regs {
unsigned long lcdcon1;
unsigned long lcdcon2;
unsigned long lcdcon3;
unsigned long lcdcon4;
unsigned long lcdcon5;
unsigned long lcdsaddr1;
unsigned long lcdsaddr2;
unsigned long lcdsaddr3;
unsigned long redlut;
unsigned long greenlut;
unsigned long bluelut;
unsigned long reserved[9];
unsigned long dithmode;
unsigned long tpal;
unsigned long lcdintpnd;
unsigned long lcdsrcpnd;
unsigned long lcdintmsk;
unsigned long lpcsel;
};
static u32 colregs[16];
static struct fb_info *s3c_fb_info;
static dma_addr_t s3c_fb_handle;
static unsigned long fb_va;
/* from pxafb.c */
static inline unsigned int chan_to_field(unsigned int chan, struct fb_bitfield *bf)
{
chan &= 0xffff;
chan >>= 16 - bf->length;
return chan << bf->offset;
}
static int s3cfb_setcolreg(unsigned regno,
unsigned red, unsigned green, unsigned blue,
unsigned transp, struct fb_info *info)
{
unsigned int val;
/* dprintk("setcol: regno=%d, rgb=%d,%d,%d\n", regno, red, green, blue); */
/* true-colour, use pseuo-palette */
if (regno < 16) {
u32 *pal = s3c_fb_info->pseudo_palette;
val = chan_to_field(red, &s3c_fb_info->var.red);
val |= chan_to_field(green, &s3c_fb_info->var.green);
val |= chan_to_field(blue, &s3c_fb_info->var.blue);
pal[regno] = val;
}
return 0;
}
static struct fb_ops s3cfb_ops = {
.owner = THIS_MODULE,
// .fb_check_var = clps7111fb_check_var,
// .fb_set_par = clps7111fb_set_par,
// .fb_setcolreg = clps7111fb_setcolreg,
// .fb_blank = clps7111fb_blank,
.fb_setcolreg = s3cfb_setcolreg,
.fb_fillrect = cfb_fillrect,
.fb_copyarea = cfb_copyarea,
.fb_imageblit = cfb_imageblit,
};
struct s3c_lcd_regs *s3c_lcd_regs;
static volatile unsigned long *gpccon;
static volatile unsigned long *gpdcon;
static volatile unsigned long *gpgcon;
int s3c_lcd_init(void)
{
extern int debug_lcd;
/* 1. 分配一个fb_info结构体 */
s3c_fb_info = framebuffer_alloc(0, NULL);
printk("%s %d\n", __FUNCTION__, __LINE__);
/* 2. 设置fb_info结构体 */
/*
2.1 设置固定的信息
2.2 设置可变的信息
2.3 设置操作函数
*/
/* 24BPP(bits per pixel), 会用到4字节, 其中浪费1字节 */
strcpy(s3c_fb_info->fix.id, "WXCAT35-TG3");
// s3c_fb_info->fix.smem_start // frame buffer's physical address
s3c_fb_info->fix.smem_len = 320*240*32/8;
s3c_fb_info->fix.type = FB_TYPE_PACKED_PIXELS;
s3c_fb_info->fix.visual = FB_VISUAL_TRUECOLOR;
s3c_fb_info->fix.line_length = 320 * 4;
s3c_fb_info->var.xres = 320;
s3c_fb_info->var.yres = 240;
s3c_fb_info->var.xres_virtual = 320;
s3c_fb_info->var.yres_virtual = 240;
s3c_fb_info->var.bits_per_pixel = 32;
s3c_fb_info->var.red.offset = 16;
s3c_fb_info->var.red.length = 8;
s3c_fb_info->var.green.offset = 8;
s3c_fb_info->var.green.length = 8;
s3c_fb_info->var.blue.offset = 0;
s3c_fb_info->var.blue.length = 8;
//s3c_fb_info->var.activate = FB_ACTIVATE;
s3c_fb_info->fbops = &s3cfb_ops;
s3c_fb_info->pseudo_palette = colregs;
/* 3. 硬件相关的操作 */
/* 配置GPIO */
gpccon = ioremap(0x56000020, 4);
gpdcon = ioremap(0x56000030, 4);
gpgcon = ioremap(0x56000060, 4);
*gpccon = 0xaaaaaaaa;
*gpdcon = 0xaaaaaaaa;
*gpgcon |= (3<<8); /* GPG4 use as lcd_pwren */
printk("%s %d\n", __FUNCTION__, __LINE__);
s3c_lcd_regs = ioremap(0X4D000000, sizeof(struct s3c_lcd_regs));
/*
* VCLK = HCLK / [(CLKVAL+1)x2] = 100M/[(CLKVAL+1)x2] = 6.4
* CLKVAL = 6.8 = 7
* TFT LCD panel
* 24bpp
*/
s3c_lcd_regs->lcdcon1 = (7<<8)|(0<<7)|(3<<5)|(0x0d<<1)|(0<<0);
printk("%s %d\n", __FUNCTION__, __LINE__);
/* VBPD: 电子枪收到VSYNC信号后,"多长时间"才能跳回第1行
* VBPD=14, LCD: tvb=15
* LINEVAL=239, LCD: 有240行
* VFPD=11, LCD: tvf=12 // 发出最后一行数据后,再过多长时间才发出VSYNC
* VSPW=2, LCD: tvp=3 // VSYNC的宽度
*/
s3c_lcd_regs->lcdcon2 = (14<<24)|(239<<14)|(11<<6)|(2<<0);
/* HBPD: 电子枪收到HSYNC信号后,"多长时间"才能跳回第1列
* HBPD=37, LCD: thb=38
* HORVAL=319, LCD: 有320行
* HFPD=19, LCD: thf=20 // 发出最后一象素数据后,再过多长时间才发出HSYNC
* HSPW=29, LCD: thp=30 // VSYNC的宽度
*/
s3c_lcd_regs->lcdcon3 = (37<<19)|(319<<8)|(19<<0);
s3c_lcd_regs->lcdcon4 = 29;
/* bit10: 在VCLK上升沿取数据
* bit9 : VSYNC低电平有效
* bit8 : HSYNC低电平有效
* bit5 : PWREN低电平有效
*/
s3c_lcd_regs->lcdcon5 = (1<<10)|(1<<9)|(1<<8)|(1<<5)|(0<<3);
/* 分配frame buffer */
fb_va = (unsigned long)dma_alloc_writecombine(NULL, s3c_fb_info->fix.smem_len, &s3c_fb_handle, GFP_KERNEL);
printk("fb_va = 0x%x, pa = 0x%x\n", fb_va, s3c_fb_handle);
s3c_fb_info->fix.smem_start = s3c_fb_handle;
s3c_fb_info->screen_base = fb_va;
/* 把framebuffer的地址告诉LCD控制器 */
s3c_lcd_regs->lcdsaddr1 = (s3c_fb_info->fix.smem_start >> 1);
s3c_lcd_regs->lcdsaddr2 = ((s3c_fb_info->fix.smem_start+320*240*4) >> 1) & 0x1fffff;
s3c_lcd_regs->lcdsaddr3 = 320*4/2;
/* 使能LCD */
s3c_lcd_regs->lcdcon1 |= (1<<0);
/* 4. register_framebuffer */
printk("%s %d\n", __FUNCTION__, __LINE__);
//debug_lcd = 1;
register_framebuffer(s3c_fb_info);
printk("%s %d\n", __FUNCTION__, __LINE__);
return 0;
}
void s3c_lcd_exit(void)
{
unregister_framebuffer(s3c_fb_info);
dma_free_writecombine(NULL, s3c_fb_info->fix.smem_len, fb_va, s3c_fb_handle);
iounmap(s3c_lcd_regs);
iounmap(gpccon);
iounmap(gpdcon);
iounmap(gpgcon);
framebuffer_release(s3c_fb_info);
}
module_init(s3c_lcd_init);
module_exit(s3c_lcd_exit);
MODULE_LICENSE("GPL");
然后把它加入到内核,以静态加载的模式启动.
最后,可以把读取内存jpeg格式数据输出到LCD屏的这部分整合到mjpg-stream或servfox去,就实现了采集图像本地显示了.