1.RGA
简单的说,RGA就是一个用来处理图片的硬件,和CPU相互独立,可以独立完成图片的处理,在瑞芯微这边提供的库是阻塞的,可以自己实现为异步的。
2.RGA错误使用
这里使用RGA,主要是用于一个快速的格式转换,在这部分实现中,阻塞或者异步都影响不大。上一章节简单说明了一下RGA的使用,经过测试那样做,有点问题,资源泄漏,就算在析构中手动释放申请的资源,后面还是会有问题,比如每一帧的转换都请求一个句柄,转换完就算释放。
void QSmartCamera::draw_viewfinder(QSharedPointer<QByteArray d)
{
QRgaYUV2RGB convert(STREAM_WIDTH,STREAM_HEIGHT,RK_FORMAT_YCbCr_420_P,(unsigned char *)d->data(),
STREAM_WIDTH,STREAM_HEIGHT,RK_FORMAT_RGB_888);
unsigned char *frame = convert.get_data();
if(frame)
{
qDebug() << "rga convert ok";
QImage image(frame,STREAM_WIDTH,STREAM_HEIGHT,QImage::Format_RGB888);
draw_image_rects(image,lastRect);
QDateTime now = QDateTime::currentDateTime();
ui->label->setPixmap(QPixmap::fromImage(create_image_text(image,now.toString(),40,QPoint(50,50))));
}
}
然后析构函数实现
QRgaYUV2RGB::~QRgaYUV2RGB()
{
qDebug() << "Deinit ------------";
rkRga.RkRgaUnmap(&bo_src);
rkRga.RkRgaUnmap(&bo_dst);
rkRga.RkRgaFreeBuffer(bo_src.fd,&bo_src);
rkRga.RkRgaFreeBuffer(bo_dst.fd,&bo_dst);
rkRga.RkRgaFree(&bo_src);
rkRga.RkRgaFree(&bo_dst);
}
这样每次使用完之后,析构会释放资源,但是发现处理五百多次之后,就无法请求到资源,并且产生内核错误信息。
3.RGA实现
这里就直接子类化了RGA
class QRgaYUV2RGB : public RockchipRga
{
public:
QRgaYUV2RGB(int sw,int sh,RgaSURF_FORMAT sf,
int dw,int dh,RgaSURF_FORMAT df);
~QRgaYUV2RGB();
public:
unsigned char *get_data(unsigned char *frame);
private:
bo_t bo_src, bo_dst;
rga_info_t src;
rga_info_t dst;
int sf_size,df_size;
};
实现如下
QRgaYUV2RGB::QRgaYUV2RGB(int sw,int sh,RgaSURF_FORMAT sf,
int dw,int dh,RgaSURF_FORMAT df)
:RockchipRga()
{
int ret = 0;
this->RkRgaInit();
ret = this->RkRgaGetAllocBuffer(&bo_src, sw, sh, 32);
ret = this->RkRgaGetAllocBuffer(&bo_dst, dw, dh, 32);
this->RkRgaGetMmap(&bo_src);
this->RkRgaGetMmap(&bo_dst);
sf_size = get_buf_size_by_w_h_f(sw,sh,sf);
df_size = get_buf_size_by_w_h_f(dw,dh,df);
memset(&src, 0, sizeof(rga_info_t));
src.fd = -1;
src.mmuFlag = 1;
memset(&dst, 0, sizeof(rga_info_t));
dst.fd = -1;
dst.mmuFlag = 1;
ret = this->RkRgaGetBufferFd(&bo_src, &src.fd);
printf("src.fd =%d \n",src.fd);
if (ret) {
printf("rgaGetsrcFd fail : %s\n", strerror(errno));
}
ret = this->RkRgaGetBufferFd(&bo_dst, &dst.fd);
printf("dst.fd =%d \n",dst.fd);
if (ret) {
printf("rgaGetdstFd error : %s\n", strerror(errno));
}
rga_set_rect(&src.rect, 0,0,sw,sh,sw,sh,sf);
rga_set_rect(&dst.rect, 0,0,dw,dh,dw,dh,df);
}
QRgaYUV2RGB::~QRgaYUV2RGB()
{
qDebug() << "Deinit ------------";
this->RkRgaUnmap(&bo_src);
this->RkRgaUnmap(&bo_dst);
this->RkRgaFreeBuffer(bo_src.fd,&bo_src);
this->RkRgaFreeBuffer(bo_dst.fd,&bo_dst);
this->RkRgaFree(&bo_src);
this->RkRgaFree(&bo_dst);
}
unsigned char *QRgaYUV2RGB::get_data(unsigned char *frame)
{
memset(bo_src.ptr,sf_size,0);
memcpy(bo_src.ptr,frame,sf_size);
memset(bo_dst.ptr,df_size,0);
if(RkRgaBlit(&src, &dst, NULL))
return nullptr;
return (unsigned char *)bo_dst.ptr;
}
这样的析构仍然是有问题的, 只是这样处理之后,得到一个常驻内存的句柄,每次通过get_data来转换数据,且这个内存是在之前申请的,所以直接返回仍然有效。
4.更新帧捕获
根据前面测测试结果,发现自定义的viewfinder有问题,目前不确定问题原因,所以暂时先取消。所有数据通过QVideoProbe得到
connect(m_probe.data(),&QVideoProbe::videoFrameProbed,[this](QVideoFrame f){
f.map(QAbstractVideoBuffer::ReadOnly);
QSharedPointer<QByteArray> a = QSharedPointer<QByteArray>(new QByteArray((char*)f.bits(),STREAM_SIZE));
f.unmap();
if(needHandle)
{
needHandle = false;
QMutexLocker ll(&locker);
frameVec.append(a);
}
emit handleFrame(a);
draw_viewfinder(a);
});
在draw_viewfinder中绘制数据到屏幕上
5.使用转换模块
在QSmartCamera添加成员QScopedPointer QRgaYUV2RGB m_rga;构造如图
m_rga.reset(new QRgaYUV2RGB(STREAM_WIDTH,STREAM_HEIGHT,RK_FORMAT_YCbCr_420_P,
STREAM_WIDTH,STREAM_HEIGHT,RK_FORMAT_RGB_888));
在捕获帧之后,通过这里刷新到label上,此时CPU,使用在25-30左右,比之前的80%+明显降低
void QSmartCamera::draw_viewfinder(QSharedPointer<QByteArray d)
{
// QRgaYUV2RGB convert(STREAM_WIDTH,STREAM_HEIGHT,RK_FORMAT_YCbCr_420_P,(unsigned char *)d->data(),
// STREAM_WIDTH,STREAM_HEIGHT,RK_FORMAT_RGB_888);
// unsigned char *frame = convert.get_data();
unsigned char *frame = m_rga->get_data((unsigned char *)d->data());
if(frame)
{
qDebug() << "rga convert ok";
QImage image(frame,STREAM_WIDTH,STREAM_HEIGHT,QImage::Format_RGB888);
draw_image_rects(image,lastRect);
QDateTime now = QDateTime::currentDateTime();
ui->label->setPixmap(QPixmap::fromImage(create_image_text(image,now.toString(),40,QPoint(50,50))));
}
}
原作者:嘉木有鱼