hardware acceleration

General questions about iSpy
jpro1001
Posts: 4
Joined: Wed Aug 05, 2015 9:33 pm

hardware acceleration

Postby jpro1001 » Wed Aug 05, 2015 9:41 pm

Anyone know why ispy doesn't seem to use my gpu? I'm using vlc and specified the --ffmpeg-hw argument. When using only vlc I do see my gpu being used in gpu-z. My source stream is a foscam clone using rtsp. Shouldn't ispy just hand off the decoding to vlc? Is there away I can see the vlc logs after ispy calls it? Any advice would be greatly appreciated as my cpu is thru the roof!

djeffers
Posts: 7
Joined: Tue Mar 10, 2015 12:13 pm

Re: hardware acceleration

Postby djeffers » Thu Aug 06, 2015 12:53 pm

I am seeing the same issue where my CPU is just about maxed out with only 6 VLC cameras. I was considering getting a accelerated video card instead of using the on board video to get around it. I have pretty much tried all the suggested items in the performance tips and I am still struggling with it.

jpro1001
Posts: 4
Joined: Wed Aug 05, 2015 9:33 pm

Re: hardware acceleration

Postby jpro1001 » Thu Aug 06, 2015 4:45 pm

Yeah, my one 1080p camera maxes out my cpu (old). I bought a $10 used gpu on ebay (geforce 210) and it knocks my cpu down to 3% when using vlc + gpu to view my stream, but sadly ispy + vlc doesn't seem to use my gpu... I'm using gpu-z to monitor my gpu usage.

jpro1001
Posts: 4
Joined: Wed Aug 05, 2015 9:33 pm

Re: hardware acceleration

Postby jpro1001 » Mon Aug 10, 2015 8:42 pm

bump......

User avatar
sporn
Posts: 1689
Joined: Sat Mar 29, 2014 4:21 am
Contact:

Re: hardware acceleration

Postby sporn » Wed Aug 12, 2015 7:15 am

Hello I typed a very long reply to this - not sure what happened to it!

Basically VLC can do GPU decode but...

To enable hardware accelerated decoding, use the VLC preferences. By default, hardware acceleration is disabled (and consequently, hardware acceleration is not yet available to external application via LibVLC).

https://wiki.videolan.org/VLC_GPU_Decoding/

... which is a bummer

so i'm not entirely sure what that --ffmpeg-hw flag actually does

That said, the performance gains from using the GPU can actually be negligible as iSpy needs to do post processing on the frames which means transferring them from the GPU memory to RAM which can sometimes take longer than just using the CPU to process them.

jpro1001
Posts: 4
Joined: Wed Aug 05, 2015 9:33 pm

Re: hardware acceleration

Postby jpro1001 » Wed Aug 12, 2015 4:59 pm

Yeah, I was concerned it might not improve that much because I do run motion tracking too. Do you know of another program that does motion tracking and has hardware acceleration? I would think it would be a common thing. What about any hardware motion tracking devices/cards? Ispy does work great for what I need, I just wish it didn't take up so much cpu... Maybe I should just buy a new computer...

dhmk
Posts: 18
Joined: Fri Jun 05, 2015 5:07 am

Re: hardware acceleration

Postby dhmk » Wed Jan 24, 2018 5:33 pm

Hi,

Now that I have 5 HD cameras iSPY takes 20-30% of CPU power. I though I could lower that a lot if GPU was used for decoding.

I have a small testing app that uses ffmpeg to display all 5 cameras. This app takes about 10-12% of CPU power. I searched the web of how to activate HW acceleration in this app and now I have positive results. I have working code. So with HW acceleration the CPU for this app went to 2.5-3.5%. A big gain. And it was not that difficult either. Of course the code is not perfect but it looks a good start. If HW is used then iSPY will improve a lot when there are many HD cameras to handle. Studying the ffmpeg code in iSPY I think the HW decoding code can be easily adapted if it is worked a bit more.

For anyone interested code in c++ follows.

Code: Select all


int decode(AVCodecContext *avctx,AVFrame *frame,int *got_frame,AVPacket *pkt){
 int ret;

 *got_frame=0;

 if(pkt) {
  ret=avcodec_send_packet(avctx,pkt);
  // In particular, we don't expect AVERROR(EAGAIN), because we read all
  // decoded frames with avcodec_receive_frame() until done.
  if(ret<0)return ret==AVERROR_EOF?0:ret;
 }

 ret=avcodec_receive_frame(avctx,frame);

 if(ret!=0 && ret!=AVERROR(EAGAIN) && ret!=AVERROR_EOF)return ret;

 if(ret==0)*got_frame=1;

 return 0;
}


AVPixelFormat GetHwFormat(AVCodecContext *s,const AVPixelFormat *pix_fmts){
 InputStream* ist=(InputStream*)s->opaque;
 ist->active_hwaccel_id=HWACCEL_DXVA2;
 ist->hwaccel_pix_fmt=AV_PIX_FMT_DXVA2_VLD;
 return ist->hwaccel_pix_fmt;
}


int _stdcall ShowCamera(char *name,char *file,HWND hWin,int &cmd){
 EnterCriticalSection(&cs);

 AVFormatContext   *pFormatCtx;
 int            i,videoindex;
 AVCodecContext   *pCodecCtx;
 AVCodec         *pCodec;
 AVFrame   *pFrame,*pFrameRGB;
 uint8_t *out_buffer;
 AVPacket *packet;
 SwsContext *img_convert_ctx;
 int ret,got_picture;
 int err;

 pFormatCtx=avformat_alloc_context();


 if(avformat_open_input(&pFormatCtx,file,NULL,NULL)!=0){
  //printf("Couldn't open input stream.\n");
  err=-1;
  goto error;
 }
 if(avformat_find_stream_info(pFormatCtx,NULL)<0){
  //printf("Couldn't find stream information.\n");
  err=-2;
  goto error;
 }

 pFormatCtx->flags|=AVFMT_FLAG_DISCARD_CORRUPT;
 pFormatCtx->flags|=AVFMT_FLAG_NOBUFFER;

 videoindex=-1;
 for(i=0; i<pFormatCtx->nb_streams; i++)
  if(pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO){
   videoindex=i;
   break;
  }
 if(videoindex==-1){
  //printf("Didn't find a video stream.\n");
  err=-3;
  goto error;
 }
 AVStream *video_st=pFormatCtx->streams[videoindex];
 pCodecCtx=video_st->codec;
 pCodecCtx->workaround_bugs=1;
 pCodecCtx->flags2|=CODEC_FLAG2_FAST|CODEC_FLAG_LOW_DELAY;

 pCodecCtx->thread_count=1;

 pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
 if(pCodec==NULL){
  //printf("Codec not found.\n");
  err=-4;
  goto error;
 }


 HDC dc=GetDC(hWin);
 RECT rc;
 GetClientRect(hWin,&rc);

 //**
 InputStream *ist=new InputStream();
 ist->hwaccel_id=HWACCEL_AUTO;
 ist->hwaccel_device="dxva2";
 ist->dec=pCodec;
 ist->dec_ctx=pCodecCtx;

 pCodecCtx->opaque=ist;
 dxva2_init(pCodecCtx);

 pCodecCtx->get_buffer2=ist->hwaccel_get_buffer;
 pCodecCtx->get_format=GetHwFormat;
 pCodecCtx->thread_safe_callbacks=1;
 //**

 if((pCodec->capabilities & AV_CODEC_CAP_TRUNCATED)==AV_CODEC_CAP_TRUNCATED)
  pCodecCtx->flags|=AV_CODEC_FLAG_TRUNCATED;

 BOOL ok=avcodec_open2(pCodecCtx,pCodec,NULL)>=0;
 if(!ok){
  err=-5;
  goto error; //Could not open codec
 }

 pFrame=av_frame_alloc();
 pFrameRGB=av_frame_alloc();


 int convertedFrameBufferSize=av_image_get_buffer_size(PIXEL_FORMAT,pCodecCtx->width,pCodecCtx->height,1);
 out_buffer=(uint8_t *)av_malloc(convertedFrameBufferSize);
 av_image_fill_arrays(pFrameRGB->data,pFrameRGB->linesize,out_buffer,PIXEL_FORMAT,pCodecCtx->width,pCodecCtx->height,1);

 img_convert_ctx=sws_getContext(pCodecCtx->width,
                                pCodecCtx->height,
                                AV_PIX_FMT_NV12,
                                rc.right,
                                rc.bottom,
                                PIXEL_FORMAT,
                                SWS_FAST_BILINEAR,
                                NULL,NULL,NULL);

 //decode
 packet=(AVPacket *)av_mallocz(sizeof(AVPacket));

 LeaveCriticalSection(&cs);
 while(cmd==0){
  av_init_packet(packet);
  int ret=av_read_frame(pFormatCtx,packet);
  if(ret<0)break;

  if(packet->stream_index==videoindex){

   ret=decode(pCodecCtx,pFrame,&got_picture,packet);

   if(ret<0)break;

   if(got_picture){
    if(dxva2_retrieve_data_call(pCodecCtx,pFrame)!=0)break;

    sws_scale(img_convert_ctx,
              pFrame->data,
              pFrame->linesize,
              0,
              pCodecCtx->height,
              pFrameRGB->data,
              pFrameRGB->linesize);

    DisplayFrame(name,dc,0,0,rc,pCodecCtx->width,pCodecCtx->height,out_buffer);
   }
  }
  av_packet_unref(packet);
  av_frame_unref(pFrame);
 }

 //finalise
 sws_freeContext(img_convert_ctx);
 av_frame_free(&pFrameRGB);
 av_frame_free(&pFrame);
 avcodec_close(pCodecCtx);
 avformat_close_input(&pFormatCtx);
 av_freep(&packet);
 ReleaseDC(hWin,dc);
 return 0;
 error:
 return err;
}


Additional code needed from the post below (Don't mind about DisplayFrame in the above code. It just copies image data to a device context):

https://stackoverflow.com/questions/346 ... with-dxva2

at:

https://drive.google.com/file/d/0B5ufHd ... sp=sharing


Return to “Questions”

Who is online

Users browsing this forum: No registered users and 1 guest