Yor are not pushing it too far. In fact, the Evoprimer is the most ideal format for doing what you are doing.
I have attempted to capture video directly from the NTSC and PAL standards. I didn't have a camera that had the USB output. By converting directly from NTSC or PAL, you can use virtually any camera and connect it to the Evoprimer.
I have had limited success with a Black and white camera. If you have a camera with USB and all the documentation, you shouldn't hav a problem. Those cameras have I2C for the commands and data lines for serial data transfer. Some use I2C or SPI to do all the functios
The EvoPrimer is an excellent format for demostrating vision systems for robotics, industrial or facial recognition.
I was only running algoriths for processing the camera data and I was not using CicleOS, I modified the VirtualCom program to do it. It was also processing the camera data in realtime as it created the image on the screen. It probably would be faster to process one entire frame, then create the image.