RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics

Recent efforts have shown that the concept of continuous learning is promising for video analytics—high accuracy can be achieved with low compute overhead, by creating a lightweight “expert” (fast and accurate) DNN model for each specific video content and somehow adapting the expert model to cope with data drift. However, current solutions either retrain models periodically (which can be slow and demand substantial compute source), or rely on selecting history models without optimizing the resource sharing among model selection and retraining, both of which limit the scalability and accuracy when scaling to more edge devices. Thus, the benefits promised in continuous learning remain unrealized RECL is a new video-analytics framework that carefully integrates model reusing and online model retraining, allowing it to quickly adapt the expert model given any video frame samples. To do this, RECL (i) shares across edge devices a (potentially growing) “model zoo” that comprises expert models previously trained for all edge devices, enabling history model reuse across video sessions, (ii) uses a fast procedure to online select a highly accurate expert model from this shared model zoo, and (iii) dynamically optimizes GPU allocation among model retraining, model selection, and timely updates of the model zoo. Our evaluation of RECL on two computer-vision tasks (object detection and classification) and over 70 hours of videos shows that RECL greatly reduces the resource need to obtain same inference accuracy than prior model-retraining and model-reusing schemes.