Skip to content

🔥🔥🔥 Latest works on video streaming/processing/analysis

Notifications You must be signed in to change notification settings

junhua-l/Awesome-Video-Streaming-and-Analysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
 
 
 
 

Repository files navigation

Awesome Video Streaming and Analysis

Awesome-Video-Streaming-and-Analysis is a curated list of awesome frameworks, applications, and systems dedicated to video streaming and analysis (processing).

With the emergence of the metaverse, an increasing number of related projects and technologies have emerged. However, many existing repositories have not been updated for some time, making it challenging to find current and relevant information. To address this gap, this repository provides an up-to-date and comprehensive collection of papers, resources, and techniques for video streaming and analysis.

Published Scope:
Streaming and analysis: SIGCOMM, NSDI, MobiCom, INFOCOM, MM, VR, WWW, MMSys, OSDI, NOSSDAV
Video processing: CVPR, ECCV, ICCV, TCSVT, TMM, ToG, TVCG, TIP, Siggraph, Vis
*Noting that this repo mainly starts from a perspective of networking.

I also pack the conference proceedings about multimedia networking into a single pdf.

Table of Contents

Resources

Back to Table of Contents

Related Resources

Related repo: Paper-Lit, Video-Streaming-Research-Papers, Deep image/video compression, Awesome-360-vision, Awesome-Streaming, Awesome-NeRF, Weekly-NeRF, Awesome-ARKit, Awesome-iot, 3D Machine learning, audio-video-streaming, Awesome Point cloud Analysis, Awesome-System-for-Machine-Learning, Awesome-Deep-Neural-Network-Compression

Related Lectures

EE364a-Convex optimization a, EE364b-Convex optimization b, CS349D: Cloud Computing techniques, CSC348K-visual computing system, EE267-virtual reality, EE359-wireless communication, EE398A-Image and Video Compression, CS244 Advanced topics in networking, ECE 5578 Multimedia Communication, 34702 Topics in Networks: Machine Learning for Networking and Systems

Tutorial and Workshops

SIGCOMM 23 Workshop on Emerging Multimedia Systems Several Special Issues in IEEE Network 23 MM 22: Advances In Quality Assessment Of Video Streaming Systems: Algorithms, Methods, Tools; Short Video Streaming Challenge

MM 21: Deep Learning for Visual Data Compression

MMSys 23, NTIRE 2023 Challenge on 360° Omnidirectional Image and Video Super-Resolution

OmniCV2022, GAZE2022, 19 MM Grand Challenge:

Limited by the author's knowledge, this section awaits replenishment.

  • Some articles may be repeated.*

2D Videos

Back to Table of Contents

Video Streaming

Year Method Detail
21 Fugu [NSDI 21] DNN (bandwidth prediction)+DP (control)
20 OnRL [Mobicom 20] Online RL
20 Stick [Infocom 20] Buffer-based+Learning-based
19 Comyco [MM 19], Concerto [Mobicom 19] Imitation Learning
19 PiTree [MM 19] Explainable Learning
18 Oboe [Sigcomm 18] Auto-tuning parameters
17 Pensieve [Sigcomm 17] Update [ICML 19] Reinforcement Learning
16 CS2P [Sigcomm 16]
16 BOLA [Infocom 16] Buffer-Based+Lyapunov Optimization
15 MPC [Sigcomm 15] MPC
14 Buffer-Based [Sigcomm 14] Buffer-Based
12 Rate-Based [CoNEXT 12] Rate-Based

Live Video Streaming

Super Resolution in Video Streaming

Video Telephony

Architectural Support for Video Streaming

Layered Video Coding and Streaming

360-degree Videos

Back to Table of Contents

360-degree Video Streaming

Live 360-degree Video Streaming

360-degree Video System

Viewport Prediction

360-degree Video Datasets

Volumetric Videos

Back to Table of Contents

Volumetric Video Streaming

Virtual Reality

Several mediums for volumetric video including point cloud, mesh, voxel, NeRF, light fields, radiance fields..... Virtual reality papers research how to render with low latency in edge/cloud architecture. They often render small objects in mobile devices and render heavy background in the server. NeRF-based Volumetric Video: \

Volumetric Video Datasets

Dynamic Point Cloud

Others:

Quality of Experience

Video Processing

Back to Table of Contents

Video Analysis

Video Analysis for 3D Vision

Video Coding with Deep Learning

This part is also known Neural video compression: This repo is currently the most comprehensive list of papers about image/video coding with deep learning

A list of summary of represented works for interested readers:

  1. end to end optimized image compression(End-to-end VAE)
  2. Hyperprior(modeling p(y,z|x), intro the distribution of prior hyperparameter)
  3. Joint Autoregressive and Hierarchical Priors for Learned Image Compression
  4. Integer Network for data compression with latent variable models
  5. channel wise autoregressive entropy models for learned image compression(diferent slice model slice dependence gg20c)
  6. learned image compression with discretized gaussian mixture likehoods and attention moduels(GMM entropy model + attention cheng2020)
  7. context adaptive entropy model for end to end optimized image compression(bits consuming and bits free entropy model)
  8. Learned Variable-Rate Multi-Frequency Image Compression using Modulated Generalized Octave Convolution(lamda sigmoid)
  9. Variable Rate Deep Image Compression With a Conditional Autoencoder(using lambda to fine model parameters,conditional autoencoder)
  10. YOU ONLY TRAIN ONCE: LOSS-CONDITIONAL TRAINING OF DEEP NETWORKS(conditional training)
  11. A Hybriad image codec with learned residuals(local attention,octave convolution, residual compression)
  12. Learned image compression with residual coding
  13. End-to-End Learned ROI Image Compression(encoder + importance map)
  14. Learning Convolutional Networks for Content-weighted Image Compression(importance map + binary feature map)
  15. Conditional probability model for deep image compression
  16. generative adversarial networks for extreme learned image compression
  17. towards conceptual compression(convolutional hyperprior)
  18. Learn to inpaint for image compression(inpaint + residual compression)
  19. deep generative models for distribution preserving lossy compression(gan)

Volumetric Video Coding and Compression

Other Video Coding

Multimedia 3D Processing

Security and Privacy Related

Unique Identification of 50,000+ Virtual Reality Users from Head & Hand Motion Data [Arxiv 23]

Video Classification

Tools

Back to Table of Contents Following are the tools and libraries that are useful to build your ideas on top of.

Multimedia Libraries

  • FFMPEG: A multimedia library with a collection of diverse video codecs, filters, and video streaming capabilities.
  • GPAC: A multimedia library that has decoding, rendering and displaying support. It also has support for 360 degree video delivery. It comes with MP4Box to package the video into DASH format segments and MP4Client a video player with adaptive video streaming solutions
  • x265: Open source implementation H.265 video codec.
  • OBS Studio: Open source broadcaster software. It is useful to stream live videos on platforms such as Facebook and Periscope etc.
  • SVT Encoders: Software (multithreaded CPU) implementation of HEVC, VP9 and AV1 encoders.
  • Saliency-aware Video Codec: X264 implementation of saliency-aware video compression.
  • SHVC: Layered coding - scalable extentions for H.265/HEVC
  • SVC: Layered coding - scalable extensions for H.264/AVC
  • VVC: Reference implementation of H.266/VVC

Multimedia Libraries

  • NeRFstudio: A collaboration friendly studio for NeRFs
  • FFMPEG: A multimedia library with a collection of diverse video codecs, filters, and video streaming capabilities.
  • GPAC: A multimedia library that has decoding, rendering and displaying support. It also has support for 360 degree video delivery. It comes with MP4Box to package the video into DASH format segments and MP4Client a video player with adaptive video streaming solutions
  • x265: Open source implementation H.265 video codec.
  • OBS Studio: Open source broadcaster software. It is useful to stream live videos on platforms such as Facebook and Periscope etc.
  • SVT Encoders: Software (multithreaded CPU) implementation of HEVC, VP9 and AV1 encoders.
  • Saliency-aware Video Codec: X264 implementation of saliency-aware video compression.
  • SHVC: Layered coding - scalable extentions for H.265/HEVC
  • SVC: Layered coding - scalable extensions for H.264/AVC
  • VVC: Reference implementation of H.266/VVC

About

🔥🔥🔥 Latest works on video streaming/processing/analysis

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published