Implementation of H4VDM

This project's goal is to implement the neural network proposed in H4VDM by Z. Xiang et al. as part of the Multimedia Data Security course held by prof. Giulia Boato at the University of Trento.

(back to top)

To get a local copy up and running follow these simple example steps.

The following resources are required for the code to work:

• Python
• Protobuf (only if you need to rebuild the protobuffers)

1. Clone the repo

   git clone https://github.com/thomasnonis/h4vdm.git
   git submodule init
   git submodule update

   Note that you need to have your ssh key set up on both GitHub and GitLab

2. Install the dependencies

   pip install -r requirements.txt

3. Install the submodule dependencies

   • Install gcc, nasm, cmake, ffmpeg
   • Run make in the h264-extractor folder to build the modified openh264 codec

4. If required, rebuild the protobuffer

   ./h264-extractor/openh264/info_shipout/compile_proto.sh

   alternatively

   apt install -y protobuf-compiler
   cd h264-extractor/openh264/info_shipout/
   protoc --python_out=. slice.proto

(back to top)

The codebase is divided in 3 main sections:

• H264 extractor
• Packages
• Runners

The H264 section (h264-extractor) is a submodule linked to https://gitlab.com/viper-purdue/h4vdm. It consists of a modified version of ffmpeg that packs the H264 parameters from an MP4 video in an easy to use protobuffer. Full credit to the original authors for this section. The only modification has been the regeneration of the python protobuf files with a newer version.

The packages section ('packages') contains the core of the work of this project. It contains all of the classes and network code.

The runners are simply the .ipynb files that are actually run to execute the code.

The codebase is written to allow the whole dataset to be lazily generated. To speed up the training phase, it is possible to generate it in advance.

If lazy generation is desired, simply run the code with build_on_init and download_on_init set to False, otherwise set them as desired. Before doing anything, make sure to set your desired dataset generation directory in packages/constands.py with the DATASET_ROOT folder. Make sure to have at least 200GB available.

When generating the dataset, the structure will be saved in a .json file to keep track of the files across multiple runs. To ignore it and forcibly regenerate the dataset or to include additional resources, ignore_local_dataset can be set to True.

To ease the process, it may be worth looping through the whole dataset to generate it before doing any training work.

See h4vdm.ipynb for a complete example.

(back to top)

• H264 parameter extraction
• H264 dataset management
• Lazy generation
• Minimize RAM usage
• Training and performance evaluation
• Randomize crop centering
• GOP extraction from random location

(back to top)

Distributed under the GNU GPLv3 license. See LICENSE.txt for more information.

(back to top)

Thomas Nonis - [email protected]

(back to top)