Draft
Conversation
Collaborator
|
The more I look at this, the more we should have a side project for an NDI frame generator. |
Contributor
Author
|
Yes, that would be nice. This generator started with the Send_Audio_and_Video example in the NDI Advanced SDK examples. There were two modifications:
When looking at the audio and video received in DistroAV from the generator, the video and audio are perfectly synced. So that was the first use of the sync test data logging. I ran into issues doing the dynamic link of the NDI runtime, and would like to have the github scripts we have in DistroAV. I can help with the code for the project, but I need help deploying it. |
Trouffman
added
enhancement
av-desync
feature
performance
!DO NOT MERGE!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add sync debug logging module and invocations of the methods at appropriate parts of DistroAV code. There are two types of logging added: log actual timestamp and timecode values with a cool down time, and detect sync test data signals (provided in this commit) and log differences in time of audio signal when the sync signal is received in video.
Motivation and Context
DistroAV source receives timecodes and timestamps from NDI, forwards timestamps to OBS, and then on the output side, DistroAV receives timestamps from OBS and sends timestamps and timecodes to NDI. Since this is happening many times per second and deals with VERY large numbers, the data can be overwhelming.
This addition to DistroAV helps by allowing the developer to log timestamps and timecode values at critical points in the the code, and have a cool down period (10 seconds) where nothing is logged. By including a developer provided message, which can be unique to the source or output name, the person looking at the log can see the values sent/received to/from NDI or OBS. Since the numbers are so big reporting is done in milliseconds.
From looking at the logs developers can tell immediately where units of timestamps and timecodes change as audio and video data move through the code.
How Has This Been Tested?
Has been tested by turning on --distroav-verbose and using NDI, media source, and VLC sources. Also tested with NDI main output and dedicated filter output.
For sync test data logging, the data must come from the SendSyncTestData.exe provided in the testing folder. When exe is run (need to copy the Processing.NDI.Lib.x64.dll from NDI Runtime to the testing folder), 1 second of white frames followed by 4 seconds of black frames. The audio is silent for black frames and has a 400hz sine wave during the white frames. With the exe running, add an NDI source to OBS and select "Sync Test BW" as the source. The sync-debug logging recognizes and reports the start of the audio signal and the start of the white frames, reporting differences in times of these signals. The sync test data can also be detected on output from OBS, in which case the developer must change the output format to BGRA to recognize the start of the white frames.
In addition to the NDI generator, a MP4 file is provided. When this video is selected as a media source, or a VLC source, and NDI output is selected, the sync debug logging will report sync offsets in the output, before sending the data to NDI.
Types of changes
Logging only, no change in behavior and only invoked when LOG_LEVEL == LOG_VERBOSE. Sync test data detection and logging is hidden behind a macro that only executes code if SYNC_DEBUG defined in sync-debug.h and LOG_LEVEL == LOG_VERBOSE. The default is on this PR can be tested without building.
Deployment recommendation
This PR can sit as not pulled into master, as the artifacts can be used for testing the logging capabilities.
Happy to make any changes needed and hope some find this info useful in addressing syncing issues in DistroAV.