Add support for RTSP PCM/WAV and G711/WAV by basantwanishraddha · Pull Request #56 · androidx/media

basantwanishraddha · 2022-03-16T06:18:34Z

Added PCM RTP packet reader and added support for PCM 8 bit,
16 bit, ALAW and MULAW playback through RTSP.

Change-Id: If0a187b55faa89850a159e17eae28358d6634799

Added PCM RTP packet reader and added support for PCM 8 bit, 16 bit, ALAW and MULAW playback through RTSP. Change-Id: If0a187b55faa89850a159e17eae28358d6634799

google-cla · 2022-03-16T06:18:39Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtpPayloadFormat.java

libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspMediaTrack.java

libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtpPayloadFormat.java

libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpPCMReader.java

Rename a few variable to be more relevant Add detailed java docs

Add RtpPcmReader tests and warning for out of order packets in RtpPcmReader. Change-Id: I1554fa0a944dad00248a0a41fefad958da073a21

claincly · 2022-03-28T10:53:07Z

LG w/ nits, will fix in our internal processes.

...ies/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpPcmReaderTest.java

Create FakeTrackOutput with deduplicateConsecutiveFormats as false Change-Id: Id3d551465a77d5148638b00ae2b164b505044ae2

claincly · 2022-03-29T12:44:41Z

Merged internally. The issue will be marked as merged automatically when we do a push. Thanks for the contribution!

PiperOrigin-RevId: 437783926

claincly · 2022-05-10T12:26:44Z

Question: how did you test the PCM streams? Especially for PCMU, they have static payload types. Did you manage to generate a dynamic payload type for PCMU?

claincly · 2022-05-10T13:46:39Z

I tried to play this media using a local live555, and it didn't work. The cause is you are not handling static payload types. And even after I hardcoded the format (PCM L16, 44100Hz, 1 channel), the audio did not play.

claincly · 2022-05-10T14:30:08Z

This is the diff I used:

--- a/src/main/java/androidx/media3/exoplayer/rtsp/MediaDescription.java
+++ b/src/main/java/androidx/media3/exoplayer/rtsp/MediaDescription.java
@@ -105,6 +105,14 @@ import java.util.HashMap;
 
   /** Builder class for {@link MediaDescription}. */
   public static final class Builder {
+
+    /** RTPMAP attribute format: {@code <payloadType> <mediaEncoding>/<clockRate>/<channelCount>}. */
+    private static final String RTP_MAP_ATTR_AUDIO_FMT = "%d %s/%d/%d";
+    private static final int RTP_STATIC_PAYLOAD_TYPE_PCMU = 0;
+    private static final int RTP_STATIC_PAYLOAD_TYPE_PCMA = 8;
+    private static final int RTP_STATIC_PAYLOAD_TYPE_L16_STEREO = 10;
+    private static final int RTP_STATIC_PAYLOAD_TYPE_L16_MONO = 11;
+
     private final String mediaType;
     private final int port;
     private final String transportProtocol;
@@ -199,15 +207,52 @@ import java.util.HashMap;
      */
     public MediaDescription build() {
       try {
-        // rtpmap attribute is mandatory in RTSP (RFC2326 Section C.1.3).
-        checkState(attributes.containsKey(ATTR_RTPMAP));
         RtpMapAttribute rtpMapAttribute =
-            RtpMapAttribute.parse(castNonNull(attributes.get(ATTR_RTPMAP)));
+            attributes.containsKey(ATTR_RTPMAP)
+            ? RtpMapAttribute.parse(castNonNull(attributes.get(ATTR_RTPMAP)))
+            : RtpMapAttribute.parse(getRtpMapStringByPayloadType(payloadType));
         return new MediaDescription(this, ImmutableMap.copyOf(attributes), rtpMapAttribute);
       } catch (ParserException e) {
         throw new IllegalStateException(e);
       }
     }
+
+    private static String getRtpMapStringByPayloadType(int rtpPayloadType) {
+      checkArgument(rtpPayloadType < 96);
+
+      switch (rtpPayloadType) {
+        case RTP_STATIC_PAYLOAD_TYPE_PCMU:
+          return Util.formatInvariant(
+              RTP_MAP_ATTR_AUDIO_FMT,
+              /* payloadType= */ 0,
+              /* mediaEncoding= */ "PCMU",
+              /* clockRate= */ 8_000,
+              /* channelCount= */ 1);
+        case RTP_STATIC_PAYLOAD_TYPE_PCMA:
+          return Util.formatInvariant(
+              RTP_MAP_ATTR_AUDIO_FMT,
+              /* payloadType= */ 8,
+              /* mediaEncoding= */ "PCMA",
+              /* clockRate= */ 8_000,
+              /* channelCount= */ 1);
+        case RTP_STATIC_PAYLOAD_TYPE_L16_STEREO:
+          return Util.formatInvariant(
+              RTP_MAP_ATTR_AUDIO_FMT,
+              /* payloadType= */ 10,
+              /* mediaEncoding= */ "L16",
+              /* clockRate= */ 44_100,
+              /* channelCount= */ 2);
+        case RTP_STATIC_PAYLOAD_TYPE_L16_MONO:
+          return Util.formatInvariant(
+              RTP_MAP_ATTR_AUDIO_FMT,
+              /* payloadType= */ 11,
+              /* mediaEncoding= */ "L16",
+              /* clockRate= */ 44_100,
+              /* channelCount= */ 1);
+        default:
+          throw new IllegalStateException("Unsupported static paylod type " + rtpPayloadType);
+      }
+    }
   }
 
   /** The media types allowed in a SDP media description. */

Edit: I noticed the test clip is little-endian signed PCM, I edited this and still no good.

media/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtpPayloadFormat.java

Line 119 in f8f1a4d

: C.ENCODING_PCM_16BIT_BIG_ENDIAN;

basantwanishraddha · 2022-05-11T11:14:32Z

The test stream that you have shared, I wasn't able to run it on VLC player as well. (Used live555 server)
I tried debugging the behavior on exoplayer, but it never reaches the onDescribeResponseReceived.
So the test stream seems to be the cause of issue.
In the case of PCM Reader we are using a WAV container, so we receive dynamic payload.
This is the audio clip that we have used to validate the PCM_MULAW mediaType.

claincly · 2022-05-12T08:40:39Z

Maybe you have used an old version of VLC/live555? My live555 binary (mac) works with the audio file I provided. if you want to have a try
l5.zip.

I'll try your clip.

That said, it's questionable to only support dynamic payload type as PCM audio streams can and are being served with static payload type.

claincly · 2022-05-12T08:55:27Z

Re-uploading the audio file
pcm.wav.zip

basantwanishraddha · 2022-05-16T06:06:58Z

I am able to play the audio file on exoplayer, after applying the diff you have shared.
No other change was required, setting it to big endian 16 bit is correct since live555 server converts little endian to big endian before streaming. (link)
Let us know if you are able to play on exoplayer after resetting it to 'Big endian 16 bit'.

Add support for RTSP PCM/WAV and G711/WAV

ac528d3

Added PCM RTP packet reader and added support for PCM 8 bit, 16 bit, ALAW and MULAW playback through RTSP. Change-Id: If0a187b55faa89850a159e17eae28358d6634799

icbaker requested a review from claincly March 21, 2022 14:34

icbaker assigned claincly Mar 21, 2022

claincly requested changes Mar 23, 2022

View reviewed changes

basantwanishraddha added 3 commits March 28, 2022 11:13

Merge branch 'main' of github.com:ittiam-systems/media into rtp_wav

bfc1fb9

Modify RtpPcmReader

500c879

Rename a few variable to be more relevant Add detailed java docs

Add PCM and G711 Reader Test

3f8a680

Add RtpPcmReader tests and warning for out of order packets in RtpPcmReader. Change-Id: I1554fa0a944dad00248a0a41fefad958da073a21

claincly reviewed Mar 28, 2022

View reviewed changes

...ies/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpPcmReaderTest.java Outdated Show resolved Hide resolved

Update RtpPcmReaderTest

e3dcf38

Create FakeTrackOutput with deduplicateConsecutiveFormats as false Change-Id: Id3d551465a77d5148638b00ae2b164b505044ae2

basantwanishraddha requested a review from claincly March 29, 2022 05:30

icbaker added a commit that referenced this pull request Apr 11, 2022

Merge pull request #56 from ittiam-systems:rtp_wav

e3a9ed6

PiperOrigin-RevId: 437783926

claincly closed this Apr 27, 2022

icbaker added a commit that referenced this pull request May 9, 2022

Merge pull request #56 from ittiam-systems:rtp_wav

930709e

PiperOrigin-RevId: 437783926

androidx locked and limited conversation to collaborators Mar 4, 2023

Conversation

basantwanishraddha commented Mar 16, 2022

Uh oh!

google-cla bot commented Mar 16, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

claincly commented Mar 28, 2022

Uh oh!

Uh oh!

claincly commented Mar 29, 2022

Uh oh!

claincly commented May 10, 2022

Uh oh!

claincly commented May 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claincly commented May 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

basantwanishraddha commented May 11, 2022

Uh oh!

claincly commented May 12, 2022

Uh oh!

claincly commented May 12, 2022

Uh oh!

basantwanishraddha commented May 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

claincly commented May 10, 2022 •

edited

Loading

claincly commented May 10, 2022 •

edited

Loading