URL query percent encoding is incorrect

achristensen07 · achristensen07 · commit 47164fceaef4 · 2026-02-04T04:46:35.000-08:00
https://bugs.webkit.org/show_bug.cgi?id=306742 rdar://169566553 Reviewed by Anne van Kesteren. We have used ucnv_setFallback(converter, true) for many, many years for our ICU-based text decoding. However, when using ICU to encode non-UTF8 text through a URL, we need to not use the fallback. This matches the behavior of Chrome and Firefox, and Chromium has a comment saying this matches Netscape behavior. This behavior is odd but already implemented and specified in https://url.spec.whatwg.org/#string-percent-encode-after-encoding Chromium implemented this by using a class for the URL encoding, ICUCharsetConverter which does not call ucnv_setFallback, and a separate class for text encoding, and TextCodecIcu, which does call ucnv_setFallback. I've already used an abstract interface for this rare case, URLTextEncoding, in order to keep it down to one implementation, so I keep that design and use ucnv_setFallback to turn off fallback when using UnencodableHandling::URLEncodedEntities for URL query encoding, then I use ucnv_setFallback to reset the state of the encoder when I'm done with the operation. Test: imported/w3c/web-platform-tests/url/resources/percent-encoding.window.html * LayoutTests/imported/w3c/web-platform-tests/url/percent-encoding.window-expected.txt: * LayoutTests/imported/w3c/web-platform-tests/url/resources/percent-encoding.json: * Source/WebCore/PAL/pal/text/TextCodecICU.cpp: (PAL::TextCodecICU::encode const): Canonical link: https://commits.webkit.org/306768@main
diff --git a/LayoutTests/imported/w3c/web-platform-tests/url/percent-encoding.window-expected.txt b/LayoutTests/imported/w3c/web-platform-tests/url/percent-encoding.window-expected.txt
@@ -13,5 +13,7 @@ PASS Input  with encoding gb18030
 PASS Input  with encoding utf-8
 PASS Input − with encoding shift_jis
 PASS Input − with encoding utf-8
+PASS Input ¢ with encoding iso-8859-2
+PASS Input ¢ with encoding utf-8
 PASS Input á| with encoding utf-8
 
diff --git a/LayoutTests/imported/w3c/web-platform-tests/url/resources/percent-encoding.json b/LayoutTests/imported/w3c/web-platform-tests/url/resources/percent-encoding.json
@@ -39,6 +39,13 @@
       "utf-8": "%E2%88%92"
     }
   },
+  {
+    "input": "\u00A2",
+    "output": {
+      "iso-8859-2": "%26%23162%3B",
+      "utf-8": "%C2%A2"
+    }
+  },
   {
     "input": "á|",
     "output": {
diff --git a/Source/WebCore/PAL/pal/text/TextCodecICU.cpp b/Source/WebCore/PAL/pal/text/TextCodecICU.cpp
@@ -305,6 +305,7 @@ Vector<uint8_t> TextCodecICU::encode(StringView string, UnencodableHandling hand
         ucnv_setFromUCallBack(m_converter.get(), urlEscapedEntityCallback, 0, 0, 0, &error);
         if (U_FAILURE(error))
             return { };
+        ucnv_setFallback(m_converter.get(), false);
         break;
     }
 
@@ -321,6 +322,10 @@ Vector<uint8_t> TextCodecICU::encode(StringView string, UnencodableHandling hand
         ucnv_fromUnicode(m_converter.get(), &target, targetLimit, &source, sourceLimit, 0, true, &error);
         result.append(byteCast<uint8_t>(std::span(buffer)).first(target - buffer.data()));
     } while (needsToGrowToProduceBuffer(error));
+
+    if (handling == UnencodableHandling::URLEncodedEntities)
+        ucnv_setFallback(m_converter.get(), true);
+
     return result;
 }
 

Original file line number	Diff line number	Diff line change
`@@ -39,6 +39,13 @@`
`39`	`39`	`"utf-8": "%E2%88%92"`
`40`	`40`	`}`
`41`	`41`	`},`
	`42`	`+ {`
	`43`	`+ "input": "\u00A2",`
	`44`	`+ "output": {`
	`45`	`+ "iso-8859-2": "%26%23162%3B",`
	`46`	`+ "utf-8": "%C2%A2"`
	`47`	`+ }`
	`48`	`+ },`
`42`	`49`	`{`
`43`	`50`	`"input": "á\|",`
`44`	`51`	`"output": {`