Skip to content

Initial implementation: Support \U\u\L\l replace modifiers#96128

Merged
roblourens merged 10 commits intomicrosoft:masterfrom
irridia:kb-filereplace-case
Jun 7, 2020
Merged

Initial implementation: Support \U\u\L\l replace modifiers#96128
roblourens merged 10 commits intomicrosoft:masterfrom
irridia:kb-filereplace-case

Conversation

@irridia
Copy link
Contributor

@irridia irridia commented Apr 25, 2020

Support \U\u\L\l modifiers to search replacement to alter word case in the per-file Find/Replace flow. This does not affect the "Replace in Files..." flow, which appears to rely on ripgrep.

The behavior is patterned after Boost, but only applied to the replacement text, not subsequent content. Thus the \E isn't relevant. This is also vaguely similar to other replacement syntax, like pcre2 and perl.

For example, to export a file full of non-exported functions in a Go file:
Find:
^func (\w+)\(
Replace:
func \u$1(
Example:
func myPrivateFunc(...
Result:
func MyPrivateFunc(...
With \U:
func MYPRIVATEFUNC(...
With \l\l\l\U:
func mypRIVATEFUNC(...

This PR partially addresses #12185

@msftclas
Copy link

msftclas commented Apr 25, 2020

CLA assistant check
All CLA requirements met.

@roblourens
Copy link
Member

roblourens commented Apr 28, 2020

Thanks for the PR! We are wrapping up the April release this week but I will review it next week for May. It looks good at first glance.

@roblourens roblourens added this to the May 2020 milestone Apr 28, 2020
@irridia irridia changed the title Initial implementation: Support \U\u\L\l\E replace modifiers Initial implementation: Support \U\u\L\l replace modifiers Apr 28, 2020
@irridia irridia marked this pull request as ready for review April 28, 2020 04:28
Copy link
Member

@roblourens roblourens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a good approach, but it's only hooked up for the find widget in the editor, could you do it for both the editor and workspace replace? All you have to do is move it to a helper method that can be called from replacePattern.ts and also from somewhere in .../search/.... You can look at what we did for the "preserve case" feature for inspiration, see how buildReplaceStringWithCasePreserved is called.

Copy link
Member

@alexdima alexdima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good work.

I am OK to merge this, and adopt it for the search across files in a separate PR.

@alexdima alexdima removed their assignment May 7, 2020
@roblourens roblourens modified the milestones: May 2020, June 2020 Jun 2, 2020
@irridia
Copy link
Contributor Author

irridia commented Jun 5, 2020

I spent some time yesterday looking through the implementations of ReplacePattern in vs/editor/contrib/find/replacePattern.ts and vs/workbench/services/search/common/replace.ts. Each of these implements different mechanisms for the CharCode switches, so a small CharCode-aware-only helper function doesn't seem viable.

The alternative IMO is to choose (or combine) one of the ReplacePattern implementations, move it to vs/base/common/search.ts (or perhaps .../replace.ts) and tweak the small handful of callers.

I'm probably not the best choice to perform a nuanced refactor since I'm a TS noob; however I can likely implement the replacement change in .../replace.ts as a parallel to .../replacePattern.ts.

Thoughts?

@roblourens
Copy link
Member

Yeah, I see why it will be a little complicated to do that and why it is different than what we did for buildReplaceStringWithCasePreserved. Maybe we can merge this PR and just do a separate one with a parallel implementation for search? Or I could do some refactoring and try to get one implementation for both.

@irridia
Copy link
Contributor Author

irridia commented Jun 5, 2020

I'd be happy to do the parallel PR on the workbench path. It's sort of continuing artificial code separation, but it will at least eliminate a new difference between the implementations.

@roblourens
Copy link
Member

That sounds good to me!

@roblourens roblourens merged commit a1de2a7 into microsoft:master Jun 7, 2020
@irridia
Copy link
Contributor Author

irridia commented Jun 9, 2020

I've spent a couple of hours on implementing case operations in .../replace.ts, but I've realized that I would need to essentially reimplement String.replace(), since that's what it fully relies on. .../replacePattern.ts already has this type of piecemeal replacement logic, so I'm thinking it would be /less/ work (and far cleaner) to use replacePattern.ReplacePattern everywhere that replace.ReplacePattern is currently used (seems like only .../searchModel.ts).

@roblourens
Copy link
Member

I guess I don't really understand the difference between the two implementations @alexdima and @sandy081, why is the editor's ReplacePattern manually resolving $ references in the replace string instead of just doing a String.replace like search's version?

@alexdima
Copy link
Member

I am sorry that I don't remember everything. Here is a comment I could find -- #18111 (comment) ...

I think I found the issue #18111 (searching for /\b\s{3}\b/) to be fundamentally caused by the researching in the matched portion of a string. In other words, searching for a regex in a large string does not yield the same result/groups as searching for the same regex in the substring that did match because regular expressions sometimes refer to characters around the matched text (like \b does here).

So I have changed the text buffer search to return captured groups while searching, so the capture groups are those of searching in a large string. But the search across files could not do that, so I believe that is the point where I forked things.

@roblourens
Copy link
Member

Oh, that's right. So you are able to get the groups as you go. Search is not able to do that because ripgrep doesn't provide that detailed info, so we have to rerun the regex on the frontend, with hacks to make it work for lookaround. So given that limitation I think we would have to either

  • rerun the regex to figure out what the groups are, construct the same info, then go through the replacePattern
  • or maintain the totally separate implementation, possibly using String.replace with a function callback to implement the case changing

Haven't looked closely enough to eliminate either of those, I think the second sounds easier, what do you think @irridia?

@irridia
Copy link
Contributor Author

irridia commented Jun 10, 2020

I'm not yet familiar with the regex handoffs between TS and ripgrep—would ripgrep have to support the same case change operations, or is TS performing the replacement and ripgrep is more of a file buffer, match-only iterator?

A String.replace() replacer() function defeats the internal parameter handling AFAICT, which means that the replacer has to walk the replacement pattern character at a time and replace the various $N instances. This isn't hugely challenging but it's very reinvent-the-wheel-y and duplicates a lot of the work and logic I see in .../replacePattern.ts.

So while I agree the second sounds easier, it also feels like it's fairly significantly increasing the difference between these fundamentally identical code paths. The current codepath is already applying the regex twice, once into _regExp and once with he text.replace()es.

Moving the vs/editor/contrib/find logic to vs/base/common might be the best use of time, even if it delays the case ops support. Just IMHO, still very much a noob to this codebase.

@roblourens
Copy link
Member

Ripgrep wouldn't have to support it. It only tells us what the matches are. So we are stuck at least rerunning the regex. And I wouldn't call the two code paths fundamentally identical, they have the same apparent effect but have to do it in different ways.

Using a replacement function would be extra work but at least then it parses the replacement patterns for you, which I think is a lot of the work.

And I am ok with implementing this feature separately but still I would support you if you want to look into trying to merge the two code paths, that would be awesome if it's possible, as long as it's not going to lead to breaking a bunch of stuff.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 22, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants