Skip to content

Optimize for performance by only using deterministic parser where needed#2062

Merged
Phergus merged 2 commits intoRPTools:developfrom
nmeier:develop
Jul 1, 2020
Merged

Optimize for performance by only using deterministic parser where needed#2062
Phergus merged 2 commits intoRPTools:developfrom
nmeier:develop

Conversation

@nmeier
Copy link
Copy Markdown
Contributor

@nmeier nmeier commented Jul 1, 2020

(TOOLTIP, expandRoll). This makes headless macros (those that work with h and r roll options) faster as no deterministic AST tree is generated within parser.

Added unit tests for macro branch roll options that previously failed.

Unit testing MapToolLineParser is difficult as it's not truly headless. There are calls to static methods that fail, so I hardened a bit where needed.

fixes #1898


This change is Reviewable

nmeier added 2 commits July 1, 2020 00:17
…ded (TOOLTIP, expandRoll). This makes headless macros (those that work with h and r roll options) faster as no deterministic AST tree is generated within parser.

Added unit tests for macro branch roll options that previously failed.

Unit testing MapToolLineParser is difficult as it's not truly headless. There are calls to static methods that fail, so I hardened a bit where needed.

fixes RPTools#1898
@Phergus Phergus linked an issue Jul 1, 2020 that may be closed by this pull request
Copy link
Copy Markdown
Contributor

@Phergus Phergus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r1.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@Phergus Phergus merged commit 8366dfe into RPTools:develop Jul 1, 2020
@Merudo
Copy link
Copy Markdown
Member

Merudo commented Jul 2, 2020

This gives my framework errors:

Illegal argument type com.google.gson.JsonObject, expecting java.math.BigDecimal or java.lang.String

On the plus sign I noticed an impressive performance improvement.

Example, this gives an error with the PR but didn't before:

[h: json = json.set("{}", "property", "value")] 
[frame("test", "value=" + json):{}]

A workaround is to replace json with string(json).

@nmeier
Copy link
Copy Markdown
Contributor Author

nmeier commented Jul 2, 2020

@Merudo checking your test on illegal argument

@nmeier
Copy link
Copy Markdown
Contributor Author

nmeier commented Jul 2, 2020

confirmed - since json arrays are not flattened all the time the addition doesn't handle the + well, I'll add a test-case. You're workaround to convert to string is right but I think we can make this backwards compatible

[h: "test" + json.set("{}", "property", "value")]

@Merudo
Copy link
Copy Markdown
Member

Merudo commented Jul 2, 2020

It's because the parser was automatically converting json arrays and objects in variables to strings, and it isn't anymore.

Example:

[h: jobj = json.set("{}", "property", "value")] 
[r: "Json Object: " + jobj] 

This returns Json Object: {"property":"value"} in 1.7 but gives an error in develop.

Which may be fine, really. The workaround is just to add string() around the variable.

If we keep this change, we should make it clear to users, though.

@Merudo
Copy link
Copy Markdown
Member

Merudo commented Jul 2, 2020

confirmed - since json arrays are not flattened all the time the addition doesn't handle the + well, I'll add a test-case. You're workaround to convert to string is right but I think we can make this backwards compatible

[h: "test" + json.set("{}", "property", "value")]

Yep, jsons should be automatically converted to strings when used with the + operator, otherwise the operation is not meaningful.

This would preserve backward compatibility while opening up other ways to deal with json elements.

@nmeier
Copy link
Copy Markdown
Contributor Author

nmeier commented Jul 2, 2020

Yeah, it does makes sense to require conversion, otherwise most of functions don't make sense.

Not good for those macro scripts using any of those functions below at this point

    net.rptools.maptool.client.functions 
        InputFunction  
        ReturnFunction  
        StrListFunctions  
        StrPropFunctions 
    net.rptools.parser.function 
        AbstractComparisonFunction 
        AbstractLogicalOperatorFunction 
        AbstractNumberFunction 
    net.rptools.parser.function.impl 
        Addition 
        Equals  (equals could do an equals on a json array, requires another parser update though)
        Eval 
        Mean 
        Median 
        NotEquals  (same, is possible to do on json array)
        StrEquals 
        StrNotEquals 

@Phergus
Copy link
Copy Markdown
Contributor

Phergus commented Jul 4, 2020

@nmeier should we expect a fix for this or should it be reverted?

@selquest
Copy link
Copy Markdown
Contributor

selquest commented Jul 5, 2020

My dev environment (Linux Mint 19.3, JDK 14) is failing a unit test starting at this merge commit - MapToolLineParserTest.testMacros():

expected line #1:`expanded roll sh[...]` doesn't match ==> expected: <
expanded roll shows ...if\(1 == 1, 1, 0\) = 1.
> but was: <
expanded roll shows ���if(1 == 1, 1, 0) = 1�1�
>

@nmeier
Copy link
Copy Markdown
Contributor Author

nmeier commented Jul 5, 2020

I think we should leave as is - making these operations backwards comp we can rule out for operations between string and the (non flattened) json objects. I think one can argue about (not)equals but the code is just full of assumptions about string/bigdecimal so I'm not sure it's worth it.

@nmeier
Copy link
Copy Markdown
Contributor Author

nmeier commented Jul 5, 2020

@selquest looking at that test-case, it works here on Windows. The special characters that maptool adds for expanded Rolls might be different in your setup.

@Merudo
Copy link
Copy Markdown
Member

Merudo commented Jul 5, 2020

I think it would make sense for the Addition functions to automatically convert json elements into strings. This should be a straightforward modification to the parser that would preserve backward compatibility.

@nmeier
Copy link
Copy Markdown
Contributor Author

nmeier commented Jul 6, 2020

added a PR that makes those operations (+, ==, etc.) work backwards so this works fine again

[r: "test" + json.set("{}", "property", "value")]

RPTools/parser#42

@Phergus
Copy link
Copy Markdown
Contributor

Phergus commented Jul 10, 2020

Parser bumped to 1.8.1.

@selquest selquest mentioned this pull request Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Macro performance

4 participants