PromQL: Unary operator and simple functions#97510
PromQL: Unary operator and simple functions#97510nikitamikhaylov merged 11 commits intoClickHouse:masterfrom
Conversation
c78b0c0 to
0bdb3dd
Compare
0bdb3dd to
01be3b2
Compare
There was a problem hiding this comment.
Pull request overview
This PR expands ClickHouse’s PromQL support by adding unary operators (+/-) and several “simple” single-argument functions (conversion, math, and date/time), together with new integration tests and some internal refactoring in the PromQL→SQL converter to support these features.
Changes:
- Add unary operator handling and a central function dispatcher (
applyFunction) to route PromQL functions to specific implementations. - Implement support for
time(),pi(),vector(),scalar(), multiple math functions (e.g.abs,sin,sqrt, …) and date/time functions (e.g.day_of_week,month,year, …). - Update integration tests and adjust evaluation-range / SQL finalization behavior (including
+Infformatting in the HTTP API).
Reviewed changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/integration/test_prometheus_protocols/test_evaluation.py | Adds test data and coverage for unary operators and new functions. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/fromSelector.cpp | Removes empty-range short-circuit; uses node range for selector bounds. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/fromFunctionTime.h / .cpp | Implements time() conversion to scalar / scalar-grid SQL. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/fromFunctionPi.h / .cpp | Implements pi() as a constant scalar. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/finalizeSQL.cpp | Removes EMPTY handling, materializes empty tags, and tightens null handling for values[1]. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/dropMetricName.cpp | Removes EMPTY store-method handling from metric-name dropping. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyUnaryOperator.h / .cpp | Adds unary + / - support for scalars and instant vectors. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyOffset.cpp | Removes EMPTY handling paths. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyMathSimpleFunction.h / .cpp | Adds a set of per-sample math functions for instant vectors. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyFunctionVector.h / .cpp | Implements vector() scalar→instant-vector conversion. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyFunctionScalar.h / .cpp | Implements scalar() instant-vector→scalar conversion (with aggregation when needed). |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyFunctionOverRange.h / .cpp | Refines function detection to string_view and adds a PQT::Function* overload. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyFunction.h / .cpp | Adds a central PromQL function dispatcher used by the converter. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/applyDateTimeFunction.h / .cpp | Adds date/time functions mapping to ClickHouse datetime extractors. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/SQLQueryPiece.h | Removes StoreMethod::EMPTY and adjusts default member initializers. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/NodeEvaluationRangeGetter.cpp | Changes subquery range computation to avoid empty inner ranges. |
| src/Storages/TimeSeries/PrometheusQueryToSQL/Converter.cpp | Routes function nodes through applyFunction and adds unary-operator node handling. |
| src/Storages/TimeSeries/PrometheusHTTPProtocolAPI.cpp | Fixes scalar +Inf rendering to match Prometheus (+Inf / -Inf). |
src/Storages/TimeSeries/PrometheusQueryToSQL/applyDateTimeFunction.cpp
Outdated
Show resolved
Hide resolved
src/Storages/TimeSeries/PrometheusQueryToSQL/applyDateTimeFunction.cpp
Outdated
Show resolved
Hide resolved
src/Storages/TimeSeries/PrometheusQueryToSQL/NodeEvaluationRangeGetter.cpp
Outdated
Show resolved
Hide resolved
3de0470 to
c2b8d10
Compare
6034d80 to
c691f7d
Compare
c691f7d to
e2bf850
Compare
| if (operator_name == "+") | ||
| return res; | ||
|
|
||
| /// We support only '+' and '-' as unary operators. | ||
| chassert(operator_name == "-"); |
There was a problem hiding this comment.
Unary operators are supported now
| if (isFunctionVector(function_name)) | ||
| return applyFunctionVector(function_node, std::move(arguments), context); | ||
|
|
||
| if (isFunctionScalar(function_name)) | ||
| return applyFunctionScalar(function_node, std::move(arguments), context); | ||
|
|
||
| if (isFunctionTime(function_name)) | ||
| return fromFunctionTime(function_node, std::move(arguments), context); | ||
|
|
||
| if (isDateTimeFunction(function_name)) | ||
| return applyDateTimeFunction(function_node, std::move(arguments), context); | ||
|
|
||
| if (isMathSimpleFunction(function_name)) | ||
| return applyMathSimpleFunction(function_node, std::move(arguments), context); | ||
|
|
||
| if (isFunctionPi(function_name)) | ||
| return fromFunctionPi(function_node, std::move(arguments), context); |
There was a problem hiding this comment.
In addition to some over-range functions a bunch of simple functions has been added here: vector(), scalar(), almost all date/time functions (day_of_week(), etc.), almost all math function (abs(), sin(), pi(), etc.).
|
Ready for review |
| } | ||
| } | ||
|
|
||
| using EvaluateWithConstArgumentFunc = Float64 (*)(Float64); |
There was a problem hiding this comment.
Interesting why do we need to evaluate constant arguments separately? We will have 2 different pathes for evaluating constant expressions then.
| } | ||
|
|
||
|
|
||
| SQLQueryPiece applyUnaryOperator( |
There was a problem hiding this comment.
Why do unary operators have a different implementation? We can treat them as simple math functions similarly how ClickHouse does it with negate.
There was a problem hiding this comment.
Unary operators in PromQL can be applied to scalars, e.g.
-5
-time()
or instant vectors, e.g.
-vector(1)
-up
And simple math functions can be applied to instant vectors only, so for example sin(vector(1)) works whereas sin(1) doesn't work.
Also unary operator as a different type of node in PrometheusQueryTree makes some code easier, for example when we transform it to string we need to consider precedences of operators.
| @@ -26,6 +26,10 @@ enum class StoreMethod | |||
| /// Can be used only with type ResultType::STRING. | |||
| CONST_STRING, | |||
|
|
|||
| /// A single scalar is stored in one row and one column named `value` (floating-point). | |||
| /// Can be used with types ResultType::SCALAR, ResultType::INSTANT_VECTOR, ResultType::RANGE_VECTOR. | |||
| SINGLE_SCALAR, | |||
There was a problem hiding this comment.
Do we really need CONST_SCALAR and a SINGLE_SCALAR? Maybe we can only leave the latter one?
There was a problem hiding this comment.
CONST_SCALAR is a kind of optimization.
For example, promql query 5 is currently translated to
SELECT toDateTime64(<current_timestamp>, 3) AS timestamp, toFloat64(5.) AS value
and if we get rid of CONST_SCALAR the same promql query will be translated to
WITH
prometheus_query_step_1 AS (
SELECT toFloat64(5.) AS value
),
SELECT toDateTime64(<current_time>, 3) AS timestamp, toFloat64(value) AS value FROM prometheus_query_step_1
There was a problem hiding this comment.
Perhaps we can keep CONST_SCALAR but use it for scalar literals only, like 5 or -5 and not for results of math functions like abs(vector(-5)) - in order to have only one path of evaluation.
nikitamikhaylov
left a comment
There was a problem hiding this comment.
LGTM. A few questions arose, but we can merge.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
PromQL: Unary operator and simple functions
Add support for unary operators
+and-and also for simple functions taking one argument:vector(),scalar(),abs(),sin(),day_of_week(), etc.Part of #89356