Skip to content

Add a column with a message pattern to text_log#44543

Merged
tavplubix merged 7 commits intomasterfrom
text_log_add_pattern
Jan 17, 2023
Merged

Add a column with a message pattern to text_log#44543
tavplubix merged 7 commits intomasterfrom
text_log_add_pattern

Conversation

@tavplubix
Copy link
Member

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Added a message_format_string column to system.text_log. The column contains a pattern that was used to format the message.

@robot-ch-test-poll4 robot-ch-test-poll4 added pr-improvement Pull request with some product improvements submodule changed At least one submodule changed in this PR. labels Dec 23, 2022
@tavplubix tavplubix added the 🎅 🎁 gift🎄 To make people wonder label Dec 23, 2022
@alexey-milovidov
Copy link
Member

This is really great!

I'd also like it if we add an array of arguments (strings ok).
We will be able to do various analytics on specific values of the arguments.

@rschu1ze rschu1ze self-assigned this Jan 2, 2023
#define LOG_FATAL(logger, ...) LOG_IMPL(logger, DB::LogsLevel::error, Poco::Message::PRIO_FATAL, __VA_ARGS__)


#define LOG_TEST_PREFORMATTED(logger, FORMAT_STRING, MESSAGE) LOG_IMPL_PREFORMATTED(logger, DB::LogsLevel::test, Poco::Message::PRIO_TEST, FORMAT_STRING, MESSAGE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not add another set of logging macros.

  1. To the reader, the difference between the existing and the new set of macros isn't immediately clear (this could be resolved with code comments),
  2. If someone requires another variant of the log macro in future, he would have to duplicate not 7 but 14 macros, leading to 28 macros in total. These things spread like cancer.
  3. There are just two files (out of hundreds) which use the new macros. I am sure that these few places can be converted to the existing macros (maybe at the cost of a negligible performance hit).

@tavplubix
Copy link
Member Author

I'd also like it if we add an array of arguments (strings ok).
We will be able to do various analytics on specific values of the arguments.

Yes, it would be a great enhancement, but it requires formatting and storing all arguments twice (current message_format_string is almost for free). But probably it's not a problem. Or we can introduce a setting...

#include <Poco/Message.h>
#include <Common/CurrentThread.h>

/// This wrapper is usefult to save formatted message into a String before sending it to a logger
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(usefult --> useful)

#include <Common/CurrentThread.h>

/// This wrapper is useful to save formatted message into a String before sending it to a logger
class LogToStrImpl
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't have a useful suggestion just wanted to say that this is a really nice idea to avoid double formatting.

@tavplubix
Copy link
Member Author

Integration tests (release) [2/4] - test_system_merges is flaky
Stress test (asan) - #45252 (comment)
Stress test (msan) - #45252 (comment)
Stress test (ubsan) - #45372

@tavplubix tavplubix merged commit 8b13b85 into master Jan 17, 2023
@tavplubix tavplubix deleted the text_log_add_pattern branch January 17, 2023 17:19
@SaltTan
Copy link
Contributor

SaltTan commented Jan 26, 2023

I installed 23.1 and was surprised to see that my text_log did not change.
I renamed it manually and a new table with the new column got created...

tavplubix added a commit that referenced this pull request Feb 23, 2023
@tavplubix tavplubix mentioned this pull request Feb 23, 2023
tavplubix added a commit that referenced this pull request Feb 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🎅 🎁 gift🎄 To make people wonder pr-improvement Pull request with some product improvements submodule changed At least one submodule changed in this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants