Skip to content

Add LDAP authentication support#11234

Merged
vitlibar merged 65 commits intoClickHouse:masterfrom
traceon:ldap-per-user-authentication
Jul 16, 2020
Merged

Add LDAP authentication support#11234
vitlibar merged 65 commits intoClickHouse:masterfrom
traceon:ldap-per-user-authentication

Conversation

@traceon
Copy link
Contributor

@traceon traceon commented May 27, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category:

  • New Feature

Changelog entry:

  • Added support of LDAP authentication for preconfigured users ("Simple Bind" method)

Detailed description:

  • Documented in a form of in-line comments in the sample config.xml and users.xml files. See <ldap_servers> and <ldap> tags.

@blinkov blinkov added doc-alert pr-feature Pull request with new product feature labels May 27, 2020
<host>localhost</host>
<port>636</port>
<auth_dn_prefix>cn=</auth_dn_prefix>
<auth_dn_suffix>, ou=users, dc=example, dc=com</auth_dn_suffix>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leading comma is bad idea

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. The logic of that concatenation if pretty straightforward. There could be cases, where automatically added comma can actually cause harm. PostgreSQL's ldapsuffix also assumes that comma.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we validate the existence of leading comma in that case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not, because, there could be cases when the suffix could be empty like in case of Active Directory?

<auth_dn_suffix>, ou=users, dc=example, dc=com</auth_dn_suffix>
<enable_tls>yes</enable_tls>
<tls_cert_verify>demand</tls_cert_verify>
<ca_cert_dir>/path/to/ca_cert_dir</ca_cert_dir>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are default ca used? Do we need to list default ca path explicitly? Is there a way to use self-signed certs?

Copy link
Contributor Author

@traceon traceon May 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, tls_cert_verify controls that. But the description of the behavior is too long to put it here. Meanwhile, it is a typical config for any OpenSSL powered functionality.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it's a typical place where people tend to have problems. :) those questions are rather for QA / documentation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be some settings (like paths) should be inherited by default from global clickhouse ssl client config

Some of other properties may be also need to be configurable
https://www.man7.org/linux/man-pages/man3/ldap_get_option.3.html#TLS_OPTIONS

auto parseExternalAuthenticators(const Poco::Util::AbstractConfiguration & config, Poco::Logger * log)
{
auto external_authenticators = std::make_unique<ExternalAuthenticators>();
parseAndAddLDAPServers(*external_authenticators, config, log);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want to put ldap config outside as we plan to use it not only for authentication?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because it logically belongs here, and it will be very ugly to make those things visible here, where they are used.

constexpr size_t DISK_ACCESS_STORAGE_INDEX = 0;
constexpr size_t USERS_CONFIG_ACCESS_STORAGE_INDEX = 1;

auto parseLDAPServer(const Poco::Util::AbstractConfiguration & config, const String & ldap_server_name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move these parsing details into class ExternalAuthenticators, it doesn't look very good here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, will do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

DOUBLE_SHA1_PASSWORD,

/// Password is checked by a [remote] LDAP server. Connection will be made at each authentication attempt.
LDAP_PASSWORD,
Copy link
Member

@vitlibar vitlibar May 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it would be nice to rename the authentication type to simple LDAP because it has nothing to do with passwords in ClickHouse when you create users. I mean you don't have to specify password when you create an user, e.g.

CREATE USER ... IDENTIFIED WITH ldap SERVER ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also recommend converting that enum to enum class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed LDAP_PASSWORD enum value to LDAP_SERVER.

/** Parses queries like
* CREATE USER [IF NOT EXISTS | OR REPLACE] name
* [IDENTIFIED [WITH {NO_PASSWORD|PLAINTEXT_PASSWORD|SHA256_PASSWORD|SHA256_HASH|DOUBLE_SHA1_PASSWORD|DOUBLE_SHA1_HASH}] BY {'password'|'hash'}]
* [IDENTIFIED [WITH {NO_PASSWORD|PLAINTEXT_PASSWORD|SHA256_PASSWORD|SHA256_HASH|DOUBLE_SHA1_PASSWORD|DOUBLE_SHA1_HASH|LDAP}] BY {'password'|'hash'|'server_name'}]
Copy link
Member

@vitlibar vitlibar May 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understood you're going to implement form

IDENTIFIED WITH ldap SERVER 'server_name' AS yyy

right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I wanted to use SERVER, but then decided to change as little as possible.
Maybe then we can use different syntax here, like: IDENTIFIED USING ldap SERVER 'server_name'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using LDAP_SERVER everywhere, so the syntax will look like this now:

... IDENTIFIED WITH LDAP_SERVER BY 'server_name' ...

Let me know, what do you think about this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lowercase would be more consistent here:

IDENTIFIED WITH ldap_server BY 'server_name'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, it is in lower case. This conversation points to an outdated version of the code.

Poco::JSON::Stringifier::stringify(auth_params_json, oss);
const auto str = oss.str();

column_auth_params.insertData(str.data(), str.size());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a table, the number of rows in each column should be the same.
So it's necessary to add something to the column column_auth_params anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I'll write {} for general case, to kind of hint that a JSON object is what we store there. This should probably also fix a regression detected by integration test I have right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For authentication type other than LDAP_SERVER the following code

std::string_view empty_json = "{}";
column_auth_params.insertData(empty_json.data(), empty_json.size());

works much faster than your code.

Copy link
Contributor Author

@traceon traceon Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, obviously. But is it worth it? Adding other new cases there will be kind of less intrusive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change that, but I believe such special cases only add entropy to the code. (And this looks like not a perf-sensitive code.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

add_library(OpenSSL::SSL ALIAS ${OPENSSL_SSL_LIBRARY})
endif ()

if (ENABLE_LDAP AND USE_INTERNAL_LDAP_LIBRARY)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is table system.build_options, I think it would be useful to add USE_LDAP in there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks.

}
case Authentication::LDAP_SERVER:
{
password = authentication.getServerName();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not a password in this case.
Maybe it's better to rename the variable password into something like with_param.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Renamed to by_value.

expect_password = (check_type != Authentication::NO_PASSWORD);
expect_password = (check_type != Authentication::NO_PASSWORD && check_type != Authentication::LDAP_SERVER);
expect_server = (check_type == Authentication::LDAP_SERVER);
break;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (check_type == Authentication::LDAP_SERVER)
  expect_server_name = true;
else if (check_type != Authentication::NO_PASSWORD)
  expect_password = true;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

auto user = connection_context.getAccessControlManager().read<User>(user_name);
const DB::Authentication::Type user_auth_type = user->authentication.getType();
if (user_auth_type != DB::Authentication::DOUBLE_SHA1_PASSWORD && user_auth_type != DB::Authentication::PLAINTEXT_PASSWORD && user_auth_type != DB::Authentication::NO_PASSWORD)
if (user_auth_type != DB::Authentication::LDAP_SERVER && user_auth_type != DB::Authentication::DOUBLE_SHA1_PASSWORD && user_auth_type != DB::Authentication::PLAINTEXT_PASSWORD && user_auth_type != DB::Authentication::NO_PASSWORD)
Copy link
Member

@vitlibar vitlibar Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like that it should be simplified.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I was just following the pattern there. Let me know, what kind of simplification you have in mind.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just invert the condition:

if (user_auth_type != DB::Authentication::LDAP_SERVER && user_auth_type != DB::Authentication::DOUBLE_SHA1_PASSWORD && user_auth_type != DB::Authentication::PLAINTEXT_PASSWORD && user_auth_type != DB::Authentication::NO_PASSWORD)

=>

if (user_auth_type == DB::Authentication::SHA256_PASSWORD)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, will do. I wonder why it wasn't set to it before my changes though...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

std::map<String, LDAPServerParams> ldap_server_params;
};

std::unique_ptr<ExternalAuthenticators> parseExternalAuthenticators(const Poco::Util::AbstractConfiguration & config, Poco::Logger * log);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this functionality to the constructor of ExternalAuthenticators?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I was following the approach of reading params, that was employed in other similar situations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

namespace DB
{

class ExternalAuthenticators
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this class seems to be too general. In fact this class is designed to manage only LDAP params. And if in the future we want to support another SSO system we'd more likely add another class. I suppose to rename this class into something like LDAPServerParamsManager.

Copy link
Contributor Author

@traceon traceon Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Idea was that if in future another authentication method is added, its params, or even a shared object that represents connection etc., could be placed here, and getLDAPServerParams(name) will be changed to something like getAuthenticatorParams(type, name) and getAuthenticator(type, name).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

traceon added 3 commits June 2, 2020 12:42
* master: (114 commits)
  Update PushingToViewsBlockOutputStream.cpp
  Update PushingToViewsBlockOutputStream.cpp
  make clang-10 happy
  Fix sync_async test (remove timeout)
  CLICKHOUSEDOCS-631: temporary_files_codec, join_on_disk_max_files_to_merge settings. (ClickHouse#11242)
  Suppress output of cancelled queries in clickhouse-client ClickHouse#9473
  Better log messages in ConfigReloader
  fix select from StorageJoin
  Fix unit tests under MSan
  Added test.
  Fix build.
  Fix arguments for AggregateFunctionQuantile/
  Update style.md
  Add a guide on error messages.
  Report dictionary name on dictionary load errors.
  more types in ASOF JOIN (ClickHouse#11301)
  Fix part_log test
  Update test.
  Add perftest.
  Parallel processing for PushingToViewsBlockOutputStream::writeSuffix
  ...
traceon and others added 2 commits July 9, 2020 15:06
* Addintion positive check when user password in LDAP changes.
global_context->getAccessControlManager().setLocalDirectory(access_control_local_path);

/// Sets external authenticators config (LDAP).
global_context->setExternalAuthenticatorsConfig(config());
Copy link
Member

@vitlibar vitlibar Jul 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it better to move that call into auto main_config_reloader = std::make_unique<ConfigReloader> section to allow rereading the configuration if it's changed without restarting ClickHouse?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reading the configuration without restarting ClickHouse would be very nice and would speed up all the tests. We currently need to restart the server whenever we change LDAP server descriptions or users in the configuration files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed a modification, that should reload ExternalAuthenticators config too. @vzakaznikov could you please include tests to verify that?

Copy link
Contributor

@vzakaznikov vzakaznikov Jul 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change causes a crash when LDAP servers are added to the configuration file and server is restarted.

2020.07.10 23:06:02.060267 [ 47 ] {} <Information> TCPHandler: Processed in 0.025695653 sec.
2020.07.10 23:06:02.060348 [ 47 ] {} <Information> TCPHandler: Done processing connection.
2020.07.10 23:06:02.073618 [ 47 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: 127.0.0.1:45272
2020.07.10 23:06:02.073677 [ 47 ] {} <Debug> TCPHandler: Connected ClickHouse client version 20.6.0, revision: 54436, user: user1.
2020.07.10 23:06:02.073716 [ 47 ] {} <Trace> ContextAccess (user1): Settings: readonly=0, allow_ddl=true, allow_introspection_functions=false
2020.07.10 23:06:02.073729 [ 47 ] {} <Trace> ContextAccess (user1): List of all grants: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, TRUNCATE, OPTIMIZE, KILL QUERY, SYSTEM, dictGet, SOURCES ON *.*
2020.07.10 23:06:02.073892 [ 8 ] {} <Trace> BaseDaemon: Received signal 11
2020.07.10 23:06:02.073998 [ 70 ] {} <Fatal> BaseDaemon: ########################################
2020.07.10 23:06:02.074021 [ 70 ] {} <Fatal> BaseDaemon: (version 20.6.1.1, build id: AC3582F40E227CA7DE9C1AB5BD8ADCEEC44DAA45) (from thread 47) (no query) Received signal Segmentation fault (11)
2020.07.10 23:06:02.074040 [ 70 ] {} <Fatal> BaseDaemon: Address: 0x10 Access: read. Address not mapped to object.
2020.07.10 23:06:02.074050 [ 70 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f581f246084 0x560943e 0xdaba6a9 0x98368bf 0xa3b1c27 0x97fb83c 0x9a56068 0xa130366 0xa133d39 0xa1352bc 0xcaf42e7 0xcaf46eb 0xcc335be 0xcc2ff2c 0x7f581f243669 0x7f581f15a323
2020.07.10 23:06:02.074087 [ 70 ] {} <Fatal> BaseDaemon: 3. pthread_mutex_lock @ 0xc084 in /usr/lib/x86_64-linux-gnu/libpthread-2.30.so
2020.07.10 23:06:02.074208 [ 70 ] {} <Fatal> BaseDaemon: 4. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/libcxx/include/atomic:746: pthread_mutex_lock @ 0x560943e in /usr/bin/clickhouse
2020.07.10 23:06:02.074712 [ 70 ] {} <Fatal> BaseDaemon: 5. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/libcxx/src/mutex.cpp:34: std::__1::mutex::lock() @ 0xdaba6a9 in /usr/bin/clickhouse
2020.07.10 23:06:02.074956 [ 70 ] {} <Fatal> BaseDaemon: 6. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/libcxx/include/map:1380: DB::ExternalAuthenticators::getLDAPServerParams(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x98368bf in /usr/bin/clickhouse
2020.07.10 23:06:02.075346 [ 70 ] {} <Fatal> BaseDaemon: 7. /home/vzakaznikov/github/ClickHouse-traceon/build/../src/Access/Authentication.cpp:86: DB::Authentication::isCorrectPassword(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ExternalAuthenticators const&) const @ 0xa3b1c27 in /usr/bin/clickhouse
2020.07.10 23:06:02.075606 [ 70 ] {} <Fatal> BaseDaemon: 8. /home/vzakaznikov/github/ClickHouse-traceon/build/../src/Access/ContextAccess.cpp:214: DB::ContextAccess::isCorrectPassword(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x97fb83c in /usr/bin/clickhouse
2020.07.10 23:06:02.076214 [ 70 ] {} <Fatal> BaseDaemon: 9. /home/vzakaznikov/github/ClickHouse-traceon/build/../src/Interpreters/Context.cpp:681: DB::Context::setUser(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Net::SocketAddress const&) @ 0x9a56068 in /usr/bin/clickhouse
2020.07.10 23:06:02.076704 [ 70 ] {} <Fatal> BaseDaemon: 10. /home/vzakaznikov/github/ClickHouse-traceon/build/../src/Server/TCPHandler.cpp:745: DB::TCPHandler::receiveHello() @ 0xa130366 in /usr/bin/clickhouse
2020.07.10 23:06:02.077358 [ 70 ] {} <Fatal> BaseDaemon: 11. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/libcxx/include/string:1426: DB::TCPHandler::runImpl() @ 0xa133d39 in /usr/bin/clickhouse
2020.07.10 23:06:02.078082 [ 70 ] {} <Fatal> BaseDaemon: 12. /home/vzakaznikov/github/ClickHouse-traceon/build/../src/Server/TCPHandler.cpp:1203: DB::TCPHandler::run() @ 0xa1352bc in /usr/bin/clickhouse
2020.07.10 23:06:02.078392 [ 70 ] {} <Fatal> BaseDaemon: 13. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0xcaf42e7 in /usr/bin/clickhouse
2020.07.10 23:06:02.078712 [ 70 ] {} <Fatal> BaseDaemon: 14. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/libcxx/include/atomic:856: Poco::Net::TCPServerDispatcher::run() @ 0xcaf46eb in /usr/bin/clickhouse
2020.07.10 23:06:02.079049 [ 70 ] {} <Fatal> BaseDaemon: 15. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59: Poco::PooledThread::run() @ 0xcc335be in /usr/bin/clickhouse
2020.07.10 23:06:02.079352 [ 70 ] {} <Fatal> BaseDaemon: 16. /home/vzakaznikov/github/ClickHouse-traceon/build/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223: Poco::ThreadImpl::runnableEntry(void*) @ 0xcc2ff2c in /usr/bin/clickhouse
2020.07.10 23:06:02.079364 [ 70 ] {} <Fatal> BaseDaemon: 17. start_thread @ 0x9669 in /usr/lib/x86_64-linux-gnu/libpthread-2.30.so
2020.07.10 23:06:02.079375 [ 70 ] {} <Fatal> BaseDaemon: 18. __clone @ 0x122323 in /usr/lib/x86_64-linux-gnu/libc-2.30.so
2020.07.10 23:06:02.079381 [ 70 ] {} <Information> SentryWriter: Not sending crash report
2020.07.10 23:06:03.655593 [ 55 ] {} <Debug> ConfigReloader: Loading config '/etc/clickhouse-server/config.xml'
2020.07.10 23:06:03.656675 [ 55 ] {} <Debug> ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration
2020.07.10 23:06:04.708822 [ 48 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: 127.0.0.1:45278
2020.07.10 23:06:04.708904 [ 48 ] {} <Debug> TCPHandler: Connected ClickHouse client version 20.6.0, revision: 54436, user: default.
2020.07.10 23:06:09.152276 [ 32 ] {} <Trace> SystemLog (system.query_log): Flushing system log, 2 entries to flush
2020.07.10 23:06:09.152284 [ 34 ] {} <Trace> SystemLog (system.query_thread_log): Flushing system log, 2 entries to flush
2020.07.10 23:06:09.152423 [ 34 ] {} <Debug> SystemLog (system.query_thread_log): Will use existing table system.query_thread_log for QueryThreadLog
2020.07.10 23:06:09.152424 [ 32 ] {} <Debug> SystemLog (system.query_log): Will use existing table system.query_log for QueryLog
2020.07.10 23:06:09.728105 [ 49 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: 127.0.0.1:45286
2020.07.10 23:06:09.728194 [ 49 ] {} <Debug> TCPHandler: Connected ClickHouse client version 20.6.0, revision: 54436, user: default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crash is fixed, but the issue still exists. Tests need to be adjusted to make sure they execute queries after the config has been parsed and applied.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vzakaznikov With the latest changes, now we have DB::Exception: LDAP server 'openldap1' is not configured. instead of DB::Exception: External authenticators are not configured, but the cause is the same: client attempts to connect, while config hasn't been loaded/applied yet.

case LDAP_SERVER:
{
if (!external_authenticators)
throw Exception("External authenticators are not configured", ErrorCodes::BAD_ARGUMENTS);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code is made more complex that it should be. You are never going to pass nullptr as external_authenticators to this function, so external_authentications should be a reference, not a pointer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it was a reference up until we decided to allow reloaded config to update it. After that, a rare case, when the config reloader hasn't read the config, external_authenticators is null.


void AccessControlManager::setExternalAuthenticatorsConfig(const Poco::Util::AbstractConfiguration & config)
{
external_authenticators = std::make_unique<ExternalAuthenticators>(config, getLogger());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the code can be easier (and safer) if you create external_authenticators in the constructor of AccessControlManager and just call external_authenticators->setConfig(config) here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not a big fan of all these setters and getters. They results in a code whose state transformation are impossible to track. I'll modify the code as you suggest, but it won't make it better, just as bad as it is now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed the change, please take a look.

traceon and others added 9 commits July 11, 2020 21:06
…-user-authentication

* commit 'ceac649c01b0158090cd271776f3219f5e7ff57c': (75 commits)
  [docs] split misc statements (ClickHouse#12403)
  Update 00405_pretty_formats.reference
  Update PrettyCompactBlockOutputFormat.cpp
  Update PrettyBlockOutputFormat.cpp
  Update DataTypeNullable.cpp
  Update 01383_remote_ambiguous_column_shard.sql
  add output_format_pretty_grid_charset setting in docs
  add setting output_format_pretty_grid_charset
  Added a test for ClickHouse#11135
  Update index.md
  RIGHT and FULL JOIN for MergeJoin (ClickHouse#12118)
  Update MergeTreeIndexFullText.cpp
  restart the tests
  [docs] add syntax highlight (ClickHouse#12398)
  query fuzzer
  Fix std::bad_typeid when JSON functions called with argument of wrong type.
  Allow typeid_cast() to cast nullptr to nullptr.
  fix another context-related segfault
  [security docs] actually, only admins can create advisories
  query fuzzer
  ...
* master: (27 commits)
  Whitespaces
  Fix typo
  Fix UBSan report in base64
  Correct default secure port for clickhouse-benchmark ClickHouse#11044
  Remove test with bug ClickHouse#10697
  Update in-functions.md (ClickHouse#12430)
  Allow nullable key in MergeTree
  Update arithmetic-functions.md
  [docs] add rabbitmq docs (ClickHouse#12326)
  Lower block sizes and look what will happen ClickHouse#9248
  Fix lifetime_bytes/lifetime_rows for Buffer direct block write
  Retrigger CI
  Fix up  test_mysql_protocol failed
  Implement lifetime_rows/lifetime_bytes for Buffer engine
  Add comment regarding proxy tunnel usage in PocoHTTPClient.cpp
  Add lifetime_rows/lifetime_bytes interface (exported via system.tables)
  Tiny IStorage refactoring
  Trigger integration-test-runner image rebuild.
  Delete log.txt
  Fix test_mysql_client/test_python_client error
  ...
@traceon traceon closed this Jul 16, 2020
@traceon traceon reopened this Jul 16, 2020
@vitlibar vitlibar merged commit 000b197 into ClickHouse:master Jul 16, 2020
@traceon traceon deleted the ldap-per-user-authentication branch July 19, 2020 17:03
@jjtjiang
Copy link

jjtjiang commented Sep 9, 2020

If there can config username and passoword when 'enable_tls'=no(Specify 'no' for plain text (ldap://) protocol)?

@traceon
Copy link
Contributor Author

traceon commented Sep 9, 2020

@jjtjiang , you mean some master/admin user and password? It is not needed for "Simple Bind" method. With that approach, the actual user's name and password, that is being authenticated by ClickHouse, are used to "bind" in LDAP server, and so they are checked that way.

Having said that, you really should not use plain text protocol, as there is a possibility that passwords will be sent in plain text and could be easily intercepted. Don't use StartTLS either, it is for compatibility with old LDAP servers that don't support pure TLS. Use pure TLS, even if you have to use self-signed certificates.

@jjtjiang
Copy link

get it.thanks.
i user docker start clickhouse but havn't set host:ip ,so it Can't contact LDAP server.
having solved this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature Pull request with new product feature submodule changed At least one submodule changed in this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants