Commit 917c1e3
Dmitry Lenev
Patch that changes approach to how we acquire metadata
locks for DML statements and changes the way MDL locks
are acquired/granted in contended case.
Instead of backing-off when a lock conflict is encountered
and waiting for it to go away before restarting open_tables()
process we now wait for lock to be released without releasing
any previously acquired locks. If conflicting lock goes away
we resume opening tables. If waiting leads to a deadlock we
try to resolve it by backing-off and restarting open_tables()
immediately.
As result both waiting for possibility to acquire and
acquiring of a metadata lock now always happen within the
same MDL API call. This has allowed to make release of a lock
and granting it to the most appropriate pending request an
atomic operation.
Thanks to this it became possible to wake up during release
of lock only those waiters which requests can be satisfied
at the moment as well as wake up only one waiter in case
when granting its request would prevent all other requests
from being satisfied. This solves thundering herd problem
which occured in cases when we were releasing some lock and
woke up many waiters for SNRW or X locks (this was the issue
in bug#52289 "performance regression for MyISAM in sysbench
OLTP_RW test".
This also allowed to implement more fair (FIFO) scheduling
among waiters with the same priority.
It also opens the door for introducing new types of requests
for metadata locks such as low-prio SNRW lock which is
necessary in order to support LOCK TABLES LOW_PRIORITY WRITE.
Notice that after this sometimes can report ER_LOCK_DEADLOCK
error in cases in which it has not happened before.
Particularly we will always report this error if waiting for
conflicting lock has happened in the middle of transaction
and resulted in a deadlock. Before this patch the error was
not reported if deadlock could have been resolved by backing
off all metadata locks acquired by the current statement.
--BZR--
revision-id: [email protected]
property-branch-nick: mysql-trunk-wfl-3
property-file-info: ld7:file_id49:mdl_sync.result-20090302184602-k11i6o8vh2s5umno-17:message302:Added test coverage for some aspects of deadlock handling in
property-file-info: metadata locking subsystem.
property-file-info: Adjusted test case after removing back-off in general case
property-file-info: when conflicting metadata lock is encountered during
property-file-info: open_tables() (now this happens only if waiting for
property-file-info: conflicting lock to go away leads to a deadlock).4:path28:mysql-test/r/mdl_sync.resulted7:file_id48:sp_sync.result-20100112141220-hrw3yyvlvrr1hnom-17:message213:Adjusted test case after removing back-off in general case
property-file-info: when conflicting metadata lock is encountered during
property-file-info: open_tables() (now this happens only if waiting for
property-file-info: conflicting lock to go away leads to a deadlock).4:path27:mysql-test/r/sp_sync.resulted7:file_id55:dml_setup_instrument-20100112014603-ubwjo2a0fk4thccy-517:message99:Adjusted test results after renaming MDL_context::
property-file-info: m_waiting_for_lock rwlock to m_LOCK_waiting_for.4:path58:mysql-test/suite/perfschema/r/dml_setup_instruments.resulted7:file_id66:sp1f-rpl_sp.result-20050505122047-6pz3qkb234acgvxly33c2rm665rolo6w7:message392:Adjusted test case after implementing new approach to
property-file-info: acquiring metadata locks in open_tables(). We no longer
property-file-info: release all MDL locks acquired by statement before waiting
property-file-info: for conflicting lock to go away. As result DROP FUNCTION
property-file-info: statement has to wait for DML statement which managed to
property-file-info: acquire metadata lock on function being dropped and now
property-file-info: waits for other conflicting metadata lock to go away.4:path36:mysql-test/suite/rpl/r/rpl_sp.resulted7:file_id64:sp1f-rpl_sp.test-20050505122048-sfpoerdz7zbb5decjlbpxcme6ew43qcp7:message392:Adjusted test case after implementing new approach to
property-file-info: acquiring metadata locks in open_tables(). We no longer
property-file-info: release all MDL locks acquired by statement before waiting
property-file-info: for conflicting lock to go away. As result DROP FUNCTION
property-file-info: statement has to wait for DML statement which managed to
property-file-info: acquire metadata lock on function being dropped and now
property-file-info: waits for other conflicting metadata lock to go away.4:path34:mysql-test/suite/rpl/t/rpl_sp.tested7:file_id47:mdl_sync.test-20090302184558-2nfnp7wh7pj6ra6e-17:message302:Added test coverage for some aspects of deadlock handling in
property-file-info: metadata locking subsystem.
property-file-info: Adjusted test case after removing back-off in general case
property-file-info: when conflicting metadata lock is encountered during
property-file-info: open_tables() (now this happens only if waiting for
property-file-info: conflicting lock to go away leads to a deadlock).4:path26:mysql-test/t/mdl_sync.tested7:file_id46:sp_sync.test-20100112141216-z3d36b7ouqevywqw-17:message213:Adjusted test case after removing back-off in general case
property-file-info: when conflicting metadata lock is encountered during
property-file-info: open_tables() (now this happens only if waiting for
property-file-info: conflicting lock to go away leads to a deadlock).4:path25:mysql-test/t/sp_sync.tested7:file_id40:mdl.cc-20080523121737-j62pi0m62eaw1hq6-17:message3296:Changed MDL subsystem to support new approach to acquring
property-file-info: metadata locks in open tables and more fair and efficient
property-file-info: scheduling of metadata locks. To implement this:
property-file-info: - Made releasing of the lock and granting it to the most
property-file-info: appropriate pending request atomic operation. As result it
property-file-info: became possible to wake up only those waiters requests from
property-file-info: which can be satisfied at the moment as well as wake-up
property-file-info: only one waiter in case when granting its request would
property-file-info: prevent all other requests from being satisfied.
property-file-info: This solved thundering herd problem which occured in cases
property-file-info: when we were releasing some lock and woke up many waiters
property-file-info: for SNRW or X locks (this was the issue in Bug #52289
property-file-info: "performance regression for MyISAM in sysbench OLTP_RW
property-file-info: test".
property-file-info: To emphasize above changes wake_up_waiters() was renamed
property-file-info: to MDL_context::reschedule_waiters().
property-file-info: - Changed code to add tickets for new requests to the back of
property-file-info: waiters queue and to select tickets to be satisfied from
property-file-info: the head of the queue if possible (this makes scheduling of
property-file-info: requests with the same priority fair). To be able to do
property-file-info: this efficiently we now use for waiting and granted queues
property-file-info: version of I_P_List class which provides fast push_back()
property-file-info: method.
property-file-info: - Members and methods of MDL_context related to sending
property-file-info: and waiting for signal were moved to separate MDL_wait
property-file-info: class.
property-file-info: - Since in order to avoid race conditions we must grant the
property-file-info: lock only to the context which was not chosen as a victim
property-file-info: of deadlock, killed or aborted due to timeout
property-file-info: MDL_wait::set_status() (former awake()) was changed not to
property-file-info: send signal if signal slot is already occupied and to
property-file-info: indicate this fact through its return value. As another
property-file-info: consequence MDL_wait::timed_wait() method was changed to
property-file-info: handle timeout (optionally) and abort due to kill as
property-file-info: signals which make signal slot occupied.
property-file-info: - Renamed MDL_context::acquire_lock_impl() to acquire_lock().
property-file-info: Changed it to be able correctly process requests for shared
property-file-info: locks when there are open HANDLERs, made this method more
property-file-info: optimized for acquisition of shared locks. As part of this
property-file-info: change moved code common between try_acquire_lock() and
property-file-info: acquire_lock() to new try_acquire_lock_impl() method.
property-file-info: Also adjusted acquire_lock()'s code to take into account
property-file-info: the fact that in cases when lock is granted as result of
property-file-info: MDL_context::reschedule_waiters() call (i.e. when it is
property-file-info: granted after waiting for lock conflict to go away)
property-file-info: updating MDL_lock state is responsibility of the thread
property-file-info: calling reschedule_waiters().
property-file-info: - Changed MDL_context::find_deadlock() to send VICTIM
property-file-info: signal even if victim is the context which has initiated
property-file-info: deadlock detection. This is required in order to avoid
property-file-info: races in cases when the same context simultaneously is
property-file-info: chosen as a victim and its request for lock is satisfied.
property-file-info: As result return value of this method became unnecessary
property-file-info: and it was changed to return void.
property-file-info: Adjusted MDL_lock::find_deadlock() method to take into
property-file-info: account that now there can be a discrepancy between
property-file-info: MDL_context::m_waiting_for value being set and real state
property-file-info: of the ticket this member points to.
property-file-info: - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
property-file-info: and MDL_context::stop_waiting() to done_waiting_for().
property-file-info: - Finally, removed MDL_context::wait_for_lock() method.4:path10:sql/mdl.cced7:file_id39:mdl.h-20080523121748-o4y2wcq3maotb9do-17:message1538:Changed MDL subsystem to support new approach to acquring
property-file-info: metadata locks in open tables and more fair and efficient
property-file-info: scheduling of metadata locks. To implement this:
property-file-info: - Members and methods of MDL_context related to sending
property-file-info: and waiting for signal were moved to separate MDL_wait
property-file-info: class.
property-file-info: - Since now in order to avoid race conditions we must grant
property-file-info: the lock only to the context which was not chosen as a
property-file-info: victim of deadlock, killed or aborted due to timeout
property-file-info: MDL_wait::set_status (former awake()) was changed not to
property-file-info: send signal if signal slot is already occupied and to
property-file-info: indicate this fact through its return value.
property-file-info: Also NORMAL_WAKE_UP signal became GRANTED, and timeouts
property-file-info: and aborts due to kill became full blown signals rather
property-file-info: than simple return values.
property-file-info: - MDL_wait::timed_wait() now takes extra parameter that
property-file-info: indicates whether signal should be set if timeout is
property-file-info: reached.
property-file-info: - Enabled fast push_back() operation in MDL_context::m_tickets
property-file-info: list to make move_ticket_after_trans_sentinel() method more
property-file-info: efficient.
property-file-info: - Removed MDL_context::wait_for_lock() method.
property-file-info: - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
property-file-info: and MDL_context::stop_waiting() to done_waiting_for().
property-file-info: - MDL_context::acquire_lock_impl() became acquire_lock().
property-file-info: - Introduced MDL_context::try_acquire_lock_impl() as a
property-file-info: place for code shared by try_acquire_lock and
property-file-info: acquire_lock().
property-file-info: - Due to fact that now VICTIM signal is sent even if victim
property-file-info: is the context which has initiated deadlock detection
property-file-info: find_deadlock() no longer needs a return value.4:path9:sql/mdl.hed7:file_id64:sp1f-sql_base.cc-19700101030959-w7tul2gb2n4jzayjwlslj3ybmf3uhk6a7:message2187:Implemented new approach to acquiring metadata locks in
property-file-info: open_tables(). We no longer perform back-off when conflicting
property-file-info: metadata lock is encountered. Instead we wait for this lock
property-file-info: to go away while holding all locks which were acquired so
property-file-info: far. Back-off is only used in situation when further waiting
property-file-info: will cause a deadlock which could be avoided by performing
property-file-info: back-off and restarting open_tables() process. Absence of
property-file-info: waiting between back-off and restart of acquiring metadata
property-file-info: locks can't lead to livelocks as MDL subsystem was changed
property-file-info: to make release of lock and granting it to waiting lock
property-file-info: an atomic action, so back-off will automatically give way
property-file-info: to other participants of deadlock loop.
property-file-info: Accordingly:
property-file-info: - open_table_get_mdl_lock() and open_and_process_routine()
property-file-info: were changed to wait for conflicting metadata lock to
property-file-info: go away without back-off. Only if such wait leads to a
property-file-info: deadlock back-off is requested. As part of this change
property-file-info: new error handler class was introduced which converts,
property-file-info: if possible, ER_LOCK_DEADLOCK error to a request for
property-file-info: back-off and re-start of open_tables() process.
property-file-info: - Open_table_context::recover_from_failed_open() was changed
property-file-info: not to wait in case of metadata lock conflict. Instead we
property-file-info: immediately proceed to re-acquiring locks.
property-file-info: - Open_table_context::request_backoff_action() now always
property-file-info: emits error if back-off is requested in the middle of
property-file-info: transaction as we can't be sure that releasing lock
property-file-info: which were acquired only by current statement will
property-file-info: resolve a deadlock. Before this patch such situations were
property-file-info: successfully detected thanks to the fact that we called
property-file-info: MDL_context::wait_for_lock() method in
property-file-info: recover_from_failed_open().
property-file-info: - In order to avoid deadlocks open_tables() code was adjusted
property-file-info: to flush open HANDLERs for which there are pending requests
property-file-info: for X locks before restarting the process of acquiring
property-file-info: metadata locks.
property-file-info: - Changed close_tables_for_reopen() not to reset MDL_request
property-file-info: for tables belonging to the tail of prelocking list. It is
property-file-info: no longer necessary as these MDL_request objects won't be
property-file-info: used for any waiting.
property-file-info: - Adjusted comment in tdc_wait_for_old_version() to avoid
property-file-info: mentioning removed MDL_context::wait_for_lock() method.4:path15:sql/sql_base.cced7:file_id45:sql_base.h-20100331135644-cgcb6oowzqyx7fi3-127:message271:As we no longer wait for conflicting metadata lock away in
property-file-info: Open_table_context::recover_from_failed_open() method,
property-file-info: Open_table_context::OT_WAIT_MDL_LOCK action was renamed to
property-file-info: OT_MDL_CONFLICT.
property-file-info: Also Open_table_context::m_failed_mdl_request became
property-file-info: unnecessary and was removed.4:path14:sql/sql_base.hed7:file_id45:sql_plist.h-20080523121803-vhna5yyou5h80k5o-17:message126:Extended I_P_List template to support efficient push_back()
property-file-info: operation if it is parameterized with an appropriate policy
property-file-info: class.4:path15:sql/sql_plist.hed7:file_id64:sp1f-sql_show.cc-19700101030959-umlljfnpplg452h7reeyqr4xnbmlkvfj7:message186:Adjusted code after removal of MDL_context::wait_for_lock()
property-file-info: method. Now if one needs to acquire metadata lock with waiting
property-file-info: one has to use a variant of MDL_context::acquire_lock() method.4:path15:sql/sql_show.ccee
testament3-sha1: 466d90b279fee5b6112f0a9e917ff359640781701 parent b4696db commit 917c1e3
13 files changed
Lines changed: 1065 additions & 510 deletions
File tree
- mysql-test
- r
- suite
- perfschema/r
- rpl
- r
- t
- t
- sql
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1765 | 1765 | | |
1766 | 1766 | | |
1767 | 1767 | | |
| 1768 | + | |
1768 | 1769 | | |
1769 | 1770 | | |
1770 | 1771 | | |
| |||
1943 | 1944 | | |
1944 | 1945 | | |
1945 | 1946 | | |
| 1947 | + | |
| 1948 | + | |
| 1949 | + | |
| 1950 | + | |
| 1951 | + | |
| 1952 | + | |
| 1953 | + | |
| 1954 | + | |
| 1955 | + | |
| 1956 | + | |
| 1957 | + | |
| 1958 | + | |
| 1959 | + | |
| 1960 | + | |
| 1961 | + | |
| 1962 | + | |
| 1963 | + | |
| 1964 | + | |
| 1965 | + | |
| 1966 | + | |
| 1967 | + | |
| 1968 | + | |
| 1969 | + | |
| 1970 | + | |
| 1971 | + | |
| 1972 | + | |
| 1973 | + | |
| 1974 | + | |
| 1975 | + | |
| 1976 | + | |
| 1977 | + | |
| 1978 | + | |
| 1979 | + | |
| 1980 | + | |
| 1981 | + | |
| 1982 | + | |
| 1983 | + | |
| 1984 | + | |
| 1985 | + | |
| 1986 | + | |
| 1987 | + | |
| 1988 | + | |
| 1989 | + | |
| 1990 | + | |
| 1991 | + | |
| 1992 | + | |
| 1993 | + | |
| 1994 | + | |
| 1995 | + | |
| 1996 | + | |
| 1997 | + | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
| 2001 | + | |
| 2002 | + | |
| 2003 | + | |
| 2004 | + | |
| 2005 | + | |
| 2006 | + | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
| 2012 | + | |
| 2013 | + | |
| 2014 | + | |
| 2015 | + | |
| 2016 | + | |
| 2017 | + | |
| 2018 | + | |
| 2019 | + | |
| 2020 | + | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
| 2032 | + | |
| 2033 | + | |
| 2034 | + | |
| 2035 | + | |
| 2036 | + | |
| 2037 | + | |
| 2038 | + | |
1946 | 2039 | | |
1947 | 2040 | | |
1948 | 2041 | | |
| |||
2175 | 2268 | | |
2176 | 2269 | | |
2177 | 2270 | | |
2178 | | - | |
| 2271 | + | |
2179 | 2272 | | |
2180 | 2273 | | |
2181 | 2274 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
62 | 63 | | |
63 | 64 | | |
64 | | - | |
| 65 | + | |
| 66 | + | |
65 | 67 | | |
66 | 68 | | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
80 | 77 | | |
81 | 78 | | |
82 | 79 | | |
83 | | - | |
84 | | - | |
85 | | - | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
89 | 90 | | |
90 | 91 | | |
91 | 92 | | |
92 | 93 | | |
93 | | - | |
| 94 | + | |
94 | 95 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1215 | 1215 | | |
1216 | 1216 | | |
1217 | 1217 | | |
1218 | | - | |
1219 | | - | |
| 1218 | + | |
| 1219 | + | |
1220 | 1220 | | |
| 1221 | + | |
1221 | 1222 | | |
1222 | 1223 | | |
1223 | | - | |
| 1224 | + | |
1224 | 1225 | | |
1225 | | - | |
| 1226 | + | |
| 1227 | + | |
| 1228 | + | |
1226 | 1229 | | |
1227 | 1230 | | |
| 1231 | + | |
1228 | 1232 | | |
1229 | 1233 | | |
| 1234 | + | |
1230 | 1235 | | |
1231 | 1236 | | |
1232 | 1237 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
657 | 657 | | |
658 | 658 | | |
659 | 659 | | |
660 | | - | |
661 | | - | |
| 660 | + | |
| 661 | + | |
662 | 662 | | |
663 | 663 | | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
664 | 668 | | |
665 | 669 | | |
666 | | - | |
| 670 | + | |
667 | 671 | | |
668 | 672 | | |
669 | | - | |
670 | 673 | | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
671 | 679 | | |
672 | 680 | | |
673 | 681 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2411 | 2411 | | |
2412 | 2412 | | |
2413 | 2413 | | |
| 2414 | + | |
2414 | 2415 | | |
2415 | 2416 | | |
2416 | 2417 | | |
| |||
2700 | 2701 | | |
2701 | 2702 | | |
2702 | 2703 | | |
| 2704 | + | |
| 2705 | + | |
| 2706 | + | |
| 2707 | + | |
| 2708 | + | |
| 2709 | + | |
| 2710 | + | |
| 2711 | + | |
| 2712 | + | |
| 2713 | + | |
| 2714 | + | |
| 2715 | + | |
| 2716 | + | |
| 2717 | + | |
| 2718 | + | |
| 2719 | + | |
| 2720 | + | |
| 2721 | + | |
| 2722 | + | |
| 2723 | + | |
| 2724 | + | |
| 2725 | + | |
| 2726 | + | |
| 2727 | + | |
| 2728 | + | |
| 2729 | + | |
| 2730 | + | |
| 2731 | + | |
| 2732 | + | |
| 2733 | + | |
| 2734 | + | |
| 2735 | + | |
| 2736 | + | |
| 2737 | + | |
| 2738 | + | |
| 2739 | + | |
| 2740 | + | |
| 2741 | + | |
| 2742 | + | |
| 2743 | + | |
| 2744 | + | |
| 2745 | + | |
| 2746 | + | |
| 2747 | + | |
| 2748 | + | |
| 2749 | + | |
| 2750 | + | |
| 2751 | + | |
| 2752 | + | |
| 2753 | + | |
| 2754 | + | |
| 2755 | + | |
| 2756 | + | |
| 2757 | + | |
| 2758 | + | |
| 2759 | + | |
| 2760 | + | |
| 2761 | + | |
| 2762 | + | |
| 2763 | + | |
| 2764 | + | |
| 2765 | + | |
| 2766 | + | |
| 2767 | + | |
| 2768 | + | |
| 2769 | + | |
| 2770 | + | |
| 2771 | + | |
| 2772 | + | |
| 2773 | + | |
| 2774 | + | |
| 2775 | + | |
| 2776 | + | |
| 2777 | + | |
| 2778 | + | |
| 2779 | + | |
| 2780 | + | |
| 2781 | + | |
| 2782 | + | |
| 2783 | + | |
| 2784 | + | |
| 2785 | + | |
| 2786 | + | |
| 2787 | + | |
| 2788 | + | |
| 2789 | + | |
| 2790 | + | |
| 2791 | + | |
| 2792 | + | |
| 2793 | + | |
| 2794 | + | |
| 2795 | + | |
| 2796 | + | |
| 2797 | + | |
| 2798 | + | |
| 2799 | + | |
| 2800 | + | |
| 2801 | + | |
| 2802 | + | |
| 2803 | + | |
| 2804 | + | |
| 2805 | + | |
| 2806 | + | |
| 2807 | + | |
| 2808 | + | |
| 2809 | + | |
| 2810 | + | |
| 2811 | + | |
| 2812 | + | |
| 2813 | + | |
| 2814 | + | |
| 2815 | + | |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
2703 | 2834 | | |
2704 | 2835 | | |
2705 | 2836 | | |
| |||
3097 | 3228 | | |
3098 | 3229 | | |
3099 | 3230 | | |
3100 | | - | |
| 3231 | + | |
3101 | 3232 | | |
3102 | 3233 | | |
3103 | 3234 | | |
| |||
0 commit comments