forked from taskflow/taskflow
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathexecutor_8hpp_source.html
More file actions
141 lines (139 loc) · 165 KB
/
executor_8hpp_source.html
File metadata and controls
141 lines (139 loc) · 165 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
<!-- HTML header for doxygen 1.8.13-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.14"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Cpp-Taskflow</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<link rel="icon" type="image/x-icon" href="favicon.ico" />
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(document).ready(initResizable);
/* @license-end */</script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectalign" style="padding-left: 0.5em;">
<div id="projectname"><a href="https://github.com/cpp-taskflow/cpp-taskflow">Cpp-Taskflow</a>
 <span id="projectnumber">2.4-master-branch</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.14 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(function() {
initMenu('',true,false,'search.php','Search');
$(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function(){initNavTree('executor_8hpp_source.html','');});
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<div class="header">
<div class="headertitle">
<div class="title">executor.hpp</div> </div>
</div><!--header-->
<div class="contents">
<div class="fragment"><div class="line"><a name="l00001"></a><span class="lineno"> 1</span> <span class="preprocessor">#pragma once</span></div><div class="line"><a name="l00002"></a><span class="lineno"> 2</span> </div><div class="line"><a name="l00003"></a><span class="lineno"> 3</span> <span class="preprocessor">#include <iostream></span></div><div class="line"><a name="l00004"></a><span class="lineno"> 4</span> <span class="preprocessor">#include <vector></span></div><div class="line"><a name="l00005"></a><span class="lineno"> 5</span> <span class="preprocessor">#include <cstdlib></span></div><div class="line"><a name="l00006"></a><span class="lineno"> 6</span> <span class="preprocessor">#include <cstdio></span></div><div class="line"><a name="l00007"></a><span class="lineno"> 7</span> <span class="preprocessor">#include <random></span></div><div class="line"><a name="l00008"></a><span class="lineno"> 8</span> <span class="preprocessor">#include <atomic></span></div><div class="line"><a name="l00009"></a><span class="lineno"> 9</span> <span class="preprocessor">#include <memory></span></div><div class="line"><a name="l00010"></a><span class="lineno"> 10</span> <span class="preprocessor">#include <deque></span></div><div class="line"><a name="l00011"></a><span class="lineno"> 11</span> <span class="preprocessor">#include <thread></span></div><div class="line"><a name="l00012"></a><span class="lineno"> 12</span> <span class="preprocessor">#include <algorithm></span></div><div class="line"><a name="l00013"></a><span class="lineno"> 13</span> <span class="preprocessor">#include <set></span></div><div class="line"><a name="l00014"></a><span class="lineno"> 14</span> <span class="preprocessor">#include <numeric></span></div><div class="line"><a name="l00015"></a><span class="lineno"> 15</span> <span class="preprocessor">#include <cassert></span></div><div class="line"><a name="l00016"></a><span class="lineno"> 16</span> </div><div class="line"><a name="l00017"></a><span class="lineno"> 17</span> <span class="preprocessor">#include "tsq.hpp"</span></div><div class="line"><a name="l00018"></a><span class="lineno"> 18</span> <span class="preprocessor">#include "notifier.hpp"</span></div><div class="line"><a name="l00019"></a><span class="lineno"> 19</span> <span class="preprocessor">#include "observer.hpp"</span></div><div class="line"><a name="l00020"></a><span class="lineno"> 20</span> <span class="preprocessor">#include "taskflow.hpp"</span></div><div class="line"><a name="l00021"></a><span class="lineno"> 21</span> </div><div class="line"><a name="l00022"></a><span class="lineno"> 22</span> <span class="keyword">namespace </span><a class="code" href="namespacetf.html">tf</a> {</div><div class="line"><a name="l00023"></a><span class="lineno"> 23</span> </div><div class="line"><a name="l00032"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html"> 32</a></span> <span class="keyword">class </span><a class="code" href="classtf_1_1Executor.html">Executor</a> {</div><div class="line"><a name="l00033"></a><span class="lineno"> 33</span> </div><div class="line"><a name="l00034"></a><span class="lineno"> 34</span>  <span class="keyword">struct </span>Worker {</div><div class="line"><a name="l00035"></a><span class="lineno"> 35</span>  <span class="keywordtype">size_t</span> id;</div><div class="line"><a name="l00036"></a><span class="lineno"> 36</span>  <span class="keywordtype">size_t</span> victim;</div><div class="line"><a name="l00037"></a><span class="lineno"> 37</span>  Domain domain;</div><div class="line"><a name="l00038"></a><span class="lineno"> 38</span>  <a class="code" href="classtf_1_1Executor.html">Executor</a>* executor;</div><div class="line"><a name="l00039"></a><span class="lineno"> 39</span>  Notifier::Waiter* waiter;</div><div class="line"><a name="l00040"></a><span class="lineno"> 40</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/numeric/random/mersenne_twister_engine.html">std::mt19937</a> rdgen { <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/numeric/random/random_device.html">std::random_device</a>{}() };</div><div class="line"><a name="l00041"></a><span class="lineno"> 41</span>  TaskQueue<Node*> wsq[NUM_DOMAINS];</div><div class="line"><a name="l00042"></a><span class="lineno"> 42</span>  Node* cache {<span class="keyword">nullptr</span>};</div><div class="line"><a name="l00043"></a><span class="lineno"> 43</span>  };</div><div class="line"><a name="l00044"></a><span class="lineno"> 44</span>  </div><div class="line"><a name="l00045"></a><span class="lineno"> 45</span>  <span class="keyword">struct </span>PerThread {</div><div class="line"><a name="l00046"></a><span class="lineno"> 46</span>  Worker* worker {<span class="keyword">nullptr</span>};</div><div class="line"><a name="l00047"></a><span class="lineno"> 47</span>  };</div><div class="line"><a name="l00048"></a><span class="lineno"> 48</span> </div><div class="line"><a name="l00049"></a><span class="lineno"> 49</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA</span></div><div class="line"><a name="l00050"></a><span class="lineno"> 50</span>  <span class="keyword">struct </span>cudaDevice {</div><div class="line"><a name="l00051"></a><span class="lineno"> 51</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/container/vector.html">std::vector<cudaStream_t></a> streams;</div><div class="line"><a name="l00052"></a><span class="lineno"> 52</span>  };</div><div class="line"><a name="l00053"></a><span class="lineno"> 53</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00054"></a><span class="lineno"> 54</span> </div><div class="line"><a name="l00055"></a><span class="lineno"> 55</span>  <span class="keyword">public</span>:</div><div class="line"><a name="l00056"></a><span class="lineno"> 56</span> </div><div class="line"><a name="l00057"></a><span class="lineno"> 57</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA </span></div><div class="line"><a name="l00058"></a><span class="lineno"> 58</span> </div><div class="line"><a name="l00061"></a><span class="lineno"> 61</span>  <span class="keyword">explicit</span> <a class="code" href="classtf_1_1Executor.html#aa09f058c7f04941de86e1d06238cdc48">Executor</a>(</div><div class="line"><a name="l00062"></a><span class="lineno"> 62</span>  <span class="keywordtype">size_t</span> N = <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency.html">std::thread::hardware_concurrency</a>(),</div><div class="line"><a name="l00063"></a><span class="lineno"> 63</span>  <span class="keywordtype">size_t</span> M = cuda_num_devices()</div><div class="line"><a name="l00064"></a><span class="lineno"> 64</span>  );</div><div class="line"><a name="l00065"></a><span class="lineno"> 65</span> <span class="preprocessor">#else</span></div><div class="line"><a name="l00066"></a><span class="lineno"> 66</span> </div><div class="line"><a name="l00069"></a><span class="lineno"> 69</span>  <span class="keyword">explicit</span> <a class="code" href="classtf_1_1Executor.html#aa09f058c7f04941de86e1d06238cdc48">Executor</a>(<span class="keywordtype">size_t</span> N = <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency.html">std::thread::hardware_concurrency</a>());</div><div class="line"><a name="l00070"></a><span class="lineno"> 70</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00071"></a><span class="lineno"> 71</span>  </div><div class="line"><a name="l00075"></a><span class="lineno"> 75</span>  <a class="code" href="classtf_1_1Executor.html#a5a511b0cc23b264826373d3dabcef670">~Executor</a>();</div><div class="line"><a name="l00076"></a><span class="lineno"> 76</span> </div><div class="line"><a name="l00084"></a><span class="lineno"> 84</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a81f35d5b0a20ac0646447eb80d97c0aa">run</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& taskflow);</div><div class="line"><a name="l00085"></a><span class="lineno"> 85</span> </div><div class="line"><a name="l00094"></a><span class="lineno"> 94</span>  <span class="keyword">template</span><<span class="keyword">typename</span> C></div><div class="line"><a name="l00095"></a><span class="lineno"> 95</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a81f35d5b0a20ac0646447eb80d97c0aa">run</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& taskflow, C&& callable);</div><div class="line"><a name="l00096"></a><span class="lineno"> 96</span> </div><div class="line"><a name="l00105"></a><span class="lineno"> 105</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">run_n</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& taskflow, <span class="keywordtype">size_t</span> N);</div><div class="line"><a name="l00106"></a><span class="lineno"> 106</span> </div><div class="line"><a name="l00116"></a><span class="lineno"> 116</span>  <span class="keyword">template</span><<span class="keyword">typename</span> C></div><div class="line"><a name="l00117"></a><span class="lineno"> 117</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">run_n</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& taskflow, <span class="keywordtype">size_t</span> N, C&& callable);</div><div class="line"><a name="l00118"></a><span class="lineno"> 118</span> </div><div class="line"><a name="l00128"></a><span class="lineno"> 128</span>  <span class="keyword">template</span><<span class="keyword">typename</span> P></div><div class="line"><a name="l00129"></a><span class="lineno"> 129</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">run_until</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& taskflow, P&& pred);</div><div class="line"><a name="l00130"></a><span class="lineno"> 130</span> </div><div class="line"><a name="l00141"></a><span class="lineno"> 141</span>  <span class="keyword">template</span><<span class="keyword">typename</span> P, <span class="keyword">typename</span> C></div><div class="line"><a name="l00142"></a><span class="lineno"> 142</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">run_until</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& taskflow, P&& pred, C&& callable);</div><div class="line"><a name="l00143"></a><span class="lineno"> 143</span>  </div><div class="line"><a name="l00147"></a><span class="lineno"> 147</span>  <span class="keywordtype">void</span> <a class="code" href="classtf_1_1Executor.html#ab9aa252f70e9a40020a1e5a89d485b85">wait_for_all</a>();</div><div class="line"><a name="l00148"></a><span class="lineno"> 148</span> </div><div class="line"><a name="l00152"></a><span class="lineno"> 152</span>  <span class="keywordtype">size_t</span> <a class="code" href="classtf_1_1Executor.html#a79223c7f32c9d715e725b8d775de2d29">num_workers</a>() <span class="keyword">const</span>;</div><div class="line"><a name="l00153"></a><span class="lineno"> 153</span>  </div><div class="line"><a name="l00160"></a><span class="lineno"> 160</span>  <span class="keywordtype">size_t</span> <a class="code" href="classtf_1_1Executor.html#a6d6c28ed58211e4c27a99571e5bf0b6c">num_topologies</a>() <span class="keyword">const</span>;</div><div class="line"><a name="l00161"></a><span class="lineno"> 161</span> </div><div class="line"><a name="l00168"></a><span class="lineno"> 168</span>  <span class="keywordtype">size_t</span> <a class="code" href="classtf_1_1Executor.html#a8acfc81ae6a64989761ce229262c9a21">num_domains</a>() <span class="keyword">const</span>;</div><div class="line"><a name="l00169"></a><span class="lineno"> 169</span> </div><div class="line"><a name="l00176"></a><span class="lineno"> 176</span>  <span class="keywordtype">int</span> <a class="code" href="classtf_1_1Executor.html#a6487d589cb1f6b078b69fd3bb1082345">this_worker_id</a>() <span class="keyword">const</span>;</div><div class="line"><a name="l00177"></a><span class="lineno"> 177</span>  </div><div class="line"><a name="l00191"></a><span class="lineno"> 191</span>  <span class="keyword">template</span><<span class="keyword">typename</span> Observer, <span class="keyword">typename</span>... Args></div><div class="line"><a name="l00192"></a><span class="lineno"> 192</span>  Observer* <a class="code" href="classtf_1_1Executor.html#a7f43cde72d3e0a17e2d006cbe7a41ff3">make_observer</a>(Args&&... args);</div><div class="line"><a name="l00193"></a><span class="lineno"> 193</span>  </div><div class="line"><a name="l00197"></a><span class="lineno"> 197</span>  <span class="keywordtype">void</span> <a class="code" href="classtf_1_1Executor.html#a3ed11b270f0c7fe1c55a81c60d20411f">remove_observer</a>();</div><div class="line"><a name="l00198"></a><span class="lineno"> 198</span> </div><div class="line"><a name="l00199"></a><span class="lineno"> 199</span> </div><div class="line"><a name="l00200"></a><span class="lineno"> 200</span>  <span class="keyword">private</span>:</div><div class="line"><a name="l00201"></a><span class="lineno"> 201</span>  </div><div class="line"><a name="l00202"></a><span class="lineno"> 202</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/condition_variable.html">std::condition_variable</a> _topology_cv;</div><div class="line"><a name="l00203"></a><span class="lineno"> 203</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/mutex.html">std::mutex</a> _topology_mutex;</div><div class="line"><a name="l00204"></a><span class="lineno"> 204</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/mutex.html">std::mutex</a> _wsq_mutex;</div><div class="line"><a name="l00205"></a><span class="lineno"> 205</span> </div><div class="line"><a name="l00206"></a><span class="lineno"> 206</span>  <span class="keywordtype">size_t</span> _num_topologies {0};</div><div class="line"><a name="l00207"></a><span class="lineno"> 207</span>  </div><div class="line"><a name="l00208"></a><span class="lineno"> 208</span>  <span class="comment">// scheduler field</span></div><div class="line"><a name="l00209"></a><span class="lineno"> 209</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/container/vector.html">std::vector<Worker></a> _workers;</div><div class="line"><a name="l00210"></a><span class="lineno"> 210</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/container/vector.html">std::vector<std::thread></a> _threads;</div><div class="line"><a name="l00211"></a><span class="lineno"> 211</span> </div><div class="line"><a name="l00212"></a><span class="lineno"> 212</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA</span></div><div class="line"><a name="l00213"></a><span class="lineno"> 213</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/container/vector.html">std::vector<cudaDevice></a> _cuda_devices;</div><div class="line"><a name="l00214"></a><span class="lineno"> 214</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00215"></a><span class="lineno"> 215</span>  </div><div class="line"><a name="l00216"></a><span class="lineno"> 216</span>  Notifier _notifier[NUM_DOMAINS];</div><div class="line"><a name="l00217"></a><span class="lineno"> 217</span> </div><div class="line"><a name="l00218"></a><span class="lineno"> 218</span>  TaskQueue<Node*> _wsq[NUM_DOMAINS];</div><div class="line"><a name="l00219"></a><span class="lineno"> 219</span> </div><div class="line"><a name="l00220"></a><span class="lineno"> 220</span>  <span class="keywordtype">size_t</span> _id_offset[NUM_DOMAINS] = {0};</div><div class="line"><a name="l00221"></a><span class="lineno"> 221</span> </div><div class="line"><a name="l00222"></a><span class="lineno"> 222</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/atomic/atomic.html">std::atomic<size_t></a> _num_actives[NUM_DOMAINS];</div><div class="line"><a name="l00223"></a><span class="lineno"> 223</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/atomic/atomic.html">std::atomic<size_t></a> _num_thieves[NUM_DOMAINS];</div><div class="line"><a name="l00224"></a><span class="lineno"> 224</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/atomic/atomic.html">std::atomic<bool></a> _done {0};</div><div class="line"><a name="l00225"></a><span class="lineno"> 225</span>  </div><div class="line"><a name="l00226"></a><span class="lineno"> 226</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/memory/unique_ptr.html">std::unique_ptr<ExecutorObserverInterface></a> _observer;</div><div class="line"><a name="l00227"></a><span class="lineno"> 227</span>  </div><div class="line"><a name="l00228"></a><span class="lineno"> 228</span>  PerThread& _per_thread() <span class="keyword">const</span>;</div><div class="line"><a name="l00229"></a><span class="lineno"> 229</span> </div><div class="line"><a name="l00230"></a><span class="lineno"> 230</span>  <span class="keywordtype">bool</span> _wait_for_task(Worker&, Node*&);</div><div class="line"><a name="l00231"></a><span class="lineno"> 231</span>  </div><div class="line"><a name="l00232"></a><span class="lineno"> 232</span>  <span class="keywordtype">void</span> _spawn(<span class="keywordtype">size_t</span>, Domain);</div><div class="line"><a name="l00233"></a><span class="lineno"> 233</span>  <span class="keywordtype">void</span> _worker_loop(Worker&);</div><div class="line"><a name="l00234"></a><span class="lineno"> 234</span>  <span class="keywordtype">void</span> _exploit_task(Worker&, Node*&);</div><div class="line"><a name="l00235"></a><span class="lineno"> 235</span>  <span class="keywordtype">void</span> _explore_task(Worker&, Node*&);</div><div class="line"><a name="l00236"></a><span class="lineno"> 236</span>  <span class="keywordtype">void</span> _schedule(Node*, <span class="keywordtype">bool</span>);</div><div class="line"><a name="l00237"></a><span class="lineno"> 237</span>  <span class="keywordtype">void</span> _schedule(PassiveVector<Node*>&);</div><div class="line"><a name="l00238"></a><span class="lineno"> 238</span>  <span class="keywordtype">void</span> _invoke(Worker&, Node*);</div><div class="line"><a name="l00239"></a><span class="lineno"> 239</span>  <span class="keywordtype">void</span> _invoke_static_work(Worker&, Node*);</div><div class="line"><a name="l00240"></a><span class="lineno"> 240</span>  <span class="keywordtype">void</span> _invoke_dynamic_work(Worker&, Node*, <a class="code" href="classtf_1_1Subflow.html">Subflow</a>&);</div><div class="line"><a name="l00241"></a><span class="lineno"> 241</span>  <span class="keywordtype">void</span> _invoke_condition_work(Worker&, Node*, <span class="keywordtype">int</span>&);</div><div class="line"><a name="l00242"></a><span class="lineno"> 242</span> </div><div class="line"><a name="l00243"></a><span class="lineno"> 243</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA</span></div><div class="line"><a name="l00244"></a><span class="lineno"> 244</span>  <span class="keywordtype">void</span> _invoke_cudaflow_work(Worker&, Node*);</div><div class="line"><a name="l00245"></a><span class="lineno"> 245</span>  <span class="keywordtype">void</span> _invoke_cudaflow_work_impl(Worker&, Node*);</div><div class="line"><a name="l00246"></a><span class="lineno"> 246</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00247"></a><span class="lineno"> 247</span> </div><div class="line"><a name="l00248"></a><span class="lineno"> 248</span>  <span class="keywordtype">void</span> _set_up_module_work(Node*, <span class="keywordtype">bool</span>&);</div><div class="line"><a name="l00249"></a><span class="lineno"> 249</span>  <span class="keywordtype">void</span> _set_up_topology(Topology*);</div><div class="line"><a name="l00250"></a><span class="lineno"> 250</span>  <span class="keywordtype">void</span> _tear_down_topology(Topology**); </div><div class="line"><a name="l00251"></a><span class="lineno"> 251</span>  <span class="keywordtype">void</span> _increment_topology();</div><div class="line"><a name="l00252"></a><span class="lineno"> 252</span>  <span class="keywordtype">void</span> _decrement_topology();</div><div class="line"><a name="l00253"></a><span class="lineno"> 253</span>  <span class="keywordtype">void</span> _decrement_topology_and_notify();</div><div class="line"><a name="l00254"></a><span class="lineno"> 254</span> };</div><div class="line"><a name="l00255"></a><span class="lineno"> 255</span> </div><div class="line"><a name="l00256"></a><span class="lineno"> 256</span> </div><div class="line"><a name="l00257"></a><span class="lineno"> 257</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA</span></div><div class="line"><a name="l00258"></a><span class="lineno"> 258</span> <span class="comment">// Constructor</span></div><div class="line"><a name="l00259"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#aa09f058c7f04941de86e1d06238cdc48"> 259</a></span> <span class="keyword">inline</span> <a class="code" href="classtf_1_1Executor.html#aa09f058c7f04941de86e1d06238cdc48">Executor::Executor</a>(<span class="keywordtype">size_t</span> N, <span class="keywordtype">size_t</span> M) :</div><div class="line"><a name="l00260"></a><span class="lineno"> 260</span>  _workers {N + M},</div><div class="line"><a name="l00261"></a><span class="lineno"> 261</span>  _cuda_devices {cuda_num_devices()},</div><div class="line"><a name="l00262"></a><span class="lineno"> 262</span>  _notifier {Notifier(N), Notifier(M)} {</div><div class="line"><a name="l00263"></a><span class="lineno"> 263</span> </div><div class="line"><a name="l00264"></a><span class="lineno"> 264</span>  <span class="keywordflow">if</span>(N == 0) {</div><div class="line"><a name="l00265"></a><span class="lineno"> 265</span>  TF_THROW(<span class="stringliteral">"no cpu workers to execute taskflows"</span>);</div><div class="line"><a name="l00266"></a><span class="lineno"> 266</span>  }</div><div class="line"><a name="l00267"></a><span class="lineno"> 267</span> </div><div class="line"><a name="l00268"></a><span class="lineno"> 268</span>  <span class="keywordflow">if</span>(M == 0) {</div><div class="line"><a name="l00269"></a><span class="lineno"> 269</span>  TF_THROW(<span class="stringliteral">"no gpu workers to execute cudaflows"</span>);</div><div class="line"><a name="l00270"></a><span class="lineno"> 270</span>  }</div><div class="line"><a name="l00271"></a><span class="lineno"> 271</span> </div><div class="line"><a name="l00272"></a><span class="lineno"> 272</span>  <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i=0; i<NUM_DOMAINS; ++i) {</div><div class="line"><a name="l00273"></a><span class="lineno"> 273</span>  _num_actives[i].store(0, std::memory_order_relaxed);</div><div class="line"><a name="l00274"></a><span class="lineno"> 274</span>  _num_thieves[i].store(0, std::memory_order_relaxed); </div><div class="line"><a name="l00275"></a><span class="lineno"> 275</span>  }</div><div class="line"><a name="l00276"></a><span class="lineno"> 276</span>  </div><div class="line"><a name="l00277"></a><span class="lineno"> 277</span>  <span class="comment">// create a per-worker stream on each cuda device</span></div><div class="line"><a name="l00278"></a><span class="lineno"> 278</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> i=0; i<_cuda_devices.size(); ++i) {</div><div class="line"><a name="l00279"></a><span class="lineno"> 279</span>  _cuda_devices[i].streams.resize(M);</div><div class="line"><a name="l00280"></a><span class="lineno"> 280</span>  cudaScopedDevice ctx(i);</div><div class="line"><a name="l00281"></a><span class="lineno"> 281</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> m=0; m<M; ++m) {</div><div class="line"><a name="l00282"></a><span class="lineno"> 282</span>  TF_CHECK_CUDA(</div><div class="line"><a name="l00283"></a><span class="lineno"> 283</span>  cudaStreamCreate(&(_cuda_devices[i].streams[m])),</div><div class="line"><a name="l00284"></a><span class="lineno"> 284</span>  <span class="stringliteral">"failed to create a cudaStream for worker "</span>, m, <span class="stringliteral">" on device "</span>, i</div><div class="line"><a name="l00285"></a><span class="lineno"> 285</span>  );</div><div class="line"><a name="l00286"></a><span class="lineno"> 286</span>  }</div><div class="line"><a name="l00287"></a><span class="lineno"> 287</span>  }</div><div class="line"><a name="l00288"></a><span class="lineno"> 288</span> </div><div class="line"><a name="l00289"></a><span class="lineno"> 289</span>  _spawn(N, HOST);</div><div class="line"><a name="l00290"></a><span class="lineno"> 290</span>  _spawn(M, CUDA);</div><div class="line"><a name="l00291"></a><span class="lineno"> 291</span> }</div><div class="line"><a name="l00292"></a><span class="lineno"> 292</span> </div><div class="line"><a name="l00293"></a><span class="lineno"> 293</span> <span class="preprocessor">#else</span></div><div class="line"><a name="l00294"></a><span class="lineno"> 294</span> <span class="comment">// Constructor</span></div><div class="line"><a name="l00295"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a4910e89d89146b6d563d598b795eb4a9"> 295</a></span> <span class="keyword">inline</span> <a class="code" href="classtf_1_1Executor.html#aa09f058c7f04941de86e1d06238cdc48">Executor::Executor</a>(<span class="keywordtype">size_t</span> N) : </div><div class="line"><a name="l00296"></a><span class="lineno"> 296</span>  _workers {N},</div><div class="line"><a name="l00297"></a><span class="lineno"> 297</span>  _notifier {Notifier(N)} {</div><div class="line"><a name="l00298"></a><span class="lineno"> 298</span>  </div><div class="line"><a name="l00299"></a><span class="lineno"> 299</span>  <span class="keywordflow">if</span>(N == 0) {</div><div class="line"><a name="l00300"></a><span class="lineno"> 300</span>  TF_THROW(<span class="stringliteral">"no cpu workers to execute taskflows"</span>);</div><div class="line"><a name="l00301"></a><span class="lineno"> 301</span>  }</div><div class="line"><a name="l00302"></a><span class="lineno"> 302</span>  </div><div class="line"><a name="l00303"></a><span class="lineno"> 303</span>  <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i=0; i<NUM_DOMAINS; ++i) {</div><div class="line"><a name="l00304"></a><span class="lineno"> 304</span>  _num_actives[i].store(0, std::memory_order_relaxed);</div><div class="line"><a name="l00305"></a><span class="lineno"> 305</span>  _num_thieves[i].store(0, std::memory_order_relaxed); </div><div class="line"><a name="l00306"></a><span class="lineno"> 306</span>  }</div><div class="line"><a name="l00307"></a><span class="lineno"> 307</span> </div><div class="line"><a name="l00308"></a><span class="lineno"> 308</span>  _spawn(N, HOST);</div><div class="line"><a name="l00309"></a><span class="lineno"> 309</span> }</div><div class="line"><a name="l00310"></a><span class="lineno"> 310</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00311"></a><span class="lineno"> 311</span> </div><div class="line"><a name="l00312"></a><span class="lineno"> 312</span> <span class="comment">// Destructor</span></div><div class="line"><a name="l00313"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a5a511b0cc23b264826373d3dabcef670"> 313</a></span> <span class="keyword">inline</span> <a class="code" href="classtf_1_1Executor.html#a5a511b0cc23b264826373d3dabcef670">Executor::~Executor</a>() {</div><div class="line"><a name="l00314"></a><span class="lineno"> 314</span>  </div><div class="line"><a name="l00315"></a><span class="lineno"> 315</span>  <span class="comment">// wait for all topologies to complete</span></div><div class="line"><a name="l00316"></a><span class="lineno"> 316</span>  <a class="code" href="classtf_1_1Executor.html#ab9aa252f70e9a40020a1e5a89d485b85">wait_for_all</a>();</div><div class="line"><a name="l00317"></a><span class="lineno"> 317</span>  </div><div class="line"><a name="l00318"></a><span class="lineno"> 318</span>  <span class="comment">// shut down the scheduler</span></div><div class="line"><a name="l00319"></a><span class="lineno"> 319</span>  _done = <span class="keyword">true</span>;</div><div class="line"><a name="l00320"></a><span class="lineno"> 320</span> </div><div class="line"><a name="l00321"></a><span class="lineno"> 321</span>  <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i=0; i<NUM_DOMAINS; ++i) {</div><div class="line"><a name="l00322"></a><span class="lineno"> 322</span>  _notifier[i].notify(<span class="keyword">true</span>);</div><div class="line"><a name="l00323"></a><span class="lineno"> 323</span>  }</div><div class="line"><a name="l00324"></a><span class="lineno"> 324</span>  </div><div class="line"><a name="l00325"></a><span class="lineno"> 325</span>  <span class="keywordflow">for</span>(<span class="keyword">auto</span>& t : _threads){</div><div class="line"><a name="l00326"></a><span class="lineno"> 326</span>  t.join();</div><div class="line"><a name="l00327"></a><span class="lineno"> 327</span>  } </div><div class="line"><a name="l00328"></a><span class="lineno"> 328</span>  </div><div class="line"><a name="l00329"></a><span class="lineno"> 329</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA </span></div><div class="line"><a name="l00330"></a><span class="lineno"> 330</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> i=0; i<_cuda_devices.size(); ++i) {</div><div class="line"><a name="l00331"></a><span class="lineno"> 331</span>  cudaScopedDevice ctx(i);</div><div class="line"><a name="l00332"></a><span class="lineno"> 332</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> m=0; m<_cuda_devices[i].streams.size(); ++m) {</div><div class="line"><a name="l00333"></a><span class="lineno"> 333</span>  cudaStreamDestroy(_cuda_devices[i].streams[m]);</div><div class="line"><a name="l00334"></a><span class="lineno"> 334</span>  }</div><div class="line"><a name="l00335"></a><span class="lineno"> 335</span>  }</div><div class="line"><a name="l00336"></a><span class="lineno"> 336</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00337"></a><span class="lineno"> 337</span> }</div><div class="line"><a name="l00338"></a><span class="lineno"> 338</span> </div><div class="line"><a name="l00339"></a><span class="lineno"> 339</span> <span class="comment">// Function: num_workers</span></div><div class="line"><a name="l00340"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a79223c7f32c9d715e725b8d775de2d29"> 340</a></span> <span class="keyword">inline</span> <span class="keywordtype">size_t</span> <a class="code" href="classtf_1_1Executor.html#a79223c7f32c9d715e725b8d775de2d29">Executor::num_workers</a>()<span class="keyword"> const </span>{</div><div class="line"><a name="l00341"></a><span class="lineno"> 341</span>  <span class="keywordflow">return</span> _workers.size();</div><div class="line"><a name="l00342"></a><span class="lineno"> 342</span> }</div><div class="line"><a name="l00343"></a><span class="lineno"> 343</span> </div><div class="line"><a name="l00344"></a><span class="lineno"> 344</span> <span class="comment">// Function: num_domains</span></div><div class="line"><a name="l00345"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a8acfc81ae6a64989761ce229262c9a21"> 345</a></span> <span class="keyword">inline</span> <span class="keywordtype">size_t</span> <a class="code" href="classtf_1_1Executor.html#a8acfc81ae6a64989761ce229262c9a21">Executor::num_domains</a>()<span class="keyword"> const </span>{</div><div class="line"><a name="l00346"></a><span class="lineno"> 346</span>  <span class="keywordflow">return</span> NUM_DOMAINS;</div><div class="line"><a name="l00347"></a><span class="lineno"> 347</span> }</div><div class="line"><a name="l00348"></a><span class="lineno"> 348</span> </div><div class="line"><a name="l00349"></a><span class="lineno"> 349</span> <span class="comment">// Function: num_topologies</span></div><div class="line"><a name="l00350"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a6d6c28ed58211e4c27a99571e5bf0b6c"> 350</a></span> <span class="keyword">inline</span> <span class="keywordtype">size_t</span> <a class="code" href="classtf_1_1Executor.html#a6d6c28ed58211e4c27a99571e5bf0b6c">Executor::num_topologies</a>()<span class="keyword"> const </span>{</div><div class="line"><a name="l00351"></a><span class="lineno"> 351</span>  <span class="keywordflow">return</span> _num_topologies;</div><div class="line"><a name="l00352"></a><span class="lineno"> 352</span> }</div><div class="line"><a name="l00353"></a><span class="lineno"> 353</span> </div><div class="line"><a name="l00354"></a><span class="lineno"> 354</span> <span class="comment">// Function: _per_thread</span></div><div class="line"><a name="l00355"></a><span class="lineno"> 355</span> <span class="keyword">inline</span> Executor::PerThread& Executor::_per_thread()<span class="keyword"> const </span>{</div><div class="line"><a name="l00356"></a><span class="lineno"> 356</span>  thread_local PerThread pt;</div><div class="line"><a name="l00357"></a><span class="lineno"> 357</span>  <span class="keywordflow">return</span> pt;</div><div class="line"><a name="l00358"></a><span class="lineno"> 358</span> }</div><div class="line"><a name="l00359"></a><span class="lineno"> 359</span> </div><div class="line"><a name="l00360"></a><span class="lineno"> 360</span> <span class="comment">// Function: this_worker_id</span></div><div class="line"><a name="l00361"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a6487d589cb1f6b078b69fd3bb1082345"> 361</a></span> <span class="keyword">inline</span> <span class="keywordtype">int</span> <a class="code" href="classtf_1_1Executor.html#a6487d589cb1f6b078b69fd3bb1082345">Executor::this_worker_id</a>()<span class="keyword"> const </span>{</div><div class="line"><a name="l00362"></a><span class="lineno"> 362</span>  <span class="keyword">auto</span> worker = _per_thread().worker;</div><div class="line"><a name="l00363"></a><span class="lineno"> 363</span>  <span class="keywordflow">return</span> worker ? <span class="keyword">static_cast<</span><span class="keywordtype">int</span><span class="keyword">></span>(worker->id) : -1;</div><div class="line"><a name="l00364"></a><span class="lineno"> 364</span> }</div><div class="line"><a name="l00365"></a><span class="lineno"> 365</span> </div><div class="line"><a name="l00366"></a><span class="lineno"> 366</span> <span class="comment">// Procedure: _spawn</span></div><div class="line"><a name="l00367"></a><span class="lineno"> 367</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_spawn(<span class="keywordtype">size_t</span> N, Domain d) {</div><div class="line"><a name="l00368"></a><span class="lineno"> 368</span>  </div><div class="line"><a name="l00369"></a><span class="lineno"> 369</span>  <span class="keyword">auto</span> <span class="keywordtype">id</span> = _threads.size();</div><div class="line"><a name="l00370"></a><span class="lineno"> 370</span> </div><div class="line"><a name="l00371"></a><span class="lineno"> 371</span>  _id_offset[d] = id;</div><div class="line"><a name="l00372"></a><span class="lineno"> 372</span> </div><div class="line"><a name="l00373"></a><span class="lineno"> 373</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> i=0; i<N; ++i, ++id) {</div><div class="line"><a name="l00374"></a><span class="lineno"> 374</span> </div><div class="line"><a name="l00375"></a><span class="lineno"> 375</span>  _workers[id].id = id;</div><div class="line"><a name="l00376"></a><span class="lineno"> 376</span>  _workers[id].victim = id;</div><div class="line"><a name="l00377"></a><span class="lineno"> 377</span>  _workers[id].domain = d;</div><div class="line"><a name="l00378"></a><span class="lineno"> 378</span>  _workers[id].executor = <span class="keyword">this</span>;</div><div class="line"><a name="l00379"></a><span class="lineno"> 379</span>  _workers[id].waiter = &_notifier[d]._waiters[i];</div><div class="line"><a name="l00380"></a><span class="lineno"> 380</span>  </div><div class="line"><a name="l00381"></a><span class="lineno"> 381</span>  _threads.emplace_back([<span class="keyword">this</span>] (Worker& w) -> <span class="keywordtype">void</span> {</div><div class="line"><a name="l00382"></a><span class="lineno"> 382</span> </div><div class="line"><a name="l00383"></a><span class="lineno"> 383</span>  PerThread& pt = _per_thread(); </div><div class="line"><a name="l00384"></a><span class="lineno"> 384</span>  pt.worker = &w;</div><div class="line"><a name="l00385"></a><span class="lineno"> 385</span> </div><div class="line"><a name="l00386"></a><span class="lineno"> 386</span>  Node* t = <span class="keyword">nullptr</span>;</div><div class="line"><a name="l00387"></a><span class="lineno"> 387</span> </div><div class="line"><a name="l00388"></a><span class="lineno"> 388</span>  <span class="comment">// must use 1 as condition instead of !done</span></div><div class="line"><a name="l00389"></a><span class="lineno"> 389</span>  <span class="keywordflow">while</span>(1) {</div><div class="line"><a name="l00390"></a><span class="lineno"> 390</span>  </div><div class="line"><a name="l00391"></a><span class="lineno"> 391</span>  <span class="comment">// execute the tasks.</span></div><div class="line"><a name="l00392"></a><span class="lineno"> 392</span>  _exploit_task(w, t);</div><div class="line"><a name="l00393"></a><span class="lineno"> 393</span> </div><div class="line"><a name="l00394"></a><span class="lineno"> 394</span>  <span class="comment">// wait for tasks</span></div><div class="line"><a name="l00395"></a><span class="lineno"> 395</span>  <span class="keywordflow">if</span>(_wait_for_task(w, t) == <span class="keyword">false</span>) {</div><div class="line"><a name="l00396"></a><span class="lineno"> 396</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00397"></a><span class="lineno"> 397</span>  }</div><div class="line"><a name="l00398"></a><span class="lineno"> 398</span>  }</div><div class="line"><a name="l00399"></a><span class="lineno"> 399</span>  </div><div class="line"><a name="l00400"></a><span class="lineno"> 400</span>  }, <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/utility/functional/ref.html">std::ref</a>(_workers[<span class="keywordtype">id</span>])); </div><div class="line"><a name="l00401"></a><span class="lineno"> 401</span>  }</div><div class="line"><a name="l00402"></a><span class="lineno"> 402</span> </div><div class="line"><a name="l00403"></a><span class="lineno"> 403</span> }</div><div class="line"><a name="l00404"></a><span class="lineno"> 404</span> </div><div class="line"><a name="l00405"></a><span class="lineno"> 405</span> <span class="comment">// Function: _explore_task</span></div><div class="line"><a name="l00406"></a><span class="lineno"> 406</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_explore_task(Worker& w, Node*& t) {</div><div class="line"><a name="l00407"></a><span class="lineno"> 407</span>  </div><div class="line"><a name="l00408"></a><span class="lineno"> 408</span>  <span class="comment">//assert(_workers[w].wsq.empty());</span></div><div class="line"><a name="l00409"></a><span class="lineno"> 409</span>  assert(!t);</div><div class="line"><a name="l00410"></a><span class="lineno"> 410</span> </div><div class="line"><a name="l00411"></a><span class="lineno"> 411</span>  <span class="keyword">const</span> <span class="keyword">auto</span> d = w.domain;</div><div class="line"><a name="l00412"></a><span class="lineno"> 412</span> </div><div class="line"><a name="l00413"></a><span class="lineno"> 413</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> l = 0;</div><div class="line"><a name="l00414"></a><span class="lineno"> 414</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> r = _workers.size() - 1;</div><div class="line"><a name="l00415"></a><span class="lineno"> 415</span> </div><div class="line"><a name="l00416"></a><span class="lineno"> 416</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> F = (_workers.size() + 1) << 1;</div><div class="line"><a name="l00417"></a><span class="lineno"> 417</span>  <span class="keyword">const</span> <span class="keywordtype">size_t</span> Y = 100;</div><div class="line"><a name="l00418"></a><span class="lineno"> 418</span> </div><div class="line"><a name="l00419"></a><span class="lineno"> 419</span>  <span class="keywordtype">size_t</span> f = 0;</div><div class="line"><a name="l00420"></a><span class="lineno"> 420</span>  <span class="keywordtype">size_t</span> y = 0;</div><div class="line"><a name="l00421"></a><span class="lineno"> 421</span> </div><div class="line"><a name="l00422"></a><span class="lineno"> 422</span>  <span class="comment">/*// explore</span></div><div class="line"><a name="l00423"></a><span class="lineno"> 423</span> <span class="comment"> while(!_done) {</span></div><div class="line"><a name="l00424"></a><span class="lineno"> 424</span> <span class="comment"> </span></div><div class="line"><a name="l00425"></a><span class="lineno"> 425</span> <span class="comment"> size_t vtm = std::uniform_int_distribution<size_t>{l, r}(w.rdgen);</span></div><div class="line"><a name="l00426"></a><span class="lineno"> 426</span> <span class="comment"> </span></div><div class="line"><a name="l00427"></a><span class="lineno"> 427</span> <span class="comment"> t = (vtm == w.id) ? _wsq[d].steal() : _workers[vtm].wsq[d].steal();</span></div><div class="line"><a name="l00428"></a><span class="lineno"> 428</span> <span class="comment"></span></div><div class="line"><a name="l00429"></a><span class="lineno"> 429</span> <span class="comment"> if(t) {</span></div><div class="line"><a name="l00430"></a><span class="lineno"> 430</span> <span class="comment"> break;</span></div><div class="line"><a name="l00431"></a><span class="lineno"> 431</span> <span class="comment"> }</span></div><div class="line"><a name="l00432"></a><span class="lineno"> 432</span> <span class="comment"></span></div><div class="line"><a name="l00433"></a><span class="lineno"> 433</span> <span class="comment"> if(f++ > F) {</span></div><div class="line"><a name="l00434"></a><span class="lineno"> 434</span> <span class="comment"> std::this_thread::yield();</span></div><div class="line"><a name="l00435"></a><span class="lineno"> 435</span> <span class="comment"> if(y++ > Y) {</span></div><div class="line"><a name="l00436"></a><span class="lineno"> 436</span> <span class="comment"> break;</span></div><div class="line"><a name="l00437"></a><span class="lineno"> 437</span> <span class="comment"> }</span></div><div class="line"><a name="l00438"></a><span class="lineno"> 438</span> <span class="comment"> }</span></div><div class="line"><a name="l00439"></a><span class="lineno"> 439</span> <span class="comment"> } */</span></div><div class="line"><a name="l00440"></a><span class="lineno"> 440</span> </div><div class="line"><a name="l00441"></a><span class="lineno"> 441</span>  <span class="keywordflow">do</span> {</div><div class="line"><a name="l00442"></a><span class="lineno"> 442</span>  </div><div class="line"><a name="l00443"></a><span class="lineno"> 443</span>  t = (w.id == w.victim) ? _wsq[d].steal() : _workers[w.victim].wsq[d].steal();</div><div class="line"><a name="l00444"></a><span class="lineno"> 444</span> </div><div class="line"><a name="l00445"></a><span class="lineno"> 445</span>  <span class="keywordflow">if</span>(t) {</div><div class="line"><a name="l00446"></a><span class="lineno"> 446</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00447"></a><span class="lineno"> 447</span>  }</div><div class="line"><a name="l00448"></a><span class="lineno"> 448</span>  </div><div class="line"><a name="l00449"></a><span class="lineno"> 449</span>  <span class="keywordflow">if</span>(f++ > F) {</div><div class="line"><a name="l00450"></a><span class="lineno"> 450</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/yield.html">std::this_thread::yield</a>();</div><div class="line"><a name="l00451"></a><span class="lineno"> 451</span>  <span class="keywordflow">if</span>(y++ > Y) {</div><div class="line"><a name="l00452"></a><span class="lineno"> 452</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00453"></a><span class="lineno"> 453</span>  }</div><div class="line"><a name="l00454"></a><span class="lineno"> 454</span>  }</div><div class="line"><a name="l00455"></a><span class="lineno"> 455</span>  </div><div class="line"><a name="l00456"></a><span class="lineno"> 456</span>  w.victim = <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/numeric/random/uniform_int_distribution.html">std::uniform_int_distribution<size_t></a>{l, r}(w.rdgen);</div><div class="line"><a name="l00457"></a><span class="lineno"> 457</span> </div><div class="line"><a name="l00458"></a><span class="lineno"> 458</span>  } <span class="keywordflow">while</span>(!_done);</div><div class="line"><a name="l00459"></a><span class="lineno"> 459</span> </div><div class="line"><a name="l00460"></a><span class="lineno"> 460</span> }</div><div class="line"><a name="l00461"></a><span class="lineno"> 461</span> </div><div class="line"><a name="l00462"></a><span class="lineno"> 462</span> <span class="comment">// Procedure: _exploit_task</span></div><div class="line"><a name="l00463"></a><span class="lineno"> 463</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_exploit_task(Worker& w, Node*& t) {</div><div class="line"><a name="l00464"></a><span class="lineno"> 464</span>  </div><div class="line"><a name="l00465"></a><span class="lineno"> 465</span>  assert(!w.cache);</div><div class="line"><a name="l00466"></a><span class="lineno"> 466</span> </div><div class="line"><a name="l00467"></a><span class="lineno"> 467</span>  <span class="keywordflow">if</span>(t) {</div><div class="line"><a name="l00468"></a><span class="lineno"> 468</span> </div><div class="line"><a name="l00469"></a><span class="lineno"> 469</span>  <span class="keyword">const</span> <span class="keyword">auto</span> d = w.domain;</div><div class="line"><a name="l00470"></a><span class="lineno"> 470</span> </div><div class="line"><a name="l00471"></a><span class="lineno"> 471</span>  <span class="keywordflow">if</span>(_num_actives[d].fetch_add(1) == 0 && _num_thieves[d] == 0) {</div><div class="line"><a name="l00472"></a><span class="lineno"> 472</span>  _notifier[d].notify(<span class="keyword">false</span>);</div><div class="line"><a name="l00473"></a><span class="lineno"> 473</span>  }</div><div class="line"><a name="l00474"></a><span class="lineno"> 474</span>  </div><div class="line"><a name="l00475"></a><span class="lineno"> 475</span>  <span class="keyword">auto</span> tpg = t->_topology;</div><div class="line"><a name="l00476"></a><span class="lineno"> 476</span>  <span class="keyword">auto</span> par = t->_parent;</div><div class="line"><a name="l00477"></a><span class="lineno"> 477</span>  <span class="keyword">auto</span> exe = <span class="keywordtype">size_t</span>{1};</div><div class="line"><a name="l00478"></a><span class="lineno"> 478</span> </div><div class="line"><a name="l00479"></a><span class="lineno"> 479</span>  <span class="keywordflow">do</span> {</div><div class="line"><a name="l00480"></a><span class="lineno"> 480</span>  _invoke(w, t);</div><div class="line"><a name="l00481"></a><span class="lineno"> 481</span> </div><div class="line"><a name="l00482"></a><span class="lineno"> 482</span>  <span class="keywordflow">if</span>(w.cache) {</div><div class="line"><a name="l00483"></a><span class="lineno"> 483</span>  t = w.cache;</div><div class="line"><a name="l00484"></a><span class="lineno"> 484</span>  w.cache = <span class="keyword">nullptr</span>;</div><div class="line"><a name="l00485"></a><span class="lineno"> 485</span>  }</div><div class="line"><a name="l00486"></a><span class="lineno"> 486</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00487"></a><span class="lineno"> 487</span>  t = w.wsq[d].pop();</div><div class="line"><a name="l00488"></a><span class="lineno"> 488</span>  <span class="keywordflow">if</span>(t) {</div><div class="line"><a name="l00489"></a><span class="lineno"> 489</span>  <span class="comment">// We only increment the counter when poping task from wsq </span></div><div class="line"><a name="l00490"></a><span class="lineno"> 490</span>  <span class="comment">// (NOT including cache!)</span></div><div class="line"><a name="l00491"></a><span class="lineno"> 491</span>  <span class="keywordflow">if</span>(t->_parent == par) {</div><div class="line"><a name="l00492"></a><span class="lineno"> 492</span>  exe++;</div><div class="line"><a name="l00493"></a><span class="lineno"> 493</span>  }</div><div class="line"><a name="l00494"></a><span class="lineno"> 494</span>  <span class="comment">// joined subflow</span></div><div class="line"><a name="l00495"></a><span class="lineno"> 495</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00496"></a><span class="lineno"> 496</span>  <span class="keywordflow">if</span>(par == <span class="keyword">nullptr</span>) {</div><div class="line"><a name="l00497"></a><span class="lineno"> 497</span>  <span class="comment">// still have tasks so the topology join counter can't be zero</span></div><div class="line"><a name="l00498"></a><span class="lineno"> 498</span>  t->_topology->_join_counter.fetch_sub(exe);</div><div class="line"><a name="l00499"></a><span class="lineno"> 499</span>  }</div><div class="line"><a name="l00500"></a><span class="lineno"> 500</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00501"></a><span class="lineno"> 501</span>  <span class="keyword">auto</span> ret = par->_join_counter.fetch_sub(exe);</div><div class="line"><a name="l00502"></a><span class="lineno"> 502</span>  <span class="keywordflow">if</span>(ret == exe) {</div><div class="line"><a name="l00503"></a><span class="lineno"> 503</span>  <span class="keywordflow">if</span>(par->domain() == d) {</div><div class="line"><a name="l00504"></a><span class="lineno"> 504</span>  w.wsq[d].push(par);</div><div class="line"><a name="l00505"></a><span class="lineno"> 505</span>  }</div><div class="line"><a name="l00506"></a><span class="lineno"> 506</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00507"></a><span class="lineno"> 507</span>  _schedule(par, <span class="keyword">false</span>);</div><div class="line"><a name="l00508"></a><span class="lineno"> 508</span>  }</div><div class="line"><a name="l00509"></a><span class="lineno"> 509</span>  }</div><div class="line"><a name="l00510"></a><span class="lineno"> 510</span>  }</div><div class="line"><a name="l00511"></a><span class="lineno"> 511</span>  exe = 1;</div><div class="line"><a name="l00512"></a><span class="lineno"> 512</span>  par = t->_parent;</div><div class="line"><a name="l00513"></a><span class="lineno"> 513</span>  }</div><div class="line"><a name="l00514"></a><span class="lineno"> 514</span>  }</div><div class="line"><a name="l00515"></a><span class="lineno"> 515</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00516"></a><span class="lineno"> 516</span>  <span class="comment">// If no more local tasks!</span></div><div class="line"><a name="l00517"></a><span class="lineno"> 517</span>  <span class="keywordflow">if</span>(par == <span class="keyword">nullptr</span>) {</div><div class="line"><a name="l00518"></a><span class="lineno"> 518</span>  <span class="keywordflow">if</span>(tpg->_join_counter.fetch_sub(exe) == exe) {</div><div class="line"><a name="l00519"></a><span class="lineno"> 519</span>  <span class="comment">// TODO: Store tpg in local variable not in w</span></div><div class="line"><a name="l00520"></a><span class="lineno"> 520</span>  _tear_down_topology(&tpg);</div><div class="line"><a name="l00521"></a><span class="lineno"> 521</span>  <span class="keywordflow">if</span>(tpg != <span class="keyword">nullptr</span>) {</div><div class="line"><a name="l00522"></a><span class="lineno"> 522</span>  t = w.wsq[d].pop();</div><div class="line"><a name="l00523"></a><span class="lineno"> 523</span>  <span class="keywordflow">if</span>(t) {</div><div class="line"><a name="l00524"></a><span class="lineno"> 524</span>  exe = 1;</div><div class="line"><a name="l00525"></a><span class="lineno"> 525</span>  }</div><div class="line"><a name="l00526"></a><span class="lineno"> 526</span>  }</div><div class="line"><a name="l00527"></a><span class="lineno"> 527</span>  }</div><div class="line"><a name="l00528"></a><span class="lineno"> 528</span>  }</div><div class="line"><a name="l00529"></a><span class="lineno"> 529</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00530"></a><span class="lineno"> 530</span>  <span class="keywordflow">if</span>(par->_join_counter.fetch_sub(exe) == exe) {</div><div class="line"><a name="l00531"></a><span class="lineno"> 531</span>  <span class="keywordflow">if</span>(par->domain() == d) {</div><div class="line"><a name="l00532"></a><span class="lineno"> 532</span>  t = par;</div><div class="line"><a name="l00533"></a><span class="lineno"> 533</span>  par = par->_parent;</div><div class="line"><a name="l00534"></a><span class="lineno"> 534</span>  exe = 1;</div><div class="line"><a name="l00535"></a><span class="lineno"> 535</span>  }</div><div class="line"><a name="l00536"></a><span class="lineno"> 536</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00537"></a><span class="lineno"> 537</span>  _schedule(par, <span class="keyword">false</span>);</div><div class="line"><a name="l00538"></a><span class="lineno"> 538</span>  }</div><div class="line"><a name="l00539"></a><span class="lineno"> 539</span>  }</div><div class="line"><a name="l00540"></a><span class="lineno"> 540</span>  }</div><div class="line"><a name="l00541"></a><span class="lineno"> 541</span>  }</div><div class="line"><a name="l00542"></a><span class="lineno"> 542</span>  }</div><div class="line"><a name="l00543"></a><span class="lineno"> 543</span>  } <span class="keywordflow">while</span>(t);</div><div class="line"><a name="l00544"></a><span class="lineno"> 544</span> </div><div class="line"><a name="l00545"></a><span class="lineno"> 545</span>  --_num_actives[d];</div><div class="line"><a name="l00546"></a><span class="lineno"> 546</span>  }</div><div class="line"><a name="l00547"></a><span class="lineno"> 547</span> }</div><div class="line"><a name="l00548"></a><span class="lineno"> 548</span> </div><div class="line"><a name="l00549"></a><span class="lineno"> 549</span> <span class="comment">// Function: _wait_for_task</span></div><div class="line"><a name="l00550"></a><span class="lineno"> 550</span> <span class="keyword">inline</span> <span class="keywordtype">bool</span> Executor::_wait_for_task(Worker& worker, Node*& t) {</div><div class="line"><a name="l00551"></a><span class="lineno"> 551</span> </div><div class="line"><a name="l00552"></a><span class="lineno"> 552</span>  <span class="keyword">const</span> <span class="keyword">auto</span> d = worker.domain;</div><div class="line"><a name="l00553"></a><span class="lineno"> 553</span> </div><div class="line"><a name="l00554"></a><span class="lineno"> 554</span>  wait_for_task:</div><div class="line"><a name="l00555"></a><span class="lineno"> 555</span> </div><div class="line"><a name="l00556"></a><span class="lineno"> 556</span>  assert(!t);</div><div class="line"><a name="l00557"></a><span class="lineno"> 557</span> </div><div class="line"><a name="l00558"></a><span class="lineno"> 558</span>  ++_num_thieves[d];</div><div class="line"><a name="l00559"></a><span class="lineno"> 559</span> </div><div class="line"><a name="l00560"></a><span class="lineno"> 560</span>  explore_task:</div><div class="line"><a name="l00561"></a><span class="lineno"> 561</span> </div><div class="line"><a name="l00562"></a><span class="lineno"> 562</span>  _explore_task(worker, t);</div><div class="line"><a name="l00563"></a><span class="lineno"> 563</span> </div><div class="line"><a name="l00564"></a><span class="lineno"> 564</span>  <span class="keywordflow">if</span>(t) {</div><div class="line"><a name="l00565"></a><span class="lineno"> 565</span>  <span class="keywordflow">if</span>(_num_thieves[d].fetch_sub(1) == 1) {</div><div class="line"><a name="l00566"></a><span class="lineno"> 566</span>  _notifier[d].notify(<span class="keyword">false</span>);</div><div class="line"><a name="l00567"></a><span class="lineno"> 567</span>  }</div><div class="line"><a name="l00568"></a><span class="lineno"> 568</span>  <span class="keywordflow">return</span> <span class="keyword">true</span>;</div><div class="line"><a name="l00569"></a><span class="lineno"> 569</span>  }</div><div class="line"><a name="l00570"></a><span class="lineno"> 570</span> </div><div class="line"><a name="l00571"></a><span class="lineno"> 571</span>  _notifier[d].prepare_wait(worker.waiter);</div><div class="line"><a name="l00572"></a><span class="lineno"> 572</span>  </div><div class="line"><a name="l00573"></a><span class="lineno"> 573</span>  <span class="comment">//if(auto vtm = _find_victim(me); vtm != _workers.size()) {</span></div><div class="line"><a name="l00574"></a><span class="lineno"> 574</span>  <span class="keywordflow">if</span>(!_wsq[d].empty()) {</div><div class="line"><a name="l00575"></a><span class="lineno"> 575</span> </div><div class="line"><a name="l00576"></a><span class="lineno"> 576</span>  _notifier[d].cancel_wait(worker.waiter);</div><div class="line"><a name="l00577"></a><span class="lineno"> 577</span>  <span class="comment">//t = (vtm == me) ? _wsq.steal() : _workers[vtm].wsq.steal();</span></div><div class="line"><a name="l00578"></a><span class="lineno"> 578</span>  </div><div class="line"><a name="l00579"></a><span class="lineno"> 579</span>  t = _wsq[d].steal();</div><div class="line"><a name="l00580"></a><span class="lineno"> 580</span>  <span class="keywordflow">if</span>(t) {</div><div class="line"><a name="l00581"></a><span class="lineno"> 581</span>  <span class="keywordflow">if</span>(_num_thieves[d].fetch_sub(1) == 1) {</div><div class="line"><a name="l00582"></a><span class="lineno"> 582</span>  _notifier[d].notify(<span class="keyword">false</span>);</div><div class="line"><a name="l00583"></a><span class="lineno"> 583</span>  }</div><div class="line"><a name="l00584"></a><span class="lineno"> 584</span>  <span class="keywordflow">return</span> <span class="keyword">true</span>;</div><div class="line"><a name="l00585"></a><span class="lineno"> 585</span>  }</div><div class="line"><a name="l00586"></a><span class="lineno"> 586</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00587"></a><span class="lineno"> 587</span>  <span class="keywordflow">goto</span> explore_task;</div><div class="line"><a name="l00588"></a><span class="lineno"> 588</span>  }</div><div class="line"><a name="l00589"></a><span class="lineno"> 589</span>  }</div><div class="line"><a name="l00590"></a><span class="lineno"> 590</span> </div><div class="line"><a name="l00591"></a><span class="lineno"> 591</span>  <span class="keywordflow">if</span>(_done) {</div><div class="line"><a name="l00592"></a><span class="lineno"> 592</span>  _notifier[d].cancel_wait(worker.waiter);</div><div class="line"><a name="l00593"></a><span class="lineno"> 593</span>  <span class="keywordflow">for</span>(<span class="keywordtype">int</span> i=0; i<NUM_DOMAINS; ++i) {</div><div class="line"><a name="l00594"></a><span class="lineno"> 594</span>  _notifier[i].notify(<span class="keyword">true</span>);</div><div class="line"><a name="l00595"></a><span class="lineno"> 595</span>  }</div><div class="line"><a name="l00596"></a><span class="lineno"> 596</span>  --_num_thieves[d];</div><div class="line"><a name="l00597"></a><span class="lineno"> 597</span>  <span class="keywordflow">return</span> <span class="keyword">false</span>;</div><div class="line"><a name="l00598"></a><span class="lineno"> 598</span>  }</div><div class="line"><a name="l00599"></a><span class="lineno"> 599</span> </div><div class="line"><a name="l00600"></a><span class="lineno"> 600</span>  <span class="keywordflow">if</span>(_num_thieves[d].fetch_sub(1) == 1) {</div><div class="line"><a name="l00601"></a><span class="lineno"> 601</span>  <span class="keywordflow">if</span>(_num_actives[d]) {</div><div class="line"><a name="l00602"></a><span class="lineno"> 602</span>  _notifier[d].cancel_wait(worker.waiter);</div><div class="line"><a name="l00603"></a><span class="lineno"> 603</span>  <span class="keywordflow">goto</span> wait_for_task;</div><div class="line"><a name="l00604"></a><span class="lineno"> 604</span>  }</div><div class="line"><a name="l00605"></a><span class="lineno"> 605</span>  <span class="comment">// check all domain queue again</span></div><div class="line"><a name="l00606"></a><span class="lineno"> 606</span>  <span class="keywordflow">for</span>(<span class="keyword">auto</span>& w : _workers) {</div><div class="line"><a name="l00607"></a><span class="lineno"> 607</span>  <span class="keywordflow">if</span>(!w.wsq[d].empty()) {</div><div class="line"><a name="l00608"></a><span class="lineno"> 608</span>  _notifier[d].cancel_wait(worker.waiter);</div><div class="line"><a name="l00609"></a><span class="lineno"> 609</span>  <span class="keywordflow">goto</span> wait_for_task;</div><div class="line"><a name="l00610"></a><span class="lineno"> 610</span>  }</div><div class="line"><a name="l00611"></a><span class="lineno"> 611</span>  }</div><div class="line"><a name="l00612"></a><span class="lineno"> 612</span>  }</div><div class="line"><a name="l00613"></a><span class="lineno"> 613</span>  </div><div class="line"><a name="l00614"></a><span class="lineno"> 614</span>  <span class="comment">// Now I really need to relinguish my self to others</span></div><div class="line"><a name="l00615"></a><span class="lineno"> 615</span>  _notifier[d].commit_wait(worker.waiter);</div><div class="line"><a name="l00616"></a><span class="lineno"> 616</span> </div><div class="line"><a name="l00617"></a><span class="lineno"> 617</span>  <span class="keywordflow">return</span> <span class="keyword">true</span>;</div><div class="line"><a name="l00618"></a><span class="lineno"> 618</span> }</div><div class="line"><a name="l00619"></a><span class="lineno"> 619</span> </div><div class="line"><a name="l00620"></a><span class="lineno"> 620</span> <span class="comment">// Function: make_observer </span></div><div class="line"><a name="l00621"></a><span class="lineno"> 621</span> <span class="keyword">template</span><<span class="keyword">typename</span> Observer, <span class="keyword">typename</span>... Args></div><div class="line"><a name="l00622"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a7f43cde72d3e0a17e2d006cbe7a41ff3"> 622</a></span> Observer* <a class="code" href="classtf_1_1Executor.html#a7f43cde72d3e0a17e2d006cbe7a41ff3">Executor::make_observer</a>(Args&&... args) {</div><div class="line"><a name="l00623"></a><span class="lineno"> 623</span>  <span class="comment">// use a local variable to mimic the constructor </span></div><div class="line"><a name="l00624"></a><span class="lineno"> 624</span>  <span class="keyword">auto</span> tmp = std::make_unique<Observer>(std::forward<Args>(args)...);</div><div class="line"><a name="l00625"></a><span class="lineno"> 625</span>  tmp->set_up(_workers.size());</div><div class="line"><a name="l00626"></a><span class="lineno"> 626</span>  _observer = std::move(tmp);</div><div class="line"><a name="l00627"></a><span class="lineno"> 627</span>  <span class="keywordflow">return</span> <span class="keyword">static_cast<</span>Observer*<span class="keyword">></span>(_observer.get());</div><div class="line"><a name="l00628"></a><span class="lineno"> 628</span> }</div><div class="line"><a name="l00629"></a><span class="lineno"> 629</span> </div><div class="line"><a name="l00630"></a><span class="lineno"> 630</span> <span class="comment">// Procedure: remove_observer</span></div><div class="line"><a name="l00631"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a3ed11b270f0c7fe1c55a81c60d20411f"> 631</a></span> <span class="keyword">inline</span> <span class="keywordtype">void</span> <a class="code" href="classtf_1_1Executor.html#a3ed11b270f0c7fe1c55a81c60d20411f">Executor::remove_observer</a>() {</div><div class="line"><a name="l00632"></a><span class="lineno"> 632</span>  _observer.reset();</div><div class="line"><a name="l00633"></a><span class="lineno"> 633</span> }</div><div class="line"><a name="l00634"></a><span class="lineno"> 634</span> </div><div class="line"><a name="l00635"></a><span class="lineno"> 635</span> <span class="comment">// Procedure: _schedule</span></div><div class="line"><a name="l00636"></a><span class="lineno"> 636</span> <span class="comment">// The main procedure to schedule a give task node.</span></div><div class="line"><a name="l00637"></a><span class="lineno"> 637</span> <span class="comment">// Each task node has two types of tasks - regular and subflow.</span></div><div class="line"><a name="l00638"></a><span class="lineno"> 638</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_schedule(Node* node, <span class="keywordtype">bool</span> bypass_hint) {</div><div class="line"><a name="l00639"></a><span class="lineno"> 639</span>  </div><div class="line"><a name="l00640"></a><span class="lineno"> 640</span>  <span class="comment">//assert(_workers.size() != 0);</span></div><div class="line"><a name="l00641"></a><span class="lineno"> 641</span> </div><div class="line"><a name="l00642"></a><span class="lineno"> 642</span>  <span class="keyword">const</span> <span class="keyword">auto</span> d = node->domain();</div><div class="line"><a name="l00643"></a><span class="lineno"> 643</span>  </div><div class="line"><a name="l00644"></a><span class="lineno"> 644</span>  <span class="comment">// caller is a worker to this pool</span></div><div class="line"><a name="l00645"></a><span class="lineno"> 645</span>  <span class="keyword">auto</span> worker = _per_thread().worker;</div><div class="line"><a name="l00646"></a><span class="lineno"> 646</span> </div><div class="line"><a name="l00647"></a><span class="lineno"> 647</span>  <span class="keywordflow">if</span>(worker != <span class="keyword">nullptr</span> && worker->executor == <span class="keyword">this</span>) {</div><div class="line"><a name="l00648"></a><span class="lineno"> 648</span>  <span class="keywordflow">if</span>(bypass_hint) {</div><div class="line"><a name="l00649"></a><span class="lineno"> 649</span>  assert(!worker->cache);</div><div class="line"><a name="l00650"></a><span class="lineno"> 650</span>  worker->cache = node;</div><div class="line"><a name="l00651"></a><span class="lineno"> 651</span>  }</div><div class="line"><a name="l00652"></a><span class="lineno"> 652</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00653"></a><span class="lineno"> 653</span>  worker->wsq[d].push(node);</div><div class="line"><a name="l00654"></a><span class="lineno"> 654</span>  <span class="keywordflow">if</span>(worker->domain != d) {</div><div class="line"><a name="l00655"></a><span class="lineno"> 655</span>  <span class="keywordflow">if</span>(_num_actives[d] == 0 && _num_thieves[d] == 0) {</div><div class="line"><a name="l00656"></a><span class="lineno"> 656</span>  _notifier[d].notify(<span class="keyword">false</span>);</div><div class="line"><a name="l00657"></a><span class="lineno"> 657</span>  }</div><div class="line"><a name="l00658"></a><span class="lineno"> 658</span>  }</div><div class="line"><a name="l00659"></a><span class="lineno"> 659</span>  }</div><div class="line"><a name="l00660"></a><span class="lineno"> 660</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00661"></a><span class="lineno"> 661</span>  }</div><div class="line"><a name="l00662"></a><span class="lineno"> 662</span> </div><div class="line"><a name="l00663"></a><span class="lineno"> 663</span>  <span class="comment">// other threads</span></div><div class="line"><a name="l00664"></a><span class="lineno"> 664</span>  {</div><div class="line"><a name="l00665"></a><span class="lineno"> 665</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard<std::mutex></a> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock.html">lock</a>(_wsq_mutex);</div><div class="line"><a name="l00666"></a><span class="lineno"> 666</span>  _wsq[d].push(node);</div><div class="line"><a name="l00667"></a><span class="lineno"> 667</span>  }</div><div class="line"><a name="l00668"></a><span class="lineno"> 668</span> </div><div class="line"><a name="l00669"></a><span class="lineno"> 669</span>  _notifier[d].notify(<span class="keyword">false</span>);</div><div class="line"><a name="l00670"></a><span class="lineno"> 670</span> }</div><div class="line"><a name="l00671"></a><span class="lineno"> 671</span> </div><div class="line"><a name="l00672"></a><span class="lineno"> 672</span> <span class="comment">// Procedure: _schedule</span></div><div class="line"><a name="l00673"></a><span class="lineno"> 673</span> <span class="comment">// The main procedure to schedule a set of task nodes.</span></div><div class="line"><a name="l00674"></a><span class="lineno"> 674</span> <span class="comment">// Each task node has two types of tasks - regular and subflow.</span></div><div class="line"><a name="l00675"></a><span class="lineno"> 675</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_schedule(PassiveVector<Node*>& nodes) {</div><div class="line"><a name="l00676"></a><span class="lineno"> 676</span> </div><div class="line"><a name="l00677"></a><span class="lineno"> 677</span>  <span class="comment">//assert(_workers.size() != 0);</span></div><div class="line"><a name="l00678"></a><span class="lineno"> 678</span>  </div><div class="line"><a name="l00679"></a><span class="lineno"> 679</span>  <span class="comment">// We need to cacth the node count to avoid accessing the nodes</span></div><div class="line"><a name="l00680"></a><span class="lineno"> 680</span>  <span class="comment">// vector while the parent topology is removed!</span></div><div class="line"><a name="l00681"></a><span class="lineno"> 681</span>  <span class="keyword">const</span> <span class="keyword">auto</span> num_nodes = nodes.size();</div><div class="line"><a name="l00682"></a><span class="lineno"> 682</span>  </div><div class="line"><a name="l00683"></a><span class="lineno"> 683</span>  <span class="keywordflow">if</span>(num_nodes == 0) {</div><div class="line"><a name="l00684"></a><span class="lineno"> 684</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00685"></a><span class="lineno"> 685</span>  }</div><div class="line"><a name="l00686"></a><span class="lineno"> 686</span> </div><div class="line"><a name="l00687"></a><span class="lineno"> 687</span>  <span class="comment">// worker thread</span></div><div class="line"><a name="l00688"></a><span class="lineno"> 688</span>  <span class="keyword">auto</span> worker = _per_thread().worker;</div><div class="line"><a name="l00689"></a><span class="lineno"> 689</span> </div><div class="line"><a name="l00690"></a><span class="lineno"> 690</span>  <span class="comment">// task counts</span></div><div class="line"><a name="l00691"></a><span class="lineno"> 691</span>  <span class="keywordtype">size_t</span> tcount[NUM_DOMAINS] = {0};</div><div class="line"><a name="l00692"></a><span class="lineno"> 692</span> </div><div class="line"><a name="l00693"></a><span class="lineno"> 693</span>  <span class="keywordflow">if</span>(worker != <span class="keyword">nullptr</span> && worker->executor == <span class="keyword">this</span>) {</div><div class="line"><a name="l00694"></a><span class="lineno"> 694</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> i=0; i<num_nodes; ++i) {</div><div class="line"><a name="l00695"></a><span class="lineno"> 695</span>  <span class="keyword">const</span> <span class="keyword">auto</span> d = nodes[i]->domain();</div><div class="line"><a name="l00696"></a><span class="lineno"> 696</span>  worker->wsq[d].push(nodes[i]);</div><div class="line"><a name="l00697"></a><span class="lineno"> 697</span>  tcount[d]++;</div><div class="line"><a name="l00698"></a><span class="lineno"> 698</span>  }</div><div class="line"><a name="l00699"></a><span class="lineno"> 699</span>  </div><div class="line"><a name="l00700"></a><span class="lineno"> 700</span>  <span class="keywordflow">for</span>(<span class="keywordtype">int</span> d=0; d<NUM_DOMAINS; ++d) {</div><div class="line"><a name="l00701"></a><span class="lineno"> 701</span>  <span class="keywordflow">if</span>(tcount[d] && d != worker->domain) {</div><div class="line"><a name="l00702"></a><span class="lineno"> 702</span>  <span class="keywordflow">if</span>(_num_actives[d] == 0 && _num_thieves[d] == 0) {</div><div class="line"><a name="l00703"></a><span class="lineno"> 703</span>  _notifier[d].notify_n(tcount[d]);</div><div class="line"><a name="l00704"></a><span class="lineno"> 704</span>  }</div><div class="line"><a name="l00705"></a><span class="lineno"> 705</span>  }</div><div class="line"><a name="l00706"></a><span class="lineno"> 706</span>  }</div><div class="line"><a name="l00707"></a><span class="lineno"> 707</span> </div><div class="line"><a name="l00708"></a><span class="lineno"> 708</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00709"></a><span class="lineno"> 709</span>  }</div><div class="line"><a name="l00710"></a><span class="lineno"> 710</span>  </div><div class="line"><a name="l00711"></a><span class="lineno"> 711</span>  <span class="comment">// other threads</span></div><div class="line"><a name="l00712"></a><span class="lineno"> 712</span>  {</div><div class="line"><a name="l00713"></a><span class="lineno"> 713</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard<std::mutex></a> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock.html">lock</a>(_wsq_mutex);</div><div class="line"><a name="l00714"></a><span class="lineno"> 714</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> k=0; k<num_nodes; ++k) {</div><div class="line"><a name="l00715"></a><span class="lineno"> 715</span>  <span class="keyword">const</span> <span class="keyword">auto</span> d = nodes[k]->domain();</div><div class="line"><a name="l00716"></a><span class="lineno"> 716</span>  _wsq[d].push(nodes[k]);</div><div class="line"><a name="l00717"></a><span class="lineno"> 717</span>  tcount[d]++;</div><div class="line"><a name="l00718"></a><span class="lineno"> 718</span>  }</div><div class="line"><a name="l00719"></a><span class="lineno"> 719</span>  }</div><div class="line"><a name="l00720"></a><span class="lineno"> 720</span>  </div><div class="line"><a name="l00721"></a><span class="lineno"> 721</span>  <span class="keywordflow">for</span>(<span class="keywordtype">int</span> d=0; d<NUM_DOMAINS; ++d) {</div><div class="line"><a name="l00722"></a><span class="lineno"> 722</span>  _notifier[d].notify_n(tcount[d]);</div><div class="line"><a name="l00723"></a><span class="lineno"> 723</span>  }</div><div class="line"><a name="l00724"></a><span class="lineno"> 724</span> }</div><div class="line"><a name="l00725"></a><span class="lineno"> 725</span> </div><div class="line"><a name="l00726"></a><span class="lineno"> 726</span> </div><div class="line"><a name="l00727"></a><span class="lineno"> 727</span> <span class="comment">// Procedure: _invoke</span></div><div class="line"><a name="l00728"></a><span class="lineno"> 728</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_invoke(Worker& worker, Node* node) {</div><div class="line"><a name="l00729"></a><span class="lineno"> 729</span> </div><div class="line"><a name="l00730"></a><span class="lineno"> 730</span>  <span class="comment">//assert(_workers.size() != 0);</span></div><div class="line"><a name="l00731"></a><span class="lineno"> 731</span> </div><div class="line"><a name="l00732"></a><span class="lineno"> 732</span>  <span class="comment">// Here we need to fetch the num_successors first to avoid the invalid memory</span></div><div class="line"><a name="l00733"></a><span class="lineno"> 733</span>  <span class="comment">// access caused by topology clear.</span></div><div class="line"><a name="l00734"></a><span class="lineno"> 734</span>  <span class="keyword">const</span> <span class="keyword">auto</span> num_successors = node->num_successors();</div><div class="line"><a name="l00735"></a><span class="lineno"> 735</span>  </div><div class="line"><a name="l00736"></a><span class="lineno"> 736</span>  <span class="comment">// acquire the parent flow counter</span></div><div class="line"><a name="l00737"></a><span class="lineno"> 737</span>  <span class="keyword">auto</span>& c = (node->_parent) ? node->_parent->_join_counter : </div><div class="line"><a name="l00738"></a><span class="lineno"> 738</span>  node->_topology->_join_counter;</div><div class="line"><a name="l00739"></a><span class="lineno"> 739</span>  </div><div class="line"><a name="l00740"></a><span class="lineno"> 740</span>  <span class="comment">// switch is faster than nested if-else due to jump table</span></div><div class="line"><a name="l00741"></a><span class="lineno"> 741</span>  <span class="keywordflow">switch</span>(node->_handle.index()) {</div><div class="line"><a name="l00742"></a><span class="lineno"> 742</span>  <span class="comment">// static task</span></div><div class="line"><a name="l00743"></a><span class="lineno"> 743</span>  <span class="keywordflow">case</span> Node::STATIC_WORK:{</div><div class="line"><a name="l00744"></a><span class="lineno"> 744</span>  _invoke_static_work(worker, node);</div><div class="line"><a name="l00745"></a><span class="lineno"> 745</span>  } </div><div class="line"><a name="l00746"></a><span class="lineno"> 746</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00747"></a><span class="lineno"> 747</span>  <span class="comment">// module task</span></div><div class="line"><a name="l00748"></a><span class="lineno"> 748</span>  <span class="keywordflow">case</span> Node::MODULE_WORK: {</div><div class="line"><a name="l00749"></a><span class="lineno"> 749</span>  <span class="keywordtype">bool</span> first_time = !node->_has_state(Node::SPAWNED);</div><div class="line"><a name="l00750"></a><span class="lineno"> 750</span>  <span class="keywordtype">bool</span> emptiness = <span class="keyword">false</span>;</div><div class="line"><a name="l00751"></a><span class="lineno"> 751</span>  _set_up_module_work(node, emptiness);</div><div class="line"><a name="l00752"></a><span class="lineno"> 752</span>  <span class="keywordflow">if</span>(first_time && !emptiness) {</div><div class="line"><a name="l00753"></a><span class="lineno"> 753</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00754"></a><span class="lineno"> 754</span>  }</div><div class="line"><a name="l00755"></a><span class="lineno"> 755</span>  }</div><div class="line"><a name="l00756"></a><span class="lineno"> 756</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00757"></a><span class="lineno"> 757</span>  <span class="comment">// dynamic task</span></div><div class="line"><a name="l00758"></a><span class="lineno"> 758</span>  <span class="keywordflow">case</span> Node::DYNAMIC_WORK: {</div><div class="line"><a name="l00759"></a><span class="lineno"> 759</span> </div><div class="line"><a name="l00760"></a><span class="lineno"> 760</span>  <span class="comment">// Need to create a subflow if it is the first time entering here</span></div><div class="line"><a name="l00761"></a><span class="lineno"> 761</span>  <span class="keywordflow">if</span>(!node->_has_state(Node::SPAWNED)) {</div><div class="line"><a name="l00762"></a><span class="lineno"> 762</span>  </div><div class="line"><a name="l00763"></a><span class="lineno"> 763</span>  <span class="keyword">auto</span>& subgraph = nstd::get<Node::DynamicWork>(node->_handle).subgraph;</div><div class="line"><a name="l00764"></a><span class="lineno"> 764</span> </div><div class="line"><a name="l00765"></a><span class="lineno"> 765</span>  subgraph.clear();</div><div class="line"><a name="l00766"></a><span class="lineno"> 766</span>  Subflow fb(subgraph); </div><div class="line"><a name="l00767"></a><span class="lineno"> 767</span> </div><div class="line"><a name="l00768"></a><span class="lineno"> 768</span>  _invoke_dynamic_work(worker, node, fb);</div><div class="line"><a name="l00769"></a><span class="lineno"> 769</span> </div><div class="line"><a name="l00770"></a><span class="lineno"> 770</span>  node->_set_state(Node::SPAWNED);</div><div class="line"><a name="l00771"></a><span class="lineno"> 771</span> </div><div class="line"><a name="l00772"></a><span class="lineno"> 772</span>  <span class="keywordflow">if</span>(!subgraph.empty()) {</div><div class="line"><a name="l00773"></a><span class="lineno"> 773</span> </div><div class="line"><a name="l00774"></a><span class="lineno"> 774</span>  PassiveVector<Node*> src; </div><div class="line"><a name="l00775"></a><span class="lineno"> 775</span> </div><div class="line"><a name="l00776"></a><span class="lineno"> 776</span>  <span class="keywordflow">for</span>(<span class="keyword">auto</span> n : subgraph._nodes) {</div><div class="line"><a name="l00777"></a><span class="lineno"> 777</span> </div><div class="line"><a name="l00778"></a><span class="lineno"> 778</span>  n->_topology = node->_topology;</div><div class="line"><a name="l00779"></a><span class="lineno"> 779</span>  n->_set_up_join_counter();</div><div class="line"><a name="l00780"></a><span class="lineno"> 780</span>  </div><div class="line"><a name="l00781"></a><span class="lineno"> 781</span>  <span class="keywordflow">if</span>(!fb.detached()) {</div><div class="line"><a name="l00782"></a><span class="lineno"> 782</span>  n->_parent = node;</div><div class="line"><a name="l00783"></a><span class="lineno"> 783</span>  }</div><div class="line"><a name="l00784"></a><span class="lineno"> 784</span> </div><div class="line"><a name="l00785"></a><span class="lineno"> 785</span>  <span class="keywordflow">if</span>(n->num_dependents() == 0) {</div><div class="line"><a name="l00786"></a><span class="lineno"> 786</span>  src.push_back(n);</div><div class="line"><a name="l00787"></a><span class="lineno"> 787</span>  }</div><div class="line"><a name="l00788"></a><span class="lineno"> 788</span>  }</div><div class="line"><a name="l00789"></a><span class="lineno"> 789</span> </div><div class="line"><a name="l00790"></a><span class="lineno"> 790</span>  <span class="keyword">const</span> <span class="keywordtype">bool</span> join = fb.joined();</div><div class="line"><a name="l00791"></a><span class="lineno"> 791</span>  <span class="keywordflow">if</span>(!join) {</div><div class="line"><a name="l00792"></a><span class="lineno"> 792</span>  <span class="comment">// Detach mode</span></div><div class="line"><a name="l00793"></a><span class="lineno"> 793</span>  node->_topology->_join_counter.fetch_add(src.size()); </div><div class="line"><a name="l00794"></a><span class="lineno"> 794</span>  }</div><div class="line"><a name="l00795"></a><span class="lineno"> 795</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00796"></a><span class="lineno"> 796</span>  <span class="comment">// Join mode</span></div><div class="line"><a name="l00797"></a><span class="lineno"> 797</span>  node->_join_counter.fetch_add(src.size());</div><div class="line"><a name="l00798"></a><span class="lineno"> 798</span> </div><div class="line"><a name="l00799"></a><span class="lineno"> 799</span>  <span class="comment">// spawned node needs another second-round execution</span></div><div class="line"><a name="l00800"></a><span class="lineno"> 800</span>  c.fetch_add(1);</div><div class="line"><a name="l00801"></a><span class="lineno"> 801</span>  }</div><div class="line"><a name="l00802"></a><span class="lineno"> 802</span> </div><div class="line"><a name="l00803"></a><span class="lineno"> 803</span>  _schedule(src);</div><div class="line"><a name="l00804"></a><span class="lineno"> 804</span> </div><div class="line"><a name="l00805"></a><span class="lineno"> 805</span>  <span class="keywordflow">if</span>(join) {</div><div class="line"><a name="l00806"></a><span class="lineno"> 806</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00807"></a><span class="lineno"> 807</span>  }</div><div class="line"><a name="l00808"></a><span class="lineno"> 808</span>  } <span class="comment">// End of first time </span></div><div class="line"><a name="l00809"></a><span class="lineno"> 809</span>  }</div><div class="line"><a name="l00810"></a><span class="lineno"> 810</span>  }</div><div class="line"><a name="l00811"></a><span class="lineno"> 811</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00812"></a><span class="lineno"> 812</span>  <span class="comment">// condition task</span></div><div class="line"><a name="l00813"></a><span class="lineno"> 813</span>  <span class="keywordflow">case</span> Node::CONDITION_WORK: {</div><div class="line"><a name="l00814"></a><span class="lineno"> 814</span> </div><div class="line"><a name="l00815"></a><span class="lineno"> 815</span>  <span class="keywordflow">if</span>(node->_has_state(Node::BRANCH)) {</div><div class="line"><a name="l00816"></a><span class="lineno"> 816</span>  node->_join_counter = node->num_strong_dependents();</div><div class="line"><a name="l00817"></a><span class="lineno"> 817</span>  }</div><div class="line"><a name="l00818"></a><span class="lineno"> 818</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00819"></a><span class="lineno"> 819</span>  node->_join_counter = node->num_dependents();</div><div class="line"><a name="l00820"></a><span class="lineno"> 820</span>  }</div><div class="line"><a name="l00821"></a><span class="lineno"> 821</span>  </div><div class="line"><a name="l00822"></a><span class="lineno"> 822</span>  <span class="keywordtype">int</span> id;</div><div class="line"><a name="l00823"></a><span class="lineno"> 823</span>  _invoke_condition_work(worker, node, <span class="keywordtype">id</span>);</div><div class="line"><a name="l00824"></a><span class="lineno"> 824</span> </div><div class="line"><a name="l00825"></a><span class="lineno"> 825</span>  <span class="keywordflow">if</span>(<span class="keywordtype">id</span> >= 0 && static_cast<size_t>(<span class="keywordtype">id</span>) < num_successors) {</div><div class="line"><a name="l00826"></a><span class="lineno"> 826</span>  <span class="keyword">auto</span> s = node->_successors[id];</div><div class="line"><a name="l00827"></a><span class="lineno"> 827</span>  s->_join_counter.store(0);</div><div class="line"><a name="l00828"></a><span class="lineno"> 828</span> </div><div class="line"><a name="l00829"></a><span class="lineno"> 829</span>  <span class="keywordflow">if</span>(s->domain() == worker.domain) {</div><div class="line"><a name="l00830"></a><span class="lineno"> 830</span>  _schedule(s, <span class="keyword">true</span>);</div><div class="line"><a name="l00831"></a><span class="lineno"> 831</span>  }</div><div class="line"><a name="l00832"></a><span class="lineno"> 832</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00833"></a><span class="lineno"> 833</span>  c.fetch_add(1);</div><div class="line"><a name="l00834"></a><span class="lineno"> 834</span>  _schedule(s, <span class="keyword">false</span>);</div><div class="line"><a name="l00835"></a><span class="lineno"> 835</span>  }</div><div class="line"><a name="l00836"></a><span class="lineno"> 836</span>  }</div><div class="line"><a name="l00837"></a><span class="lineno"> 837</span>  return ;</div><div class="line"><a name="l00838"></a><span class="lineno"> 838</span>  } <span class="comment">// no need to add a break here due to the immediate return</span></div><div class="line"><a name="l00839"></a><span class="lineno"> 839</span> </div><div class="line"><a name="l00840"></a><span class="lineno"> 840</span>  <span class="comment">// cudaflow task</span></div><div class="line"><a name="l00841"></a><span class="lineno"> 841</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA</span></div><div class="line"><a name="l00842"></a><span class="lineno"> 842</span>  <span class="keywordflow">case</span> Node::CUDAFLOW_WORK: {</div><div class="line"><a name="l00843"></a><span class="lineno"> 843</span>  _invoke_cudaflow_work(worker, node);</div><div class="line"><a name="l00844"></a><span class="lineno"> 844</span>  }</div><div class="line"><a name="l00845"></a><span class="lineno"> 845</span>  <span class="keywordflow">break</span>; </div><div class="line"><a name="l00846"></a><span class="lineno"> 846</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00847"></a><span class="lineno"> 847</span> </div><div class="line"><a name="l00848"></a><span class="lineno"> 848</span>  <span class="comment">// monostate</span></div><div class="line"><a name="l00849"></a><span class="lineno"> 849</span>  <span class="keywordflow">default</span>:</div><div class="line"><a name="l00850"></a><span class="lineno"> 850</span>  <span class="keywordflow">break</span>;</div><div class="line"><a name="l00851"></a><span class="lineno"> 851</span>  }</div><div class="line"><a name="l00852"></a><span class="lineno"> 852</span>  </div><div class="line"><a name="l00853"></a><span class="lineno"> 853</span> </div><div class="line"><a name="l00854"></a><span class="lineno"> 854</span>  <span class="comment">// We MUST recover the dependency since subflow may have </span></div><div class="line"><a name="l00855"></a><span class="lineno"> 855</span>  <span class="comment">// a condition node to go back (cyclic).</span></div><div class="line"><a name="l00856"></a><span class="lineno"> 856</span>  <span class="comment">// This must be done before scheduling the successors, otherwise this might cause </span></div><div class="line"><a name="l00857"></a><span class="lineno"> 857</span>  <span class="comment">// race condition on the _dependents</span></div><div class="line"><a name="l00858"></a><span class="lineno"> 858</span>  <span class="keywordflow">if</span>(node->_has_state(Node::BRANCH)) {</div><div class="line"><a name="l00859"></a><span class="lineno"> 859</span>  <span class="comment">// If this is a case node, we need to deduct condition predecessors</span></div><div class="line"><a name="l00860"></a><span class="lineno"> 860</span>  node->_join_counter = node->num_strong_dependents();</div><div class="line"><a name="l00861"></a><span class="lineno"> 861</span>  }</div><div class="line"><a name="l00862"></a><span class="lineno"> 862</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00863"></a><span class="lineno"> 863</span>  node->_join_counter = node->num_dependents();</div><div class="line"><a name="l00864"></a><span class="lineno"> 864</span>  }</div><div class="line"><a name="l00865"></a><span class="lineno"> 865</span> </div><div class="line"><a name="l00866"></a><span class="lineno"> 866</span>  node->_unset_state(Node::SPAWNED);</div><div class="line"><a name="l00867"></a><span class="lineno"> 867</span> </div><div class="line"><a name="l00868"></a><span class="lineno"> 868</span>  <span class="comment">// At this point, the node storage might be destructed.</span></div><div class="line"><a name="l00869"></a><span class="lineno"> 869</span>  Node* cache {<span class="keyword">nullptr</span>};</div><div class="line"><a name="l00870"></a><span class="lineno"> 870</span> </div><div class="line"><a name="l00871"></a><span class="lineno"> 871</span>  <span class="keywordflow">for</span>(<span class="keywordtype">size_t</span> i=0; i<num_successors; ++i) {</div><div class="line"><a name="l00872"></a><span class="lineno"> 872</span>  <span class="keywordflow">if</span>(--(node->_successors[i]->_join_counter) == 0) {</div><div class="line"><a name="l00873"></a><span class="lineno"> 873</span>  <span class="keywordflow">if</span>(node->_successors[i]->domain() != worker.domain) {</div><div class="line"><a name="l00874"></a><span class="lineno"> 874</span>  c.fetch_add(1);</div><div class="line"><a name="l00875"></a><span class="lineno"> 875</span>  _schedule(node->_successors[i], <span class="keyword">false</span>);</div><div class="line"><a name="l00876"></a><span class="lineno"> 876</span>  }</div><div class="line"><a name="l00877"></a><span class="lineno"> 877</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00878"></a><span class="lineno"> 878</span>  <span class="keywordflow">if</span>(cache) {</div><div class="line"><a name="l00879"></a><span class="lineno"> 879</span>  c.fetch_add(1);</div><div class="line"><a name="l00880"></a><span class="lineno"> 880</span>  _schedule(cache, <span class="keyword">false</span>);</div><div class="line"><a name="l00881"></a><span class="lineno"> 881</span>  }</div><div class="line"><a name="l00882"></a><span class="lineno"> 882</span>  cache = node->_successors[i];</div><div class="line"><a name="l00883"></a><span class="lineno"> 883</span>  }</div><div class="line"><a name="l00884"></a><span class="lineno"> 884</span>  }</div><div class="line"><a name="l00885"></a><span class="lineno"> 885</span>  }</div><div class="line"><a name="l00886"></a><span class="lineno"> 886</span> </div><div class="line"><a name="l00887"></a><span class="lineno"> 887</span>  <span class="keywordflow">if</span>(cache) {</div><div class="line"><a name="l00888"></a><span class="lineno"> 888</span>  _schedule(cache, <span class="keyword">true</span>);</div><div class="line"><a name="l00889"></a><span class="lineno"> 889</span>  }</div><div class="line"><a name="l00890"></a><span class="lineno"> 890</span> }</div><div class="line"><a name="l00891"></a><span class="lineno"> 891</span> </div><div class="line"><a name="l00892"></a><span class="lineno"> 892</span> <span class="comment">// Procedure: _invoke_static_work</span></div><div class="line"><a name="l00893"></a><span class="lineno"> 893</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_invoke_static_work(Worker& worker, Node* node) {</div><div class="line"><a name="l00894"></a><span class="lineno"> 894</span>  <span class="keywordflow">if</span>(_observer) {</div><div class="line"><a name="l00895"></a><span class="lineno"> 895</span>  _observer->on_entry(worker.id, TaskView(node));</div><div class="line"><a name="l00896"></a><span class="lineno"> 896</span>  nstd::get<Node::StaticWork>(node->_handle).work();</div><div class="line"><a name="l00897"></a><span class="lineno"> 897</span>  _observer->on_exit(worker.id, TaskView(node));</div><div class="line"><a name="l00898"></a><span class="lineno"> 898</span>  }</div><div class="line"><a name="l00899"></a><span class="lineno"> 899</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00900"></a><span class="lineno"> 900</span>  nstd::get<Node::StaticWork>(node->_handle).work();</div><div class="line"><a name="l00901"></a><span class="lineno"> 901</span>  }</div><div class="line"><a name="l00902"></a><span class="lineno"> 902</span> }</div><div class="line"><a name="l00903"></a><span class="lineno"> 903</span> </div><div class="line"><a name="l00904"></a><span class="lineno"> 904</span> <span class="comment">// Procedure: _invoke_dynamic_work</span></div><div class="line"><a name="l00905"></a><span class="lineno"> 905</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_invoke_dynamic_work(Worker& worker, Node* node, Subflow& sf) {</div><div class="line"><a name="l00906"></a><span class="lineno"> 906</span>  <span class="keywordflow">if</span>(_observer) {</div><div class="line"><a name="l00907"></a><span class="lineno"> 907</span>  _observer->on_entry(worker.id, TaskView(node));</div><div class="line"><a name="l00908"></a><span class="lineno"> 908</span>  nstd::get<Node::DynamicWork>(node->_handle).work(sf);</div><div class="line"><a name="l00909"></a><span class="lineno"> 909</span>  _observer->on_exit(worker.id, TaskView(node));</div><div class="line"><a name="l00910"></a><span class="lineno"> 910</span>  }</div><div class="line"><a name="l00911"></a><span class="lineno"> 911</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00912"></a><span class="lineno"> 912</span>  nstd::get<Node::DynamicWork>(node->_handle).work(sf);</div><div class="line"><a name="l00913"></a><span class="lineno"> 913</span>  }</div><div class="line"><a name="l00914"></a><span class="lineno"> 914</span> }</div><div class="line"><a name="l00915"></a><span class="lineno"> 915</span> </div><div class="line"><a name="l00916"></a><span class="lineno"> 916</span> <span class="comment">// Procedure: _invoke_condition_work</span></div><div class="line"><a name="l00917"></a><span class="lineno"> 917</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_invoke_condition_work(Worker& worker, Node* node, <span class="keywordtype">int</span>& <span class="keywordtype">id</span>) {</div><div class="line"><a name="l00918"></a><span class="lineno"> 918</span>  <span class="keywordflow">if</span>(_observer) {</div><div class="line"><a name="l00919"></a><span class="lineno"> 919</span>  _observer->on_entry(worker.id, TaskView(node));</div><div class="line"><a name="l00920"></a><span class="lineno"> 920</span>  <span class="keywordtype">id</span> = nstd::get<Node::ConditionWork>(node->_handle).work();</div><div class="line"><a name="l00921"></a><span class="lineno"> 921</span>  _observer->on_exit(worker.id, TaskView(node));</div><div class="line"><a name="l00922"></a><span class="lineno"> 922</span>  }</div><div class="line"><a name="l00923"></a><span class="lineno"> 923</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00924"></a><span class="lineno"> 924</span>  <span class="keywordtype">id</span> = nstd::get<Node::ConditionWork>(node->_handle).work();</div><div class="line"><a name="l00925"></a><span class="lineno"> 925</span>  }</div><div class="line"><a name="l00926"></a><span class="lineno"> 926</span> }</div><div class="line"><a name="l00927"></a><span class="lineno"> 927</span> </div><div class="line"><a name="l00928"></a><span class="lineno"> 928</span> <span class="preprocessor">#ifdef TF_ENABLE_CUDA</span></div><div class="line"><a name="l00929"></a><span class="lineno"> 929</span> <span class="comment">// Procedure: _invoke_cudaflow_work</span></div><div class="line"><a name="l00930"></a><span class="lineno"> 930</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_invoke_cudaflow_work(Worker& worker, Node* node) {</div><div class="line"><a name="l00931"></a><span class="lineno"> 931</span> </div><div class="line"><a name="l00932"></a><span class="lineno"> 932</span>  assert(worker.domain == node->domain());</div><div class="line"><a name="l00933"></a><span class="lineno"> 933</span> </div><div class="line"><a name="l00934"></a><span class="lineno"> 934</span>  <span class="keywordflow">if</span>(_observer) {</div><div class="line"><a name="l00935"></a><span class="lineno"> 935</span>  _observer->on_entry(worker.id, TaskView(node));</div><div class="line"><a name="l00936"></a><span class="lineno"> 936</span>  _invoke_cudaflow_work_impl(worker, node);</div><div class="line"><a name="l00937"></a><span class="lineno"> 937</span>  _observer->on_exit(worker.id, TaskView(node));</div><div class="line"><a name="l00938"></a><span class="lineno"> 938</span>  }</div><div class="line"><a name="l00939"></a><span class="lineno"> 939</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l00940"></a><span class="lineno"> 940</span>  _invoke_cudaflow_work_impl(worker, node);</div><div class="line"><a name="l00941"></a><span class="lineno"> 941</span>  }</div><div class="line"><a name="l00942"></a><span class="lineno"> 942</span> }</div><div class="line"><a name="l00943"></a><span class="lineno"> 943</span> </div><div class="line"><a name="l00944"></a><span class="lineno"> 944</span> <span class="comment">// Procedure: _invoke_cudaflow_work_impl</span></div><div class="line"><a name="l00945"></a><span class="lineno"> 945</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_invoke_cudaflow_work_impl(Worker& w, Node* node) {</div><div class="line"><a name="l00946"></a><span class="lineno"> 946</span> </div><div class="line"><a name="l00947"></a><span class="lineno"> 947</span>  <span class="keyword">auto</span>& h = nstd::get<Node::cudaFlowWork>(node->_handle);</div><div class="line"><a name="l00948"></a><span class="lineno"> 948</span> </div><div class="line"><a name="l00949"></a><span class="lineno"> 949</span>  h.graph.clear();</div><div class="line"><a name="l00950"></a><span class="lineno"> 950</span> </div><div class="line"><a name="l00951"></a><span class="lineno"> 951</span>  cudaFlow cf(h.graph, [repeat=1] () <span class="keyword">mutable</span> { return repeat-- == 0; });</div><div class="line"><a name="l00952"></a><span class="lineno"> 952</span> </div><div class="line"><a name="l00953"></a><span class="lineno"> 953</span>  h.work(cf); </div><div class="line"><a name="l00954"></a><span class="lineno"> 954</span> </div><div class="line"><a name="l00955"></a><span class="lineno"> 955</span>  <span class="keywordflow">if</span>(h.graph.empty()) {</div><div class="line"><a name="l00956"></a><span class="lineno"> 956</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00957"></a><span class="lineno"> 957</span>  }</div><div class="line"><a name="l00958"></a><span class="lineno"> 958</span>  </div><div class="line"><a name="l00959"></a><span class="lineno"> 959</span>  <span class="comment">// transforms cudaFlow to a native cudaGraph under the specified device</span></div><div class="line"><a name="l00960"></a><span class="lineno"> 960</span>  <span class="comment">// and launches the graph through a given or an internal device stream</span></div><div class="line"><a name="l00961"></a><span class="lineno"> 961</span>  <span class="keyword">const</span> <span class="keywordtype">int</span> d = cf._device;</div><div class="line"><a name="l00962"></a><span class="lineno"> 962</span> </div><div class="line"><a name="l00963"></a><span class="lineno"> 963</span>  cudaScopedDevice ctx(d);</div><div class="line"><a name="l00964"></a><span class="lineno"> 964</span>  </div><div class="line"><a name="l00965"></a><span class="lineno"> 965</span>  <span class="keyword">auto</span> s = cf._stream ? *(cf._stream) : </div><div class="line"><a name="l00966"></a><span class="lineno"> 966</span>  _cuda_devices[d].streams[w.id - _id_offset[w.domain]];</div><div class="line"><a name="l00967"></a><span class="lineno"> 967</span> </div><div class="line"><a name="l00968"></a><span class="lineno"> 968</span>  h.graph._make_native_graph();</div><div class="line"><a name="l00969"></a><span class="lineno"> 969</span> </div><div class="line"><a name="l00970"></a><span class="lineno"> 970</span>  cudaGraphExec_t exec;</div><div class="line"><a name="l00971"></a><span class="lineno"> 971</span> </div><div class="line"><a name="l00972"></a><span class="lineno"> 972</span>  TF_CHECK_CUDA(</div><div class="line"><a name="l00973"></a><span class="lineno"> 973</span>  cudaGraphInstantiate(&exec, h.graph._native_handle, <span class="keyword">nullptr</span>, <span class="keyword">nullptr</span>, 0),</div><div class="line"><a name="l00974"></a><span class="lineno"> 974</span>  <span class="stringliteral">"failed to create an executable cudaGraph"</span></div><div class="line"><a name="l00975"></a><span class="lineno"> 975</span>  );</div><div class="line"><a name="l00976"></a><span class="lineno"> 976</span>  </div><div class="line"><a name="l00977"></a><span class="lineno"> 977</span>  <span class="keywordflow">while</span>(!cf._predicate()) {</div><div class="line"><a name="l00978"></a><span class="lineno"> 978</span>  TF_CHECK_CUDA(</div><div class="line"><a name="l00979"></a><span class="lineno"> 979</span>  cudaGraphLaunch(exec, s), <span class="stringliteral">"failed to launch cudaGraph on stream "</span>, s</div><div class="line"><a name="l00980"></a><span class="lineno"> 980</span>  );</div><div class="line"><a name="l00981"></a><span class="lineno"> 981</span> </div><div class="line"><a name="l00982"></a><span class="lineno"> 982</span>  TF_CHECK_CUDA(</div><div class="line"><a name="l00983"></a><span class="lineno"> 983</span>  cudaStreamSynchronize(s), <span class="stringliteral">"failed to synchronize stream "</span>, s</div><div class="line"><a name="l00984"></a><span class="lineno"> 984</span>  );</div><div class="line"><a name="l00985"></a><span class="lineno"> 985</span>  }</div><div class="line"><a name="l00986"></a><span class="lineno"> 986</span> </div><div class="line"><a name="l00987"></a><span class="lineno"> 987</span>  TF_CHECK_CUDA(</div><div class="line"><a name="l00988"></a><span class="lineno"> 988</span>  cudaGraphExecDestroy(exec), <span class="stringliteral">"failed to destroy an executable cudaGraph"</span></div><div class="line"><a name="l00989"></a><span class="lineno"> 989</span>  );</div><div class="line"><a name="l00990"></a><span class="lineno"> 990</span> }</div><div class="line"><a name="l00991"></a><span class="lineno"> 991</span> <span class="preprocessor">#endif</span></div><div class="line"><a name="l00992"></a><span class="lineno"> 992</span> </div><div class="line"><a name="l00993"></a><span class="lineno"> 993</span> <span class="comment">// Procedure: _set_up_module_work</span></div><div class="line"><a name="l00994"></a><span class="lineno"> 994</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_set_up_module_work(Node* node, <span class="keywordtype">bool</span>& ept) {</div><div class="line"><a name="l00995"></a><span class="lineno"> 995</span> </div><div class="line"><a name="l00996"></a><span class="lineno"> 996</span>  <span class="comment">// second time to enter this context</span></div><div class="line"><a name="l00997"></a><span class="lineno"> 997</span>  <span class="keywordflow">if</span>(node->_has_state(Node::SPAWNED)) {</div><div class="line"><a name="l00998"></a><span class="lineno"> 998</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l00999"></a><span class="lineno"> 999</span>  }</div><div class="line"><a name="l01000"></a><span class="lineno"> 1000</span>  </div><div class="line"><a name="l01001"></a><span class="lineno"> 1001</span>  <span class="comment">// first time to enter this context</span></div><div class="line"><a name="l01002"></a><span class="lineno"> 1002</span>  node->_set_state(Node::SPAWNED);</div><div class="line"><a name="l01003"></a><span class="lineno"> 1003</span> </div><div class="line"><a name="l01004"></a><span class="lineno"> 1004</span>  <span class="keyword">auto</span> module = nstd::get<Node::ModuleWork>(node->_handle).module;</div><div class="line"><a name="l01005"></a><span class="lineno"> 1005</span> </div><div class="line"><a name="l01006"></a><span class="lineno"> 1006</span>  <span class="keywordflow">if</span>(module->empty()) {</div><div class="line"><a name="l01007"></a><span class="lineno"> 1007</span>  ept = <span class="keyword">true</span>;</div><div class="line"><a name="l01008"></a><span class="lineno"> 1008</span>  <span class="keywordflow">return</span>;</div><div class="line"><a name="l01009"></a><span class="lineno"> 1009</span>  }</div><div class="line"><a name="l01010"></a><span class="lineno"> 1010</span> </div><div class="line"><a name="l01011"></a><span class="lineno"> 1011</span>  PassiveVector<Node*> src;</div><div class="line"><a name="l01012"></a><span class="lineno"> 1012</span> </div><div class="line"><a name="l01013"></a><span class="lineno"> 1013</span>  <span class="keywordflow">for</span>(<span class="keyword">auto</span> n: module->_graph._nodes) {</div><div class="line"><a name="l01014"></a><span class="lineno"> 1014</span> </div><div class="line"><a name="l01015"></a><span class="lineno"> 1015</span>  n->_topology = node->_topology;</div><div class="line"><a name="l01016"></a><span class="lineno"> 1016</span>  n->_parent = node;</div><div class="line"><a name="l01017"></a><span class="lineno"> 1017</span>  n->_set_up_join_counter();</div><div class="line"><a name="l01018"></a><span class="lineno"> 1018</span> </div><div class="line"><a name="l01019"></a><span class="lineno"> 1019</span>  <span class="keywordflow">if</span>(n->num_dependents() == 0) {</div><div class="line"><a name="l01020"></a><span class="lineno"> 1020</span>  src.push_back(n);</div><div class="line"><a name="l01021"></a><span class="lineno"> 1021</span>  }</div><div class="line"><a name="l01022"></a><span class="lineno"> 1022</span>  }</div><div class="line"><a name="l01023"></a><span class="lineno"> 1023</span> </div><div class="line"><a name="l01024"></a><span class="lineno"> 1024</span>  node->_join_counter.fetch_add(src.size());</div><div class="line"><a name="l01025"></a><span class="lineno"> 1025</span>  </div><div class="line"><a name="l01026"></a><span class="lineno"> 1026</span>  <span class="keywordflow">if</span>(node->_parent == <span class="keyword">nullptr</span>) {</div><div class="line"><a name="l01027"></a><span class="lineno"> 1027</span>  node->_topology->_join_counter.fetch_add(1);</div><div class="line"><a name="l01028"></a><span class="lineno"> 1028</span>  }</div><div class="line"><a name="l01029"></a><span class="lineno"> 1029</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l01030"></a><span class="lineno"> 1030</span>  node->_parent->_join_counter.fetch_add(1);</div><div class="line"><a name="l01031"></a><span class="lineno"> 1031</span>  }</div><div class="line"><a name="l01032"></a><span class="lineno"> 1032</span>  </div><div class="line"><a name="l01033"></a><span class="lineno"> 1033</span>  <span class="comment">// src can't be empty (banned outside)</span></div><div class="line"><a name="l01034"></a><span class="lineno"> 1034</span>  _schedule(src);</div><div class="line"><a name="l01035"></a><span class="lineno"> 1035</span> }</div><div class="line"><a name="l01036"></a><span class="lineno"> 1036</span> </div><div class="line"><a name="l01037"></a><span class="lineno"> 1037</span> <span class="comment">// Function: run</span></div><div class="line"><a name="l01038"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a81f35d5b0a20ac0646447eb80d97c0aa"> 1038</a></span> <span class="keyword">inline</span> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a81f35d5b0a20ac0646447eb80d97c0aa">Executor::run</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& f) {</div><div class="line"><a name="l01039"></a><span class="lineno"> 1039</span>  <span class="keywordflow">return</span> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">run_n</a>(f, 1, [](){});</div><div class="line"><a name="l01040"></a><span class="lineno"> 1040</span> }</div><div class="line"><a name="l01041"></a><span class="lineno"> 1041</span> </div><div class="line"><a name="l01042"></a><span class="lineno"> 1042</span> <span class="comment">// Function: run</span></div><div class="line"><a name="l01043"></a><span class="lineno"> 1043</span> <span class="keyword">template</span> <<span class="keyword">typename</span> C></div><div class="line"><a name="l01044"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a94b45b69dc247c82c2ada8c90424b172"> 1044</a></span> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a81f35d5b0a20ac0646447eb80d97c0aa">Executor::run</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& f, C&& c) {</div><div class="line"><a name="l01045"></a><span class="lineno"> 1045</span>  <span class="keywordflow">return</span> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">run_n</a>(f, 1, std::forward<C>(c));</div><div class="line"><a name="l01046"></a><span class="lineno"> 1046</span> }</div><div class="line"><a name="l01047"></a><span class="lineno"> 1047</span> </div><div class="line"><a name="l01048"></a><span class="lineno"> 1048</span> <span class="comment">// Function: run_n</span></div><div class="line"><a name="l01049"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6"> 1049</a></span> <span class="keyword">inline</span> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">Executor::run_n</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& f, <span class="keywordtype">size_t</span> repeat) {</div><div class="line"><a name="l01050"></a><span class="lineno"> 1050</span>  <span class="keywordflow">return</span> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">run_n</a>(f, repeat, [](){});</div><div class="line"><a name="l01051"></a><span class="lineno"> 1051</span> }</div><div class="line"><a name="l01052"></a><span class="lineno"> 1052</span> </div><div class="line"><a name="l01053"></a><span class="lineno"> 1053</span> <span class="comment">// Function: run_n</span></div><div class="line"><a name="l01054"></a><span class="lineno"> 1054</span> <span class="keyword">template</span> <<span class="keyword">typename</span> C></div><div class="line"><a name="l01055"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#aff558312e64d19f98f26c91349c317da"> 1055</a></span> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">Executor::run_n</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& f, <span class="keywordtype">size_t</span> repeat, C&& c) {</div><div class="line"><a name="l01056"></a><span class="lineno"> 1056</span>  <span class="keywordflow">return</span> <a class="code" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">run_until</a>(f, [repeat]() <span class="keyword">mutable</span> { <span class="keywordflow">return</span> repeat-- == 0; }, std::forward<C>(c));</div><div class="line"><a name="l01057"></a><span class="lineno"> 1057</span> }</div><div class="line"><a name="l01058"></a><span class="lineno"> 1058</span> </div><div class="line"><a name="l01059"></a><span class="lineno"> 1059</span> <span class="comment">// Function: run_until </span></div><div class="line"><a name="l01060"></a><span class="lineno"> 1060</span> <span class="keyword">template</span><<span class="keyword">typename</span> P></div><div class="line"><a name="l01061"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6"> 1061</a></span> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">Executor::run_until</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& f, P&& pred) {</div><div class="line"><a name="l01062"></a><span class="lineno"> 1062</span>  <span class="keywordflow">return</span> <a class="code" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">run_until</a>(f, std::forward<P>(pred), [](){});</div><div class="line"><a name="l01063"></a><span class="lineno"> 1063</span> }</div><div class="line"><a name="l01064"></a><span class="lineno"> 1064</span> </div><div class="line"><a name="l01065"></a><span class="lineno"> 1065</span> <span class="comment">// Function: _set_up_topology</span></div><div class="line"><a name="l01066"></a><span class="lineno"> 1066</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_set_up_topology(Topology* tpg) {</div><div class="line"><a name="l01067"></a><span class="lineno"> 1067</span> </div><div class="line"><a name="l01068"></a><span class="lineno"> 1068</span>  tpg->_sources.clear();</div><div class="line"><a name="l01069"></a><span class="lineno"> 1069</span>  </div><div class="line"><a name="l01070"></a><span class="lineno"> 1070</span>  <span class="comment">// scan each node in the graph and build up the links</span></div><div class="line"><a name="l01071"></a><span class="lineno"> 1071</span>  <span class="keywordflow">for</span>(<span class="keyword">auto</span> node : tpg->_taskflow._graph._nodes) {</div><div class="line"><a name="l01072"></a><span class="lineno"> 1072</span> </div><div class="line"><a name="l01073"></a><span class="lineno"> 1073</span>  node->_topology = tpg;</div><div class="line"><a name="l01074"></a><span class="lineno"> 1074</span>  node->_clear_state();</div><div class="line"><a name="l01075"></a><span class="lineno"> 1075</span> </div><div class="line"><a name="l01076"></a><span class="lineno"> 1076</span>  <span class="keywordflow">if</span>(node->num_dependents() == 0) {</div><div class="line"><a name="l01077"></a><span class="lineno"> 1077</span>  tpg->_sources.push_back(node);</div><div class="line"><a name="l01078"></a><span class="lineno"> 1078</span>  }</div><div class="line"><a name="l01079"></a><span class="lineno"> 1079</span> </div><div class="line"><a name="l01080"></a><span class="lineno"> 1080</span>  node->_set_up_join_counter();</div><div class="line"><a name="l01081"></a><span class="lineno"> 1081</span>  }</div><div class="line"><a name="l01082"></a><span class="lineno"> 1082</span> </div><div class="line"><a name="l01083"></a><span class="lineno"> 1083</span>  tpg->_join_counter.store(tpg->_sources.size(), std::memory_order_relaxed);</div><div class="line"><a name="l01084"></a><span class="lineno"> 1084</span> }</div><div class="line"><a name="l01085"></a><span class="lineno"> 1085</span> </div><div class="line"><a name="l01086"></a><span class="lineno"> 1086</span> <span class="comment">// Function: _tear_down_topology</span></div><div class="line"><a name="l01087"></a><span class="lineno"> 1087</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_tear_down_topology(Topology** tpg) {</div><div class="line"><a name="l01088"></a><span class="lineno"> 1088</span> </div><div class="line"><a name="l01089"></a><span class="lineno"> 1089</span>  <span class="keyword">auto</span> &f = (*tpg)->_taskflow;</div><div class="line"><a name="l01090"></a><span class="lineno"> 1090</span> </div><div class="line"><a name="l01091"></a><span class="lineno"> 1091</span>  <span class="comment">//assert(&tpg == &(f._topologies.front()));</span></div><div class="line"><a name="l01092"></a><span class="lineno"> 1092</span> </div><div class="line"><a name="l01093"></a><span class="lineno"> 1093</span>  <span class="comment">// case 1: we still need to run the topology again</span></div><div class="line"><a name="l01094"></a><span class="lineno"> 1094</span>  <span class="keywordflow">if</span>(! (*tpg)->_pred() ) {</div><div class="line"><a name="l01095"></a><span class="lineno"> 1095</span>  <span class="comment">//tpg->_recover_num_sinks();</span></div><div class="line"><a name="l01096"></a><span class="lineno"> 1096</span> </div><div class="line"><a name="l01097"></a><span class="lineno"> 1097</span>  assert((*tpg)->_join_counter == 0);</div><div class="line"><a name="l01098"></a><span class="lineno"> 1098</span>  (*tpg)->_join_counter = (*tpg)->_sources.size();</div><div class="line"><a name="l01099"></a><span class="lineno"> 1099</span> </div><div class="line"><a name="l01100"></a><span class="lineno"> 1100</span>  _schedule((*tpg)->_sources); </div><div class="line"><a name="l01101"></a><span class="lineno"> 1101</span>  }</div><div class="line"><a name="l01102"></a><span class="lineno"> 1102</span>  <span class="comment">// case 2: the final run of this topology</span></div><div class="line"><a name="l01103"></a><span class="lineno"> 1103</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l01104"></a><span class="lineno"> 1104</span>  </div><div class="line"><a name="l01105"></a><span class="lineno"> 1105</span>  <span class="keywordflow">if</span>((*tpg)->_call != <span class="keyword">nullptr</span>) {</div><div class="line"><a name="l01106"></a><span class="lineno"> 1106</span>  (*tpg)->_call();</div><div class="line"><a name="l01107"></a><span class="lineno"> 1107</span>  }</div><div class="line"><a name="l01108"></a><span class="lineno"> 1108</span> </div><div class="line"><a name="l01109"></a><span class="lineno"> 1109</span>  f._mtx.lock();</div><div class="line"><a name="l01110"></a><span class="lineno"> 1110</span> </div><div class="line"><a name="l01111"></a><span class="lineno"> 1111</span>  <span class="comment">// If there is another run (interleave between lock)</span></div><div class="line"><a name="l01112"></a><span class="lineno"> 1112</span>  <span class="keywordflow">if</span>(f._topologies.size() > 1) {</div><div class="line"><a name="l01113"></a><span class="lineno"> 1113</span> </div><div class="line"><a name="l01114"></a><span class="lineno"> 1114</span>  assert((*tpg)->_join_counter == 0);</div><div class="line"><a name="l01115"></a><span class="lineno"> 1115</span> </div><div class="line"><a name="l01116"></a><span class="lineno"> 1116</span>  <span class="comment">// Set the promise</span></div><div class="line"><a name="l01117"></a><span class="lineno"> 1117</span>  (*tpg)->_promise.set_value();</div><div class="line"><a name="l01118"></a><span class="lineno"> 1118</span>  f._topologies.pop_front();</div><div class="line"><a name="l01119"></a><span class="lineno"> 1119</span>  f._mtx.unlock();</div><div class="line"><a name="l01120"></a><span class="lineno"> 1120</span>  </div><div class="line"><a name="l01121"></a><span class="lineno"> 1121</span>  <span class="comment">// decrement the topology but since this is not the last we don't notify</span></div><div class="line"><a name="l01122"></a><span class="lineno"> 1122</span>  _decrement_topology();</div><div class="line"><a name="l01123"></a><span class="lineno"> 1123</span> </div><div class="line"><a name="l01124"></a><span class="lineno"> 1124</span>  *tpg = &(f._topologies.front());</div><div class="line"><a name="l01125"></a><span class="lineno"> 1125</span> </div><div class="line"><a name="l01126"></a><span class="lineno"> 1126</span>  _set_up_topology(*tpg);</div><div class="line"><a name="l01127"></a><span class="lineno"> 1127</span>  _schedule((*tpg)->_sources);</div><div class="line"><a name="l01128"></a><span class="lineno"> 1128</span> </div><div class="line"><a name="l01129"></a><span class="lineno"> 1129</span>  <span class="comment">//f._topologies.front()._bind(f._graph);</span></div><div class="line"><a name="l01130"></a><span class="lineno"> 1130</span>  <span class="comment">//*tpg = &(f._topologies.front());</span></div><div class="line"><a name="l01131"></a><span class="lineno"> 1131</span> </div><div class="line"><a name="l01132"></a><span class="lineno"> 1132</span>  <span class="comment">//assert(f._topologies.front()._join_counter == 0);</span></div><div class="line"><a name="l01133"></a><span class="lineno"> 1133</span> </div><div class="line"><a name="l01134"></a><span class="lineno"> 1134</span>  <span class="comment">//f._topologies.front()._join_counter = f._topologies.front()._sources.size();</span></div><div class="line"><a name="l01135"></a><span class="lineno"> 1135</span> </div><div class="line"><a name="l01136"></a><span class="lineno"> 1136</span>  <span class="comment">//_schedule(f._topologies.front()._sources);</span></div><div class="line"><a name="l01137"></a><span class="lineno"> 1137</span>  }</div><div class="line"><a name="l01138"></a><span class="lineno"> 1138</span>  <span class="keywordflow">else</span> {</div><div class="line"><a name="l01139"></a><span class="lineno"> 1139</span>  assert(f._topologies.size() == 1);</div><div class="line"><a name="l01140"></a><span class="lineno"> 1140</span> </div><div class="line"><a name="l01141"></a><span class="lineno"> 1141</span>  <span class="comment">// Need to back up the promise first here becuz taskflow might be </span></div><div class="line"><a name="l01142"></a><span class="lineno"> 1142</span>  <span class="comment">// destroy before taskflow leaves</span></div><div class="line"><a name="l01143"></a><span class="lineno"> 1143</span>  <span class="keyword">auto</span> p {<a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/utility/move.html">std::move</a>((*tpg)->_promise)};</div><div class="line"><a name="l01144"></a><span class="lineno"> 1144</span> </div><div class="line"><a name="l01145"></a><span class="lineno"> 1145</span>  f._topologies.pop_front();</div><div class="line"><a name="l01146"></a><span class="lineno"> 1146</span> </div><div class="line"><a name="l01147"></a><span class="lineno"> 1147</span>  f._mtx.unlock();</div><div class="line"><a name="l01148"></a><span class="lineno"> 1148</span> </div><div class="line"><a name="l01149"></a><span class="lineno"> 1149</span>  <span class="comment">// We set the promise in the end in case taskflow leaves before taskflow</span></div><div class="line"><a name="l01150"></a><span class="lineno"> 1150</span>  p.set_value();</div><div class="line"><a name="l01151"></a><span class="lineno"> 1151</span> </div><div class="line"><a name="l01152"></a><span class="lineno"> 1152</span>  _decrement_topology_and_notify();</div><div class="line"><a name="l01153"></a><span class="lineno"> 1153</span> </div><div class="line"><a name="l01154"></a><span class="lineno"> 1154</span>  <span class="comment">// Reset topology so caller can stop execution</span></div><div class="line"><a name="l01155"></a><span class="lineno"> 1155</span>  *tpg = <span class="keyword">nullptr</span>;</div><div class="line"><a name="l01156"></a><span class="lineno"> 1156</span>  }</div><div class="line"><a name="l01157"></a><span class="lineno"> 1157</span>  }</div><div class="line"><a name="l01158"></a><span class="lineno"> 1158</span> }</div><div class="line"><a name="l01159"></a><span class="lineno"> 1159</span> </div><div class="line"><a name="l01160"></a><span class="lineno"> 1160</span> <span class="comment">// Function: run_until</span></div><div class="line"><a name="l01161"></a><span class="lineno"> 1161</span> <span class="keyword">template</span> <<span class="keyword">typename</span> P, <span class="keyword">typename</span> C></div><div class="line"><a name="l01162"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#a85f5cd590d2dbe15d1f2f1f8d920a7e6"> 1162</a></span> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> <a class="code" href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">Executor::run_until</a>(<a class="code" href="classtf_1_1Taskflow.html">Taskflow</a>& f, P&& pred, C&& c) {</div><div class="line"><a name="l01163"></a><span class="lineno"> 1163</span> </div><div class="line"><a name="l01164"></a><span class="lineno"> 1164</span>  _increment_topology();</div><div class="line"><a name="l01165"></a><span class="lineno"> 1165</span> </div><div class="line"><a name="l01166"></a><span class="lineno"> 1166</span>  <span class="comment">// Special case of predicate</span></div><div class="line"><a name="l01167"></a><span class="lineno"> 1167</span>  <span class="keywordflow">if</span>(f.<a class="code" href="classtf_1_1Taskflow.html#a224301842d94c44fd7fe99d8ac2ba241">empty</a>() || pred()) {</div><div class="line"><a name="l01168"></a><span class="lineno"> 1168</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/promise.html">std::promise<void></a> promise;</div><div class="line"><a name="l01169"></a><span class="lineno"> 1169</span>  promise.set_value();</div><div class="line"><a name="l01170"></a><span class="lineno"> 1170</span>  _decrement_topology_and_notify();</div><div class="line"><a name="l01171"></a><span class="lineno"> 1171</span>  <span class="keywordflow">return</span> promise.get_future();</div><div class="line"><a name="l01172"></a><span class="lineno"> 1172</span>  }</div><div class="line"><a name="l01173"></a><span class="lineno"> 1173</span>  </div><div class="line"><a name="l01174"></a><span class="lineno"> 1174</span>  <span class="comment">// Multi-threaded execution.</span></div><div class="line"><a name="l01175"></a><span class="lineno"> 1175</span>  <span class="keywordtype">bool</span> run_now {<span class="keyword">false</span>};</div><div class="line"><a name="l01176"></a><span class="lineno"> 1176</span>  Topology* tpg;</div><div class="line"><a name="l01177"></a><span class="lineno"> 1177</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/future.html">std::future<void></a> future;</div><div class="line"><a name="l01178"></a><span class="lineno"> 1178</span>  </div><div class="line"><a name="l01179"></a><span class="lineno"> 1179</span>  {</div><div class="line"><a name="l01180"></a><span class="lineno"> 1180</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard<std::mutex></a> lock(f._mtx);</div><div class="line"><a name="l01181"></a><span class="lineno"> 1181</span> </div><div class="line"><a name="l01182"></a><span class="lineno"> 1182</span>  <span class="comment">// create a topology for this run</span></div><div class="line"><a name="l01183"></a><span class="lineno"> 1183</span>  <span class="comment">//tpg = &(f._topologies.emplace_back(f, std::forward<P>(pred), std::forward<C>(c)));</span></div><div class="line"><a name="l01184"></a><span class="lineno"> 1184</span>  f._topologies.emplace_back(f, std::forward<P>(pred), std::forward<C>(c));</div><div class="line"><a name="l01185"></a><span class="lineno"> 1185</span>  tpg = &(f._topologies.back());</div><div class="line"><a name="l01186"></a><span class="lineno"> 1186</span>  future = tpg->_promise.get_future();</div><div class="line"><a name="l01187"></a><span class="lineno"> 1187</span>  </div><div class="line"><a name="l01188"></a><span class="lineno"> 1188</span>  <span class="keywordflow">if</span>(f._topologies.size() == 1) {</div><div class="line"><a name="l01189"></a><span class="lineno"> 1189</span>  run_now = <span class="keyword">true</span>;</div><div class="line"><a name="l01190"></a><span class="lineno"> 1190</span>  <span class="comment">//tpg->_bind(f._graph);</span></div><div class="line"><a name="l01191"></a><span class="lineno"> 1191</span>  <span class="comment">//_schedule(tpg->_sources);</span></div><div class="line"><a name="l01192"></a><span class="lineno"> 1192</span>  }</div><div class="line"><a name="l01193"></a><span class="lineno"> 1193</span>  }</div><div class="line"><a name="l01194"></a><span class="lineno"> 1194</span>  </div><div class="line"><a name="l01195"></a><span class="lineno"> 1195</span>  <span class="comment">// Notice here calling schedule may cause the topology to be removed sonner </span></div><div class="line"><a name="l01196"></a><span class="lineno"> 1196</span>  <span class="comment">// before the function leaves.</span></div><div class="line"><a name="l01197"></a><span class="lineno"> 1197</span>  <span class="keywordflow">if</span>(run_now) {</div><div class="line"><a name="l01198"></a><span class="lineno"> 1198</span>  _set_up_topology(tpg);</div><div class="line"><a name="l01199"></a><span class="lineno"> 1199</span>  _schedule(tpg->_sources);</div><div class="line"><a name="l01200"></a><span class="lineno"> 1200</span>  }</div><div class="line"><a name="l01201"></a><span class="lineno"> 1201</span> </div><div class="line"><a name="l01202"></a><span class="lineno"> 1202</span>  <span class="keywordflow">return</span> future;</div><div class="line"><a name="l01203"></a><span class="lineno"> 1203</span> }</div><div class="line"><a name="l01204"></a><span class="lineno"> 1204</span> </div><div class="line"><a name="l01205"></a><span class="lineno"> 1205</span> <span class="comment">// Procedure: _increment_topology</span></div><div class="line"><a name="l01206"></a><span class="lineno"> 1206</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_increment_topology() {</div><div class="line"><a name="l01207"></a><span class="lineno"> 1207</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard<std::mutex></a> lock(_topology_mutex);</div><div class="line"><a name="l01208"></a><span class="lineno"> 1208</span>  ++_num_topologies;</div><div class="line"><a name="l01209"></a><span class="lineno"> 1209</span> }</div><div class="line"><a name="l01210"></a><span class="lineno"> 1210</span> </div><div class="line"><a name="l01211"></a><span class="lineno"> 1211</span> <span class="comment">// Procedure: _decrement_topology_and_notify</span></div><div class="line"><a name="l01212"></a><span class="lineno"> 1212</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_decrement_topology_and_notify() {</div><div class="line"><a name="l01213"></a><span class="lineno"> 1213</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard<std::mutex></a> lock(_topology_mutex);</div><div class="line"><a name="l01214"></a><span class="lineno"> 1214</span>  <span class="keywordflow">if</span>(--_num_topologies == 0) {</div><div class="line"><a name="l01215"></a><span class="lineno"> 1215</span>  _topology_cv.notify_all();</div><div class="line"><a name="l01216"></a><span class="lineno"> 1216</span>  }</div><div class="line"><a name="l01217"></a><span class="lineno"> 1217</span> }</div><div class="line"><a name="l01218"></a><span class="lineno"> 1218</span> </div><div class="line"><a name="l01219"></a><span class="lineno"> 1219</span> <span class="comment">// Procedure: _decrement_topology</span></div><div class="line"><a name="l01220"></a><span class="lineno"> 1220</span> <span class="keyword">inline</span> <span class="keywordtype">void</span> Executor::_decrement_topology() {</div><div class="line"><a name="l01221"></a><span class="lineno"> 1221</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard<std::mutex></a> <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/lock.html">lock</a>(_topology_mutex);</div><div class="line"><a name="l01222"></a><span class="lineno"> 1222</span>  --_num_topologies;</div><div class="line"><a name="l01223"></a><span class="lineno"> 1223</span> }</div><div class="line"><a name="l01224"></a><span class="lineno"> 1224</span> </div><div class="line"><a name="l01225"></a><span class="lineno"> 1225</span> <span class="comment">// Procedure: wait_for_all</span></div><div class="line"><a name="l01226"></a><span class="lineno"><a class="line" href="classtf_1_1Executor.html#ab9aa252f70e9a40020a1e5a89d485b85"> 1226</a></span> <span class="keyword">inline</span> <span class="keywordtype">void</span> <a class="code" href="classtf_1_1Executor.html#ab9aa252f70e9a40020a1e5a89d485b85">Executor::wait_for_all</a>() {</div><div class="line"><a name="l01227"></a><span class="lineno"> 1227</span>  <a class="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/doxygen/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/unique_lock.html">std::unique_lock<std::mutex></a> lock(_topology_mutex);</div><div class="line"><a name="l01228"></a><span class="lineno"> 1228</span>  _topology_cv.wait(lock, [&](){ <span class="keywordflow">return</span> _num_topologies == 0; });</div><div class="line"><a name="l01229"></a><span class="lineno"> 1229</span> }</div><div class="line"><a name="l01230"></a><span class="lineno"> 1230</span> </div><div class="line"><a name="l01231"></a><span class="lineno"> 1231</span> } <span class="comment">// end of namespace tf -----------------------------------------------------</span></div><div class="line"><a name="l01232"></a><span class="lineno"> 1232</span> </div><div class="line"><a name="l01233"></a><span class="lineno"> 1233</span> </div><div class="ttc" id="classtf_1_1Executor_html_a6487d589cb1f6b078b69fd3bb1082345"><div class="ttname"><a href="classtf_1_1Executor.html#a6487d589cb1f6b078b69fd3bb1082345">tf::Executor::this_worker_id</a></div><div class="ttdeci">int this_worker_id() const</div><div class="ttdoc">queries the id of the caller thread in this executor </div><div class="ttdef"><b>Definition:</b> executor.hpp:361</div></div>
<div class="ttc" id="classtf_1_1Executor_html_a81f35d5b0a20ac0646447eb80d97c0aa"><div class="ttname"><a href="classtf_1_1Executor.html#a81f35d5b0a20ac0646447eb80d97c0aa">tf::Executor::run</a></div><div class="ttdeci">std::future< void > run(Taskflow &taskflow)</div><div class="ttdoc">runs the taskflow once </div><div class="ttdef"><b>Definition:</b> executor.hpp:1038</div></div>
<div class="ttc" id="classtf_1_1Executor_html_a3ed11b270f0c7fe1c55a81c60d20411f"><div class="ttname"><a href="classtf_1_1Executor.html#a3ed11b270f0c7fe1c55a81c60d20411f">tf::Executor::remove_observer</a></div><div class="ttdeci">void remove_observer()</div><div class="ttdoc">removes the associated observer </div><div class="ttdef"><b>Definition:</b> executor.hpp:631</div></div>
<div class="ttc" id="atomic_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/atomic/atomic.html">std::atomic< size_t ></a></div></div>
<div class="ttc" id="uniform_int_distribution_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/numeric/random/uniform_int_distribution.html">std::uniform_int_distribution</a></div></div>
<div class="ttc" id="classtf_1_1Executor_html_a8acf7515e8e8fdda366ace68bcd65aa6"><div class="ttname"><a href="classtf_1_1Executor.html#a8acf7515e8e8fdda366ace68bcd65aa6">tf::Executor::run_until</a></div><div class="ttdeci">std::future< void > run_until(Taskflow &taskflow, P &&pred)</div><div class="ttdoc">runs the taskflow multiple times until the predicate becomes true and then invokes a callback ...</div><div class="ttdef"><b>Definition:</b> executor.hpp:1061</div></div>
<div class="ttc" id="classtf_1_1Executor_html_a5a511b0cc23b264826373d3dabcef670"><div class="ttname"><a href="classtf_1_1Executor.html#a5a511b0cc23b264826373d3dabcef670">tf::Executor::~Executor</a></div><div class="ttdeci">~Executor()</div><div class="ttdoc">destructs the executor </div><div class="ttdef"><b>Definition:</b> executor.hpp:313</div></div>
<div class="ttc" id="yield_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/yield.html">std::this_thread::yield</a></div><div class="ttdeci">T yield(T... args)</div></div>
<div class="ttc" id="condition_variable_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/condition_variable.html">std::condition_variable</a></div></div>
<div class="ttc" id="future_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/future.html">std::future</a></div></div>
<div class="ttc" id="namespacetf_html"><div class="ttname"><a href="namespacetf.html">tf</a></div><div class="ttdef"><b>Definition:</b> error.hpp:9</div></div>
<div class="ttc" id="promise_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/promise.html">std::promise</a></div></div>
<div class="ttc" id="hardware_concurrency_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency.html">std::thread::hardware_concurrency</a></div><div class="ttdeci">T hardware_concurrency(T... args)</div></div>
<div class="ttc" id="mutex_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/mutex.html">std::mutex</a></div></div>
<div class="ttc" id="classtf_1_1Executor_html_a7f43cde72d3e0a17e2d006cbe7a41ff3"><div class="ttname"><a href="classtf_1_1Executor.html#a7f43cde72d3e0a17e2d006cbe7a41ff3">tf::Executor::make_observer</a></div><div class="ttdeci">Observer * make_observer(Args &&... args)</div><div class="ttdoc">constructs an observer to inspect the activities of worker threads </div><div class="ttdef"><b>Definition:</b> executor.hpp:622</div></div>
<div class="ttc" id="classtf_1_1Executor_html_aa09f058c7f04941de86e1d06238cdc48"><div class="ttname"><a href="classtf_1_1Executor.html#aa09f058c7f04941de86e1d06238cdc48">tf::Executor::Executor</a></div><div class="ttdeci">Executor(size_t N=std::thread::hardware_concurrency(), size_t M=cuda_num_devices())</div><div class="ttdoc">constructs the executor with N/M cpu/gpu worker threads </div><div class="ttdef"><b>Definition:</b> executor.hpp:259</div></div>
<div class="ttc" id="lock_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/lock.html">std::lock</a></div><div class="ttdeci">T lock(T... args)</div></div>
<div class="ttc" id="classtf_1_1Taskflow_html"><div class="ttname"><a href="classtf_1_1Taskflow.html">tf::Taskflow</a></div><div class="ttdoc">main entry to create a task dependency graph </div><div class="ttdef"><b>Definition:</b> taskflow.hpp:18</div></div>
<div class="ttc" id="random_device_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/numeric/random/random_device.html">std::random_device</a></div></div>
<div class="ttc" id="classtf_1_1Taskflow_html_a224301842d94c44fd7fe99d8ac2ba241"><div class="ttname"><a href="classtf_1_1Taskflow.html#a224301842d94c44fd7fe99d8ac2ba241">tf::Taskflow::empty</a></div><div class="ttdeci">bool empty() const</div><div class="ttdoc">queries the emptiness of the taskflow </div><div class="ttdef"><b>Definition:</b> taskflow.hpp:132</div></div>
<div class="ttc" id="move_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/utility/move.html">std::move</a></div><div class="ttdeci">T move(T... args)</div></div>
<div class="ttc" id="classtf_1_1Executor_html_a8acfc81ae6a64989761ce229262c9a21"><div class="ttname"><a href="classtf_1_1Executor.html#a8acfc81ae6a64989761ce229262c9a21">tf::Executor::num_domains</a></div><div class="ttdeci">size_t num_domains() const</div><div class="ttdoc">queries the number of worker domains </div><div class="ttdef"><b>Definition:</b> executor.hpp:345</div></div>
<div class="ttc" id="unique_lock_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/unique_lock.html">std::unique_lock</a></div></div>
<div class="ttc" id="vector_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/container/vector.html">std::vector< cudaStream_t ></a></div></div>
<div class="ttc" id="unique_ptr_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/memory/unique_ptr.html">std::unique_ptr</a></div></div>
<div class="ttc" id="ref_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/utility/functional/ref.html">std::ref</a></div><div class="ttdeci">T ref(T... args)</div></div>
<div class="ttc" id="lock_guard_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/thread/lock_guard.html">std::lock_guard</a></div></div>
<div class="ttc" id="classtf_1_1Executor_html"><div class="ttname"><a href="classtf_1_1Executor.html">tf::Executor</a></div><div class="ttdoc">execution interface for running a taskflow graph </div><div class="ttdef"><b>Definition:</b> executor.hpp:32</div></div>
<div class="ttc" id="classtf_1_1Executor_html_a79223c7f32c9d715e725b8d775de2d29"><div class="ttname"><a href="classtf_1_1Executor.html#a79223c7f32c9d715e725b8d775de2d29">tf::Executor::num_workers</a></div><div class="ttdeci">size_t num_workers() const</div><div class="ttdoc">queries the number of worker threads (can be zero) </div><div class="ttdef"><b>Definition:</b> executor.hpp:340</div></div>
<div class="ttc" id="classtf_1_1Executor_html_a6d6c28ed58211e4c27a99571e5bf0b6c"><div class="ttname"><a href="classtf_1_1Executor.html#a6d6c28ed58211e4c27a99571e5bf0b6c">tf::Executor::num_topologies</a></div><div class="ttdeci">size_t num_topologies() const</div><div class="ttdoc">queries the number of running topologies at the time of this call </div><div class="ttdef"><b>Definition:</b> executor.hpp:350</div></div>
<div class="ttc" id="mersenne_twister_engine_html"><div class="ttname"><a href="http://en.cppreference.com/w/cpp/numeric/random/mersenne_twister_engine.html">std::mt19937</a></div></div>
<div class="ttc" id="classtf_1_1Subflow_html"><div class="ttname"><a href="classtf_1_1Subflow.html">tf::Subflow</a></div><div class="ttdoc">building methods of a subflow graph in dynamic tasking </div><div class="ttdef"><b>Definition:</b> flow_builder.hpp:860</div></div>
<div class="ttc" id="classtf_1_1Executor_html_adca6cd0ce1bd7e6fa2ed2a55c9ae15e6"><div class="ttname"><a href="classtf_1_1Executor.html#adca6cd0ce1bd7e6fa2ed2a55c9ae15e6">tf::Executor::run_n</a></div><div class="ttdeci">std::future< void > run_n(Taskflow &taskflow, size_t N)</div><div class="ttdoc">runs the taskflow for N times </div><div class="ttdef"><b>Definition:</b> executor.hpp:1049</div></div>
<div class="ttc" id="classtf_1_1Executor_html_ab9aa252f70e9a40020a1e5a89d485b85"><div class="ttname"><a href="classtf_1_1Executor.html#ab9aa252f70e9a40020a1e5a89d485b85">tf::Executor::wait_for_all</a></div><div class="ttdeci">void wait_for_all()</div><div class="ttdoc">wait for all pending graphs to complete </div><div class="ttdef"><b>Definition:</b> executor.hpp:1226</div></div>
</div><!-- fragment --></div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="navelem"><a class="el" href="dir_88dad41ea55ca2177e141d32a93e931c.html">taskflow</a></li><li class="navelem"><a class="el" href="dir_220cd4d9b8cb38c840b455d5d75c25bb.html">core</a></li><li class="navelem"><b>executor.hpp</b></li>
<li class="footer">Generated by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.14 </li>
</ul>
</div>
</body>
</html>