forked from azk0019/CourseProject
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy path7pass40model.log
More file actions
108 lines (108 loc) · 16.9 KB
/
7pass40model.log
File metadata and controls
108 lines (108 loc) · 16.9 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
2020-12-03 10:10:49,944:DEBUG:Start of model:
2020-12-03 10:10:49,947:INFO:using autotuned alpha, starting with [0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025]
2020-12-03 10:10:49,954:INFO:using serial LDA version on this node
2020-12-03 10:10:50,066:INFO:running online (multi-pass) LDA training, 40 topics, 7 passes over the supplied corpus of 2673 documents, updating model once every 2673 documents, evaluating perplexity every 2673 documents, iterating 100x with a convergence threshold of 0.001000
2020-12-03 10:10:50,066:WARNING:too few updates, training might not converge; consider increasing the number of passes or iterations to improve accuracy
2020-12-03 10:10:50,148:DEBUG:bound: at document #0
2020-12-03 10:11:06,784:INFO:-13.031 per-word bound, 8370.1 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:11:06,784:INFO:PROGRESS: pass 0, at document #2673/2673
2020-12-03 10:11:06,784:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:11:20,825:DEBUG:1183/2673 documents converged within 100 iterations
2020-12-03 10:11:20,869:INFO:optimized alpha [0.023997417, 0.022365075, 0.02265548, 0.024750292, 0.02300152, 0.02416003, 0.028231757, 0.024704386, 0.022539789, 0.02276371, 0.022919267, 0.022236513, 0.0236634, 0.022907931, 0.024827901, 0.022857342, 0.022511698, 0.02512807, 0.027442096, 0.02362644, 0.023573097, 0.023251658, 0.023846254, 0.022580836, 0.022094555, 0.024255276, 0.026208304, 0.024693985, 0.023227809, 0.023998868, 0.023837466, 0.024817476, 0.023068005, 0.024864975, 0.022960968, 0.023569733, 0.026904924, 0.024604265, 0.025996527, 0.02336336]
2020-12-03 10:11:20,869:DEBUG:updating topics
2020-12-03 10:11:20,969:INFO:topic #24 (0.022): 0.009*"gore" + 0.008*"bush" + 0.006*"gun" + 0.005*"war" + 0.005*"bushs" + 0.005*"control" + 0.005*"gores" + 0.004*"republican" + 0.004*"plan" + 0.004*"party"
2020-12-03 10:11:20,970:INFO:topic #11 (0.022): 0.018*"bush" + 0.011*"lazio" + 0.010*"george" + 0.010*"bushs" + 0.009*"tax" + 0.007*"plan" + 0.006*"surplus" + 0.005*"security" + 0.005*"clinton" + 0.005*"support"
2020-12-03 10:11:20,970:INFO:topic #36 (0.027): 0.042*"bush" + 0.027*"gore" + 0.011*"campaign" + 0.008*"president" + 0.007*"al" + 0.007*"percent" + 0.007*"voters" + 0.007*"george" + 0.007*"bushs" + 0.007*"gores"
2020-12-03 10:11:20,971:INFO:topic #18 (0.027): 0.041*"gore" + 0.025*"bush" + 0.016*"president" + 0.009*"campaign" + 0.009*"vice" + 0.008*"gores" + 0.007*"bushs" + 0.006*"al" + 0.006*"george" + 0.006*"tax"
2020-12-03 10:11:20,971:INFO:topic #6 (0.028): 0.036*"bush" + 0.035*"gore" + 0.014*"president" + 0.008*"vice" + 0.007*"campaign" + 0.007*"george" + 0.007*"al" + 0.006*"bushs" + 0.005*"gores" + 0.005*"voters"
2020-12-03 10:11:20,974:INFO:topic diff=19.211441, rho=1.000000
2020-12-03 10:11:21,127:DEBUG:bound: at document #0
2020-12-03 10:11:34,538:INFO:-8.839 per-word bound, 457.8 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:11:34,538:INFO:PROGRESS: pass 1, at document #2673/2673
2020-12-03 10:11:34,538:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:11:44,410:DEBUG:2346/2673 documents converged within 100 iterations
2020-12-03 10:11:44,450:INFO:optimized alpha [0.021839105, 0.020232648, 0.020680707, 0.023173036, 0.02084285, 0.023324938, 0.030171135, 0.022890547, 0.020403504, 0.020574259, 0.021014668, 0.020154223, 0.021534164, 0.020610567, 0.022903342, 0.020692378, 0.02039595, 0.023347845, 0.027993789, 0.021601563, 0.021935198, 0.021117564, 0.022296615, 0.020443263, 0.019801343, 0.022556698, 0.02533473, 0.023286955, 0.021075666, 0.022124777, 0.021903083, 0.023414077, 0.020902088, 0.022930935, 0.02079672, 0.021674754, 0.026472494, 0.022659326, 0.024777863, 0.021373142]
2020-12-03 10:11:44,450:DEBUG:updating topics
2020-12-03 10:11:44,543:INFO:topic #24 (0.020): 0.013*"war" + 0.009*"bush" + 0.007*"arcadia" + 0.007*"arcadias" + 0.006*"bushs" + 0.006*"mattel" + 0.005*"republican" + 0.004*"novel" + 0.004*"gores" + 0.004*"party"
2020-12-03 10:11:44,543:INFO:topic #11 (0.020): 0.040*"lazio" + 0.016*"lazios" + 0.016*"bush" + 0.015*"tax" + 0.012*"george" + 0.012*"bushs" + 0.012*"plan" + 0.010*"surplus" + 0.010*"security" + 0.008*"social"
2020-12-03 10:11:44,544:INFO:topic #36 (0.026): 0.044*"bush" + 0.028*"gore" + 0.012*"campaign" + 0.011*"percent" + 0.010*"voters" + 0.008*"bushs" + 0.007*"president" + 0.007*"george" + 0.007*"al" + 0.006*"gores"
2020-12-03 10:11:44,544:INFO:topic #18 (0.028): 0.042*"gore" + 0.021*"bush" + 0.015*"president" + 0.009*"vice" + 0.009*"gores" + 0.009*"campaign" + 0.006*"al" + 0.006*"bushs" + 0.006*"clinton" + 0.005*"tax"
2020-12-03 10:11:44,544:INFO:topic #6 (0.030): 0.036*"bush" + 0.034*"gore" + 0.014*"president" + 0.009*"campaign" + 0.007*"vice" + 0.007*"al" + 0.006*"george" + 0.006*"bushs" + 0.006*"voters" + 0.005*"clinton"
2020-12-03 10:11:44,547:INFO:topic diff=7.138132, rho=0.946551
2020-12-03 10:11:44,703:DEBUG:bound: at document #0
2020-12-03 10:11:56,180:INFO:-8.365 per-word bound, 329.8 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:11:56,180:INFO:PROGRESS: pass 2, at document #2673/2673
2020-12-03 10:11:56,180:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:12:04,595:DEBUG:2557/2673 documents converged within 100 iterations
2020-12-03 10:12:04,652:INFO:optimized alpha [0.02005297, 0.01870766, 0.019213604, 0.021932049, 0.019160544, 0.023723515, 0.03163843, 0.0216825, 0.018768935, 0.018833954, 0.019848933, 0.018724123, 0.019849407, 0.018801078, 0.021412227, 0.018960783, 0.01891521, 0.021895155, 0.028327193, 0.020143157, 0.020869967, 0.019460572, 0.02170936, 0.018827826, 0.01800532, 0.021226738, 0.024476545, 0.022445962, 0.019390218, 0.020670727, 0.020512946, 0.022583116, 0.019354813, 0.021574028, 0.019190744, 0.02035893, 0.026253916, 0.021229496, 0.02358798, 0.01987509]
2020-12-03 10:12:04,652:DEBUG:updating topics
2020-12-03 10:12:04,776:INFO:topic #24 (0.018): 0.014*"war" + 0.012*"bush" + 0.007*"arcadia" + 0.007*"arcadias" + 0.007*"novel" + 0.007*"bushs" + 0.006*"mattel" + 0.005*"catherine" + 0.005*"actual" + 0.004*"love"
2020-12-03 10:12:04,777:INFO:topic #1 (0.019): 0.020*"page" + 0.019*"front" + 0.016*"drug" + 0.016*"re" + 0.016*"gore" + 0.015*"bush" + 0.012*"al" + 0.010*"president" + 0.007*"medicare" + 0.006*"major"
2020-12-03 10:12:04,777:INFO:topic #36 (0.026): 0.046*"bush" + 0.029*"gore" + 0.015*"percent" + 0.012*"voters" + 0.011*"campaign" + 0.008*"bushs" + 0.008*"poll" + 0.007*"george" + 0.007*"president" + 0.007*"al"
2020-12-03 10:12:04,777:INFO:topic #18 (0.028): 0.044*"gore" + 0.019*"bush" + 0.015*"president" + 0.010*"vice" + 0.009*"gores" + 0.008*"campaign" + 0.006*"al" + 0.006*"clinton" + 0.005*"governor" + 0.005*"bushs"
2020-12-03 10:12:04,778:INFO:topic #6 (0.032): 0.036*"bush" + 0.035*"gore" + 0.014*"president" + 0.009*"campaign" + 0.007*"vice" + 0.007*"al" + 0.006*"george" + 0.006*"voters" + 0.006*"bushs" + 0.006*"clinton"
2020-12-03 10:12:04,781:INFO:topic diff=3.802336, rho=0.933033
2020-12-03 10:12:04,949:DEBUG:bound: at document #0
2020-12-03 10:12:16,462:INFO:-8.264 per-word bound, 307.4 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:12:16,463:INFO:PROGRESS: pass 3, at document #2673/2673
2020-12-03 10:12:16,463:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:12:25,079:DEBUG:2604/2673 documents converged within 100 iterations
2020-12-03 10:12:25,137:INFO:optimized alpha [0.018621668, 0.017623628, 0.018048933, 0.020947427, 0.017785158, 0.024801169, 0.033123933, 0.020883441, 0.017532488, 0.017443981, 0.019172583, 0.017665327, 0.018486304, 0.017313242, 0.020156773, 0.017555403, 0.017871726, 0.020693036, 0.02861877, 0.019069852, 0.020214107, 0.018111141, 0.02179289, 0.017577246, 0.0165607, 0.02020297, 0.023845451, 0.022028957, 0.018048756, 0.019475456, 0.019501014, 0.022105405, 0.018249815, 0.020715522, 0.01799843, 0.019438395, 0.026226357, 0.020128775, 0.02253536, 0.018659092]
2020-12-03 10:12:25,137:DEBUG:updating topics
2020-12-03 10:12:25,262:INFO:topic #24 (0.017): 0.015*"war" + 0.012*"bush" + 0.007*"novel" + 0.007*"arcadia" + 0.007*"arcadias" + 0.006*"bushs" + 0.006*"mattel" + 0.005*"actual" + 0.005*"catherine" + 0.005*"violence"
2020-12-03 10:12:25,262:INFO:topic #13 (0.017): 0.023*"bush" + 0.012*"gore" + 0.009*"pine" + 0.008*"florida" + 0.007*"campaign" + 0.007*"texas" + 0.006*"karner" + 0.006*"gores" + 0.006*"committee" + 0.005*"andrew"
2020-12-03 10:12:25,263:INFO:topic #36 (0.026): 0.047*"bush" + 0.030*"gore" + 0.018*"percent" + 0.014*"voters" + 0.011*"campaign" + 0.009*"poll" + 0.009*"bushs" + 0.007*"george" + 0.007*"president" + 0.007*"al"
2020-12-03 10:12:25,263:INFO:topic #18 (0.029): 0.045*"gore" + 0.017*"bush" + 0.015*"president" + 0.010*"vice" + 0.009*"gores" + 0.008*"campaign" + 0.007*"clinton" + 0.006*"al" + 0.005*"governor" + 0.004*"bushs"
2020-12-03 10:12:25,263:INFO:topic #6 (0.033): 0.037*"bush" + 0.035*"gore" + 0.014*"president" + 0.010*"campaign" + 0.007*"vice" + 0.007*"al" + 0.006*"voters" + 0.006*"george" + 0.006*"clinton" + 0.006*"bushs"
2020-12-03 10:12:25,266:INFO:topic diff=1.446494, rho=0.922681
2020-12-03 10:12:25,426:DEBUG:bound: at document #0
2020-12-03 10:12:37,037:INFO:-8.211 per-word bound, 296.3 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:12:37,037:INFO:PROGRESS: pass 4, at document #2673/2673
2020-12-03 10:12:37,037:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:12:45,384:DEBUG:2598/2673 documents converged within 100 iterations
2020-12-03 10:12:45,453:INFO:optimized alpha [0.017458992, 0.01689524, 0.017181808, 0.020117246, 0.016651435, 0.02627978, 0.03464037, 0.020368587, 0.016579581, 0.016354175, 0.01889579, 0.01681845, 0.017369354, 0.01606981, 0.019089421, 0.016367922, 0.01722945, 0.01969716, 0.0288375, 0.018284524, 0.019930309, 0.01701626, 0.022396415, 0.016553696, 0.015351415, 0.019370677, 0.023292195, 0.021971358, 0.01694222, 0.018539269, 0.018831838, 0.02190707, 0.017440708, 0.02019127, 0.017066645, 0.018756784, 0.02636202, 0.01927035, 0.021666419, 0.017663091]
2020-12-03 10:12:45,454:DEBUG:updating topics
2020-12-03 10:12:45,591:INFO:topic #24 (0.015): 0.015*"war" + 0.012*"bush" + 0.008*"novel" + 0.007*"arcadia" + 0.007*"arcadias" + 0.006*"mattel" + 0.006*"bushs" + 0.006*"violence" + 0.005*"actual" + 0.005*"catherine"
2020-12-03 10:12:45,591:INFO:topic #13 (0.016): 0.023*"bush" + 0.011*"pine" + 0.010*"gore" + 0.008*"karner" + 0.007*"texas" + 0.007*"campaign" + 0.006*"gores" + 0.006*"blue" + 0.006*"florida" + 0.005*"andrew"
2020-12-03 10:12:45,592:INFO:topic #36 (0.026): 0.048*"bush" + 0.031*"gore" + 0.020*"percent" + 0.016*"voters" + 0.011*"campaign" + 0.011*"poll" + 0.009*"bushs" + 0.007*"george" + 0.007*"president" + 0.007*"al"
2020-12-03 10:12:45,592:INFO:topic #18 (0.029): 0.046*"gore" + 0.016*"president" + 0.015*"bush" + 0.010*"vice" + 0.010*"gores" + 0.007*"campaign" + 0.007*"clinton" + 0.006*"al" + 0.005*"governor" + 0.004*"people"
2020-12-03 10:12:45,592:INFO:topic #6 (0.035): 0.037*"bush" + 0.036*"gore" + 0.014*"president" + 0.010*"campaign" + 0.007*"vice" + 0.007*"voters" + 0.006*"al" + 0.006*"clinton" + 0.006*"george" + 0.005*"gores"
2020-12-03 10:12:45,596:INFO:topic diff=0.729786, rho=0.914308
2020-12-03 10:12:45,747:DEBUG:bound: at document #0
2020-12-03 10:12:57,154:INFO:-8.175 per-word bound, 288.9 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:12:57,154:INFO:PROGRESS: pass 5, at document #2673/2673
2020-12-03 10:12:57,154:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:13:05,207:DEBUG:2624/2673 documents converged within 100 iterations
2020-12-03 10:13:05,262:INFO:optimized alpha [0.016505573, 0.016407946, 0.0165141, 0.019459257, 0.015718915, 0.02792344, 0.03610004, 0.019947765, 0.01584344, 0.015467037, 0.018824074, 0.016129598, 0.016427469, 0.015038976, 0.018174531, 0.015358333, 0.01697342, 0.018867614, 0.029076083, 0.017670304, 0.01984347, 0.016085545, 0.023290535, 0.015725844, 0.014335889, 0.018695304, 0.022826176, 0.022102699, 0.016020134, 0.01780234, 0.018479073, 0.021920161, 0.016876448, 0.019913571, 0.016351297, 0.018250337, 0.026649073, 0.018573364, 0.020920303, 0.016829757]
2020-12-03 10:13:05,263:DEBUG:updating topics
2020-12-03 10:13:05,394:INFO:topic #24 (0.014): 0.015*"war" + 0.012*"bush" + 0.008*"novel" + 0.007*"arcadia" + 0.007*"arcadias" + 0.006*"mattel" + 0.006*"violence" + 0.006*"bushs" + 0.006*"actual" + 0.005*"catherine"
2020-12-03 10:13:05,394:INFO:topic #13 (0.015): 0.022*"bush" + 0.012*"pine" + 0.009*"gore" + 0.008*"karner" + 0.007*"texas" + 0.006*"blue" + 0.006*"campaign" + 0.006*"gores" + 0.006*"andrew" + 0.005*"florida"
2020-12-03 10:13:05,394:INFO:topic #5 (0.028): 0.038*"bush" + 0.022*"plan" + 0.020*"gore" + 0.020*"health" + 0.019*"medicare" + 0.019*"drug" + 0.013*"prescription" + 0.013*"bushs" + 0.012*"care" + 0.009*"education"
2020-12-03 10:13:05,395:INFO:topic #18 (0.029): 0.047*"gore" + 0.016*"president" + 0.014*"bush" + 0.010*"vice" + 0.010*"gores" + 0.007*"campaign" + 0.007*"clinton" + 0.007*"al" + 0.005*"governor" + 0.004*"people"
2020-12-03 10:13:05,395:INFO:topic #6 (0.036): 0.037*"bush" + 0.036*"gore" + 0.014*"president" + 0.010*"campaign" + 0.007*"vice" + 0.007*"voters" + 0.006*"al" + 0.006*"clinton" + 0.006*"gores" + 0.005*"george"
2020-12-03 10:13:05,398:INFO:topic diff=0.474147, rho=0.907288
2020-12-03 10:13:05,550:DEBUG:bound: at document #0
2020-12-03 10:13:16,814:INFO:-8.149 per-word bound, 283.8 perplexity estimate based on a held-out corpus of 2673 documents with 381037 words
2020-12-03 10:13:16,814:INFO:PROGRESS: pass 6, at document #2673/2673
2020-12-03 10:13:16,814:DEBUG:performing inference on a chunk of 2673 documents
2020-12-03 10:13:24,737:DEBUG:2625/2673 documents converged within 100 iterations
2020-12-03 10:13:24,797:INFO:optimized alpha [0.01571306, 0.016082268, 0.01604299, 0.018917583, 0.01492529, 0.029756809, 0.03761518, 0.019692134, 0.015271994, 0.014746613, 0.018955916, 0.0156205585, 0.015632484, 0.014165326, 0.01742318, 0.014501929, 0.017023219, 0.018148586, 0.02934553, 0.017254218, 0.019890891, 0.015287584, 0.024411824, 0.0150122605, 0.013469277, 0.018149437, 0.022495452, 0.022389386, 0.015240696, 0.017209949, 0.018380618, 0.02211078, 0.016478317, 0.019806206, 0.015811134, 0.01782804, 0.026935214, 0.017971497, 0.020279098, 0.016146697]
2020-12-03 10:13:24,798:DEBUG:updating topics
2020-12-03 10:13:24,930:INFO:topic #24 (0.013): 0.016*"war" + 0.012*"bush" + 0.009*"novel" + 0.007*"arcadias" + 0.007*"arcadia" + 0.007*"violence" + 0.006*"mattel" + 0.006*"actual" + 0.006*"catherine" + 0.006*"engagement"
2020-12-03 10:13:24,931:INFO:topic #13 (0.014): 0.021*"bush" + 0.012*"pine" + 0.009*"karner" + 0.008*"gore" + 0.007*"texas" + 0.007*"blue" + 0.006*"campaign" + 0.006*"andrew" + 0.006*"gores" + 0.005*"irish"
2020-12-03 10:13:24,931:INFO:topic #18 (0.029): 0.048*"gore" + 0.016*"president" + 0.013*"bush" + 0.010*"vice" + 0.010*"gores" + 0.007*"clinton" + 0.007*"campaign" + 0.007*"al" + 0.005*"governor" + 0.004*"people"
2020-12-03 10:13:24,932:INFO:topic #5 (0.030): 0.038*"bush" + 0.021*"plan" + 0.020*"gore" + 0.020*"health" + 0.019*"drug" + 0.019*"medicare" + 0.013*"prescription" + 0.012*"bushs" + 0.012*"care" + 0.010*"education"
2020-12-03 10:13:24,932:INFO:topic #6 (0.038): 0.037*"gore" + 0.036*"bush" + 0.014*"president" + 0.010*"campaign" + 0.008*"vice" + 0.007*"voters" + 0.006*"al" + 0.006*"clinton" + 0.006*"gores" + 0.005*"democrats"
2020-12-03 10:13:24,935:INFO:topic diff=0.351341, rho=0.901250
2020-12-03 10:13:24,998:DEBUG:End of model:
2020-12-03 10:13:25,000:INFO:saving LdaState object under .\LDA_data\40baseline2sav.state, separately None
2020-12-03 10:13:25,000:DEBUG:{'uri': '.\\LDA_data\\40baseline2sav.state', 'mode': 'wb', 'buffering': -1, 'encoding': None, 'errors': None, 'newline': None, 'closefd': True, 'opener': None, 'ignore_ext': False, 'transport_params': None}
2020-12-03 10:13:25,032:INFO:saved .\LDA_data\40baseline2sav.state
2020-12-03 10:13:25,032:DEBUG:{'uri': '.\\LDA_data\\40baseline2sav.id2word', 'mode': 'wb', 'buffering': -1, 'encoding': None, 'errors': None, 'newline': None, 'closefd': True, 'opener': None, 'ignore_ext': False, 'transport_params': None}
2020-12-03 10:13:25,038:INFO:saving LdaModel object under .\LDA_data\40baseline2sav, separately ['expElogbeta', 'sstats']
2020-12-03 10:13:25,038:INFO:storing np array 'expElogbeta' to .\LDA_data\40baseline2sav.expElogbeta.npy
2020-12-03 10:13:25,041:INFO:not storing attribute id2word
2020-12-03 10:13:25,041:INFO:not storing attribute state
2020-12-03 10:13:25,042:INFO:not storing attribute dispatcher
2020-12-03 10:13:25,042:DEBUG:{'uri': '.\\LDA_data\\40baseline2sav', 'mode': 'wb', 'buffering': -1, 'encoding': None, 'errors': None, 'newline': None, 'closefd': True, 'opener': None, 'ignore_ext': False, 'transport_params': None}
2020-12-03 10:13:25,044:INFO:saved .\LDA_data\40baseline2sav