1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title>LLVM Test Results for $DateString</title></head>
<body bgcolor=white>
<center><font size=+3 face=Verdana><b>LLVM Test Results for $DateString</b></font></center>
<hr height=1>
<table width=100%>
<tr><td valign=top align=center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77">
<font size=+1><b>Sections:</b></font><br>
</td></tr><tr><td bgcolor="#FFCC99" align=center>
<a href="#Overview">Overview</a><br>
<a href="#Changes">Changes</a><br>
<a href="#Dejagnu">Dejagnu Tests</a><br>
<a href="#Trends">Trends</a><br>
<a href="#Programs">Programs</a><br>
</td></tr></table></td></tr></table>
<p>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77"
<font size=+1><b>Previous:</b></font><br>
</td></tr><tr><td bgcolor="#FFCC99">
$PrevDaysList
</td></tr></table></td></tr></table>
<p>
<font size=+1><b>Back to:</b></font><br>
<a href="http://llvm.cs.uiuc.edu/testresults/">Test Results</a><br>
<a href="http://llvm.cs.uiuc.edu/">LLVM Page</a><p>
</td><td valign=top>
<center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
<font size=+2 face=Verdana><b><a name="Overview">Today's Test Results Overview</font></b>
</td></tr></table></td></tr></table></center><p>
<!-- Running LOC graph -->
<table align=right>
<tr><td>
<a href="running_loc_large.png"
><img border=0 width=480 height=360 src="running_loc.png"></a>
</td></tr>
<tr><td align=center>Lines Of Code over Time<br>
<font size=-1><a href="running_loc_large.png">Click for larger view</a></font>
</td></tr>
</table>
<h2>Nightly Test Overview:</h2>
<ul>
<li>Start: <b>$TestStartTime GMT</b></li>
<li>Finish: <b>$TestFinishTime GMT</b></li>
<li>Platform: <b>$TestPlatform</b></li>
</ul>
<h2>CVS Tree Overview:</h2>
<ul>
<li><a href="$DATE-CVS-Log.txt">CVS Checkout Log</a>
<ul>
<b>$NumDirsInCVS</b> dirs, <b>$NumFilesInCVS</b> files, <b>$LOC</b>
lines of code, checked out in <b>$CVSCheckoutTime</b> seconds<br></ul>
<li><a href="$DATE-Build-Log.txt">Compilation Log</a>
<table>
<tr><td><b>Item</b></td><td><b>CPU Time</b></td><td><b>Wall Clock</b></td></tr>
<tr><td>Configure CVS Tree</td><td>$ConfigTime</td><td>$ConfigWallTime</td></tr>
<tr><td>Build CVS Tree</td><td>$BuildTime</td><td>$BuildWallTime</td></tr>
<tr><td>Run Dejagnu Tests</td><td>$DejagnuTime</td><td>$DejagnuWallTime</td></tr>
</table></li>
<li>Number of object files compiled: <b>$NumObjects</b></li>
<li>Number of libraries linked: <b>$NumLibraries</b></li>
<li>Number of executables linked:<b> $NumExecutables</b></li>
<li>Build Error: $BuildError</li>
</ul>
<h2>Warnings during the build:</h2>
<ul>$WarningsList
</ul><p>
<br><br><center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
<font size=+2 face=Verdana><b><a name="Changes">Changes from Yesterday</font></b>
</td></tr></table></td></tr></table></center><p>
<h2>Changes to CVS:</h2>
<ul>
<li>Users who committed to CVS: <b>$UserCommitList</b>
<li>Users who updated from CVS: <b>$UserUpdateList</b>
<li>Added Files: $AddedFilesList
<li>Modified Files: $ModifiedFilesList
<li>Removed Files: $RemovedFilesList
</ul><p>
<h2>Changes to Warnings:</h2>
<ul>
<li>Warnings Added: $WarningsAdded
<li>Warnings Removed: $WarningsRemoved
</ul><p>
<h2>Changes in the test suite:</h2>
<ul>
<li>New Tests: $TestsAdded
<li>Removed Tests: $TestsRemoved
<li>Newly passing tests: $TestsFixed
<li>Newly failing tests: $TestsBroken
</ul>
</td></tr></tbody></table>
<br/><br/><center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
<font size=+2 face=Verdana><b><a name="Dejagnu">Dejagnu Test Results</font></b>
</td></tr></table></td></tr></table></center>
<br/>
$DejagnuTestResults
<p>A complete log of testing <a href="$DATE-Dejagnu-testrun.log">Feature and Regression</a> is available for further analysis.</p>
<br><br><center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
<font size=+2 face=Verdana><b><a name="Trends">Changes Over Time</font></b>
</td></tr></table></td></tr></table></center><p>
Here are some charts showing how the LLVM optimizer and code generators are
changing over time. For now we use the Olden benchmark suite to measure this,
but eventually we will switch to using SPEC CPU2000. All programs are run with
"LARGE_PROBLEM_SIZE" enabled. Click on any of the charts to get a larger
version.<p>
<h2>Compilation Measurements:</h2>
<table border="0" align=center>
<tr>
<td width=50% align=center>
<a href="running_Olden_bytecode_large.png"><img width=480 height=360 border=0 src="running_Olden_bytecode.png"></a><br>
Size of LLVM bytecode files
</td>
<td width=50% align=center>
<a href="running_Olden_machcode_large.png"><img width=480 height=360 border=0 src="running_Olden_machcode.png"></a><br>
Size of native machine code for each program (generated by the JIT)
</td></tr>
<tr>
<td align=center>
<a href="running_Olden_opt_time_large.png"><img width=480 height=360 border=0 src="running_Olden_opt_time.png"></a><br>
Time to run the LLVM optimizer on each program
</td>
<td></td></tr>
</table>
<h2>Program Execution Measurements:</h2>
<table border="0" align=center>
<tr>
<td width=50% align=center>
<a href="running_Olden_cbe_time_large.png"><img width=480 height=360 border=0 src="running_Olden_cbe_time.png"></a><br>
Execution time for CBE generated executable
</td>
<td width=50% align=center>
<a href="running_Olden_llc_time_large.png"><img width=480 height=360 border=0 src="running_Olden_llc_time.png"></a><br>
Execution time for the LLC generated executable
</td></tr>
<tr>
<td align=center>
<a href="running_Olden_jit_time_large.png"><img width=480 height=360 border=0 src="running_Olden_jit_time.png"></a><br>
Execution time for program in the JIT
</td>
<td></td></tr>
</table>
<br><br><center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
<font size=+2 face=Verdana><b><a name="Programs">Program Tests</font></b>
</td></tr></table></td></tr></table></center><p>
This section tests LLVM on a variety of programs in the test suite. This
includes benchmark suites like the Olden, McCat, Ptrdist, and SPEC benchmarks as
well as a few random programs with test inputs. This section is meant to track
how stable LLVM is as a whole. A failure in the execution of any test is marked
with an asterisk: `*'. The columns of the tables are:<p>
<ol>
<li><a name="Program">Program</a> - The name of the program for that row.</li>
<li><a name="GCCAS">GCCAS</a> - Time to run LLVM optimizers on the program.</li>
<li><a name="Bytecode">Bytecode</a> - The size of the bytecode for the
program</li>
<li><a name="Instrs">Instrs</a> - The number of LLVM instructions in the
compiled bytecode</li>
<li><a name="LLC<br>compile">LLC compile</a> - The time taken compile with
LLC (the static backend)</li>
<li><a name="JIT<br>codegen">JIT codegen</a> - The amount of time spent in the
JIT itself, instead of executing the program.</li>
<li><a name="Machine<br>code">Machine code</a> - The number of bytes of machine
code generated by the JIT.</li>
<li><a name="GCC">GCC</a> - The time taken to execute the program when compiled
with GCC -O2.</li>
<li><a name="CBE">CBE</a> - The time taken to execute the program after
compilation through the C backend, compiled with -O2.</li>
<li><a name="LLC">LLC</a> - How long does the program generated by the static
backend LLC take to execute </li>
<li><a name="JIT">JIT</a> - The amount of time spent running the
program with the JIT; this includes the code generation phase (listed above)
and actually running the program.</li>
<li><a name="GCC/LLC">GCC/LLC</a> - The speed-up of the LLC output vs the native
GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li>
<li><a name="GCC/CBE">GCC/CBE</a> - The speed-up of the CBE output vs the native
GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li>
<li><a name="LLC-BETA">LLC-BETA</a> - How long does the program generated by the static
backend LLC take to execute the program, when compiled with new experimental
features. This is temporary, for tuning.</li>
</ol><p>
A complete log of testing
<a href="$DATE-SingleSource-ProgramTest.txt.gz">SingleSource</a>,
<a href="$DATE-MultiSource-ProgramTest.txt.gz">MultiSource</a>, and
<a href="$DATE-External-ProgramTest.txt.gz">External</a> programs are
available for further analysis.
<h2>Programs/External</h2>
<center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
$ExternalProgramsTable
</td></tr></table></center>
<h2>Programs/MultiSource</h2>
<center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
$MultiSourceProgramsTable
</td></tr></table></center>
<h2>Programs/SingleSource</h2>
<center>
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
$SingleSourceProgramsTable
</td></tr></table></center>
</td></tr></html>
|
