.   +---------------------+      
+---|---------------------|-------.    .---_||||||_-----------------.   
|:--|-----\\\\\//////-----|--. o   ----                          (\)| 
|#|||------         ------|--.||      [  Peter A. H. Peterson       | 
|-::|-----|.-,   ,-.|-----|===========[  Assistant Professor of CS  |  
|o+o|----(| @ |   @ |)----|:::::::::::[  329 Heller Hall            | 
||o||-----#  '-'    #-----|:::::::::::[  1114 Kirby Drive           | 
|||||------\"==="  /------|===========[  Duluth, MN 55812           | 
|||||-------`-----'|------| | o       [  pahp@d.umn.edu             | 
||/ |.----/'\'-----'/\---.|-+-'       [  pedro@tastytronic.net_  (/)|   
||^_|     '''|     |''    |-----'''----||||-||||||||||-||||||||||.  |  
||| +---------------------+-----------'                          | :| 
|:|                                                              |:.|
| '------------.  [back to research]                             |.:|
| home         |                                                 |:.| 
'------------. |  Representative Synthetic Benchmarking          |.:|
.------------' |                                                 |:.|      
| cv           |  One can hardly work in the area of operating   |.:|      
| .------------'  systems, security, or power measurement,       |:.|      
| '------------.  without becoming deeply interested in, and     |.:|      
| research     |  perhaps dismayed by, the state of              |:.|      
'------------. |  benchmarking and system testing. One would     |.:|      
.------------' |  like to test systems meant for the real world  |:.|      
| interests    |  with representative workloads. But what        |.:|      
| .------------'  benchmark workloads are representative of      |:.|      
| '------------.  real users?                                    |.:|      
| blog         |                                                 |:.|      
'------------. |  Because of the difficulty of these and other   |.:|      
.------------' |  questions, researchers often fall back on      |:.|      
| reading      |  microbenchmarks, standardized benchmarks       |.:|      
| .------------'  (such as SPEC), and old standbys such as the   |:.|      
| '------------.  Modified Andrew Benchmark. While each of       |.:|      
| quotes       |  these benchmarks have their place, the         |:.|      
'------------. |  conventional wisdom is that these workloads    |.:|      
.------------' |  are not representative of how real users use   |:.|      
| contact      |  their computers. Instead, benchmarks like      |.:|      
|              |  this were designed primarily to quantify raw   |:.|      
|''''''''''''''|  computational power.                           |.:|      
| 001010101101 |                                                 |:.|      
| 011011011001 |  Today's laptop is faster than yesterday's      |.:|      
| 010011011011 |  supercomputer, and another bit of              |:.|      
| 110011010111 |  conventional wisdom is that most personal      |.:|      
| 100100001100 |  computers are underutilized. If that is the    |:.|      
| 101001100111 |  case, then users can benefit -- with little    |.:|      
| 001101100101 |  to no visible  performance degradation --      |:.|      
| 001111000000 |  when this surplus  computational power is      |.:|      
| 001010101101 |  used to perform other tasks on behalf of the   |:.|      
| 011011011001 |  user. Software-based full disk encryption      |.:|      
| 010011011011 |  (FDE) is one such example; while it once       |:.|      
| 110011010111 |  seemed completely impractical, it is quite     |.:|      
| 100100001100 |  common and essentially invisible today. Other  |:.|      
| 101001100111 |  potential uses for unused cycles include       |.:|      
| 001101100101 |  speculation, machine learning, compression,    |:.|      
| 001111000000 |  other cryptographic uses, and more.            |.:|      
| 001010101101 |                                                 |:.|      
| 011011011001 |  These are potentially exciting areas for       |.:|      
| 010011011011 |  research, but without defensible research      |:.|      
| 110011010111 |  showing acceptable performance, it can be      |.:|      
| 100100001100 |  difficult to convince skeptics that the        |:.|      
| 101001100111 |  technique can be practical, even if they       |.:|      
| 001101100101 |  accept its functional value. And without       |:.|      
| 001111000000 |  well-founded "real world workloads," it is     |.:|      
| 001010101101 |  difficult to perform these experiments. I'm    |:.|      
| 011011011001 |  interested in quantifying the "typical user    |.:|      
| 010011011011 |  workload," so that we can use these workloads  |:.|      
| 110011010111 |  (in conjunction with other techniques) to      |.:|      
| 100100001100 |  test experimental systems.                     |:.|      
| 101001100111 |                                                 |.:|      
| 001101100101 |  Additionally, a strong notion of typical user  |:.|      
| 001111000000 |  workloads would also be valuable for power     |.:|      
| 001010101101 |  research, because they could be used to drive  |:.|      
| 011011011001 |  systems for power measurement. Research into   |.:|      
| 010011011011 |  capturing, generating, and using these kinds   |:.|      
| 110011010111 |  of workloads is ongoing.                       |.:|      
| 100100001100 |                                                 |:.|      
| 101001100111 |  [back to research]                             |.:|      
| 001101100101 |                                                 |:.|      
|'''''''''''''''-------------------------------------------------'::'. 
|::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::| 
`=======---------.______.-------:::::::-:------------------------. ? | 
                                                                  ---'