########################################################################## ### 17 Nov 98: I have not forgotten this site! I do have some new results to post (not that many actually). I plan to update the tables before the ned of the month ... Al Aburto (aburto@spawar.navy.mil) ### 20 Nov 97: Updated whetnew.tbl. New Whetstone results from Roy Longbottom. Unshar'd shuffle.shar (sorry forgot that one previously). I'll try to get Pentium II 266 MHz and 300 MHz results in as soon as possible. aburto@spawar.navy.mil ### 30 Oct 97: Put each benchmark (test program) into its own directory. Removed all the ".shar" files. Remade bench.tar.gz. I'll add (soon) bench.zip for PC users ... Al Aburto ### 17 Oct 97: Updated all the files finally! Need Pentium II results. I thank Paul Caskey for offering to help me find better ways to run and tabulate the results. We're working to make the entire process more efficient... Al Aburto, aburto@spawar.navy.mil NOTE: ftp.nosc.mil has new IP address: 128.49.16.7 ### 19 May 97: Added whetnew.tbl --- almost forgot it! Working on next update to all the other tables ... ### 18 Mar 97: Revised version of whetstone (whets.c) provided. Changed whets.tbl to whetold.tbl Another C Linpack program added (linpackc.shar), from Roy Longbottom (101323.2241@compuserve.com). ### 10 Mar 97: Added blackjack game test program (bj.shar) and results table (bj.tbl). ### 03 Mar 97: Updated all tables ... Added POSIX.1 timer from: John Pelan, johnp@am.qub.ac.uk ### 10 Jan 97: Fixed typos in shuffle.c and in some of the tables as recommended by Roy Longbottom (clinpack.dpr, flops, dhry). Will provide new update to all the tables "soon". Finished the BlackJack analysis program. Will post that program (bj.c) soon too. ### 20 Nov 96: Updated whets.tbl ### 12 Nov 96: Updated tables with some new results for the Pentium Pro, Pentium P5-100, 486DX2, 386DX, and Sun's Ultra 4000. Added 2 new simple test programs: (1) shuffle.c Uses a card shuffling procedure to test a Random Number Generators (RNG) performance. The RNG tested is Mother() which has an extremely long cycle length. (2) fibonacci.c Another one of those classic test programs ... ### 07 Nov 96: Added the Whetstone C benchmark (whets.c). It is provided by Roy Longbottom, 101323.2241@compuserve.com. Please send results (whets.res) and comments to Roy so he can update his spreadsheet and table of results (whets.tbl). This version of whets.c is approved by Harold Curnow who is the orginal author of whetstone. The first version of the Whetstone benchmark is dated 21 Nov 1972. Please read whets.c for more information and instructions on compiling and running. ### 30 Oct 96: Latest set of results are provided. --- I'm working on a Blackjack game simulation and and analysis program. The program will play Blackjack assuming a 'basic playing strategy' and a simple betting strategy and accumulate win/loss/push and other statistics over at least 100,000 hands played. I hope it will make a good benchmark too... --- The Whetstone (whets.c) benchmark will be added soon. ### 10 Oct 96: I have received alot of data in September. Tables are current to: 07 Sep 96 only!! Added WINDOWS NT Timer (WIN32). Added a PC unshar.zip file so PC users can undo the '.shar' files... ### 27 Aug 96: Added a very comprehensive table of Whetstone benchmark results (whets.tbl) provided by Roy Longbottom, UK, 101323.2241@compuserve.com ### 05 Aug 96: Another mirror site of the programs and tables: ftp://ftp.mpi-sb.mpg.de/pub/benchmarks/mirror/ftp.nosc.mil ### 18 Jul 96: Updated tables with some new results for some of the tests: Pentium 200 MHz, 166 MHz, 133 MHz, 100 MHz ### 13 Apr 96: Added Pentium, Cyrix 5x86, UltraSPARC 167MHz, and PowerPC 604 results. The UltraSPARC 167MHz flops results using Sun's 'acc' compiler are vastly improved compared to the 'gcc' results. An UltraSPARC flops executable using Sun's acc compiler is available in the uuencoded file "US_1.txt". ### 30 Jan 96: I have removed the SPEC89/92 tables as they were out of date and incomplete. The entire set of SPEC92 results can be obtained from the netlib Performance Database Server (PDS): http://performance.netlib.org/performance/html/PDSreports.html Results from ftp.nosc.mil (pub/aburto) are also mirror'd on the netlib PDS. ### 18 Dec 95: Updated timers on all programs. Made 'shar' files for each program. ### 14 Dec 95: DEC Alpine corrected to Aspen Alpine. ### 07 Dec 95: Added new results for: 80486DX4, 100 MHz, Sun HyperSPARC, 125 MHz, DEC Alpine 275 XS. ########################################################################## This directory contains a collection of the source code and the results of a number of public benchmarks (in 'c'). It is maintained by Al Aburto of the Naval Command, Control and Ocean Surveillance Center (NCCOSC) RDT&E Division (NRaD) in San Diego, CA. If you have contributions (program results or programs) or any comments please send them to aburto@nosc.mil. All 'c' programs and tables of results ('.tbl') are available via anonymous ftp to 'ftp.nosc.mil' in directory 'pub/aburto'. The IP address for ftp.nosc.mil is: 128.49.192.51 PLEASE NOTE: The file 'bench.tar.gz' is a gzip compressed tar file of all programs and results. This is the only BINARY file. All other files are ASCII. To get bench.tar.gz be sure to set ftp to BINARY mode. A brief description of the test programs available follows: (1) sim An integer program that compares DNA segments for similarity. Source: sim.shar Result: sim.tbl (2) fhourstones An integer program to find solutions to the 'connect-4' game. Source: c4.shar Result: c4.tbl (3) dhrystone A synthetic integer program which provides a MIPS rating based upon a 'typical' instruction mix. Very sensitive to optimization and results will appear erratic unless degree of optimization is carefully tracked. Source: dhry.shar Result: dhry.tbl (4) nsieve An integer program which generates prime numbers based on the Sieve of Eratosthenes using array sizes from 8 KBytes to 2 MBytes. A nsieve MIPS rating is provided but the nsieve instruction mix is NOT 'typical'. Nsieve can reveal the 'knees' in performance relative to memory and cache size as illustrated below (item 16). Source: nsieve.c Result: nsieve.tbl (5) heapsort An integer program which uses the 'heap sort' method of sorting a random array of long integers up to 2 MBytes in size. A heapsort MIPS rating is provided but it is NOT representative of a 'typical' instruction mix. Source: heapsort.c Result: heapsort.tbl (6) hanoi An integer program which solves the Towers of Hanoi puzzle using recursive function calls. Source: hanoi.c Result: hanoi.tbl (7) queens An integer program that solves the "14 Queens Problem". That is, the program finds all ways to place 14 queens on a 14 X 14 square "chess" board such that no queen can capture another. Source: queens.c Result: queens.tbl (8) flops Estimates MFLOPS rating for specific FADD, FSUB, FMUL, and FDIV instruction mixes. Four distinct MFLOPS ratings are provided based on the FDIV weightings from 25% to 0% and using register-register operations. Works with both scalar and vector machines. Since the program trys to maximize register usage the results are NOT sensitive to main memory speed. In this sense flops yields a peak rating. Source: flops.c Result: flops_1.tbl, flops_2.tbl, flops_3.tbl, flops_4.tbl (9) clinpack This is the Linpack program (floating-point) converted to c. Results here are sensitive to cache effects and memory speed. Source: clinpack.c Result: clinpack.dpr, clinpack.dpu, clinpack.spr, clinpack.spu (10) fft An FFT test program from Ron Mayer (mayer@acuson.com). Program times a 131072 point FFT. Source: fft.shar Result: fft.tbl (11) tfftdp This program performs FFT's using the Duhamel-Hollman method for FFT's from 32 to 262,144 points in size. Source: tfftdp.c Result: tfftdp.tbl (12) matrix multiply (mm) This program (mm.c) contains 9 different algorithms for doing matrix multiplication (500 X 500 standard size). Results illustrate the enormous effects of cache thrashing versus algorithm, machine, compiler, and compiler options. Source: mm.c Result: mm_1.tbl, mm_2.tbl (13) SPEC results I've stopped collecting SPEC results. The SPEC92 results are however provided in the Performance Database Server (PDS): http://performance.netlib.org/performance/html/PDSreports.html (14) Timer programs Timer routines in 'c' are also available. Source: timers_a.c, timers_b.c (15) FAQ's Some helpful FAQ files are also available: FAQ: spec.faq, nistlib.faq, comp.benchmarks.faq (16) Nsieve example results: ----------------------------------------------------------------------- Sieve of Eratosthenes (Scaled to 10 Iterations) Version 1.2b, 26 Sep 1992 R4400 100 MHz, 16 KByte I/D internal caches, 1 MByte external cache cc -DUNIX -mips2 -sopt-so=4 -O3 -non_shared nsieve.c -o nsieve Array Size Number Last Prime Linear RunTime MIPS caches (Bytes) of Primes Time(sec) (Sec) 8191 1899 16381 0.022 0.022 76.2 10000 2261 19997 0.027 0.026 76.8___ 16KB 20000 4202 39989 0.053 0.060 69.0 40000 7836 79999 0.106 0.152 55.0 80000 14683 160001 0.213 0.326 51.7 160000 27607 319993 0.425 0.680 50.2 320000 52073 639997 0.850 1.432 48.1 640000 98609 1279997 1.700 3.451 40.3___ 1 MB 1280000 187133 2559989 3.401 11.772 23.9 2560000 356243 5119997 6.801 44.810 12.6 ----------------------------------------------------------------------- R4000PC 100 MHz CPU, 8 KByte I/D internal caches, NO external cache cc -DUNIX -mips2 -sopt-so=4 -O3 -non_shared nsieve.c -o nsieve Array Size Number Last Prime Linear RunTime MIPS (Bytes) of Primes Time(sec) (Sec) 8191 1899 16381 0.023 0.023 72.1___ 8 KB 10000 2261 19997 0.028 0.047 43.6 20000 4202 39989 0.056 0.184 22.4 40000 7836 79999 0.112 0.435 19.1 80000 14683 160001 0.225 0.944 17.9 160000 27607 319993 0.449 1.984 17.2 320000 52073 639997 0.899 4.133 16.7 640000 98609 1279997 1.797 8.733 15.9 1280000 187133 2559989 3.594 18.228 15.4 2560000 356243 5119997 7.188 38.085 14.9 Al Aburto aburto@nosc.mil