Here are few Performance and Scaling tools
https://opensource.com/article/19/6/performance-scaling-tools
Category Archives: performance & Scale
SPEC Cloud™ IaaS 2016
The SPEC Cloud™ IaaS 2016 benchmark is SPEC’s first benchmark suite to measure cloud performance. The benchmark suite’s use is targeted at cloud providers, cloud consumers, hardware vendors, virtualization software vendors, application software vendors, and academic researchers.
The SPEC Cloud™ IaaS 2016 Benchmark addresses the performance of infrastructure-as-a-service (IaaS) cloud platforms. IaaS cloud platforms can either be public or private.
The SPEC Cloud™ IaaS 2016 benchmark is available for purchase via the SPEC order form.
Austin Openstack summit talk
https://www.openstack.org/videos/video/dell-performance-measuring-with-spec-cloud
Other links
http://www.reuters.com/article/idUSnMKWyNDfRa+1da+MKW20160503
pbench: A Benchmarking and Performance Analysis Framework
- What is pbench?
Pbench is a harness that allows data collection from a variety of tools while running a benchmark.
Pbench has some built-in script that run some common benchmarks, but the data collection can be run separately as well
with a benchmark that is not built-in to pbench, or a pbench script can be written for the benchmark. Such contributions are more than welcome!
2. Available tools:
The configured default set of tools (what you would get by running pbench-register-tool-set) is:
- sar, iostat, mpstat, pidstat, proc-vmstat, proc-interrupts, perf
In addition, there are tools that can be added to the default set with pbench-register-tool:
- blktrace, cpuacct, dm-cache, docker, kvmstat, kvmtrace, lockstat, numastat, perf, porc-sched_debug, proc-vmstat, qemu-migrate, rabbit, strace, sysfs, systemtap, tcpdump, turbostat, virsh-migrate, vmstat
3. Available benchmark scripts
pbench_dbenchpbench_fiopbench_linpackpbench_migratepbench_tpccpbench_uperfuser-benchmark( to run any job)
5. How to install?
http://distributed-system-analysis.github.io/pbench/doc/agent/installation.html
6. Repos:
https://copr.fedorainfracloud.org/coprs/ndokos/pbench/
7. User Guide
http://distributed-system-analysis.github.io/pbench/doc/agent/user-guide.html
8. source
https://github.com/distributed-system-analysis/pbench
9. Mailing list
https://groups.google.com/forum/#!forum/pbench
10) Example pbench_fio run from host on clients ( Could be remote machines or guests)
pbench config settings:
/opt/pbench-agent/bench-scripts/pbench_fio ( can customize ioengine, ..etc)
Ex:
block_sizes=”4,64,1024″
targets=”/tmp/fio”
runtime=30
ramptime=5
#iodepth=32
iodepth=1
#ioengine=”libaio”
ioengine=”sync”
Clear previously registered tools (if any)
source /opt/pbench-agent/profile
clear-results
kill-tools
clear-tools
#register tool set
register-tool-set ( which covers all default tools)
#If any other tools needed
register-tool –name=perf — –record-opts=”record -g –pid=14355 -e kvm:kvm_inj_virq -e kvm:kvm_entry -e kvm:kvm_exit”
register-tool –name kvmstat
register-tool –name tcpdump
register-tool –name blktrace — –devices=/dev/nvme1n1p1
#run benchmark
pbench_fio –clients=<clients on which tests need to be done. For ex: VM’s> –test-types=<test types> –block-sizes=<different sizes with a comma> –targets=<disks> –samples=<num of samples> –config=<config name>
pbench_fio –clients=$clients –test-types=write,read,randread,randwrite,randrw –block-sizes=4,16,64,256,1024,4096,16384 –targets=/dev/vdc –samples=5 –config=fio-config
Can add more disks if more jobs need to be run.
samples=n: Run 5 samples and pbench calclulate std deviation and gives you closer match.
#kill tools and move results
move-results
kill-tools
clear-tools
Results:
iteration summary tools readwrite_IOPS rw_kB_sec read_IOPS read_kB_sec read_95th_lat_us write_IOPS write_kB_sec write_95th_lat_us
1-write-4KiB 201319.13[+/-2.3%] 805276.42[+/-2.3%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 201319.13[+/-2.3%] 805276.42[+/-2.3%] 2486540.80[+/-2.5%]
2-write-16KiB 59416.95[+/-3.8%] 950669.97[+/-3.8%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 59416.95[+/-3.8%] 950669.97[+/-3.8%] 6065548.80[+/-2.7%]
3-write-64KiB 13235.63[+/-13.2%] 847083.63[+/-13.2%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 13235.63[+/-13.2%] 847083.63[+/-13.2%]21400166.40[+/-20.8%]
4-write-256KiB 3755.32[+/-3.8%] 961464.90[+/-3.8%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 3755.32[+/-3.8%] 961464.90[+/-3.8%] 79495168.00[+/-5.2%]
5-write-1024KiB 988.33[+/-2.1%] 1012008.91[+/-2.1%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 988.33[+/-2.1%] 1012008.91[+/-2.1%]314354892.80[+/-5.4%]
6-write-4096KiB 272.84[+/-1.7%] 1117408.43[+/-1.6%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 272.84[+/-1.7%] 1117408.43[+/-1.6%]1635133030.40[+/-2.7%]
iteration readwrite_IOPS rw_kB_sec read_IOPS read_kB_sec read_95th_lat_us write_IOPS write_kB_sec write_95th_lat_us
7-read-4KiB 443081.35[+/-1.5%] 1772325.42[+/-1.5%] 443081.35[+/-1.5%] 1772325.42[+/-1.5%] 1569043.20[+/-2.5%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
8-read-16KiB 145205.19[+/-0.4%] 2323284.26[+/-0.4%] 145205.19[+/-0.4%] 2323284.26[+/-0.4%] 5470771.20[+/-12.3%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
9-read-64KiB 33796.18[+/-5.2%] 2162961.77[+/-5.2%] 33796.18[+/-5.2%] 2162961.77[+/-5.2%]21228390.40[+/-11.2%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
10-read-256KiB 9030.00[+/-1.4%] 2311667.51[+/-1.4%] 9030.00[+/-1.4%] 2311667.51[+/-1.4%]103413760.00[+/-4.4%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
11-read-1024KiB 2358.56[+/-1.3%] 2415179.90[+/-1.3%] 2358.56[+/-1.3%] 2415179.90[+/-1.3%]185453772.80[+/-3.8%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
12-read-4096KiB 597.96[+/-2.0%] 2449138.14[+/-2.0%] 597.96[+/-2.0%] 2449138.14[+/-2.0%]746619699.20[+/-1.5%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
iteration readwrite_IOPS rw_kB_sec read_IOPS read_kB_sec read_95th_lat_us write_IOPS write_kB_sec write_95th_lat_us
13-randread-4KiB 461194.31[+/-0.9%] 1844777.40[+/-0.9%] 461194.31[+/-0.9%] 1844777.40[+/-0.9%] 1369676.80[+/-2.9%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
14-randread-16KiB 143443.52[+/-0.1%] 2295097.46[+/-0.1%] 143443.52[+/-0.1%] 2295097.46[+/-0.1%] 5756672.00[+/-2.8%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
15-randread-64KiB 30334.00[+/-11.2%] 1941378.17[+/-11.2%] 30334.00[+/-11.2%] 1941378.17[+/-11.2%]18773094.40[+/-14.6%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
16-randread-256KiB 8990.84[+/-0.9%] 2301671.35[+/-0.9%] 8990.84[+/-0.9%] 2301671.35[+/-0.9%]100553113.60[+/-3.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
17-randread-1024KiB 2301.50[+/-1.7%] 2356708.83[+/-1.7%] 2301.50[+/-1.7%] 2356708.83[+/-1.7%]184976998.40[+/-3.2%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
18-randread-4096KiB 587.94[+/-1.3%] 2408056.26[+/-1.3%] 587.94[+/-1.3%] 2408056.26[+/-1.3%]757801779.20[+/-1.3%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%]
iteration readwrite_IOPS rw_kB_sec read_IOPS read_kB_sec read_95th_lat_us write_IOPS write_kB_sec write_95th_lat_us
19-randwrite-4KiB 227753.59[+/-4.2%] 911014.29[+/-4.2%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 227753.59[+/-4.2%] 911014.29[+/-4.2%] 2133932.80[+/-5.6%]
20-randwrite-16KiB 59156.56[+/-7.7%] 946503.20[+/-7.7%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 59156.56[+/-7.7%] 946503.20[+/-7.7%] 6273779.20[+/-13.1%]
21-randwrite-64KiB 13194.78[+/-11.4%] 844464.84[+/-11.4%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 13194.78[+/-11.4%] 844464.84[+/-11.4%]22670233.60[+/-22.6%]
22-randwrite-256KiB 3626.32[+/-9.8%] 928403.87[+/-9.8%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 3626.32[+/-9.8%] 928403.87[+/-9.8%]89994444.80[+/-15.1%]
23-randwrite-1024KiB 961.35[+/-9.6%] 984385.43[+/-9.6%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 961.35[+/-9.6%] 984385.43[+/-9.6%]315842150.40[+/-21.3%]
24-randwrite-4096KiB 198.75[+/-8.9%] 814045.25[+/-8.9%] 0.00[+/-0.0%] 0.00[+/-0.0%] 0.00[+/-0.0%] 198.75[+/-8.9%] 814045.25[+/-8.9%]2429529497.60[+/-13.7%]
iteration readwrite_IOPS rw_kB_sec read_IOPS read_kB_sec read_95th_lat_us write_IOPS write_kB_sec write_95th_lat_us
25-randrw-4KiB 209010.84[+/-3.6%] 836043.29[+/-3.6%] 103595.54[+/-3.5%] 414382.20[+/-3.5%] 2793664.00[+/-3.0%] 105415.30[+/-3.7%] 421661.09[+/-3.7%] 2113689.60[+/-4.4%]
26-randrw-16KiB 66672.72[+/-6.8%] 1066765.22[+/-6.8%] 33757.08[+/-6.6%] 540113.54[+/-6.6%] 6673433.60[+/-8.8%] 32915.65[+/-6.9%] 526651.68[+/-6.9%] 6249600.00[+/-10.3%]
27-randrw-64KiB 13629.35[+/-14.3%] 872278.02[+/-14.3%] 7258.89[+/-14.4%] 464570.25[+/-14.4%] 20632985.60[+/-9.7%] 6370.46[+/-14.1%] 407707.76[+/-14.1%] 21101363.20[+/-9.7%]
28-randrw-256KiB 4991.29[+/-11.3%] 1277774.89[+/-11.3%] 2732.44[+/-10.0%] 699505.92[+/-10.0%]84772556.80[+/-19.5%] 2258.84[+/-12.7%] 578268.96[+/-12.7%]89707827.20[+/-23.2%]
29-randrw-1024KiB 1215.91[+/-6.0%] 1245092.34[+/-6.0%] 606.65[+/-5.9%] 621245.43[+/-5.9%]326084608.00[+/-9.1%] 609.25[+/-6.1%] 623846.91[+/-6.1%]337001267.20[+/-10.2%]
30-randrw-4096KiB 328.24[+/-7.1%] 1344195.53[+/-7.2%] 148.70[+/-14.2%] 608808.18[+/-14.2%]1185399603.20[+/-1.1%] 179.54[+/-1.4%] 735387.35[+/-1.4%]1534726963.20[+/-4.4%]
Summary, tools of each run provide respective debug info details.
For ex:
1-write-4KiB clients details readwrite-IOPS readwrite-kB/sec read-IOPS read-kB/sec read-95th-lat-ms write-IOPS write-kB/sec write-95th-lat-ms
all_clients 202003.10 808011.98 0.00 0.00 0.00 202003.10 808011.98 4.88
virbr0-122-233 12391.71 49566.75 0.00 0.00 0.00 12391.71 49566.75 4.93
virbr0-122-26 12218.42 48873.65 0.00 0.00 0.00 12218.42 48873.65 4.96
virbr0-122-123 12835.02 51339.93 0.00 0.00 0.00 12835.02 51339.93 4.77
virbr0-122-84 12515.69 50062.74 0.00 0.00 0.00 12515.69 50062.74 4.90
virbr0-122-242 12689.89 50759.62 0.00 0.00 0.00 12689.89 50759.62 4.89
virbr0-122-184 12473.84 49895.40 0.00 0.00 0.00 12473.84 49895.40 4.93
virbr0-122-34 12413.58 49654.27 0.00 0.00 0.00 12413.58 49654.27 4.93
virbr0-122-170 12806.87 51227.44 0.00 0.00 0.00 12806.87 51227.44 4.82
virbr0-122-152 12745.22 50980.79 0.00 0.00 0.00 12745.22 50980.79 4.85
virbr0-122-236 12553.71 50214.89 0.00 0.00 0.00 12553.71 50214.89 4.95
virbr0-122-237 12840.07 51360.35 0.00 0.00 0.00 12840.07 51360.35 4.83
virbr0-122-186 12629.69 50518.86 0.00 0.00 0.00 12629.69 50518.86 4.91
virbr0-122-40 13056.13 52224.43 0.00 0.00 0.00 13056.13 52224.43 4.71
virbr0-122-223 12946.96 51787.85 0.00 0.00 0.00 12946.96 51787.85 4.75
virbr0-122-108 12421.49 49685.84 0.00 0.00 0.00 12421.49 49685.84 4.94
virbr0-122-90 12464.81 49859.18 0.00 0.00 0.00 12464.81 49859.18 4.94
Tools provide all graphical representation of every debug tool registered.
Thanks to Andrew, Peter, ndk, archit and other perf folks in Redhat who developed tool
Pradeep K Surisetty 