.. _before_benchmark_setup:

*******************************
Before Setting up the Benchmark
*******************************

The tester must consider the following before running the benchmark.

==========================
Where does the CBTOOL Run?
==========================

CBTOOL and benchmark drivers must be set up together on the same machine. 
This machine is hereafter referred to as the benchmark harness machine.

If the cloud under test is whitebox, the benchmark harness machine must not
be part of the cloud.

If the cloud under test is blackbox, the benchmark harness machine should not
share a physical machine with the instances to the extent possible.

A diagram depicting the harness and instances is shown below.

.. image:: images/harness_cloud.png
    :width: 500pt

===================================================================
Timezone and NTP Server For Instances and Benchmark Harness Machine
===================================================================

It is important that the benchmark harness machine and the benchmark 
test instances have a very similar idea of "time."  By default, 
benchmark instances will be configured to get time via NTP (Network 
Time Protocol) from the benchmark harness machine.  The benchmark 
harness machine must therefore be able to serve time via NTP, which 
means it too must be time synchronized - for example by being 
configured to obtain time from some number of NTP servers.

If you wish to have benchmark test instances obtain time from NTP 
server(s) other than the benchmark harness machine, that is accomplished 
by editing the osgcloud_rules.yaml configuration file present in
~/osgcloud/driver/ directory. A comma separated list of NTP servers can
be specified in the following parameter::

	timeserver: NTP_Server1_IP_Address_Or_Hostname,NTP_Server2_IP_Address_Or_Hostname

It is highly recommended that the timezone of the benchmark harness machine 
and instances be UTC. Please note that to setup timezones in instance images,
a separate configuration is required that may vary across operating system
distributions.


Caution about NTP servers
-------------------------
The benchmark test instances must be able to synchronize time with one 
or more NTP servers. If the instances are unable to reach the NTP server(s),
or synchronize time with them, then "baseline" and "elasticity" phases of 
the benchmark may hang.


========================================
A Typical Flow for Running the Benchmark
========================================

A tester may use the following flow as a reference for the overall testing 
process.

.. image:: images/test_flow1.png
    :width: 550pt


.. image:: images/test_flow2.png
    :width: 550pt


.. image:: images/test_flow3.png
    :width: 550pt