statistician module

class AlertParam(short_median=12, long_median=120, threshold=1.5, time_resolution=10)

Stores the parameters used in the alert detection process.

short_median and long_median are the size of the windows for the moving averages.

An alert will be raised if short_outflow_average > long_outflow_average * threshold.

Warning

The unit base of short_median and long_median is time_resolution, that means that long_median=2 with time_resolution=10 will calculate an outflow average on the last 2 * 10 = 20 seconds.

class QueueWriter(output_queue, parse=True, pace10=1, factor=2)

Bases: threading.Thread

Used to fill the Statistician queue, to simulate a fast reading and compare the reading speed with or without parsing.

Note

pace10 is the pace for 100ms, ie 10*pace10 entries are put in the queue every second.

run()

Puts n=pace10 lines every 10th of a second in the output_queue

class Statistician(input_queue, sleeping_time=0.1, parse=False, alert_param=<statistician.AlertParam instance>)

Bases: threading.Thread

This thread object is responsible for the statistics maintenance, it has a Statistics object to store them and raise the alerts.

It possesses the alert parameters.

Read lines are ‘thread-safely’ received thanks to an input_queue. They should be parsed by default, but this can be changed with the parse parameter.

Variables:should_run (bool) – If False, the thread will shortly end stop its operation. Used to cleanly end the program.

Note

Alerts are checked every AlertParam.time_resolution.

run()

Checks if an alert should be raised, checks the input queue, update the stats if necessary and starts again.

state()

Returns:Describes the present thread state Return type:string

class Statistics

Object used by the statistician as its “notebook”. This is were the stats are saved.

It calls the Displayer if an alert should be raised or shut down. It is called when the stats should be printed.

Variables:

• section (dictionary) – The keys are the hit sections, values are the number of hits for each section.
• total_bytes (int) – Sum of the bytes sent.
• total_hits (int) – Total number of hits.
• should_run (bool) – If False, the thread will shortly end stop its operation. Used to cleanly end the program.
• long_term_bytes_buffer (int) – Stores the sum of bytes sent during a certain time, then is appended to the long_term_bytes list and reseted.
• long_term_bytes (list of int) – List that stores the evolution of the number of sent bytes, used to compute moving average.
• alert_raised (bool) – True if an alert has been raised and not shut down.

Note

The use of statistics.lock makes this object thread-safe.

Warning

number_of_hits, total_bytes and total_hits are used for the printed stats, ‘total’ is in fact ‘total since the last display’.

emergency(alert_param)

Returns a dictionary with the alert parameters if there is one. Called by update_long_term

get_last_stats()

Returns a stats dict, used for the regular stats printing

reset_short_stat()

Called by the displayer after get_last_stats to reset the ‘printing stats’

upadate_stat(HTTP_dict)

Update the stats with the given parse line (ie the HTTP_dict)

update_long_term(alert_param)

Update the long_term_bytes list. Checks if an alert should be raised (or shut down), and raises it if necessary.

Called by the Statistician every AlertParam.time_resolution.