Figures & data
Fig. 1. Stochastic GridWorld.
Fig. 2. The effects of model changes on the action selection strategies for multiple agents learning in parallel: (a) ε-greedy;=0.15, (b) softmax selection, and (c) unbiased sampling.
Fig. 3. A high level comparison detailing the KLD between the distributions (learned and true) and the Q-values for agent learners in parallel: (a) comparison of selection strategies for five agent learners and (b) average Q values for five agent learners with varying selection strategies.
Fig. 4. The effects of model changes on the action selection strategies for multiple agents learning in parallel: (a) ε-greedy selection (ε=0.05), (b) ε-greedy selection (ε=0.15), (c) ε-greedy selection (ε=0.25), (d) unbiased sampling, and (e) comparison of the performance of the selection strategies for 10 agents in parallel.
Fig. 5. Data traces provided by Cedexis measuring application response time, total requests per country and total requests per region all over a single day: (a) number of requests per country, (b) number of requests satisfied per region, and (c) application response time performance histogram by region (Amazon EC2).
Fig. 6. Performance of the parallel learning architecture, demonstrating improvements as additional learners are added: (a) Virtual resource allocation, unbiased sampling, multiple agent learners, learning in parallel. (b) A single run of unbiased sampling for 300 trials.
