LinUCBHybridPolicy: Policy: LinUCB with hybrid linear models
In contextual: Simulation and Analysis of Contextual Multi-Armed Bandit Policies

Description Details Usage Arguments Parameters Methods References See Also

LinUCBHybridPolicy is an R implementation of "Algorithm 2 LinUCB" from Li (2010) "A contextual-bandit approach to personalized news article recommendation.".

Each time step t, LinUCBHybridOptimizedPolicy runs a linear regression per arm that produces coefficients for each context feature d. Next, it observes the new context, and generates a predicted payoff or reward together with a confidence interval for each available arm. It then proceeds to choose the arm with the highest upper confidence bound.

1	policy <- LinUCBHybridPolicy(alpha = 1.0)

alpha: double, a positive real value R+; Hyper-parameter adjusting the balance between exploration and exploitation.
name: character string specifying this policy. name is, among others, saved to the History log and displayed in summaries and plots.

A: d*d identity matrix
b: a zero vector of length d

new(alpha = 1): Generates a new LinUCBHybridPolicy object. Arguments are defined in the Argument section above.

set_parameters(): each policy needs to assign the parameters it wants to keep track of to list self$theta_to_arms that has to be defined in set_parameters()'s body. The parameters defined here can later be accessed by arm index in the following way: theta[[index_of_arm]]$parameter_name

get_action(context): here, a policy decides which arm to choose, based on the current values of its parameters and, potentially, the current context.

set_reward(reward, context): in set_reward(reward, context), a policy updates its parameter values based on the reward received, and, potentially, the current context.

Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010, April). A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web (pp. 661-670). ACM.

Core contextual classes: Bandit, Policy, Simulator, Agent, History, Plot

Bandit subclass examples: BasicBernoulliBandit, ContextualLogitBandit, OfflineReplayEvaluatorBandit

Policy subclass examples: EpsilonGreedyPolicy, ContextualLinTSPolicy

contextual documentation built on July 26, 2020, 1:06 a.m.