POMDPModel

Creates a POMDP style factor graph model. This class is really just a wrapper around the GeniusModel class with constrained functionality to enable the user to create a POMDP model. Strictly speaking, one can create a POMDP with the GeniusModel class but the convenience functions in this class make the process easier and include checks to make sure all the necessary model components are present.

To import the POMDPModel class use: `from genius_client_sdk.pomdp import POMDPModel

Usage pattern

POMDPModel()

Attributes

Variable	Type	Description
agent_url	str	The URL of the agent associated with the model.
flags	dict	A dictionary storing which required POMDP components have been added to the model.
vfg	dict	The dictionary representation of the JSON model.
version	str	The Genius model version.

Methods

add_action_variable

Adds an action variable to the POMDP model. If there are multiple state factors then there should be an action variable that can control the transition of each of these state factors.

Parameter	Type	Default	Description
name	str	Required	The name of the action variable.
values	list[str]	Required	The categories of the action variable.

Example:

model = POMDPModel()
actions = ["UP", "RIGHT", "DOWN", "LEFT", "STAY"]
model.add_action_variable(name="movement", values=actions)

add_likelihood_factor

Adds a likelihood factor to the POMDP model. If there are multiple observation modalities to be modeled then a separate likelihood factor must be added for each one.

Parameter	Type	Default	Description
values	np.ndarray	Required	A likelihood probability matrix.
target	str	Required	An observation or observation modality variable.
parents	list[str]	Required	The states or state factors the observation depends on.
counts	np.ndarray	Optional	The counts for the factor.

Example:

model = POMDPModel()

likelihood_vars = ["observation", "position"]
likelihood_probs = np.array(
    [
        [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0],
    ]
)

model.add_likelihood_factor(
    values=likelihood_probs, 
    target=likelihood_vars[0], 
    parents=likelihood_vars[1])

add_observation_variable

Adds an observation variable to the model. If there are multiple observation modalities then this method can be called repeatedly to add additional modalities.

Parameter	Type	Default	Description
name	str	Required	The name of the observation variable.
values	list[str]	Required	The categories of the observation variable.

Example:

model = POMDPModel()
observations = ["door", "shelf", "pallet", "table", "belt", "light", "box", "paper", "ramp"]
model.add_observation_variable(name="observation", values=observations)

add_preference_factor

Adds a preference factor to the model. There should be one preference factor per observation modality. Preference factors are always marginal distributions so no parents are needed.

Parameter	Type	Default	Description
values	np.ndarray	Required	A preference factor probability vector.
target	str	Required	An observation or observation modality variable.
parents	list[str]	Required	The parent variables for the factor.
counts	np.ndarray	Optional	The counts for the factor.

Example:

model = POMDPModel()

preferences_vars = ["observation"]
preference_probs = np.array([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0])

model.add_preference_factor(values=preference_probs, target=preferences_vars)

add_prior_factor

Adds a state prior factor to the model. An agent does not need to have a state prior but if it does, there could be one for each state factor. State priors are always marginal distributions so no parents are needed.

Parameter	Type	Default	Description
values	np.ndarray	Required	A state prior probability vector.
target	str	Required	An state or state factor variable.
parents	list[str]	Required	The parent variables for the factor.
counts	np.ndarray	Optional	The counts for the factor.

Example:

model = POMDPModel()

state_prior_vars = ["position"]
state_prior_probs = np.array([0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0])

model.add_prior_factor(values=state_prior_probs, target=state_prior_vars)

add_state_variable

Adds a state variable to the model. If there are multiple state factors then this method can be called repeatedly to add additional state factors

Parameter	Type	Default	Description
name	str	Required	The name of the state variable.
values	list[str]	Required	The categories of the state variable.

Example:

model = POMDPModel()

states = ["load", "aisle", "store", "office", "convey", "charge", "pack", "desk", "end"]
model.add_state_variable(name="state", values=states)

add_transition_factor

Adds a state-transition factor to the model. There should be one state-transition per state factor in the model.

Parameter	Type	Default	Description
values	np.ndarray	Required	A state-transition matrix.
target	str	Required	A state or state factor representing the current state.
parents	list[str]	Required	A pair of dependencies for the target. The first dependency should be identical to the target since it represents the transition of this target to a new state. The second dependency should be an action variable.
counts	np.ndarray	Optional	The counts for the factor.

Example:

model = POMDPModel()

transition_vars = ["position", "position", "action"]
transition_probs = np.array(
      [[[1., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 1., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 1., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 1., 0., 0., 0.]],

       [[0., 1., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 1., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 1., 0., 0., 0.],
        [0., 0., 0., 0., 0., 1., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 1., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 1.],
        [0., 0., 0., 0., 0., 0., 0., 0., 1.]],

       [[0., 0., 0., 1., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 1., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 1., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 1., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 1.],
        [0., 0., 0., 0., 0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 1., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 1.]],

       [[1., 0., 0., 0., 0., 0., 0., 0., 0.],
        [1., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 1., 0., 0., 0., 0., 0.],
        [0., 0., 0., 1., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 1., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 1., 0.]],

       [[1., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 1., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 1., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 1., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 1., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 1., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 1.]]]).T
        
 model.add_transition_factor(
    values=transition_probs, 
    target=transition_vars[0], 
    parents=transition_vars[1:])

validate

Validates that a model created with the POMDPModel class is valid. This includes a check that all model elements are present.

Parameter	Type	Default	Description
model_type	ModelType	FactorGraph	Defines the type of model the model should be validated as.
correct_errors	bool	False	If True, the validate endpoint will attempt to correct fixable errors.
verbose	bool	True	Whether the logger should report if a model was successfully validated.

Returns:

Type	Description
list[errors.ValidationError]	A list of errors which were detected.