oa::PpoTrainer

Complete environment-neutral categorical PPO loop. The environment remains caller-owned; this class owns collection storage, policy bookkeeping, GAE, and optimizer updates. It composes `ItRolloutTraining` because collection and update phases are not the same lifecycle as DQN or SAC.

ClC++ public headers

Public Methods

oa::Status oa::PpoTrainer::abortCollection()

PolicyResult oa::PpoTrainer::act(const oa::Matrix & inObservation)

const RolloutBatch & oa::PpoTrainer::batch()

oa::Status oa::PpoTrainer::beginCollection()

const PpoTrainerConfig & oa::PpoTrainer::config()

oa::Status oa::PpoTrainer::endCollection()

bool oa::PpoTrainer::isDone()

bool oa::PpoTrainer::isValid()

oa::Status oa::PpoTrainer::load(const oa::String & inPath)

const PpoTrainerMetrics & oa::PpoTrainer::metrics()

bool oa::PpoTrainer::needsCollection()

oa::Status oa::PpoTrainer::observe(const oa::Matrix & inObservation, const oa::Matrix & inNextObservation, const oa::Matrix & inReward, const oa::Matrix & inTerminated, const oa::Matrix & inTruncated, const PolicyResult & inPolicy)

RolloutTrainingPhase oa::PpoTrainer::phase()

oa::Status oa::PpoTrainer::save(const oa::String & inPath)

const oa::ItTraining & oa::PpoTrainer::trainingLoop()

oa::ItTraining & oa::PpoTrainer::trainingLoop()

oa::Status oa::PpoTrainer::update()

Static Public Methods

static oa::Result<oa::UniquePtr<PpoTrainer>> oa::PpoTrainer::create(oa::Engine & inEngine, ActorCritic & inModel, oa::Optimizer & inOptimizer, const PpoTrainerConfig & inConfig)

Constructor & Destructor Documentation

void oa::PpoTrainer::~PpoTrainer()

No public source comment is attached to this declaration.

Public Method Documentation

oa::Status oa::PpoTrainer::abortCollection()

Rolls back collection control state after the caller cancels the unsubmitted command transaction. valid from beginCollection through endCollection, until the first update actually begins.

Returns

oa::Status

The declared return value.

PolicyResult oa::PpoTrainer::act(
	const oa::Matrix & inObservation
)

No public source comment is attached to this declaration.

Parameters

inObservation

const oa::Matrix &

—

Returns

PolicyResult

The declared return value.

const RolloutBatch & oa::PpoTrainer::batch()

No public source comment is attached to this declaration.

Returns

const RolloutBatch &

The declared return value.

oa::Status oa::PpoTrainer::beginCollection()

No public source comment is attached to this declaration.

Returns

oa::Status

The declared return value.

const PpoTrainerConfig & oa::PpoTrainer::config()

No public source comment is attached to this declaration.

Returns

const PpoTrainerConfig &

The declared return value.

oa::Status oa::PpoTrainer::endCollection()

No public source comment is attached to this declaration.

Returns

oa::Status

The declared return value.

bool oa::PpoTrainer::isDone()

No public source comment is attached to this declaration.

Returns

bool

The declared return value.

bool oa::PpoTrainer::isValid()

No public source comment is attached to this declaration.

Returns

bool

The declared return value.

oa::Status oa::PpoTrainer::load(
	const oa::String & inPath
)

No public source comment is attached to this declaration.

Parameters

inPath

const oa::String &

—

Returns

oa::Status

The declared return value.

const PpoTrainerMetrics & oa::PpoTrainer::metrics()

No public source comment is attached to this declaration.

Returns

const PpoTrainerMetrics &

The declared return value.

bool oa::PpoTrainer::needsCollection()

No public source comment is attached to this declaration.

Returns

bool

The declared return value.

oa::Status oa::PpoTrainer::observe(
	const oa::Matrix & inObservation,
	const oa::Matrix & inNextObservation,
	const oa::Matrix & inReward,
	const oa::Matrix & inTerminated,
	const oa::Matrix & inTruncated,
	const PolicyResult & inPolicy
)

No public source comment is attached to this declaration.

Parameters

inObservation

const oa::Matrix &

—

inNextObservation

const oa::Matrix &

—

inReward

const oa::Matrix &

—

inTerminated

const oa::Matrix &

—

inTruncated

const oa::Matrix &

—

inPolicy

const PolicyResult &

—

Returns

oa::Status

The declared return value.

RolloutTrainingPhase oa::PpoTrainer::phase()

No public source comment is attached to this declaration.

Returns

RolloutTrainingPhase

The declared return value.

oa::Status oa::PpoTrainer::save(
	const oa::String & inPath
)

No public source comment is attached to this declaration.

Parameters

inPath

const oa::String &

—

Returns

oa::Status

The declared return value.

const oa::ItTraining & oa::PpoTrainer::trainingLoop()

No public source comment is attached to this declaration.

Returns

const oa::ItTraining &

The declared return value.

oa::ItTraining & oa::PpoTrainer::trainingLoop()

The ordinary optimizer-update iterator shared by supervised and RL training. attach TrainingSession here for live control/observation.

Returns

oa::ItTraining &

The declared return value.

oa::Status oa::PpoTrainer::update()

Performs one PPO update epoch. call until needsCollection() or isDone().

Returns

oa::Status

The declared return value.

static oa::Result<oa::UniquePtr<PpoTrainer>> oa::PpoTrainer::create(
	oa::Engine & inEngine,
	ActorCritic & inModel,
	oa::Optimizer & inOptimizer,
	const PpoTrainerConfig & inConfig
)

No public source comment is attached to this declaration.

Parameters

inEngine

oa::Engine &

—

inModel

ActorCritic &

—

inOptimizer

oa::Optimizer &

—

inConfig

const PpoTrainerConfig &

—

Returns

oa::Result<oa::UniquePtr<PpoTrainer>>

The declared return value.