Conversation
kontramind
reviewed
Aug 26, 2026
kontramind
reviewed
Aug 26, 2026
kontramind
reviewed
Aug 26, 2026
…'use_q_ide.py' the problem was related to a quick and dirty fix during training.
…here the orthogonality check used the wron variable. The related section of the code (w_speed is ndarray) has so fra not been used.
…rting is now done from episode-runs on trained data. The removed code was in any case inaccessible, due to changes in reward_stats.
reward_stats is now dict[str, list[float]], not list[list[float]]; the old row-indexing silently produced garbage plots instead of erroring. Matches the pattern already used in algorithm.py.
analyse_trained() made r_limit keyword-only but this call site still passed it positionally, raising TypeError on every call.
…rror
The orthogonality-check ValueError was missing the f prefix, so
{w_speed}/{self.wire.direction} printed literally instead of
interpolating.
Pre-existing formatting drift from a028b57 that CI's ruff format check caught; unrelated to the mechanical fixes in the prior commits.
_save_reward_plot() read env.unwrapped.reward_stats, whose shape is owned by the Q-learning side and just changed from list to dict under it, silently corrupting the plot (list.extend(dict) iterates keys). Switch to EpRewardLogCallback's own per-interval rows (the same data already written to csv_path) via a new public `rows` property, so PPO no longer depends on Q-learning's internal reward_stats shape at all.
aleksandarbabicdnv
approved these changes
Aug 27, 2026
aleksandarbabicdnv
left a comment
aleksandarbabicdnv
left a comment
Collaborator
There was a problem hiding this comment.
Looks good to me
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mayor upgrade mainly on q_agent and controlled_crane_pendulum to get Q-Learning to work on the anti-pendulum controller. Trained models are recorded in models/ folder.