update direct stepwise train, pass trainning with mock one step trace duplicated 52 times; reward OK, curve improved as the step grows.

Browse Source

...

This commit is contained in:

yuyr

2025-06-30 15:06:38 +00:00

parent 5e632c53ac

commit 4295f30f9a

56 changed files with 1409 additions and 1 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_clip_ratio_high_max.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_clip_ratio_high_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_clip_ratio_low_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_clip_ratio_low_min.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_clip_ratio_region_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_completions_clipped_ratio.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 13 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_completions_max_length.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_completions_mean_length.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_completions_min_length.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 14 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_kl.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_loss.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 11 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_reward.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 10 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_reward_std.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 11 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_rewards_WebAccuracy_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 14 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_rewards_WebAccuracy_std.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 14 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_runtime.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_samples_per_second.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 15 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/eval_steps_per_second.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_clip_ratio_high_max.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_clip_ratio_high_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_clip_ratio_low_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_clip_ratio_low_min.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 12 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_clip_ratio_region_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 13 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_completions_clipped_ratio.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 14 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_completions_max_length.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 22 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_completions_mean_length.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 27 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_completions_min_length.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 26 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_epoch.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 17 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_grad_norm.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_kl.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 20 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_learning_rate.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 22 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_loss.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 25 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_memory(GiB).png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 15 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_reward.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 20 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_reward_std.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 20 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_rewards_WebAccuracy_mean.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 24 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_rewards_WebAccuracy_std.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 22 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_total_flos.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 10 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_train_loss.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 11 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_train_runtime.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 11 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_train_samples_per_second.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 17 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_train_speed(iter_s).png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 19 KiB

BIN

direct_stepwise_train/v7-20250630-143128/images/train_train_steps_per_second.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 16 KiB