MoonTrajectory

MoonTrajectory is a small MoonBit library for reinforcement-learning trajectories. It focuses on a practical core that is useful on its own and easy to extend:

• Transition for single-step environment data
• Episode for ordered rollout storage and discounted returns
• ReplayBuffer for uniform experience replay
• PrioritizedReplayBuffer for proportional prioritized replay

The current implementation is intentionally compact, but the shape of the API is designed for later DQN, PPO, and offline-RL extensions.

Project Links

• GitHub: https://github.com/hmyhmyhmyss/MoonTrajectory
• GitLink: https://gitlink.org.cn/hmyhmy78/MoonTrajectory

Competition Fit

This project is built to match the OSC2026 MoonBit track requirements:

• MoonBit is the main implementation language
• the repository is structured as a clean library, not a one-off script
• the code is fully original and does not reuse third-party source code
• README, license, tests, demo, and CI are all present
• the commit history is intentionally split into meaningful development steps

Submission Checklist

• public repository
• OSI-approved license
• runnable example
• source provenance note
• no extra contributors
• default branch set to master when pushed to the empty remote

Why this topic

This project sits in a mature and reusable area:

• it solves a common infrastructure problem in RL workflows
• it has a clear upgrade path toward n-step returns, frame stacking, segment-tree sampling, and persistence
• it can serve as a base package for future MoonBit algorithm demos

Layout

• trajectory.mbt: core Transition and Episode types
• replay_buffer.mbt: circular replay buffer and uniform sampling
• prioritized_replay.mbt: basic proportional prioritized replay
• rng.mbt: tiny deterministic sampler used by tests and examples
• cmd/main/main.mbt: smoke demo

Quick start

moon test
moon run cmd/main

Public API

Transition

Represents one step of interaction with an environment.

Episode

Stores an ordered list of transitions and keeps track of:

• total reward
• termination state
• truncation state

It also exposes discounted_returns(gamma) for return computation.

ReplayBuffer

A circular buffer with:

• fixed capacity
• overwrite-on-full behavior
• logical ordering from oldest to newest
• deterministic sampling for repeatable tests

PrioritizedReplayBuffer

A basic prioritized replay implementation with:

• proportional sampling
• configurable alpha, beta, and epsilon
• per-item priority updates
• normalized importance-sampling weights

The first version uses a simple scan-based sampler instead of a tree structure so the code stays easy to read and audit.

Competition notes

This repository is set up to match the MoonBit open-source competition expectations:

• MoonBit is the primary language
• the license is OSI-approved
• the project has a README, tests, and a runnable example
• the API is public and the scope is broad enough to grow
• the implementation is original and does not port third-party source code
• the repository links can be verified directly from this file

Before submission, update these metadata fields for the final publishing account:

• moon.mod name
• moon.mod repository
• cmd/main/moon.pkg import path if the module name changes

License

Apache-2.0

Status

This is the first public implementation pass. The next natural extensions are:

• n-step transition aggregation
• episode padding and masking
• replay serialization
• segment-tree prioritized sampling
• richer statistics for training loops