Autolab ManiSkill Tuning

When to use this skill

Use this skill when working on the Autolab simulation stack, including:

• ManiSkill task evaluation and controller tuning
• VSLAM evaluation and parameter search
• Run monitoring, result comparison, and summary generation
• Writing durable agent memory and checkpoints

Operating rules

• Start every session by calling get_system_prompt() before any other action.
• Use MCP tools for simulation interactions. Prefer them over shell commands.
• Work autonomously. Do not ask the user for tuning or evaluation decisions.
• Fix errors before moving to the next step.
• Save the final report to results/summary.txt.
• Use run_shell_command only for file writes or tooling gaps.

Standard workflow

1. Discover tasks

Call list_tasks() to inspect:

• task names
• parameter grids
• default parameters
• optimization metric

Choose:

• path_follow or room_nav for ManiSkill
• vslam_eval only if the ROS2 / ORB-SLAM3 toolchain is available

2. Verify baseline execution

Run one episode with default settings before tuning:

• run_evaluate(task_type="path_follow", episodes=1)
• or run_shell_command(...) for vslam_eval only when needed

If the run fails:

• inspect stderr
• identify the root cause
• retry after fixing the environment or parameters

3. Tune parameters

Prefer the background tuner for longer jobs:

• start_tuning_run(task_type="path_follow", episodes=3)
• poll with check_run_status(pid)
• read results with get_tuning_results(task_type="path_follow")

Use the blocking evaluator for short runs:

• run_evaluate(...)

Choose the best result by:

• lowest mean_cte
• completion_rate >= 0.8 when possible

4. Verify the best config

Re-run the best parameter set for more episodes:

• run_evaluate(task_type="path_follow", episodes=5, params={...})

For VSLAM, verify with the best bag_rate and settle_time.

5. Write the summary

Write results/summary.txt with:

• baseline parameters
• baseline metrics
• best parameters
• best metrics
• percentage improvement
• short notes on behavior and failure modes

Memory and checkpoints

Use these tools when information is reusable:

• write_memory(agent_id, content, ...)
• search_memories(agent_id, query, ...)
• save_checkpoint(agent_id, checkpoint_key, state, ...)
• load_checkpoint(agent_id, checkpoint_key, ...)

Store:

• successful parameter combinations
• task-specific failure modes
• environment quirks
• reproducible setup steps

Common mistakes

• Don’t skip get_system_prompt()
• Don’t move to tuning before baseline verification
• Don’t rely on shell when an MCP tool exists
• Don’t publish a summary without a verified rerun
• Don’t overwrite prior memory without checking for existing context
• Don’t use vslam_eval unless the ROS2 / ORB-SLAM3 stack is available

Summary format

ManiSkill

=== Tuning Results — <task name> ===

Baseline parameters:  <param=value ...>
Baseline mean_cte:    <value>
Baseline completion:  <value>

Best parameters:      <param=value ...>
Best mean_cte:        <value>
Best completion:      <value>

Improvement:          <X>% reduction in mean cross-track error

Notes:
<short observations>

VSLAM

=== Tuning Results — vslam_eval ===

Baseline parameters:  bag_rate=1.0  settle_time=3.0  sequence_name=V1_01_easy
Baseline ape_rmse:    <value>
Baseline ape_mean:    <value>

Best parameters:      bag_rate=<value>  settle_time=<value>
Best ape_rmse:        <value>
Best ape_mean:        <value>

Improvement:          <X>% reduction in APE RMSE

Notes:
<short observations>