{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Elevator Dispatch with Multi-Agent RL\n",
        "\n",
        "**From [rlbook.ai](https://rlbook.ai/applications/elevator-dispatch)**\n",
        "\n",
        "Train a multi-agent DQN to control elevators in a building. Learn about:\n",
        "- Multi-agent coordination without communication\n",
        "- Reward shaping for complex objectives\n",
        "- Independent Q-learning with shared replay\n",
        "- Baseline policy comparisons\n",
        "\n",
        "This notebook lets you train and evaluate elevator dispatch policies interactively!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Setup\n",
        "\n",
        "Install the rlbook package and dependencies."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Install rlbook package from GitHub\n",
        "!pip install -q git+https://github.com/ebilgin/rlbook.git#subdirectory=code\n",
        "\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import torch\n",
        "\n",
        "from rlbook.envs import ElevatorDispatch\n",
        "from rlbook.agents import ElevatorDQN\n",
        "from rlbook.agents.elevator_dqn import random_policy, nearest_car_policy, scan_policy\n",
        "\n",
        "print(f\"NumPy: {np.__version__}\")\n",
        "print(f\"PyTorch: {torch.__version__}\")\n",
        "print(f\"Device: {'cuda' if torch.cuda.is_available() else 'cpu'}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Part 1: Explore the Environment\n",
        "\n",
        "Let's create an elevator dispatch environment and see how it works."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Create environment\n",
        "env = ElevatorDispatch(\n",
        "    n_floors=10,\n",
        "    n_elevators=3,\n",
        "    traffic_pattern=\"morning_rush\",\n",
        "    max_timesteps=50,  # Short episode for demo\n",
        "    render_mode=\"ansi\"\n",
        ")\n",
        "\n",
        "print(\"Environment created!\")\n",
        "print(f\"Floors: {env.n_floors}\")\n",
        "print(f\"Elevators: {env.n_elevators}\")\n",
        "print(f\"Traffic: {env.traffic_pattern.name}\")\n",
        "print(f\"\\nObservation space (per elevator): {env.observation_space}\")\n",
        "print(f\"Action space (all elevators): {env.action_space}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Run a Random Episode\n",
        "\n",
        "See what happens with random elevator actions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "obs, info = env.reset(seed=42)\n",
        "\n",
        "# Run 10 steps with random actions\n",
        "for step in range(10):\n",
        "    # Random actions for all elevators\n",
        "    actions = [np.random.randint(0, env.n_floors) for _ in range(env.n_elevators)]\n",
        "    \n",
        "    obs, reward, done, truncated, info = env.step(actions)\n",
        "    \n",
        "    # Print every 3 steps\n",
        "    if step % 3 == 0:\n",
        "        print(f\"\\n=== Step {step} ===\")\n",
        "        print(f\"Actions: {actions}\")\n",
        "        print(f\"Reward: {reward:.1f}\")\n",
        "        print(f\"Waiting: {info['waiting_passengers']}, Served: {info['delivered_passengers']}\")\n",
        "        if step == 9:\n",
        "            print(f\"\\nFinal avg wait time: {info['avg_wait_time']:.1f}s\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Visualize the Building\n",
        "\n",
        "The environment has an ASCII renderer. Let's see it!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "env.reset(seed=42)\n",
        "\n",
        "# Run a few steps and render\n",
        "for _ in range(5):\n",
        "    actions = [np.random.randint(0, env.n_floors) for _ in range(env.n_elevators)]\n",
        "    env.step(actions)\n",
        "\n",
        "# Show current state\n",
        "print(env.render())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Part 2: Baseline Policies\n",
        "\n",
        "Before training an RL agent, let's see how simple rule-based policies perform."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def evaluate_policy(env, policy_fn, n_episodes=10, policy_name=\"Policy\"):\n",
        "    \"\"\"Evaluate a policy over multiple episodes.\"\"\"\n",
        "    wait_times = []\n",
        "    passengers_served = []\n",
        "    \n",
        "    for episode in range(n_episodes):\n",
        "        obs, info = env.reset()\n",
        "        \n",
        "        for step in range(env.max_timesteps):\n",
        "            actions = policy_fn(env)\n",
        "            obs, reward, done, truncated, info = env.step(actions)\n",
        "            \n",
        "            if done or truncated:\n",
        "                break\n",
        "        \n",
        "        wait_times.append(info['avg_wait_time'])\n",
        "        passengers_served.append(info['delivered_passengers'])\n",
        "    \n",
        "    return {\n",
        "        'avg_wait': np.mean(wait_times),\n",
        "        'std_wait': np.std(wait_times),\n",
        "        'avg_served': np.mean(passengers_served),\n",
        "    }\n",
        "\n",
        "# Create longer environment for evaluation\n",
        "eval_env = ElevatorDispatch(\n",
        "    n_floors=10,\n",
        "    n_elevators=3,\n",
        "    traffic_pattern=\"morning_rush\",\n",
        "    max_timesteps=300\n",
        ")\n",
        "\n",
        "print(\"Evaluating baseline policies...\")\n",
        "print(\"(This takes ~30 seconds)\\n\")\n",
        "\n",
        "baselines = {\n",
        "    \"Random\": random_policy,\n",
        "    \"Nearest Car\": nearest_car_policy,\n",
        "    \"SCAN\": scan_policy,\n",
        "}\n",
        "\n",
        "results = {}\n",
        "for name, policy in baselines.items():\n",
        "    results[name] = evaluate_policy(eval_env, policy, n_episodes=10, policy_name=name)\n",
        "    print(f\"{name:15s} - Wait: {results[name]['avg_wait']:5.1f}s (±{results[name]['std_wait']:.1f}), \"\n",
        "          f\"Served: {results[name]['avg_served']:.0f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Observations:**\n",
        "- Random is terrible (no surprise!)\n",
        "- Nearest Car is decent - assigns closest elevator\n",
        "- SCAN is best baseline - continues in direction like a hard drive\n",
        "\n",
        "Can RL do better? Let's find out!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Part 3: Train a DQN Agent\n",
        "\n",
        "Now we'll train a multi-agent DQN. Each elevator learns its own Q-network, but they share experiences.\n",
        "\n",
        "**Training parameters:**\n",
        "- 200 episodes (increase for better results, but slower)\n",
        "- ε starts at 1.0, decays to 0.01\n",
        "- Shared replay buffer (50k transitions)\n",
        "- [128, 128] hidden layers\n",
        "\n",
        "**Expected training time:**\n",
        "- ~5 minutes on Colab GPU\n",
        "- ~15 minutes on Colab CPU"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Create training environment\n",
        "train_env = ElevatorDispatch(\n",
        "    n_floors=10,\n",
        "    n_elevators=3,\n",
        "    traffic_pattern=\"morning_rush\",\n",
        "    max_timesteps=300\n",
        ")\n",
        "\n",
        "# Get observation dimension\n",
        "obs, _ = train_env.reset()\n",
        "obs_dim = obs[\"elevator_0\"].shape[0]\n",
        "\n",
        "# Create agent\n",
        "agent = ElevatorDQN(\n",
        "    n_floors=10,\n",
        "    n_elevators=3,\n",
        "    observation_dim=obs_dim,\n",
        "    hidden_dims=(128, 128),\n",
        "    learning_rate=1e-3,\n",
        "    gamma=0.99,\n",
        "    epsilon=1.0,\n",
        "    epsilon_decay=0.99,  # Faster decay for shorter training\n",
        "    epsilon_min=0.01,\n",
        "    buffer_size=50000,\n",
        "    batch_size=64\n",
        ")\n",
        "\n",
        "print(f\"Agent created!\")\n",
        "print(f\"Observation dim: {obs_dim}\")\n",
        "print(f\"Q-networks: {agent.n_elevators} (one per elevator)\")\n",
        "print(f\"Parameters per network: {sum(p.numel() for p in agent.q_networks[0].parameters()):,}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Training Loop\n",
        "\n",
        "Run the training! This cell will take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "n_episodes = 200  # Increase to 500-1000 for better results\n",
        "\n",
        "episode_rewards = []\n",
        "avg_wait_times = []\n",
        "epsilon_history = []\n",
        "\n",
        "print(f\"Training for {n_episodes} episodes...\\n\")\n",
        "\n",
        "for episode in range(n_episodes):\n",
        "    obs, info = train_env.reset()\n",
        "    episode_reward = 0\n",
        "    \n",
        "    for step in range(train_env.max_timesteps):\n",
        "        # Select actions\n",
        "        actions = agent.select_actions(obs, training=True)\n",
        "        \n",
        "        # Take step\n",
        "        next_obs, reward, done, truncated, info = train_env.step(actions)\n",
        "        \n",
        "        # Store transition\n",
        "        agent.store_transitions(obs, actions, reward, next_obs, done or truncated)\n",
        "        \n",
        "        # Train\n",
        "        loss = agent.train_step()\n",
        "        \n",
        "        episode_reward += reward\n",
        "        obs = next_obs\n",
        "        \n",
        "        if done or truncated:\n",
        "            break\n",
        "    \n",
        "    # Decay epsilon\n",
        "    agent.decay_epsilon()\n",
        "    \n",
        "    # Record metrics\n",
        "    episode_rewards.append(episode_reward)\n",
        "    avg_wait_times.append(info['avg_wait_time'])\n",
        "    epsilon_history.append(agent.epsilon)\n",
        "    \n",
        "    # Print progress\n",
        "    if (episode + 1) % 20 == 0:\n",
        "        recent_reward = np.mean(episode_rewards[-20:])\n",
        "        recent_wait = np.mean(avg_wait_times[-20:])\n",
        "        print(f\"Episode {episode + 1:3d}/{n_episodes} | \"\n",
        "              f\"Reward: {recent_reward:6.1f} | \"\n",
        "              f\"Wait: {recent_wait:5.1f}s | \"\n",
        "              f\"ε: {agent.epsilon:.3f} | \"\n",
        "              f\"Buffer: {len(agent.replay_buffer)}\")\n",
        "\n",
        "print(\"\\nTraining complete!\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Visualize Training Progress"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "fig, axes = plt.subplots(1, 3, figsize=(15, 4))\n",
        "\n",
        "# Smooth data for plotting\n",
        "def smooth(data, window=20):\n",
        "    return np.convolve(data, np.ones(window)/window, mode='valid')\n",
        "\n",
        "# Episode rewards\n",
        "axes[0].plot(smooth(episode_rewards), color='#06b6d4', linewidth=2)\n",
        "axes[0].set_xlabel('Episode')\n",
        "axes[0].set_ylabel('Episode Reward')\n",
        "axes[0].set_title('Training Reward (20-episode avg)')\n",
        "axes[0].grid(alpha=0.3)\n",
        "\n",
        "# Wait times\n",
        "axes[1].plot(smooth(avg_wait_times), color='#f97316', linewidth=2)\n",
        "axes[1].set_xlabel('Episode')\n",
        "axes[1].set_ylabel('Avg Wait Time (s)')\n",
        "axes[1].set_title('Average Wait Time (20-episode avg)')\n",
        "axes[1].grid(alpha=0.3)\n",
        "\n",
        "# Epsilon decay\n",
        "axes[2].plot(epsilon_history, color='#8b5cf6', linewidth=2)\n",
        "axes[2].set_xlabel('Episode')\n",
        "axes[2].set_ylabel('Epsilon')\n",
        "axes[2].set_title('Exploration Rate')\n",
        "axes[2].grid(alpha=0.3)\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.show()\n",
        "\n",
        "print(f\"Final performance (last 20 episodes):\")\n",
        "print(f\"  Average reward: {np.mean(episode_rewards[-20:]):.1f}\")\n",
        "print(f\"  Average wait time: {np.mean(avg_wait_times[-20:]):.1f}s\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Part 4: Evaluate the Trained Agent\n",
        "\n",
        "How does our trained DQN compare to the baselines?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def evaluate_dqn(env, agent, n_episodes=10):\n",
        "    \"\"\"Evaluate trained DQN agent.\"\"\"\n",
        "    wait_times = []\n",
        "    passengers_served = []\n",
        "    \n",
        "    for episode in range(n_episodes):\n",
        "        obs, info = env.reset()\n",
        "        \n",
        "        for step in range(env.max_timesteps):\n",
        "            actions = agent.select_actions(obs, training=False)  # No exploration\n",
        "            obs, reward, done, truncated, info = env.step(actions)\n",
        "            \n",
        "            if done or truncated:\n",
        "                break\n",
        "        \n",
        "        wait_times.append(info['avg_wait_time'])\n",
        "        passengers_served.append(info['delivered_passengers'])\n",
        "    \n",
        "    return {\n",
        "        'avg_wait': np.mean(wait_times),\n",
        "        'std_wait': np.std(wait_times),\n",
        "        'avg_served': np.mean(passengers_served),\n",
        "    }\n",
        "\n",
        "# Evaluate\n",
        "dqn_results = evaluate_dqn(eval_env, agent, n_episodes=10)\n",
        "\n",
        "# Compare all policies\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"FINAL COMPARISON\")\n",
        "print(\"=\"*60)\n",
        "print(f\"{'Algorithm':<20} {'Avg Wait (s)':<15} {'Passengers Served':<20}\")\n",
        "print(\"-\"*60)\n",
        "\n",
        "for name, res in results.items():\n",
        "    print(f\"{name:<20} {res['avg_wait']:<15.2f} {res['avg_served']:<20.1f}\")\n",
        "\n",
        "print(f\"{'DQN (Trained)':<20} {dqn_results['avg_wait']:<15.2f} {dqn_results['avg_served']:<20.1f}\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "# Calculate improvement\n",
        "best_baseline = min(results.items(), key=lambda x: x[1]['avg_wait'])\n",
        "improvement = (best_baseline[1]['avg_wait'] - dqn_results['avg_wait']) / best_baseline[1]['avg_wait'] * 100\n",
        "\n",
        "print(f\"\\nDQN improves over best baseline ({best_baseline[0]}) by {improvement:.1f}%!\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Visualize the Comparison"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Prepare data for plotting\n",
        "algorithms = list(results.keys()) + ['DQN (Trained)']\n",
        "wait_times_plot = [results[name]['avg_wait'] for name in results.keys()] + [dqn_results['avg_wait']]\n",
        "colors = ['#ef4444', '#f97316', '#eab308', '#06b6d4']\n",
        "\n",
        "# Create bar chart\n",
        "fig, ax = plt.subplots(figsize=(10, 6))\n",
        "bars = ax.bar(algorithms, wait_times_plot, color=colors, alpha=0.8, edgecolor='black')\n",
        "\n",
        "# Highlight the best\n",
        "bars[-1].set_color('#10b981')\n",
        "bars[-1].set_linewidth(3)\n",
        "\n",
        "ax.set_ylabel('Average Wait Time (seconds)', fontsize=12)\n",
        "ax.set_title('Elevator Dispatch Performance Comparison', fontsize=14, fontweight='bold')\n",
        "ax.grid(axis='y', alpha=0.3)\n",
        "\n",
        "# Add value labels on bars\n",
        "for bar in bars:\n",
        "    height = bar.get_height()\n",
        "    ax.text(bar.get_x() + bar.get_width()/2., height,\n",
        "            f'{height:.1f}s',\n",
        "            ha='center', va='bottom', fontweight='bold')\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Part 5: Analyze Learned Behavior\n",
        "\n",
        "What did the agent learn? Let's run an episode and watch what happens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Create environment with rendering\n",
        "demo_env = ElevatorDispatch(\n",
        "    n_floors=10,\n",
        "    n_elevators=3,\n",
        "    traffic_pattern=\"morning_rush\",\n",
        "    max_timesteps=50,  # Short for demo\n",
        "    render_mode=\"ansi\"\n",
        ")\n",
        "\n",
        "obs, info = demo_env.reset(seed=42)\n",
        "\n",
        "print(\"Running trained agent...\\n\")\n",
        "print(\"Watch how the elevators coordinate!\\n\")\n",
        "\n",
        "for step in range(20):\n",
        "    actions = agent.select_actions(obs, training=False)\n",
        "    obs, reward, done, truncated, info = demo_env.step(actions)\n",
        "    \n",
        "    # Print every 5 steps\n",
        "    if step % 5 == 0:\n",
        "        print(f\"\\n{'='*50}\")\n",
        "        print(f\"Step {step}\")\n",
        "        print(f\"Actions: {actions} (target floors)\")\n",
        "        print(demo_env.render())\n",
        "        print(f\"Reward: {reward:.1f}\")\n",
        "    \n",
        "    if done or truncated:\n",
        "        break\n",
        "\n",
        "print(f\"\\nFinal metrics:\")\n",
        "print(f\"  Wait time: {info['avg_wait_time']:.1f}s\")\n",
        "print(f\"  Passengers served: {info['delivered_passengers']}\")\n",
        "print(f\"  Utilization: {info['elevator_utilization']*100:.1f}%\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**What to look for:**\n",
        "- Do elevators naturally partition the floors? (e.g., one handles low, one mid, one high)\n",
        "- Do they position themselves anticipating future requests?\n",
        "- Do they avoid clustering together?\n",
        "\n",
        "This coordination **emerged** from learning - we never programmed it explicitly!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Part 6: Extensions & Experiments\n",
        "\n",
        "Try these on your own:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Exercise 1: Different Traffic Patterns\n",
        "\n",
        "How does the agent perform on evening_rush (upper floors → lobby)?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# TODO: Create environment with traffic_pattern=\"evening_rush\"\n",
        "# TODO: Evaluate the agent trained on morning_rush\n",
        "# Does it generalize? Or does performance drop?\n",
        "\n",
        "# Your code here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Exercise 2: Reward Shaping\n",
        "\n",
        "Try modifying the reward function. What happens if you:\n",
        "- Increase the starvation penalty (line 377 in elevator.py)\n",
        "- Increase the delivery bonus\n",
        "- Add a \"fairness\" term (penalize variance in wait times)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# TODO: Modify the environment's _calculate_reward method\n",
        "# TODO: Retrain and compare performance\n",
        "\n",
        "# Your code here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Exercise 3: Larger Buildings\n",
        "\n",
        "What happens with 20 floors and 5 elevators?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# TODO: Create environment with n_floors=20, n_elevators=5\n",
        "# TODO: Train an agent (may take longer)\n",
        "# TODO: Does coordination become harder?\n",
        "\n",
        "# Your code here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Exercise 4: Network Architecture\n",
        "\n",
        "How does performance change with different hidden layer sizes?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# TODO: Try hidden_dims=(64, 64) vs (256, 256)\n",
        "# TODO: Compare training speed and final performance\n",
        "\n",
        "# Your code here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Summary\n",
        "\n",
        "In this notebook, you:\n",
        "\n",
        "1. ✅ **Explored** a multi-elevator environment with realistic traffic patterns\n",
        "2. ✅ **Compared** baseline policies (Random, Nearest Car, SCAN)\n",
        "3. ✅ **Trained** a multi-agent DQN with independent Q-learning\n",
        "4. ✅ **Evaluated** the learned policy and achieved better performance than baselines\n",
        "5. ✅ **Analyzed** emergent coordination behaviors\n",
        "\n",
        "**Key Takeaways:**\n",
        "- Multi-agent RL can learn implicit coordination without communication\n",
        "- Reward shaping (wait time + delivery bonus + starvation prevention) is critical\n",
        "- Independent Q-learning with shared replay is a simple but effective approach\n",
        "- RL outperforms rule-based algorithms when the environment is complex\n",
        "\n",
        "**Next Steps:**\n",
        "- Read the full application guide: [rlbook.ai/applications/elevator-dispatch](https://rlbook.ai/applications/elevator-dispatch)\n",
        "- Explore advanced multi-agent methods (QMIX, MADDPG)\n",
        "- Try deploying to a real building (sim-to-real challenges!)\n",
        "- Experiment with the extensions above\n",
        "\n",
        "**Questions or feedback?** Join the discussion at [discord.gg/mJ7n3zNf7r](https://discord.gg/mJ7n3zNf7r)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "## Save Your Trained Model (Optional)\n",
        "\n",
        "Save the trained agent to continue later or share with others."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Save the model\n",
        "agent.save(\"elevator_dqn_trained.pt\")\n",
        "print(\"Model saved to elevator_dqn_trained.pt\")\n",
        "\n",
        "# To load later:\n",
        "# new_agent = ElevatorDQN(...same parameters...)\n",
        "# new_agent.load(\"elevator_dqn_trained.pt\")"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.0"
    },
    "colab": {
      "provenance": [],
      "toc_visible": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}