From cb0dc0d8796390fca9cc2b0c422b7151ba2cacad Mon Sep 17 00:00:00 2001 From: Bill Thiede Date: Thu, 4 Dec 2025 16:52:02 -0800 Subject: [PATCH] Use custom image w/ cargo-aoc --- README.md | 173 +++++----------------------------------------------- aocsync.py | 38 ++++++++++-- config.yaml | 6 +- 3 files changed, 52 insertions(+), 165 deletions(-) diff --git a/README.md b/README.md index cabed25..6123910 100644 --- a/README.md +++ b/README.md @@ -1,174 +1,33 @@ # AOC Sync -A Python script that polls multiple git repositories containing Advent of Code implementations written in Rust using `cargo-aoc` format. It automatically updates repositories when changes are detected, runs benchmarks, and generates a beautiful HTML comparison page showing performance metrics across users, years, and days. +A tool to poll multiple git repositories containing Advent of Code implementations and generate performance comparison reports. -## Features +## Building the Podman Image -- **Automatic Git Polling**: Monitors multiple repositories for changes -- **Flexible Repository Structure**: Supports both single-repo (all years) and multi-repo (one per year) configurations -- **Automatic Runtime Measurement**: Runs `cargo aoc` for all implemented days and parses runtime information -- **Performance Parsing**: Extracts timing data from cargo-aoc output -- **Data Storage**: SQLite database for historical performance data -- **HTML Reports**: Beautiful, responsive HTML comparison pages -- **Gap Handling**: Gracefully handles missing years, days, and parts -- **Configurable Comparisons**: Filter by specific years and days +The default configuration uses a custom Podman image (`aocsync:latest`) that has `cargo-aoc` pre-installed for faster execution. -## Requirements +To build the image: -- Python 3.7+ -- Git -- Rust and Cargo -- `cargo-aoc` installed (install with `cargo install cargo-aoc`) - -## Installation - -1. Clone this repository: ```bash -git clone -cd aocsync +podman build -t aocsync:latest -f Dockerfile . ``` -2. Install Python dependencies: -```bash -pip install -r requirements.txt -``` - -3. Ensure `cargo-aoc` is installed: -```bash -cargo install cargo-aoc -``` - -Note: The script runs `cargo aoc` (not `cargo aoc bench`) and parses runtime information from the output. +Alternatively, you can use the `rust:latest` image, but it will install `cargo-aoc` on each run (slower). ## Configuration -Edit `config.yaml` to configure repositories to monitor: - -```yaml -# Poll interval in seconds -poll_interval: 300 - -# Output directory for generated HTML -output_dir: "output" - -# Data storage directory -data_dir: "data" - -# Repositories to monitor -repositories: - # Single repository with all years - - name: "user1" - url: "https://github.com/user1/advent-of-code" - type: "single" - local_path: "repos/user1" - - # Multiple repositories, one per year - - name: "user2" - type: "multi-year" - years: - - year: 2023 - url: "https://github.com/user2/aoc-2023" - local_path: "repos/user2-2023" - - year: 2024 - url: "https://github.com/user2/aoc-2024" - local_path: "repos/user2-2024" - -# Optional: Filter specific years to compare -compare_years: [2023, 2024] - -# Optional: Filter specific days to compare -# compare_days: [1, 2, 3, 4, 5] -``` +See `config.yaml` for configuration options, including: +- Repository URLs and paths +- Docker/Podman container settings +- Build and registry cache directories +- Resource limits ## Usage -### Run Once - -To sync repositories and generate a report once: - ```bash -python aocsync.py --once +# Run sync (only processes changed repositories) +python3 aocsync.py + +# Force rerun all benchmarks +python3 aocsync.py --force ``` - -### Force Rerun All Days - -To force rerun all days even if repositories haven't changed: - -```bash -python aocsync.py --once --force -``` - -or - -```bash -python aocsync.py --once --rerun-all -``` - -This is useful for refreshing all data or testing changes to the script. - -### Continuous Polling - -To continuously poll repositories (default): - -```bash -python aocsync.py -``` - -Or with a custom config file: - -```bash -python aocsync.py --config myconfig.yaml -``` - -## Output - -- **Database**: Performance data is stored in `data/results.db` (SQLite) -- **HTML Report**: Generated at `output/index.html` (configurable via `output_dir`) - -The HTML report includes: -- Performance comparison tables for each year/day/part -- Visual highlighting of fastest and slowest implementations -- Relative speed comparisons (X times faster/slower) -- Responsive design for viewing on any device - -## How It Works - -1. **Git Polling**: Checks each configured repository for changes by comparing local and remote commits -2. **Repository Update**: Clones new repositories or updates existing ones when changes are detected -3. **Day Detection**: Automatically finds implemented days by scanning for `day*.rs` files and Cargo.toml entries -4. **Runtime Measurement**: Runs `cargo aoc --day X` for each implemented day -5. **Parsing**: Extracts timing data from cargo-aoc output (handles nanoseconds, microseconds, milliseconds, seconds) -6. **Storage**: Stores results in SQLite database with timestamps -7. **Report Generation**: Generates HTML comparison page showing latest results - -## Repository Structure Detection - -The script automatically detects: -- **Year**: From repository path, name, or Cargo.toml -- **Days**: From `src/bin/day*.rs`, `src/day*.rs`, or Cargo.toml entries -- **Parts**: From cargo-aoc benchmark output (Part 1, Part 2) - -## Troubleshooting - -### Cargo-aoc not found -Ensure `cargo-aoc` is installed and in your PATH: -```bash -cargo install cargo-aoc -``` - -### Git authentication issues -For private repositories, ensure your git credentials are configured or use SSH URLs. - -### Benchmark timeouts -If benchmarks take too long, the script has a 5-minute timeout per day. Adjust in the code if needed. - -### Missing performance data -If some users/days/parts don't show up: -- Check that `cargo aoc --day X` runs successfully in the repository -- Verify the repository structure matches cargo-aoc conventions -- Ensure `cargo aoc` outputs timing information (check if it's configured to show runtime) -- Check logs for parsing errors - -## License - -[Your License Here] diff --git a/aocsync.py b/aocsync.py index c86b451..92ea197 100755 --- a/aocsync.py +++ b/aocsync.py @@ -95,7 +95,7 @@ class Config: 'registry_cache_dir': docker_config.get('registry_cache_dir', ''), 'memory': docker_config.get('memory', '2g'), 'cpus': docker_config.get('cpus', '2'), - 'image': docker_config.get('image', 'rust:latest') + 'image': docker_config.get('image', 'aocsync:latest') } @@ -618,6 +618,11 @@ class CargoAOCRunner: try: # Build Podman command + podman_image = docker_config.get('image', 'aocsync:latest') + + # Check if image has cargo-aoc pre-installed (aocsync:latest) or needs installation + needs_cargo_aoc_install = podman_image == 'rust:latest' or not podman_image.startswith('aocsync') + podman_cmd = [ 'podman', 'run', '--rm', # Remove container after execution @@ -642,11 +647,32 @@ class CargoAOCRunner: # Use tmpfs for registry cache (cleared after each run) podman_cmd.extend(['--tmpfs', '/root/.cargo/registry:rw,noexec,nosuid,size=100m']) + # Handle cargo-aoc installation location (only needed if installing on-the-fly) + if needs_cargo_aoc_install: + # Need writable /root/.cargo/bin to install cargo-aoc + if registry_cache_dir: + # If using persistent registry cache, also persist cargo bin + cargo_bin_path = Path(registry_cache_dir).parent / 'cargo-bin' + cargo_bin_path.mkdir(parents=True, exist_ok=True) + podman_cmd.extend(['-v', f'{cargo_bin_path}:/root/.cargo/bin:rw']) + else: + # Use tmpfs for cargo bin (will be cleared, but allows installation) + podman_cmd.extend(['--tmpfs', '/root/.cargo/bin:rw,noexec,nosuid,size=50m']) + # Add Podman image and command - podman_cmd.extend([ - docker_config.get('image', 'rust:latest'), - 'cargo', 'aoc', '--day', str(day) - ]) + if needs_cargo_aoc_install: + # Install cargo-aoc first if not pre-installed (slower, but works with rust:latest) + # Check if already installed to avoid reinstalling every time + podman_cmd.extend([ + podman_image, + 'sh', '-c', 'if ! command -v cargo-aoc >/dev/null 2>&1; then cargo install --quiet cargo-aoc 2>/dev/null || true; fi; cargo aoc --day ' + str(day) + ]) + else: + # Use pre-installed cargo-aoc (faster, requires aocsync:latest image) + podman_cmd.extend([ + podman_image, + 'cargo', 'aoc', '--day', str(day) + ]) # Set CARGO_TARGET_DIR to use the mounted build directory env = os.environ.copy() @@ -723,7 +749,7 @@ class CargoAOCRunner: 'registry_cache_dir': '', 'memory': '2g', 'cpus': '2', - 'image': 'rust:latest' + 'image': 'aocsync:latest' } result = CargoAOCRunner._run_cargo_aoc_in_container(work_dir, day, repo_root, docker_config) diff --git a/config.yaml b/config.yaml index ce88132..07c658b 100644 --- a/config.yaml +++ b/config.yaml @@ -27,8 +27,10 @@ docker: memory: "2g" # Memory limit cpus: "2" # CPU limit - # Podman image to use - image: "rust:latest" + # Podman image to use (should have cargo-aoc installed) + # Default: "aocsync:latest" (build with: podman build -t aocsync:latest -f Dockerfile .) + # Alternative: "rust:latest" (will install cargo-aoc on first run, slower) + image: "aocsync:latest" # Repositories to monitor repositories: