Use custom image w/ cargo-aoc
This commit is contained in:
parent
91d5db6453
commit
cb0dc0d879
173
README.md
173
README.md
@ -1,174 +1,33 @@
|
||||
# AOC Sync
|
||||
|
||||
A Python script that polls multiple git repositories containing Advent of Code implementations written in Rust using `cargo-aoc` format. It automatically updates repositories when changes are detected, runs benchmarks, and generates a beautiful HTML comparison page showing performance metrics across users, years, and days.
|
||||
A tool to poll multiple git repositories containing Advent of Code implementations and generate performance comparison reports.
|
||||
|
||||
## Features
|
||||
## Building the Podman Image
|
||||
|
||||
- **Automatic Git Polling**: Monitors multiple repositories for changes
|
||||
- **Flexible Repository Structure**: Supports both single-repo (all years) and multi-repo (one per year) configurations
|
||||
- **Automatic Runtime Measurement**: Runs `cargo aoc` for all implemented days and parses runtime information
|
||||
- **Performance Parsing**: Extracts timing data from cargo-aoc output
|
||||
- **Data Storage**: SQLite database for historical performance data
|
||||
- **HTML Reports**: Beautiful, responsive HTML comparison pages
|
||||
- **Gap Handling**: Gracefully handles missing years, days, and parts
|
||||
- **Configurable Comparisons**: Filter by specific years and days
|
||||
The default configuration uses a custom Podman image (`aocsync:latest`) that has `cargo-aoc` pre-installed for faster execution.
|
||||
|
||||
## Requirements
|
||||
To build the image:
|
||||
|
||||
- Python 3.7+
|
||||
- Git
|
||||
- Rust and Cargo
|
||||
- `cargo-aoc` installed (install with `cargo install cargo-aoc`)
|
||||
|
||||
## Installation
|
||||
|
||||
1. Clone this repository:
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd aocsync
|
||||
podman build -t aocsync:latest -f Dockerfile .
|
||||
```
|
||||
|
||||
2. Install Python dependencies:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Ensure `cargo-aoc` is installed:
|
||||
```bash
|
||||
cargo install cargo-aoc
|
||||
```
|
||||
|
||||
Note: The script runs `cargo aoc` (not `cargo aoc bench`) and parses runtime information from the output.
|
||||
Alternatively, you can use the `rust:latest` image, but it will install `cargo-aoc` on each run (slower).
|
||||
|
||||
## Configuration
|
||||
|
||||
Edit `config.yaml` to configure repositories to monitor:
|
||||
|
||||
```yaml
|
||||
# Poll interval in seconds
|
||||
poll_interval: 300
|
||||
|
||||
# Output directory for generated HTML
|
||||
output_dir: "output"
|
||||
|
||||
# Data storage directory
|
||||
data_dir: "data"
|
||||
|
||||
# Repositories to monitor
|
||||
repositories:
|
||||
# Single repository with all years
|
||||
- name: "user1"
|
||||
url: "https://github.com/user1/advent-of-code"
|
||||
type: "single"
|
||||
local_path: "repos/user1"
|
||||
|
||||
# Multiple repositories, one per year
|
||||
- name: "user2"
|
||||
type: "multi-year"
|
||||
years:
|
||||
- year: 2023
|
||||
url: "https://github.com/user2/aoc-2023"
|
||||
local_path: "repos/user2-2023"
|
||||
- year: 2024
|
||||
url: "https://github.com/user2/aoc-2024"
|
||||
local_path: "repos/user2-2024"
|
||||
|
||||
# Optional: Filter specific years to compare
|
||||
compare_years: [2023, 2024]
|
||||
|
||||
# Optional: Filter specific days to compare
|
||||
# compare_days: [1, 2, 3, 4, 5]
|
||||
```
|
||||
See `config.yaml` for configuration options, including:
|
||||
- Repository URLs and paths
|
||||
- Docker/Podman container settings
|
||||
- Build and registry cache directories
|
||||
- Resource limits
|
||||
|
||||
## Usage
|
||||
|
||||
### Run Once
|
||||
|
||||
To sync repositories and generate a report once:
|
||||
|
||||
```bash
|
||||
python aocsync.py --once
|
||||
# Run sync (only processes changed repositories)
|
||||
python3 aocsync.py
|
||||
|
||||
# Force rerun all benchmarks
|
||||
python3 aocsync.py --force
|
||||
```
|
||||
|
||||
### Force Rerun All Days
|
||||
|
||||
To force rerun all days even if repositories haven't changed:
|
||||
|
||||
```bash
|
||||
python aocsync.py --once --force
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
python aocsync.py --once --rerun-all
|
||||
```
|
||||
|
||||
This is useful for refreshing all data or testing changes to the script.
|
||||
|
||||
### Continuous Polling
|
||||
|
||||
To continuously poll repositories (default):
|
||||
|
||||
```bash
|
||||
python aocsync.py
|
||||
```
|
||||
|
||||
Or with a custom config file:
|
||||
|
||||
```bash
|
||||
python aocsync.py --config myconfig.yaml
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
- **Database**: Performance data is stored in `data/results.db` (SQLite)
|
||||
- **HTML Report**: Generated at `output/index.html` (configurable via `output_dir`)
|
||||
|
||||
The HTML report includes:
|
||||
- Performance comparison tables for each year/day/part
|
||||
- Visual highlighting of fastest and slowest implementations
|
||||
- Relative speed comparisons (X times faster/slower)
|
||||
- Responsive design for viewing on any device
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Git Polling**: Checks each configured repository for changes by comparing local and remote commits
|
||||
2. **Repository Update**: Clones new repositories or updates existing ones when changes are detected
|
||||
3. **Day Detection**: Automatically finds implemented days by scanning for `day*.rs` files and Cargo.toml entries
|
||||
4. **Runtime Measurement**: Runs `cargo aoc --day X` for each implemented day
|
||||
5. **Parsing**: Extracts timing data from cargo-aoc output (handles nanoseconds, microseconds, milliseconds, seconds)
|
||||
6. **Storage**: Stores results in SQLite database with timestamps
|
||||
7. **Report Generation**: Generates HTML comparison page showing latest results
|
||||
|
||||
## Repository Structure Detection
|
||||
|
||||
The script automatically detects:
|
||||
- **Year**: From repository path, name, or Cargo.toml
|
||||
- **Days**: From `src/bin/day*.rs`, `src/day*.rs`, or Cargo.toml entries
|
||||
- **Parts**: From cargo-aoc benchmark output (Part 1, Part 2)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Cargo-aoc not found
|
||||
Ensure `cargo-aoc` is installed and in your PATH:
|
||||
```bash
|
||||
cargo install cargo-aoc
|
||||
```
|
||||
|
||||
### Git authentication issues
|
||||
For private repositories, ensure your git credentials are configured or use SSH URLs.
|
||||
|
||||
### Benchmark timeouts
|
||||
If benchmarks take too long, the script has a 5-minute timeout per day. Adjust in the code if needed.
|
||||
|
||||
### Missing performance data
|
||||
If some users/days/parts don't show up:
|
||||
- Check that `cargo aoc --day X` runs successfully in the repository
|
||||
- Verify the repository structure matches cargo-aoc conventions
|
||||
- Ensure `cargo aoc` outputs timing information (check if it's configured to show runtime)
|
||||
- Check logs for parsing errors
|
||||
|
||||
## License
|
||||
|
||||
[Your License Here]
|
||||
|
||||
32
aocsync.py
32
aocsync.py
@ -95,7 +95,7 @@ class Config:
|
||||
'registry_cache_dir': docker_config.get('registry_cache_dir', ''),
|
||||
'memory': docker_config.get('memory', '2g'),
|
||||
'cpus': docker_config.get('cpus', '2'),
|
||||
'image': docker_config.get('image', 'rust:latest')
|
||||
'image': docker_config.get('image', 'aocsync:latest')
|
||||
}
|
||||
|
||||
|
||||
@ -618,6 +618,11 @@ class CargoAOCRunner:
|
||||
|
||||
try:
|
||||
# Build Podman command
|
||||
podman_image = docker_config.get('image', 'aocsync:latest')
|
||||
|
||||
# Check if image has cargo-aoc pre-installed (aocsync:latest) or needs installation
|
||||
needs_cargo_aoc_install = podman_image == 'rust:latest' or not podman_image.startswith('aocsync')
|
||||
|
||||
podman_cmd = [
|
||||
'podman', 'run',
|
||||
'--rm', # Remove container after execution
|
||||
@ -642,9 +647,30 @@ class CargoAOCRunner:
|
||||
# Use tmpfs for registry cache (cleared after each run)
|
||||
podman_cmd.extend(['--tmpfs', '/root/.cargo/registry:rw,noexec,nosuid,size=100m'])
|
||||
|
||||
# Handle cargo-aoc installation location (only needed if installing on-the-fly)
|
||||
if needs_cargo_aoc_install:
|
||||
# Need writable /root/.cargo/bin to install cargo-aoc
|
||||
if registry_cache_dir:
|
||||
# If using persistent registry cache, also persist cargo bin
|
||||
cargo_bin_path = Path(registry_cache_dir).parent / 'cargo-bin'
|
||||
cargo_bin_path.mkdir(parents=True, exist_ok=True)
|
||||
podman_cmd.extend(['-v', f'{cargo_bin_path}:/root/.cargo/bin:rw'])
|
||||
else:
|
||||
# Use tmpfs for cargo bin (will be cleared, but allows installation)
|
||||
podman_cmd.extend(['--tmpfs', '/root/.cargo/bin:rw,noexec,nosuid,size=50m'])
|
||||
|
||||
# Add Podman image and command
|
||||
if needs_cargo_aoc_install:
|
||||
# Install cargo-aoc first if not pre-installed (slower, but works with rust:latest)
|
||||
# Check if already installed to avoid reinstalling every time
|
||||
podman_cmd.extend([
|
||||
docker_config.get('image', 'rust:latest'),
|
||||
podman_image,
|
||||
'sh', '-c', 'if ! command -v cargo-aoc >/dev/null 2>&1; then cargo install --quiet cargo-aoc 2>/dev/null || true; fi; cargo aoc --day ' + str(day)
|
||||
])
|
||||
else:
|
||||
# Use pre-installed cargo-aoc (faster, requires aocsync:latest image)
|
||||
podman_cmd.extend([
|
||||
podman_image,
|
||||
'cargo', 'aoc', '--day', str(day)
|
||||
])
|
||||
|
||||
@ -723,7 +749,7 @@ class CargoAOCRunner:
|
||||
'registry_cache_dir': '',
|
||||
'memory': '2g',
|
||||
'cpus': '2',
|
||||
'image': 'rust:latest'
|
||||
'image': 'aocsync:latest'
|
||||
}
|
||||
result = CargoAOCRunner._run_cargo_aoc_in_container(work_dir, day, repo_root, docker_config)
|
||||
|
||||
|
||||
@ -27,8 +27,10 @@ docker:
|
||||
memory: "2g" # Memory limit
|
||||
cpus: "2" # CPU limit
|
||||
|
||||
# Podman image to use
|
||||
image: "rust:latest"
|
||||
# Podman image to use (should have cargo-aoc installed)
|
||||
# Default: "aocsync:latest" (build with: podman build -t aocsync:latest -f Dockerfile .)
|
||||
# Alternative: "rust:latest" (will install cargo-aoc on first run, slower)
|
||||
image: "aocsync:latest"
|
||||
|
||||
# Repositories to monitor
|
||||
repositories:
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user