AnalyzeProjects

AnalyzeProjects evaluates a controlled portfolio of local projects with OpenAI and produces per-project JSON plus a compiled Markdown summary. ProjectIndex.md is the sole inclusion authority: the analyzer does not discover extra projects from drive roots.

Current evaluation inventory

The checked-in index contains exactly 27 projects:

Group	Count	Evaluation basis
DRS	9	Desktop Application Release Standard
CTS	8	Command Tool Standard
WDS	0	Website Development Standard
LDS	0	Library Development Standard
Standards	10	Internal clarity, completeness, consistency, and implementability

ConversionTools and CratesDataset are included in the CTS evaluation inventory. AptlantisLogos is evaluated as CTS because it is a whole-drive support/standardization project, not a WDS website project. The ten standards are read from D:\.city_hall, not the historical .city_hall tree, and exclude D:\.city_hall\Blanks, whose contents are manifest templates rather than an evaluable standard document.

How evaluation works

1. Parse and validate ProjectIndex.md.
2. Require 27 unique, existing project directories with the exact 9/8/0/0/10 group distribution.
3. Select <folder>.manifest.toml when present. A single alternative direct manifest is accepted; a manifest is optional.
4. Read each DRS, CTS, and WDS governing standard once.
5. Load PROJECT-READMAP.toml when present, reserve mandatory read-map entries first, and sample declared priority, evidence, and secondary roots within the configured file and character budgets.
6. Send the project evidence and its group-specific governance context to gpt-5-mini-2025-08-07.
7. Validate and normalize the JSON response, write individual results, then compile the summary.

Standards projects use a separate rubric. Application release requirements are not automatically imposed on governance documents.

Sampling and truncation safety

Every transferred file block is labeled either complete file or sampled excerpt, including the exact excerpt size when shortened. An excerpt ending is a context-transfer boundary, not evidence of a damaged repository file. The prompt prohibits treating that boundary as a defect, and post-processing removes transfer-related truncation claims from actionable findings.

When a project supplies PROJECT-READMAP.toml, the analyzer treats it as the project discovery contract. Mandatory entries are sampled before exploratory budget is spent; declared priority files and evidence/secondary roots define the candidate pool; declared excludes are merged with configured excluded directories. Projects without a read map keep the legacy relevance-scored fallback walk.

Each result records:

• project_group
• governing_standard
• repository_url
• manifest_path when available
• sampling.sampled_files, sampled characters, and configured limits

The original assessment fields remain compatible with existing dashboard consumers.

Configuration and credentials

Copy config.example.toml to the ignored local config.toml when bootstrapping a checkout. Set the OpenAI key in the process environment; credentials are not read from configuration files:

$env:OPENAI_API_KEY = [Environment]::GetEnvironmentVariable("OPENAI_API_KEY", "Machine")

The analyzer fails before scanning when the key is unavailable. If a credential was previously stored in config.toml, rotate it through the OpenAI dashboard. If OpenAI rejects the configured key with 401 or 403, the run stops immediately instead of retrying every project.

Run

From D:\CTS\Analyze Projects:

python Summarizer.py

The configured output remains under the existing dashboard summary location. Recent result files may be reused when skip_already_processed is enabled.

Comparison runs

A full run re-evaluates every project against the model. To instead check the portfolio for drift since the last run — without re-spending a full evaluation on projects that have not changed — use comparison mode:

python Summarizer.py --compare

For each project, the analyzer samples current evidence locally (no API cost) and hashes it into a content_fingerprint. If the fingerprint matches the project's last saved result, the model is not called at all and the previous evaluation is carried forward with comparison_status: "unchanged". If the content changed, the model is sent the same governance rubric plus the previous evaluation, and asked to reconcile only what has changed rather than re-assessing from scratch; the result is tagged comparison_status: "changed" (or "new" if there was no previous result) and records a previous_snapshot of the prior status/completion for reference.

Comparison runs write an additional comparison_summary.md alongside the usual summary.md, listing unchanged, changed, and new projects and the status/completion delta for each. Set mode = "compare" in config.toml to make comparison the default, or pass --full on the command line to force a full re-evaluation regardless of config.

Verification

The test suite uses temporary directories and mocked HTTP requests; it does not make paid API calls:

python -m unittest discover -s tests -v

To inspect the index without invoking the model:

python -c "import Summarizer; c=Summarizer.load_config('config.toml'); print([(t.project_group, t.project_name) for t in Summarizer.discover_projects(c)])"

Malformed entries, duplicate paths, missing directories, missing standards, and incorrect group totals are fatal rather than silently skipped.