Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e0d2d3e
Add SCL expected configuration factory
masarray Jul 15, 2026
afb3647
Add configuration comparison summaries
masarray Jul 15, 2026
8392588
Integrate SCL configuration comparison into SV observations
masarray Jul 15, 2026
09c03c9
Persist SV configuration comparison in observation snapshots
masarray Jul 15, 2026
3537431
Add deterministic P3C configuration comparison tests
masarray Jul 15, 2026
9a90c91
Fix P3C integration test frame construction
masarray Jul 15, 2026
f85c9a6
Document completed SCL configuration integration
masarray Jul 15, 2026
755990b
Clean SV observation consistency check
masarray Jul 15, 2026
433d1eb
Add profile detection to SV observation snapshots
masarray Jul 15, 2026
83e954a
Add single-reset collection updates for Subscriber visuals
masarray Jul 15, 2026
959fd55
Expose compact Subscriber analysis state
masarray Jul 15, 2026
bc6a168
Add compact profile and evidence state to Subscriber streams
masarray Jul 15, 2026
65480aa
Wire compact P3D state into Subscriber refresh pipeline
masarray Jul 15, 2026
09d80c6
Add compact P3D status and expandable evidence panel
masarray Jul 15, 2026
9c8c5aa
Add deterministic P3D presentation contract tests
masarray Jul 15, 2026
ebaddf3
Expose bulk collections through stable read-only bindings
masarray Jul 15, 2026
1066062
Document compact Subscriber evidence UX
masarray Jul 15, 2026
b1600a1
Use a two-second Subscriber observation window
masarray Jul 15, 2026
d9d3ffe
Lock the two-second Subscriber observation defaults
masarray Jul 15, 2026
cb06dcb
Add deterministic SV evidence report schema and serializers
masarray Jul 15, 2026
03455ce
Build Subscriber evidence reports from runtime and observation snapshots
masarray Jul 15, 2026
4cbd2b7
Export paired Markdown and JSON SV evidence reports
masarray Jul 15, 2026
ef1d1ba
Test SV evidence report JSON and Markdown contracts
masarray Jul 15, 2026
8ce3085
Embed CI source revision in application informational version
masarray Jul 15, 2026
22b62c7
Document Subscriber evidence report contract
masarray Jul 15, 2026
ff2282a
Clarify unknown values in SV report schema
masarray Jul 15, 2026
4b1c682
Add P3F evidence report comparison engine
masarray Jul 15, 2026
b1bf78e
Add deterministic P3F comparison tests
masarray Jul 15, 2026
3d1ec20
Add P3F evidence comparison action
masarray Jul 15, 2026
a8431f4
Wire P3F report comparison workflow
masarray Jul 15, 2026
c0ed7b4
Document P3F evidence comparison workflow
masarray Jul 15, 2026
612a196
Add temporary P3F compile diagnostics
masarray Jul 15, 2026
44aaba7
Fix temporary compiler log capture
masarray Jul 15, 2026
79dc2b5
Harden P3F evidence comparator
masarray Jul 15, 2026
524a298
Add P3F comparison serializers
masarray Jul 15, 2026
cd881d5
Remove temporary P3F compile diagnostics
masarray Jul 15, 2026
a394de6
Add one-shot PR 39 review fix workflow
masarray Jul 15, 2026
142ab44
Address PR 39 evidence binding and report review feedback
github-actions[bot] Jul 15, 2026
2c1cb51
Remove one-shot PR 39 review fix workflow
masarray Jul 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
<RestoreLockedMode Condition="'$(CI)' == 'true'">true</RestoreLockedMode>
<VersionPrefix>0.4.0</VersionPrefix>
<SourceRevisionId Condition="'$(SourceRevisionId)' == '' and '$(GITHUB_SHA)' != ''">$(GITHUB_SHA)</SourceRevisionId>
<IncludeSourceRevisionInInformationalVersion>true</IncludeSourceRevisionInInformationalVersion>
<RepositoryUrl>https://github.com/masarray/arsvin</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PackageLicenseExpression>GPL-3.0-or-later</PackageLicenseExpression>
<Authors>Ari Sulistiono</Authors>
<Company>Ari Sulistiono</Company>
<Copyright>Copyright © 2026 Ari Sulistiono</Copyright>
</PropertyGroup>
</Project>
</Project>
110 changes: 93 additions & 17 deletions docs/sv-profile-infrastructure.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Imported PCAP ────┘ ↓
└── SvConfigurationComparer
```

The profile infrastructure is part of the shared `ARSVIN.Engine` assembly and has no dependency on WPF or Npcap. Subscriber live capture and PCAP replay now use the same manager and stable stream identity, so both paths produce the same facts and diagnostics contract.
The profile infrastructure is part of the shared `ARSVIN.Engine` assembly and has no dependency on WPF or Npcap. Subscriber live capture and PCAP replay use the same manager and stable stream identity, so both paths produce the same facts and diagnostics contract.

## Observed facts

Expand Down Expand Up @@ -75,30 +75,106 @@ A high score is an engineering classification result, not a conformance certific

## Configuration-versus-wire comparison

`SvConfigurationComparer` compares configured expectations with observed facts for addressing, identifiers, dataset, revision, packing, payload, and declared sampling fields.
`SvExpectedStreamConfigurationFactory` converts the SCL-bound `SampledValuesPublisherProfile` into a transport-neutral `SvExpectedStreamConfiguration`. The expected configuration includes addressing, identifiers, revision, packing, payload length, declared sampling fields, and ordered dataset signature.

`SvStreamObservationManager` compares that expected configuration with the rolling `SvObservedStreamFacts` for every bound live or PCAP stream. The resulting immutable snapshot carries:

- the expected configuration,
- field-level findings,
- comparison mode,
- exact/warning/error counts,
- a compact summary such as `Exact`, `2 warnings`, or `1 error`.

Two modes are available:

- **Strict** — mismatches become errors suitable for validation and live-transmission preflight.
- **Compatible** — mismatches become warnings suitable for troubleshooting and unfamiliar devices.
- **Compatible** — the Subscriber default; mismatches become warnings suitable for troubleshooting and unfamiliar devices.
- **Strict** — explicit opt-in for validation, preflight, and formal test cases; mismatches become errors.

Neither mode stops receive-side capture or decoding. Unknown and conflicting streams remain visible.

## Current integration boundary
Before accepting an SCL candidate, the observation manager requires APPID, destination MAC, and VLAN to identify the same configured stream. A candidate that fails this address gate is rejected instead of contaminating observed facts with the wrong dataset layout.

## Subscriber compact state

The selected stream exposes one compact analysis strip instead of permanent large evidence cards:

```text
PROFILE Generic Layer-2 SV
CONFIDENCE Unknown · insufficient evidence
SCL MATCH Exact | N warnings | N errors
WINDOW duration · observed samples
```

Detailed detector evidence, configuration findings, and observation diagnostics remain collapsed behind an expandable evidence panel. Capture and visualization continue without requiring repeated manual selection.

Waveform, phasor, and RMS collections use one reset notification per UI refresh. The visual layer withholds partial waveform windows until a complete two-cycle set is available, then retains the most recent complete window if the next refresh is incomplete. Compatible SCL warnings remain warnings and do not force the stream into a blocking `BAD` state.

## Evidence report bundle

Subscriber Export creates two files from the same report snapshot:

- a Markdown engineering report for review and handover,
- a JSON evidence document using schema `arsvin.sv-subscriber-evidence/v1` for automation, archival, and later comparison.

`SvStreamObservationManager` now:
Both files include:

- generation time, product version, informational version, repository, and source commit,
- live, PCAP, or mixed input provenance,
- SCL path, adapter, user filter, capture duration, frame counts, parse errors, and filtered-frame counts,
- per-stream identity, runtime integrity, quality, cursor state, waveform readiness, and phasors,
- first and last observation timestamps, frame and sample counts, measured rates, sample-counter transitions, and wrap evidence,
- every stable observed fact with its provenance (`WireObserved`, `CaptureCalculated`, `SclDerived`, or trusted context),
- profile definition, evidence maturity, confidence, weighted match evidence, and source metadata,
- expected SCL configuration, comparison mode, exact/warning/error summary, and field-level findings,
- runtime and observation diagnostics.

Unknown nullable values remain explicit `null`; unknown text fields remain empty strings in JSON and render as `unknown` in Markdown. They are not silently removed or interpreted as matches. The report schema validates generation time, product identity, stream count, and unique stream keys before serialization.

GitHub Actions injects `GITHUB_SHA` into `SourceRevisionId`; the .NET informational version therefore carries the build commit for release and CI artifacts. Local builds without source revision metadata report the commit as `unknown` rather than inventing one.

## Evidence report comparison

The Subscriber **Compare** action accepts a baseline and candidate `arsvin.sv-subscriber-evidence/v1` JSON report and writes a paired comparison bundle:

- Markdown for engineering review,
- JSON using schema `arsvin.sv-subscriber-evidence-comparison/v1` for automation and regression gates.

Comparison uses the full stable stream key first. Unmatched streams are then paired through a logical identity made from APPID, destination MAC, VLAN, `svID`, and dataset reference. Source MAC is deliberately excluded from that fallback identity so publisher/NIC failover is reported as a source change rather than a false removed-plus-added stream.

The comparison classifies:

- report-level schema, software, commit, capture-source, SCL-source, and health changes,
- added, removed, changed, and unchanged logical streams,
- health regression or recovery,
- source-MAC failover,
- `confRev`, ASDU packing, sample-rate, and sample-mode changes,
- sequence gaps, duplicates, out-of-order frames, payload issues, and SCL mismatches,
- observation-window size and measured-rate changes,
- SCL binding and configuration-comparison regression,
- profile identity and confidence changes,
- payload, counter-wrap, nominal-frequency, dataset-signature, provenance, and diagnostic changes.

Changes are assigned `Info`, `Warning`, or `Error` severity. Removed streams, health transitions to `BAD`/`ERROR`, new out-of-order or payload failures, blocking configuration errors, profile conflicts, and dataset-signature changes are treated as high-signal regressions. Measured-rate changes within the one-percent comparison tolerance do not create false warnings.

## Current integration boundary

- accepts parsed frames from live Npcap capture and classic-PCAP replay;
- groups frames by source, destination, VLAN, APPID, `svID`, and dataset reference;
- deliberately keeps `confRev` outside the key so revision changes remain observable;
- retains input provenance (`LiveCapture` or `PcapReplay`);
- adds SCL-derived dataset signatures when a stream is bound; and
- exposes immutable rolling facts to the Subscriber runtime snapshot.
`SvStreamObservationManager` and Subscriber now:

- accept parsed frames from live Npcap capture and classic-PCAP replay;
- group frames by source, destination, VLAN, APPID, `svID`, and dataset reference;
- deliberately keep `confRev` outside the key so revision changes remain observable;
- retain input provenance (`LiveCapture` or `PcapReplay`);
- validate the SCL candidate against the observed address tuple;
- convert a bound SCL stream into `SvExpectedStreamConfiguration`;
- run Compatible comparison by default, with Strict available per observation call;
- evaluate the built-in evidence-aware profile catalog;
- expose compact profile, confidence, SCL match, and observation-window state;
- expose detailed evidence only on demand;
- keep full-window waveform, phasor, and RMS visualization stable through bulk collection refreshes;
- serialize the same observation, profile, configuration, provenance, source, and build evidence into paired Markdown and JSON reports; and
- compare baseline and candidate evidence reports through deterministic, severity-ranked Markdown and JSON output.

## Next integration

1. Convert selected SCL stream bindings into `SvExpectedStreamConfiguration`.
2. Run strict or compatible configuration comparison per observed stream.
3. Present one compact profile/confidence and SCL-match state per selected stream.
4. Show evidence and mismatch details on demand rather than adding permanent visual noise.
5. Add profile-specific definitions only after source review and deterministic evidence.
1. Add profile-specific definitions only after source review and deterministic evidence.
2. Accumulate real comparison history before defining organization-specific pass/fail policy or CI blocking thresholds.
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,27 @@ public sealed record SvConfigurationComparisonResult
= Array.Empty<SvConfigurationFinding>();

public bool HasBlockingErrors => Findings.Any(item => item.Severity == SvConfigurationFindingSeverity.Error);
public int InfoCount => Findings.Count(item => item.Severity == SvConfigurationFindingSeverity.Info);
public int ErrorCount => Findings.Count(item => item.Severity == SvConfigurationFindingSeverity.Error);
public int WarningCount => Findings.Count(item => item.Severity == SvConfigurationFindingSeverity.Warning);
public bool IsExactMatch => Findings.Count == 0;

public string Summary
{
get
{
if (IsExactMatch)
return "Exact";
if (ErrorCount > 0)
return CountText(ErrorCount, "error");
if (WarningCount > 0)
return CountText(WarningCount, "warning");
return CountText(InfoCount, "info");
}
}

private static string CountText(int count, string label)
=> $"{count} {label}{(count == 1 ? string.Empty : "s")}";
}

public sealed class SvConfigurationComparer
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
using AR.Iec61850.SampledValues;
using AR.Iec61850.Scl;

namespace AR.Iec61850.SampledValues.Profiles;

public static class SvExpectedStreamConfigurationFactory
{
public static SvExpectedStreamConfiguration Create(SampledValuesPublisherProfile profile)
{
ArgumentNullException.ThrowIfNull(profile);

return new SvExpectedStreamConfiguration
{
EtherType = 0x88BA,
AppId = profile.AppId,
DestinationMac = profile.Destination.ToString(),
VlanId = profile.Vlan?.VlanId,
VlanPriority = profile.Vlan?.PriorityCodePoint,
SvId = profile.Stream.SvId,
DataSetReference = profile.Stream.DataSetReference,
ConfigurationRevision = profile.Stream.ConfigurationRevision,
AsduPerFrame = profile.AsduPerFrame,
PayloadBytesPerAsdu = profile.PayloadLayout.PayloadByteLength,
DeclaredSampleRate = profile.Stream.SampleRate == 0
? null
: profile.Stream.SampleRate,
DeclaredSampleMode = MapSampleMode(profile.Stream.SampleMode),
DataSetSignature = profile.Entries.Select(ToSignature).ToArray()
};
}

private static ushort? MapSampleMode(string sampleMode)
{
if (string.IsNullOrWhiteSpace(sampleMode))
return null;

return sampleMode.Trim() switch
{
"SmpPerPeriod" => 0,
"SmpPerSec" => 1,
"SecPerSmp" => 2,
_ => null
};
}

private static SvDatasetElementSignature ToSignature(SclDataSetEntry entry)
=> new()
{
BType = entry.BType,
Cdc = entry.Cdc,
IsQuality = entry.IsQuality,
IsTimestamp = entry.IsTimestamp
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -55,16 +55,25 @@ public sealed record SvStreamObservationSnapshot
public SvObservationInputKind LastInputKind { get; init; }
public bool IsBoundToScl { get; init; }
public string ControlBlockReference { get; init; } = string.Empty;
public SvExpectedStreamConfiguration? ExpectedConfiguration { get; init; }
public SvConfigurationComparisonResult? ConfigurationComparison { get; init; }
public SvProfileDetectionResult? ProfileDetection { get; init; }
public string ConfigurationMatchSummary => ConfigurationComparison?.Summary ?? "Not configured";
public IReadOnlyList<string> Diagnostics { get; init; } = Array.Empty<string>();
}

public sealed class SvStreamObservationManager
{
public const int DefaultMaximumObservations = 12_288;
public static readonly TimeSpan DefaultMaximumAge = TimeSpan.FromSeconds(2);

private sealed class StreamState
{
private readonly object _gate = new();
private readonly HashSet<SvObservationInputKind> _inputKinds = [];
private readonly Queue<string> _diagnostics = new();
private SvExpectedStreamConfiguration? _expectedConfiguration;
private SvComparisonMode _comparisonMode = SvComparisonMode.Compatible;

public StreamState(int maximumObservations, TimeSpan maximumAge)
{
Expand All @@ -80,6 +89,7 @@ public void Add(
SvFrameObservation observation,
SvObservationInputKind inputKind,
SampledValuesPublisherProfile? profile,
SvComparisonMode comparisonMode,
IEnumerable<string> diagnostics)
{
Accumulator.Add(observation);
Expand All @@ -91,6 +101,8 @@ public void Add(
{
IsBoundToScl = true;
ControlBlockReference = profile.Stream.ControlBlockReference;
_expectedConfiguration = SvExpectedStreamConfigurationFactory.Create(profile);
_comparisonMode = comparisonMode;
}

foreach (var diagnostic in diagnostics.Where(item => !string.IsNullOrWhiteSpace(item)))
Expand All @@ -110,6 +122,16 @@ public SvStreamObservationSnapshot Snapshot(SvObservedStreamKey key)
var facts = Accumulator.BuildFacts();
lock (_gate)
{
var comparison = _expectedConfiguration is null
? null
: new SvConfigurationComparer().Compare(
_expectedConfiguration,
facts,
_comparisonMode);
var profileDetection = new SvProfileDetector().DetectBest(
facts,
SvProfileCatalog.BuiltIn);

return new SvStreamObservationSnapshot
{
Key = key,
Expand All @@ -118,6 +140,9 @@ public SvStreamObservationSnapshot Snapshot(SvObservedStreamKey key)
LastInputKind = LastInputKind,
IsBoundToScl = IsBoundToScl,
ControlBlockReference = ControlBlockReference,
ExpectedConfiguration = _expectedConfiguration,
ConfigurationComparison = comparison,
ProfileDetection = profileDetection,
Diagnostics = facts.Diagnostics.Concat(_diagnostics).Distinct(StringComparer.Ordinal).ToArray()
};
}
Expand All @@ -129,13 +154,13 @@ public SvStreamObservationSnapshot Snapshot(SvObservedStreamKey key)
private readonly TimeSpan _maximumAge;

public SvStreamObservationManager(
int maximumObservations = SvObservationAccumulator.DefaultMaximumObservations,
int maximumObservations = DefaultMaximumObservations,
TimeSpan? maximumAge = null)
{
if (maximumObservations < 2)
throw new ArgumentOutOfRangeException(nameof(maximumObservations));

_maximumAge = maximumAge ?? SvObservationAccumulator.DefaultMaximumAge;
_maximumAge = maximumAge ?? DefaultMaximumAge;
if (_maximumAge <= TimeSpan.Zero)
throw new ArgumentOutOfRangeException(nameof(maximumAge));

Expand All @@ -150,7 +175,8 @@ public bool TryObserve(
SvObservationInputKind inputKind,
SampledValuesPublisherProfile? profile,
out SvStreamObservationSnapshot snapshot,
double? nominalFrequencyHz = null)
double? nominalFrequencyHz = null,
SvComparisonMode comparisonMode = SvComparisonMode.Compatible)
{
ArgumentNullException.ThrowIfNull(frame);
snapshot = new SvStreamObservationSnapshot();
Expand All @@ -161,8 +187,9 @@ public bool TryObserve(
return false;

var key = SvObservedStreamKey.FromFrame(frame);
var diagnostics = ValidateFrameConsistency(asdus);
var signature = profile?.Entries.Select(ToSignature).ToArray()
var diagnostics = ValidateFrameConsistency(asdus).ToList();
var boundProfile = ValidateProfileBinding(frame, profile, diagnostics);
var signature = boundProfile?.Entries.Select(ToSignature).ToArray()
?? Array.Empty<SvDatasetElementSignature>();
var payloadLengths = asdus.Select(item => item.SamplePayload.Length).Distinct().ToArray();
var payloadLength = payloadLengths.Length == 1 ? payloadLengths[0] : first.SamplePayload.Length;
Expand All @@ -189,7 +216,7 @@ public bool TryObserve(
var state = _streams.GetOrAdd(
key,
_ => new StreamState(_maximumObservations, _maximumAge));
state.Add(observation, inputKind, profile, diagnostics);
state.Add(observation, inputKind, boundProfile, comparisonMode, diagnostics);
snapshot = state.Snapshot(key);
return true;
}
Expand All @@ -216,6 +243,29 @@ public IReadOnlyList<SvStreamObservationSnapshot> SnapshotAll()

public void Clear() => _streams.Clear();

private static SampledValuesPublisherProfile? ValidateProfileBinding(
SampledValuesFrame frame,
SampledValuesPublisherProfile? profile,
ICollection<string> diagnostics)
{
if (profile is null)
return null;

var appIdMatches = profile.AppId == frame.AppId;
var destinationMatches = string.Equals(
profile.Destination.ToString(),
frame.Destination.ToString(),
StringComparison.OrdinalIgnoreCase);
var vlanMatches = profile.Vlan?.VlanId == frame.Vlan?.VlanId;
if (appIdMatches && destinationMatches && vlanMatches)
return profile;

diagnostics.Add(
$"Rejected SCL candidate {profile.Stream.ControlBlockReference}: " +
"APPID, destination MAC, and VLAN must identify the same configured stream before comparison.");
return null;
}

private static IReadOnlyList<string> ValidateFrameConsistency(IReadOnlyList<SampledValueAsdu> asdus)
{
var diagnostics = new List<string>();
Expand Down
Loading
Loading