Skip to content

add enabled_logs variable which can be redefed to limit the set of logs that gets JSON streaming logs created for it - #12

Merged
ckreibich merged 5 commits into
corelight:masterfrom
mmguero-dev:master
Dec 3, 2025
Merged

add enabled_logs variable which can be redefed to limit the set of logs that gets JSON streaming logs created for it#12
ckreibich merged 5 commits into
corelight:masterfrom
mmguero-dev:master

Conversation

@mmguero

@mmguero mmguero commented Nov 26, 2025

Copy link
Copy Markdown
Contributor
# Filter the list of files
redef JSONStreaming::enabled_logs = set("http","files");

This PR adds the ability to do the above. By default enabled_logs is an empty set, meaning "do all of the logs" (the current default behavior). But by setting enabled_logs to a set containing one or more log names (e.g., conn, http, dns, files, etc.) then only those json_streaming_ logs get created. The rest of the logic (for rotating, etc.) is untouched.

I also added testing/tests/logs-filtered.zeek to verify.

Comment thread scripts/main.zeek Outdated
else if ( filt?$path_func )
filt$path = "json_streaming_" + filt$path_func(stream, "", []);

# Skip this filter if it's not in the enabled set (unless enabled_logs is empty)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question here Seth — are the individual filters really the granularity you're after? I ask because the parsing is a bit complex and your example of set("http","files") suggests so. If you're actually after log streams, not their attached filters, we could do

const JSONStreaming::enabled_logs: set[Log::ID] = set() &redef;

instead. Then the set could be e.g. set(HTTP::LOG, Files::LOG) etc, and we'd check higher up as we iterate over Log::active_streams.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right. I'll adjust my PR next week after the holiday. Thanks for the review.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ckreibich Thanks for the suggestion I've updated the PR with this change.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks Seth! Github has been unwilling to kick off that CI run for a whopping 8h now, so I'll hold out a little longer. 🤞

@ckreibich

Copy link
Copy Markdown
Member

Hrrrm, looks like I no longer have maintainer access to the repo — I'd like to enable workflows for your PR. I'll need to ask around re. that one.

@mmguero
mmguero marked this pull request as draft November 27, 2025 01:29
@mmguero
mmguero marked this pull request as ready for review December 1, 2025 16:09
@ckreibich

Copy link
Copy Markdown
Member

I just can't get Github to actually run this package's workflows. It tests fine over here locally, so I'll merge now. Thanks Seth!

@ckreibich
ckreibich merged commit af109f0 into corelight:master Dec 3, 2025
0 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants