Fix syncookies statistics and tests - #63
Conversation
ad52be7 to
1c684ba
Compare
consuelo2210
left a comment
There was a problem hiding this comment.
I reviewed only C/C++ part and left python part to @symstu-tempesta. I have the only question about expanding statistic:
Could we consider keeping this within the existing XfwTrafficStat abstraction? From an extensibility perspective, it seems preferable to add XFW_SYNCOOKIE_FAILED there and use count_traffic_stat(...), rather than introducing a separate field and a separate code path. This would also make adding future traffic statistics simpler and avoid special cases in Prometheus.
680109c to
78b9a55
Compare
consuelo2210
left a comment
There was a problem hiding this comment.
LGTM: just make the alignment consistent after the name changes in drop_stats.h.
Also, commit 78b9a55 has an incorrect description: we don't report XDP_DROP stats to Prometheus.
029cfd7 to
6790dda
Compare
Previously we configured passive_timer and flood_timer with 0 by default and tcp_syncookies_flood_mode() and tcp_syncookies_passive_mode() used jiffies (1/HZ sec). This leaded to immediate, and worse - scheduler dependent, switch between the modes. Effectively, flood_timer=0 passive_timer=0 in the tests leaded to flacky behavior. Now by default set the timers to 1 second and discourage 0 values in the wiki. Also make the TCPsyncookies Prometheus counters consistent with the kernel statistics and report generated, received and failed syncookies. This makes the testing an future observability easier. Keep the existing statistics split by packet-processing outcome: generated SYN cookies are TX statistics, failed SYN cookies remain DROP incidents, and received SYN cookies are traffic statistics.
Adopt slightly modified patch by Maksym Stukalo from the Escudo project: add Tempesta xFW Prometheus statistics along the currently monitored /proc/net/netstat - we need all the counters to validate the xFW behavior. Change only one test to check the concept - if it's right, then I'll update the rest of the tests accordingly.
Call XFW.metrics() and run_start() to initialize the metrics values after slef-requests. Small cleanup for fetching syncookies values from /proc/net/netstat Original patch also adjusts the values for syncookies_read_kern_stats(), but in the current version we distinguish between the kernel and xFW statistics and analyze both of them.
* Move TCP syncookie failed counter to traffic stats * Separate common and drop statistics Add comments and rename constants in XfwDropStat to distinguish Prometheus statistic (XDP_DROP, XDP_PASS and common traffic) and ClickHouse events (DROP statistics).
4c0685c to
4d24a67
Compare
* It's difficult to set exact metric values because of kernel and XFW work specifics * Added comments * Used variables instead of magic numbers * Added check_xfw_stats to shorten the code for comparing XFW metrics and kernel stats
4d24a67 to
d3406dd
Compare
| invalid_acks = sum(acknowledged for _, acknowledged in flood_results) | ||
| # failed [lo, hi] in params is a factor of invalid_acks: [1, 1] exact, [0, 1] any up to that. | ||
| failed_lo, failed_hi = expected_xfw["xfw_syncookie_failed_packets"] | ||
| expected = { | ||
| **expected_xfw, | ||
| "xfw_syncookie_failed_packets": [ | ||
| failed_lo * invalid_acks, | ||
| failed_hi * invalid_acks + 1, | ||
| ], | ||
| } |
There was a problem hiding this comment.
This is unnecessary code originally generated by an LLM.
/proc/net/netstatand make SYNcookies easier to monitor and testXfwDropStatto distinguish Prometheus statistic (XDP_DROP,XDP_PASSand common traffic) and ClickHouse events (DROPstatistics)