This is best demonstrated with an example.
Here is a typical week in Student X's semester:

Note that there are two courses that meet only on Mondays (CSCI 3901H lecture, CSCI 4061 lab). If we use the "Add to Calendar" button on this week, the scraping process yields the following calendar information:
{
"additionalMeetings": [],
"courses": [
{ "calendarStrings": { ... }, "courseName": "MATH 5616H", "courseNum": "57571", },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", },
{ "calendarStrings": { ... }, "courseName": "CSCI 3081W", "courseNum": "59084", },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59779", },
{ "calendarStrings": { ... }, "courseName": "HSEM 2621H", "courseNum": "67213", },
{ "calendarStrings": { ... }, "courseName": "CSCI 3081W", "courseNum": "59520", }
]
}
This is behaving exactly as desired.
However, let's try something slightly different: let's hit the "Add to Calendar" button while open to the first week of the semester:

Note that CSCI 3901 lecture and CSCI 4061 lab are completely absent from this week because Monday is before the start of the semester. Since these courses aren't in the sample week, general course info is never read for them, so they are never expressed as recurring meetings. Instead, they're expressed as numerous additional meetings. This is what the scraping process returns:
{
"additionalMeetings": [
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-01-22T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-01-22T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-01-29T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-01-29T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-02-05T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-02-05T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-02-12T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-02-12T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-02-19T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-02-19T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-02-26T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-02-26T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-03-11T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-03-11T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-03-18T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-03-18T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-03-25T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-03-25T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-04-01T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-04-01T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-04-08T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-04-08T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-04-15T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-04-15T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-04-22T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-04-22T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59781", "date": "2024-04-29T00:00:00.000Z", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3901H", "courseNum": "66290", "date": "2024-04-29T00:00:00.000Z", ... }
],
"courses": [
{ "calendarStrings": { ... }, "courseName": "CSCI 3081W", "courseNum": "59084", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 4061", "courseNum": "59779", ... },
{ "calendarStrings": { ... }, "courseName": "HSEM 2621H", "courseNum": "67213", ... },
{ "calendarStrings": { ... }, "courseName": "MATH 5616H", "courseNum": "57571", ... },
{ "calendarStrings": { ... }, "courseName": "CSCI 3081W", "courseNum": "59520", ... }
]
}
This is potentially confusing/annoying for Student X, since in the calendar they get, all the meetings of CSCI 3901H are completely independent calendar events. Like if they wanted to change CSCI 3901H's the event title, they'd have to manually edit every single instance.
The crux of this problem is making sure that we pick a good sample week--one that isn't omitting any of Student X's courses.
Problem
This is best demonstrated with an example.

Here is a typical week in Student X's semester:
Note that there are two courses that meet only on Mondays (CSCI 3901H lecture, CSCI 4061 lab). If we use the "Add to Calendar" button on this week, the scraping process yields the following calendar information:
This is behaving exactly as desired.
However, let's try something slightly different: let's hit the "Add to Calendar" button while open to the first week of the semester:

Note that CSCI 3901 lecture and CSCI 4061 lab are completely absent from this week because Monday is before the start of the semester. Since these courses aren't in the sample week, general course info is never read for them, so they are never expressed as recurring meetings. Instead, they're expressed as numerous additional meetings. This is what the scraping process returns:
This is potentially confusing/annoying for Student X, since in the calendar they get, all the meetings of CSCI 3901H are completely independent calendar events. Like if they wanted to change CSCI 3901H's the event title, they'd have to manually edit every single instance.
The crux of this problem is making sure that we pick a good sample week--one that isn't omitting any of Student X's courses.
Possible solutions
Neat solution ("lookup by term"): for each term (spring, summer, fall), store a "safe" date that's a week or two into the term. When person hits the "Add to Calendar" button, we know what term they have open from
weekToJson, so look up the corresponding "safe week" to feed toscrapeASemester.Messier but more flexible solution ("calendar probing"): peek at several weeks (perhaps the currently open week, the previous week, and a few following weeks) and just see which week has the most meetings. This might be a bit messy, and also this is more fetch requests, which slows down the scraper considerably (well, maybe we could somehow memoize these results to save work for the scraper, but that'd be more complexity.)