2025-02-06 Meeting notes

Last meeting

• Covered tools and data pipeline.

  • 2 repos: api-tools and nebula-api

    • api-tools: scraping, parsing, and uploading

    • nebula-api: talks between mongodb and user

Tasks/issues

• Some L1 tasks have opened up from people who didn’t do them last semester

  • Fix/suppress some annoying chromedp errors

  • fix parser bug pulling the room as “No Meeting“ and others

  • add /events/:day/:building/:room/sections route

  • unit testing!

    • Josh will give some examples by next meeting for anyone to do the rest of them

  • on hold tasks - feel free to do if interested

    • astra isn’t as useful as we thought so schemas for that scraper are on hold

    • changing api-tools to not have to upload and replace all the data on mongodb each scrape+parse+upload

How the team works

• Josh puts out tasks in batches, pick one up and work on it

• Feel free to work in groups

rooms-route branch

• route

  • adds OPTIONS and GET routes

• controller

  • pulls data from mongodb

• work already done on the database

  • aggregations in the database to transform data

  • sections creates the rooms and events views

    • rooms shows every classroom

    • events shows every section on a day

  • doing this in mongo helps keep the code from being messy