Week 5 – Down and dirty with data

This week, we continued with the project we’d been working on last week. As I mentioned in last week’s post, we’d been split up into two team – the data team and the front end team. Though we worked together last week, we spent this week mostly by ourselves, finishing the tasks we were given by Chip. The two of us on the data team, Quinn and I, worked together to create a stable database which the front end team could then use to create the quiz that the users would then take.

We started off by using the POJOs from last week and creating a library which we could call to parse them. We wrote a parser class for each POJO we had created, which we could then use to parse the data we were receiving from the POJOs so that the file would return the data in a specific file format.

To proceed, we needed to have our program be able to read the data from where it was stored, which is the database. Since a lot of the data we would be using was in the process of being ported to the AWS cloud, we decided to use DynamoDB as our database, which is Amazon’s default NoSQL database. One performance benefit we can get from DynamoDB is that while SQL databases are strongly consistent, DynamoDB is eventually consistent, which leads to high performance benefits when transferring large amounts of data, as the system does not have to keep track of all the data in the system all the time while needing to modify it immediately.

Once we had become familiar with DynamoDB, we spent a day or two writing more parsers for different output datatypes, and implemented a system by which the users – who, in this case, are the exam designers – could choose which files they wanted to be displayed and how.

We then finished off the week by writing programs that could convert the data we already had in text files into the data format that we needed, as well as an interface that could directly ask the user for the data and write it into the system. This, for me, was the most fun part of the week, as we had previously only known how to write the data directly to the database without using any programs, but through this we were able to change what was in the database and keep expanding it so that we could have our entire question bank in the database without having to go through the trouble of writing it all straight to the database.

This week was definitely the hardest in terms of work we had to do, but it was also one of the most fun weeks, as seeing the project slowly come up in front of my eyes is a wonderful feeling, and I’m excited to see where we can take this project!

Leave a Reply

Your email address will not be published. Required fields are marked *