Metrics and boilerplate code

Second last week at the office! Things are winding down, and the office is getting ready for us to leave (by which I mean they’re excited about us not eating their snacks all the time). The project the other interns have been working on is drawing to a close, and we’re finishing all our pending work and tying up all our loose ends.

This week, Quinn and I were told to help out Connor with his job. We learned that since we’re moving up into the cloud, and all our servers wouldn’t be physically accessible anymore, that we would have to add metrics to all of our code so that we would be able to track how often it was being run, servers that were being hit more often, the distribution of the load among the servers in the cloud, etc. To this end, Quinn and I were given most of the functional servers that are currently up in the cloud and asked to review them and implement any changes that weren’t already made, such as adding logging and connecting the programs to Firehose so that we would be able to keep track of the logs that the programs were generating, and keep track of all the data that was being sent through to the servers.

We attacked this task with gusto. Quinn was given slightly more responsibilities than I was, as I also has the build to maintain and keep running – with all the changes that keep being made to the build, somebody needed to keep an eye on it to make sure those changes did what they were supposed to do. As the week progressed, it became harder to balance both the tasks, as the build grew slightly more fussy, and had to be run a certain way, and I had also run up against a particularly ancient program that had been running on the company servers for a very long while, and had to make massive changes to so that it was compatible with all the other technologies that we were trying to introduce. The changes weren’t very technically challenging, but I had a hard time trying to figure out exactly how to push them into place so that it would play well with the other code that depended on this.

All in all, it was an interesting week – we were given a lot of responsibility, but the work itself didn’t require us to be very innovative, so to say. There was a lot of boilerplate code that had to be inserted, and we helped with that, but it was very interesting to see this side of the code and the coders as well, and to find out that not all code is just practical problem solving, but that there are other elements to it too. Last week incoming, stay tuned to find out how we end it!

Finished Logging!

I finally finished the logging for the build! It wasn’t easy, and I kept running into one challenge after another, but ultimately I was able to finish it.

QA and I set a goal of Thursday, by which I was to try and finish the logging for the build, so that we would be able to do a test run on Friday, and have it all good by the weekend. Chip was out this week, so we also decided that we would surprise him with the whole thing done when he got back. With this in mind, I sat down on Monday, trying to get at least the morning segment of the build done by the end of the day.

The morning segment was fairly straightforward, all I had to do was make sure all the steps for two of the daily collections had been run. The early afternoon steps were slightly more problematic, as I had to dig into huge blocks of code to find out exactly which functions were being run for which menu call, and then had to time those functions and log their occurrences to HipChat. Though this initially seemed to be a tedious task, I eventually figured out that this was teaching me how to find certain segments and be able to link it to all the functions that it calls/are calling it, which can be really useful for when the code crashes (which it occasionally did while I was trying to compile my changes. I then progressed to the late afternoon build, which is the most complex segment of the build, and has multiple concurrent processes, which all need to be tracked, as they are interdependent. This was pretty challenging, as some of these processes also collided with processes being run at other times of the day, but with different inputs. This led to a few complications, but as previously mentioned, once I figured out which methods depended on variable inputs and how they functioned with the inputs, I was able to connect them to HipChat with relative ease. All it then took was finishing the job, which just required more of the same thing, and making sure that the logs didn’t collide and the build didn’t crash.

That’s all for this week, I’m not really sure what I’m going to be working on this coming week, but if the last few weeks are any indication of it, then I’m sure it’s going to be challenging as well as informative! Keep following to find out what I end up doing with my upcoming week!

More logging, some realizations

I spent most of this week adding more logging to the build, but this time, I was working with groovy instead of Java. I’ve already outlined all my thoughts on groovy and why I find it less than ideal to work with, but ultimately the work has to get done, so I buckled up and galumphed through all the groovy and tried to figure out how it worked, so that the logging would be more effective.

If I haven’t mentioned it already, the build is huge. Massive. There are multiple parts to it, and each part has thousands of lines of code, which are initially not very easy to decipher unless you’re already familiar with the overall structure and function of the build, which itself can take a while. Fortunately, since I’ve been working with the build for a while now, I gained a cursory understanding of how it works, and that seemed to me to be enough to be armed with as I dove into the code. Making a few elementary changes first also helped, and just reading the code (along with some help from QA and Laura, the people who actually work with the build) led to a slightly better understanding of how it worked. It was a very interesting experience, as the build has had to grow with the markets, and the methods of data collection have morphed and evolved as more technologies have introduced different, and possibly better ways of solving the same problem.

The build is majestic. It has, as part of it, code written as far back as 2001, and as recently as two days ago. All these parts work together to supply the company with data continuously. I’m working on implementing something that has a relatively low chance of breaking the build, but even so, making changes to it is a strange feeling, almost like I’m changing something that I shouldn’t be messing with, as it’s far beyond my current capabilities to manage it. Working on it also teaches me more about stocks and how they work, because as I sift through the code looking for where and how I am to insert the necessary logging, I end up internalizing some of the code that explains how the data is collected, and what the different types of data mean.

Well, that’s a very flamboyant way of describing what I’m doing. It’s pretty routine, and looking at large chunks of code for hours on end can get a little enervating, but it’s ultimately worth it. Contrary to how it sounds, it honestly is a lot of fun, and is something I look forward to doing everyday, as I always end up learning something new. I’m hoping to be done with my current project in about a week, so check in next week and find out how I’m doing!