Monthly Archives: July 2017

More logging, some realizations

I spent most of this week adding more logging to the build, but this time, I was working with groovy instead of Java. I’ve already outlined all my thoughts on groovy and why I find it less than ideal to work with, but ultimately the work has to get done, so I buckled up and galumphed through all the groovy and tried to figure out how it worked, so that the logging would be more effective.

If I haven’t mentioned it already, the build is huge. Massive. There are multiple parts to it, and each part has thousands of lines of code, which are initially not very easy to decipher unless you’re already familiar with the overall structure and function of the build, which itself can take a while. Fortunately, since I’ve been working with the build for a while now, I gained a cursory understanding of how it works, and that seemed to me to be enough to be armed with as I dove into the code. Making a few elementary changes first also helped, and just reading the code (along with some help from QA and Laura, the people who actually work with the build) led to a slightly better understanding of how it worked. It was a very interesting experience, as the build has had to grow with the markets, and the methods of data collection have morphed and evolved as more technologies have introduced different, and possibly better ways of solving the same problem.

The build is majestic. It has, as part of it, code written as far back as 2001, and as recently as two days ago. All these parts work together to supply the company with data continuously. I’m working on implementing something that has a relatively low chance of breaking the build, but even so, making changes to it is a strange feeling, almost like I’m changing something that I shouldn’t be messing with, as it’s far beyond my current capabilities to manage it. Working on it also teaches me more about stocks and how they work, because as I sift through the code looking for where and how I am to insert the necessary logging, I end up internalizing some of the code that explains how the data is collected, and what the different types of data mean.

Well, that’s a very flamboyant way of describing what I’m doing. It’s pretty routine, and looking at large chunks of code for hours on end can get a little enervating, but it’s ultimately worth it. Contrary to how it sounds, it honestly is a lot of fun, and is something I look forward to doing everyday, as I always end up learning something new. I’m hoping to be done with my current project in about a week, so check in next week and find out how I’m doing!

The week started off with me doing the same thing I did last week, which was porting code from Java to Groovy, and running the daily data build. After becoming familiar with the build, Chip revealed to me (and the rest of the data team) why he had me join the team – while the other team members all worked on keeping the build stable and making incremental improvements, he wanted me to implement a more centralized and accessible logging system for the entire build, so that in the future, keeping tabs on the build and knowing the necessary time  and order for the execution would be a lot easier to manage.

This was exciting – a chance to put something into the build that would last multiple years, and actually be useful and increase the productivity of those who interacted with the build – I was thrilled to work on this. Chip had already written a program that helped append the log messages to the messaging app that StockCharts uses, HipChat. Since the HipChat appender was already created, all I had to do initially was connect the current java part of the build to the program. This turned out to be a lot simpler than expected, and I soon finished, pushing my code to production. (By the way, pushing code that you know is going to be in production for the fist time is an intense feeling, much akin to the feeling you get when you’re expecting a reply from someone and are just about to open their message.)

I didn’t break the build! Next, it was time to move on to groovy, which should pose more challenges. However, just as I was about to start, I was warned by some other members of the team not to make any changes to the groovy build temporarily – they were working on a certain feature, and they didn’t want any version control nightmares, or any broken code. So, as I sat around, I decided to try and automate the daily build. I was having to run the daily build everyday while finishing my coding assignments as well, and it was a tedious task. I reckoned that with a little touch of crontab and the right commands pushed through, the daily build wouldn’t need people to run it ( except the parts that needed to be reviewed, of course). I sat down and researched cron jobs, and learned a lot about cron and the unix shell in the process – the amount we can do with our computers is amazing!

That’s pretty much all for this week though. Will I finally start writing some groovy logging? Will I be able to automate the build in the manner I want to? Find out in the next episode of Rajesh’s Summer Adventure!

Week 8 – Running the data build

We finished the quiz application! Well, almost. Having finished creating all the functional requirements of the quiz, all that remains is for us to put the final touches on the application, then connect it with the final website so that it goes live. It’s pretty exciting to see something we’ve been working on for so long finally come to fruition, but at the same time, we’re all also pretty happy to be done with working on this project. It’ll be interesting for all of us to move on from this project, and see where the rest of the internship takes us.

On that note, since the final touches that are to be made are mostly to do with the front end aspects of the application, Quinn and I don’t really have much to do. As such, Quinn was given another project to work on, and I was assigned to work with the data team! I work with QA, another Whitman alum, on the data builds and the daily data processes. Most of our work is making sure there are no anomalies in the code and the output, and so we execute production code and make sure everything is up to the mark. On the side, however, we work on porting existing groovy code over to java, which is both fun and confusing – groovy is similar enough to java that you think you would be able to do it without much thinking, but different enough that you run into all kinds of obstacles when trying to make something work in java. A lot of the libraries used in groovy are slightly different from their java counterparts, which makes it so that we often have to come up with new solutions to make something work, as the libraries used in the existing codebase are different from the ones that are available to us.

That’s been all this week, it’s become a routine of running the existing code in groovy and porting it over to java. Stay tuned to find out what happens this coming week!