Distributed Lookup Tables – Alan LaMielle – NCAR SIParCS 2009 from Alan LaMielle on Vimeo.

This is a talk I gave that summarizes my work over the summer of 2009 in the SIParCS program at NCAR in Boulder, Colorado. I looked at reducing the non-scalable memory footprint of a set of lookup tables in an atmospheric model within CCSM—the community climate system model. Our approach was to utilize a feature of MPI 2 called one-sided MPI by distributing the lookup tables across MPI tasks. One-sided MPI allows one to remotely access memory of other MPI tasks.

Surprisingly, making a roll that tastes good was quite easy.  Not surprisingly, making a roll that looked good was very hard.  I can definitely understand why people can study sushi making their whole lives and never master the art.  There are so many interesting tastes, ingredients, and techniques to explore–the variations are so numerous!  I’m definitely going to have to make more of this type of cuisine in the future.

Following the sushi making/eating and beer drinking, we headed out (around 10pm) to play ‘bocce ball’ around Evan’s apartment complex.  I put it in quotes because, despite knowing the exact rules of the real game, I doubt we were actually following them at all.  Mostly we just made up a game using the ceramic balls from the bocce ball set Evan had purchased a few days before.  It reminded me of when I was much younger and we would make up games on our own and play for hours.  Apparently it takes late nights, sushi, and beer to do this now.  Regardless, it was lots of fun!

I’ve been working on a codebase at NCAR for my summer SiParCS internship that is based in an Subversion repository.  However, the code I have is unreleased and I haven’t been given commit access to the repo.  Also, I hate the feeling of working on code that isn’t properly versioned as I’ve lost work one too many times and never want that to happen again.

While working, I found myself needing multiple versions of some code I was working on (where each version was gathering different sets of data).  Rather than commenting in/out various sections of the code, git branches would work perfectly in this situation.

Keep in mind that the code I’m working on resides on an AIX front-end node to a reasonably powerful supercomputer.  The tool support on this machine is lacking, I have no zsh, updated vim, git, etc.  To get around this tool deficiency, I mount the code I’m working on locally on my machine using sshfs.  This works great!  However…  I still have no version control, so I decide to attempt to add the code to a git repository.

I started by removing the .svn directories using one of my favorite shell aliases, svnnuke.  I then created a git repo with git init.  So far so good.  Finally, I attempted to add all of the files to the repo for the initial commit with git add ..  However, this produced the error: 'fatal: Unable to write new index file'.  After searching through the git source for the location of the error string and trying many variations of various git commands, I was stuck.

This morning I approached the problem again, from a different angle.  I realized the problem was most likely not with git, but with the fact that I’m working over sshfs.  Therefore, I may solve the problem with how I am mounting the remote filesystem.  I googled for ‘git sshfs’ and found an article discussing Bazaar and SSHFS.  This article containted the solution to my problem: mount the remote filesystem using sshfs with the -oworkaround=rename option.  Poof!  Git is suddenly working and I’m hacking on the code versioned under git and all its glory!

Midway through the day we decided we’d like to try Bear peak.  We ducked out of work a bit early and after grabbing our gear and waiting a bit for Ben, we headed out around 4:30.  As we knew, this is definitely too late to head out for a hike as early/late afternoon storms have made themselves a part of Colorado life for the past few months.  Despite this, we were hopeful and tried to make a decent pace up the trail.

The first mile or so along the mesa trail isn’t so bad.  We held a pretty good pace for this section on the Mesa trail.  Apparently there are two ways to reach the top of Bear Peak: Along the Bear Canyon trail (longer and less steep) or along the Fern Canyon trail (much shorter but much steeper).  We chose the latter option.

Most of the way up the Fern Canyon trail was full of rock steps and switchbacks due to the decently steep incline we were ascending.  After multiple breaks, we made it to a decent lookout over the west side of the climb and could see some very pretty snow covered peaks and another peak to our north:

Peak north of the Bear Peak trail

Up to this point we had heard a few rumbles of thunder.  As we could see west, the building storm clouds were visible.  The combination of the continuing thunder, slight sprinkles, and occasional lightning were enough to convince us to turn back.  I’m not one who likes to mess with lightning in elevated terrain.  As we turned around to descend, a nice view of the plains just south of boulder presented itself:

View of the plains from the Bear Peak trail

Also, a picture of us descending (from right to left) Nick (fixing his shoe), Ben, Kuo, Dan, and Evan:

Descending Bear Peak

After a knee/shin jarring descent and a quick hike back down the Mesa trail towards NCAR, I snapped a quick few images of Boulder from the trail just west of the office:

Start/End of the Mesa trail just north of NCAR

By this time it was quite close to raining pretty good, so some of us hurried back to the building.  Just as Ben and I made it to the back entrance (across the bridge) it started raining.  Perfect timing!

After hiking Evan, Kate, and I headed to Southern Sun for dinner and a few beers.  Also, Evan and I decided we were sick of the weather denying us a full hike, so we are planning on leaving around 6:30 tomorrow morning to try again.  It is 11pm now, we’ll see if we actually make it.  I’m sure I’ll at least post photos if we do.

Side note: I found a Linux GUI/CLI tool called Hugin that does photo stitching very easily.  The third and fourth images of this post were made using Hugin and turned out pretty well.  I plan to make more in the future as they are much more impressive than a single image.

The day started with an almost exactly 1 hour drive down to Boulder after picking Kate up from her place.  Even at 7am, traffic wasn’t bad enough to slow us down too much.  We’ve averaged around 1 hour over the ~3 trips we’ve made down there.  I’m guessing it is now safe to say that it takes an hour to get from Fort Collins to Boulder (with our apartment or NCAR as the destination).

As we were early (we were supposed to be at NCAR at 9am), we stopped at our not-yet-paid-for apartment and found that the office was closed.  Resolving to call at lunch to get apartmenty things settled, we headed up the hill towards NCAR.

In contrast with the previous times we’d been to the NCAR campus, it was fairly windy and cold today when we arrived.  We headed in to the building and found our way to the cafeteria.  We were early (by around 30 mins) and met and talked with some of the other SiParCS interns who were also early.  Once 9am rolled around we headed to a conference room and started the day.

Most of the morning was filled with filling out forms, tours, and getting machines setup, configured, and working.  On the outside the building is really intersting, very cool architecture.  On the inside, you find that the cool architecture requires a complex layout that is initially difficult to find your way around.  Luckily I think I figured most of the useful routes out, but I’m guessing I’ll still have trouble at some point.  In terms of machine setup, I had to be the difficult one and use my personal laptop (along with one other inten).  To get access to various NCAR intranet resources, I needed to get VPN working.  However, the vpnc settings I was using weren’t working for some reason.  Later in the day we finally setup kvpnc and got VPN working with it.  (Later I figured out that I needed to specify my password each time using my recently acquired cryptocard, and got VPN working with vpnc in networkmanager, whooo!).

I ate lunch with Kate, Nick (the third intern working under John) and John in the NCAR cafeteria.  Kate and I learned that food is pretty expensive there ($20 for both of us total) and decided to shop later in the day for lunchy food and save ourselves some money.  The food is decent, just not worth $10 a meal.

After lunch I actually got started on my project, which I’ll describe in more detail at some point in a future post.  Basically right now I’m writing some Python scripts to process lookup table access traces from a run of CAM (the Community Atmospheric Model).  I’m hoping I can get some results to discuss with John by tomorrow.

Kate, Nick, and I decided to attend a welcoming reception and left around 3:30.  Nick headed straight there while Kate and I went and took care of apartmenty things first.  After paying rent and dropping off our stuff in our newly claimed apartment, we headed to the reception.  We were enticed by free food, expecting some sort of BBQ or something, but it turned out to be a more fancy-pants snacky event.  We grabbed some of the food (including coconut chicken on a stick!), listened to some speeches, chatted with Kuo, Nick, Even and Kate (all SiParcS interns) and decided we needed some real food.  We headed to our new favorite brewery, Southern Sun.

Southern Sun is a local Boulder brewery just across from our apartment.  We headed there for food, and ended up getting more than we could handle.  I ordered the Blackberry Wheat beer they had on tap (super tasty!) and Kate ordered a Belgium tripel (also super tasty!).  For food, we had an interesting soup that tasted exactly like a grilled cheese sandwich, very nice!  On top of that, we ordered the chicken nachos, and realized after it came out we had made a mistake:

Massive stack of nachos at Southern Sun Brewery

Needless to say, we didn’t manage to finish them, but what we did have was delicious.

After dropping Kuo off at home, Kate and I headed to our apartment and chilled for the evening and went to bed.  It was a long day and I’m pretty tired.  This is looking to be quite a fun summer.  I’m excited to get to work tomorrow on my project!

As I won’t be in town and working in the CS department every day, I’ve been trying to get a few last minute things squared away with IEGen and ready for Michelle.  We’re trying to get a journal paper finished and submitted (as we have been for ~6 months now…).  As I won’t be working for her directly anymore (thought I will be working on IEGen in the evenings) I’m trying to get certain things finished up so she doesn’t have to do as much learning of various pieces of my code.  I’m almost where I want to be, but I still need to get a few things finished up and ready before Monday.

Things are falling into place to get started working Monday at NCAR.  I’m really excited!  This should be a fun 10 weeks.

After we finished up signing all of the apartment paperwork we had about 30 mins to kill.  We stopped by Caffe Sole for a coffee and snack.  This may be the place I hang out if/when I’m not at the lab working.

We met with John around 11.  We chatted for a bit, got a brief tour of the building, found our future (though hopefully too needed) cubes, saw some big (though in the grand scheme of things not that big) machines that they use to run some simulations on, and had a (free!) lunch.  I really like the atmosphere (no pun intended!) there, it is very open, everyone is friendly and seems very approachable, and I immediately felt included and a part of the lab.  After lunch we chatted with most of the CS researchers there.  Every time I meet a new person or group of people that work in my field (or something close) I always seem to get along instantly with them.  I suppose it is the thinking alike thing that causes this…  Regardless, everyone seemed really nice, smart, and will be great to work with.

After another hour drive back, I met briefly with Michelle about the status of IEGen and her progress with the related full space tiling work.  Things are going well and I may actually be able to run some experiments soon!

Finally, we ended the day at Jim’s Wings on Elizabeth St, very close to Adam’s (soon to be) old apartment.  Hot wings go great with a few pints of beer.  We also discovered a small local ice cream shop where we enjoyed some tasty treats before I headed home on my bike.

I’m tired, it’s time for bed.