Supercomputer as stepping stone CRAIG STEPHENSON: So I’m Craig Stephenson, I’m 41. I’m from Fairbanks, Alaska, originally. Born here. My relationship with Bob here is friend and frequent colleague. BOB TORGERSON: Hi, my name is Bob Torgerson. I’m 38. I was born and raised in Fairbanks, Alaska, also. And yeah, old friends and colleagues with Craig. So Craig, knowing where we’re from, I wanted to ask you — how did you get your start here at the university? CRAIG STEPHENSON: So basically, just growing up in Fairbanks, Alaska, I’d always been interested in computers and I’d heard that UAF had a good computer science program. So there was no question in my mind that I would just stay here, get in-state tuition, go to school here, get my computer science degree. I applied for a student position at the Arctic Region Supercomputing Center, ARSC, which is a place that I’d always heard about growing up here, that there was a big supercomputer up on the hill at UAF. Just this idea that I was already into computers and knowing that there was this really powerful supercomputer, one of the fastest ones in the country, fastest one in the state, probably. So I applied there, got my job there. It was a bit of a culture shock because, having heard about how powerful this thing was, working in the office, all you saw were people working in terminal windows, just the most primitive thing you could possibly imagine. It was just kind of this magical place where people were getting all this work done just by typing words into this black box. Then from there I just worked there for two years as a student, four years as staff, and then worked at IARC — International Arctic Research Center — and then worked at the Research Computing Systems — RCS — and now I’m at SNAP — Scenarios Network for Alaska and Arctic Planning. BOB TORGERSON: Lot of acronyms we’ve been through. CRAIG STEPHENSON: So that’s how I got here, basically. So how about you? BOB TORGERSON: My story is similar. I was born and raised here in Fairbanks, was always interested in technology, ever since I was a little kid. I remember the first computer that my dad bought us was an old Compaq and he thought it was the biggest thing since sliced bread. So did I, so it was very quick for me to fall in love with these things and start to teach myself a little bit of programming when I was growing up. I actually started with Pascal programming, strangely enough, which nobody uses anymore that I’m aware of. I remember I made my family a Christmas card from 3D-generated, well, it was actually 2D-generated, imagery. Just circles that were drawn via Pascal. And there was snow that would come down for this snowman. CRAIG STEPHENSON: Sounds awesome. BOB TORGERSON: Yeah. So that got me into my passion for computer programming and going to the university here happened just simply because I wanted to stay in Fairbanks. I love this town. One year in 2010, I happened to go to a job fair because I was getting to the end of my term as a senior, and wondering, “What am I going to do with my life now?” and I ran into Oralee Nudson, and she became my mentor and boss when she hired me to work up at ARSC, also. ARSC being that supercomputing center that we talked about previously. I started as a student in 2010 and got my master’s degree by 2012, and then after that I proceeded to get hired on full time. Which was really neat. The only downside being that the DoD had pulled funding from us about the same time that I was hired. CRAIG STEPHENSON: I remember that. BOB TORGERSON: Yeah, it was sad times. There were a lot of people going in and out pretty quickly, it felt like, mostly out. So I took over a position for a storage admin after another storage admin had either decided to leave or just wanted to move on to different things. I did that for ARSC for three years until they closed in 2015 and we officially became the Research Computing Systems in 2015, and worked for them for another five years as their main storage architect. And then in 2020 as the pandemic hit, very much uncertainty about the future, Bruce Crevensten approached me about the possibility of a remote job, which at the time was, yeah, a real blessing. It was like that was the first foray into what could be remote work going forward. Hired on in July of 2020 and have been working there ever since so I’m coming up on my five-year anniversary for SNAP here. CRAIG STEPHENSON: Yeah, you were one of the ones that convinced me to come to SNAP. BOB TORGERSON: Oh, yeah. CRAIG STEPHENSON: Yeah, yeah. So that worked out really well for both of us, I think. BOB TORGERSON: Yeah, for sure. CRAIG STEPHENSON: We both have computer science degrees and a lot of our jobs have involved a lot of IT over the years but this is just pure programming most of the time. Yeah. It’s just nice. BOB TORGERSON: Yeah, it’s certainly a change of pace. The difference between this is that there’s never an on-call situation for me. And being in RCS for that long, or for ARSC, there was always on call, and when you were the only storage person, you were always on call, regardless of whether or not you were on call for that week. I remember there was one time, a Thanksgiving morning, I was sitting there watching a Macy’s Day Parade go by and I get a bunch of buzzes on my phone. I look at them and they were all, I had hoped it was friends or family saying Happy Thanksgiving. It was actually that the data center had just gone dark. And so everything in there had gone down hard, and one thing I know about computers is that they don’t like to go from being hot spinning to just cold off all at once. Ended up spending the entire day for Thanksgiving from about, I want to say 9 a.m. until almost 9 p.m. bringing everything back up, fixing all the problems that came with — we had to actually swap hard drives of systems so that they would boot up in a different chassis. I remember these very distinct conversations we had like, well this is the Hail Mary, we’ve got to try this or it’s not coming back. CRAIG STEPHENSON: Doesn’t sound like the best Thanksgiving. BOB TORGERSON: No. But that’s what I really appreciate about this, is that we’re not a 24/7 shop. We do have downtimes that are acceptable and, yeah, we don’t have to be on call for any one given period. I know for a while there at RCS, you had multiple, multiple projects that you were constantly being asked to do. Did you pull one together at the last second, that you can remember? CRAIG STEPHENSON: Well a lot of them had pretty sharp deadlines but the thing that comes to mind the most is the aurora forecast at the Geophysical Institute. I was partly involved in developing that and mostly involved in keeping it online. During the four years I was at RCS, it just got more and more popular, even tourist companies were linking to it from their pages and things like that. We could see in the Google Analytics that we were getting like 2 million people a month viewing that thing. And eventually it reached a point where aurora activity was so high that it could be seen from Montana and stuff, and it got linked from the front page of CNN. And then our server started melting. Once that started happening, we’re getting enough legitimate traffic and potentially some denial-of-service traffic, just keeping that thing online was extremely challenging. Long story short, I found out that if somebody bombards the server with like 600 requests in five minutes, you can just ban them for an hour, the problem goes away. So that was my way of fixing that website so that it stayed online while it was on the front page of CNN. But I’d never run into a situation like that before. BOB TORGERSON: No, we, yeah, don’t often get like 2 million views, or that number of viewers so rapidly, I would say. CRAIG STEPHENSON: It’s just interesting because you spend a lot of time when you’re doing this kind of work trying to figure out how to keep things performant, able to cater to lots of people at the same time, but that’s the first real example in real world that like, “Oh wow, I have to do this for real, now. We need to figure this out.” It was pretty satisfying.