“They have hunted like this for countless generations” – David Attenborough, The Intense 8 Hour Hunt, on the San Bushmen of the Kalahari Desert

How I Ran a Marathon

Hi all,

I want to write another blog post – this time, on effective study habits ( because I’ve actually been asked this before by a couple other folks ).

Ok, so let’s begin. When I practiced Leetcode/Algorithmic problems, I kept consistency with a predicate, regimented routine of 15-minutes to 30-minutes daily, every day of the week ( yep that often included Saturday and Sundays ). Now of course, there were days or weeks where I didn’t do problems ( trust me, I did not solve problems on days when I was sick, the days I spent on vacation with family, my hiking weekend day trips, or family quality time ).

But here’s the thing.

I’ve never sprinted and crammed!

I’ve never been like other engineers out there in the world who talked loudly about spending three intense months of practicing 3-4 hours per day. I could never see myself doing that. I am the type who burns out after around the 30-minute to 45-minute mark ( except on a good day, where I give two fragmented 30-minute sessions , or the days I’ve done technical onsite, which operates differently). Everyone is “wired and build differently”, and to be candid, I am not built for this type of intensity.

But I am good at being persistent. And I think there’s benefits to the slow-burning approach over the intensity. I’m confident that my volume is the same – to quickly tabulate the equivalencies, 3-hours over 2 months = 1 hour over six months = 30 minutes over 12 months ( of daily practice, assuming a month is 30 days ).

And I think this approach actually turned out much better. Let me review them!

The Benefits

1. Leverages Spaced Repetition – There’s more days of practice and there’s more time in between each practice sessions to review and go over concepts. You’re less likely to forget what you learned if you have more time to learn material ( versus learning it all at once )
2. Accrue More Thinking Time – if you space out your studying, you get more thinking time. All that other subconscious daily time – the shower, on a walk, talking to friends – also makes it way available over a long-term horizon. You get to leverage more time in your life, ironically, to finding solutions to your problems – subconscious and conscious.
3. Reduced burnout probability – studying for concepts intensely typically leads to burnout. Spaced repetition and small units of practice are less mentally-taxing on the individual, so keep persistent is easier.
4. Higher Success Rate – the more time you have at your disposal, the higher your probability of finding a solution to a problem. You’re also more likely to solve 1-2 problems in a day than solve five-six problems in three hours ( they used to do this in competitive programming, and it’s why I probably thought I was bad at in my first year of college )
5. Builds up Healthy Studying/Practice Habits – studying and practicing interview skills isn’t something everyone in the world “naturally does” – most of us ( because there are exceptions ) are not born like this ; it takes time build the habits and the muscle memory to get into the groove of practicing problems. I think it’s because I spaced out my practice patterns over the long run that I always found it easy to immediately “get back into the zone”, as they say, and bust out a problem if I needed to practice again for interviews ( or do them for fun ).

The Drawbacks ( which aren’t much IMHO )

1. You ( may ) have to study ( a bit ) over weekends or holidays – the approach doesn’t give the most headway for “free days”. But think about the other benefits ( and I’m not asking that much time from you – trust me, I’m just as hedonistic and I want you to enjoy living. ) Also I took my approach to include spaced repetition practice on weekends, but I could’ve stuck and enforced boundaries on Mondays-Fridays only.
2. You’re going to start thinking more analytically – which is a good thing in the hiding. I mean, the more time you spend doing smart things, the smarter you get ( seems to be the right adage )?

“What really lights you up” – a good friend 🙂

That’s an excellent question!

Reasons and motives are aplenty – building a brand, monetization of secondary passive income streams supplementing active income, and practicing or honing my writing skills and my communication skills ( the written word is harder than the spoken word ). Sure, these are some of my personal motivators. But I have other motivators too – more intrinsic ones.

The Mentor-Mentee Relationship

I’ve always held strong mentor figures in my life as admirable – there’s something profoundly amazing about mentor-mentee relationships, where an older master of an experienced domain area teaches a younger student skills and shares their wisdoms and learnings accrued a long their way. Back in my undergraduate days – the days of learning and the days of research – I felt amazed at how passionate, inspirational professors who really knew their craft like the back of their hand could get their students to learn challenging skills. Some were so good, that it felt like the time that I spent with them literally made me “smarter” ( as the dictum goes, iron sharpens iron ). The time I spent with them still makes me think about how I like to approach work and how I like to operate as an engineer – to always be curious and interested, and to make sure to ask good questions.

To Disseminate my Accumulated Wisdoms

Presuming they are, that is to say :-P.

And personally, I want to share and disseminate the knowledge I’ve learnt a long the way to the next generation of engineers. I think in the long-run, writing a blog ( perhaps even a future O’Reilly-esque book or two ) would prove far more personally fulfilling. I’ve spent quite a number of years in my life deep in the dense thickets of software engineering, wearing hats across multiple domains – infrastructure, dev ops, algorithms, systems, product road-mapping, and leadership. I’m personally antagonistic to witnessing the totality of my accumulated knowledge either (a) go to waste or (b) remain forever locked and isolated within a couple of individuals across the team’s I’ve worked on. Provided that anything I share doesn’t leak private or confidential information, I’m open to getting it out into the world.

A Story of Teaching and of Learning

There’s also something amazing about teaching – and that’s making someone else in the world bright.

Let me be a raconteur – a narrator – and tell you a story. The setting is Middle Eastern restaurant back in Menlo Park, California, and it’s a cold, dry night : typically of the Bay area’s weather around that time of year. Fortuity had it that my practice buddy was able to meet up that night for a meal. I met him while I was preparing for my technical interviews – I used to practice mock 1:1 sessions with peers over algoexpert.io and pramp.io ( I made some lifelong friendships too ! ). He was one of the select few who I engaged in consistent back-and-forth 1:1 practice sessions.

At the end of our meal, he said something along these lines :

“I learned a lot from you.”

and hearing that was meaningful. It communicated something to me. It communicated to me that the time I spent in someone else’s life made a positive difference. That I managed to help someone get somewhere in the world ( in this case, an big tech company offer letter ). If someone else in the world tells me “I learned a lot from you”, it tells me that I did my job well. These words are a powerful thing to say to someone else.

“Drill down”

In the beginning, there’s just ERROR codes

Hi all,

I want to write a post inspired by real-life developer work that took place back at Capital One.

Alright, let’s imagine that junior engineer Andrea needs to write APIs for Enterprise data-layer held objects. She’s been tasked to write APIs so that internal developers on other teams can execute CRUD – create, read, update, and delete – operations on Enterprise objects. The APIs not only enhance scalability, but allow for further processing and verification steps ( e.g. ACLs, rate limiting, resource sharing )

When she initially wrote the APIs, she did think about error handling, but from the scope of HTTP status codes ( see https://en.wikipedia.org/wiki/List_of_HTTP_status_codes ), such as the 400 bad request error. This error is introduced for cases that backend servers can not process requests due to possible client-side errors ( e.g. badly formatted inputs or deceptive inputs ).

In the initial version, everything seems dandy and working fine. And for a context of a standalone web application with a single client and a single server, it’s ( mostly ) sufficient to return numeric status codes. But problems arise up in the future.

Trouble brews months later !

Alright, so senior engineer Javier has to develop our his team’s application, which operates on the same set of Enterprise objects. This means that his application behaves as a client, and executes Andrea’s API calls, exposed as conventional HTTP /GET, /POST, /PUT, and /DELETE methods. Javier recognizes on client-side development that he needs to introduce error handling paths ( e.g. halting his program or engaging in graceful degradation and capturing bad payloads in a dead letter queue ) in the event of 4xx errors when executing /GET calls.

But there’s a gotcha.

He’s running into a problem where he runs into a 4xx issue, but it turns out that the client-side payload is actually well-formed. Everything is correct there. What’s going on? 4xx should be a client-side issue, right? I clearly shouldn’t be failing a request – or event the processing of my events – if things are all and good here.

Yes … and no

So it turns out that the payload is correct, BUT, the Enterprise asset the payload operates on is stale. The assets themselves are out-of-date by a couple of months, and underneath the hood, that /GET call executes a SQL query to filter out and retrieve an asset based on unique primary keys.

And the problem is both the scaffolding – everything around the query – and the SQL query are “technically” correct. It’s actually an issue with the underlying database not being up-to-date ( due to a laundry list of reasons I will avoid delving into 😛 ). Perhaps a Database admin or other user forgot to purge out old, stale records.

So tell me what the hot-fix is?

After some back-and-forth conversations, Javier and Andrea recognize that a new version of the APIs are needed in production settings with modifications. The return of the 4xx is correct, but we need more information. We need a short, 150-character tweet-sized textual error response as well ( and maybe even an additional layer of API enum code ). The response and code should reveal more information such as :

• “Failure code <A> : Did not locate Enterprise objects; client-side payload is valid.”
• “Failure code <B> : Located Enterprise objects; badly-formed client-side payload.”

Andrea wants to change the APIs, but changing APIs requires extensive approval and testing. As such, she arranges a meeting not only with Javier, but with his team’s product owners and other senior engineers to generate “buy-in” and get approval to release the same /GET call, but on a version two form that returns more information.

After this release, Javier project get unblocked. He’s able to return back to his client-side code and adds the error-handling paths, based on a function of not only the 4xx status code, but also the textual error response and additional API enum Failure code. His failure-handling code resembles the following structure

if(HTTP.STATUS_CODE = 4XX):
     if(HTTP.STATUS_TEXT.FAILURE_CODE == <B> OR HTTP.STATUS_TEXT.STR == "Located Enterprise objects, but ... "):
        logger.LOG_ERROR("Ran into malformed payload - delay processing of events.")
        logger.LOG_FATAL("Shutting down event streaming processing");
        return;
    elif(HTTP.STATUS_TEXT.FAILURE_CODE == <A> OR HTTP.STATUS_TEXT.STR == "Unable to locate Enterprise object; client-side payload is good."):
        logger.LOG_ERROR("Ran into stale asset case with assetId={assetId}. Logging into DLQ {DLQ_id} with name = {DLQ_name} and topic = {DLQ_topic}.")        
        DLQClient.appendToDLQ(DLQ_id, DLQ_name, DLQ_topic, assetId);
        continue;

The real-life story

In this scenario, I’m senior engineer Javier. I spent a couple of hours deep in debugging systems, triaging the root cause of failed Kafka stream events. I had to determine the cause of failure for events that stalled event stream processing – was attribution due to the staleness of downstream assets ( server-side issues ) or due to malformed inputs ( client-side issues )? In both scenarios, I introduced custom Cloudwatch-level logging – with 90 days of retention – to address the server-side issues ; this enabled enable human-in-the-loop intervention on stale assets. As for client-side issues, I biased towards graceful degradation and halting the event stream; I preferred notifying upstream producers as soon as possible that they sent malformed payloads.

This logging had repercussions – I worked with a event stream processing +1 million events/second, and to many failing stalls meant that events remained unprocessed and backed up. The probability of stale asset events superseded the probability of malformed client payloads ( I don’t have a specific multiplier by how much, but intuition intimated it ). By changing error handling, I reduced the frequency of halting event stream processing, thus enabling the processing of more customer traffic.

The Challenges I ran into

There were a good number of challenges I ran into with the design – let me review them :

• Generating buy-in and consensus – I had to get other senior engineers, a staff engineer, and my two direct managers to agree on the architecture & design approaches. I set the agendas and led the meetings
• Communicating unexpected delays and blockers – #TODO
• Spending time to look into alternatives to pre-empt future issues – #TODO

I created my blog with multiple intentions in mind.

Firstly – to disseminate technical information, interests, projects, and content to outside readers. Topics and themes encompassed include algorithmic interviewing, system design interviewing, niche architectural topics, useful command-line hacks, and others.

Secondly – to share the knowledge and the skills that I’ve accumulated over the years working across multiple industries and verticals in software engineering. At some point, I want to see myself mentor and share more about what I know, as well as raise the next generation of technically-savvy engineers to be competent enough to take the world’s challenges.

Thirdly – to encourage a knowledge sharing world. At some point, I may or may not engage in content monetization on all or a targeted segment of articles, but in its current inception, posts are free content.

My Blog’s Value Statements

I am open to any criticism or feedback regarding this blog too. I’m also open to commenters, readers, or viewers who want to share interesting articles or other forms of knowledge.

I ( If I do plan on enabling commentary ) want to abide by content moderation and keep all commentary positive. Humor is welcome, BUT, I ( or an AI-tool ) will flag comments with negative sentiment and ban users identified as mistreating others based off of religion, national origin, gender, race, sexual orientations, and so on. All consumers of material are to be treated equally and deserve fair and impartial equity.