harisrid Tech News

Spanning across many domains – data, systems, algorithms, and personal

recent posts

about

Graduated from the University of Texas, Austin’ 2018 Bachelors in computer science and bachelors in mathematics. Currently living in beautiful sunny San Francisco, CA, where the skies are clear and the grass is greener! Outside of work, I like to hike, explore new restaurants, go boxing, spend quality time with friends, and travel the world!
  • TOTW/18 – Unlock Real-World Skills and Bias to Real-World-Esque Interview Problems. You can assess candidates better.

    TOTW/18 – Unlock Real-World Skills and Bias to Real-World-Esque Interview Problems. You can assess candidates better.

    Because I’m just as sick of seeing questions that remind me of inapplicable problems, like shooting penguins out of cannons in your AP Physics Textbooks, and I’d rather solve something that entails more pragmatic engagement.

    Why real-world applicability?

    Today, I want to share one of my favorite leetcode problems to solve, and why I think it’s a worthwhile problem to stress test possible candidates. Since I’m someone who’s solved a boatload of Leetcode problems, I’m always thinking about how interviewers can select for better problems and index for candidates who demonstrate better real-world, on the job performance thinking skills.

    The problem is Leetcode 428. Serialize and Deserialize a N-Ary Tree. I’ve attached the description underneath, but in this problem, TC ( the canidate ) needs to be creative and conjure up their own approach to serializing ( converting from an in-memory data structure to a string ) a n-ary tree and then deserializing said tree ( converting back from a string to the original structure ) . It’s technically a HARD category problem, but in my honest opinion, it borders a hard MEDIUM difficulty question. I would definitely ask it to a Google L4+ candidate ( using current leveling systems ). I think this is a good problem because we stress-test the following criteria :

    1. Leetcode problem 428 : A pragmatic hard that one can expect to frequently encounter.
    The Stress-Test Criteria
    • Word problem translation – the description closely mirrors real-world case studies and ambiguous problems engineers encounter. Without the constraints mention, TC can spend some time asking really good clarifying questions.
    • Design thinking open-endedness – candidates can get super creative with their approach ; their really is no “one-size-fits-all” strategy, meaning that there’s room for an interviewer to possibly learn how to do a problem differently ( and maybe better ). My approach involves a root-left-right style with more paranthethicals : it resembles (1 (3 (5) (6) ) (2) (4))
    • Real-world applicability – thinking of how to handle serialization and deserialiation, or at least handle converting structures across multiple formats ( e.g. JSON to memory or in-memory to Protobuf ), is frequently encountered when transmitting payloads across environments.
    • Recursion – in the general, most good interview problems stress test recursive thinking.
    • String handling -TC needs to think about parsing strings – how to handle the different tokens ( integer values, ‘(‘, ‘)’, ‘,’ ). This may entail the use of Regexes or a combination of library functions and delimeters. It also involves conjuring up a rules engine to adjust a pointer in string input, based on the token under processing.
    • Top-down/ bottom-up chunkification – TC needs to think about how to segment the input in their recursive-esque approach.
    • Compression – interviewers can gauge how well TC think of making minimal serialization strings. This matters in the real-world with at-scale settings, where input compressions translate to revenue.
    • Minimal data structures – the problem is solutionable without data structures ( implicit stack space recursion ) or with a stack ( explicit memory recursion ). There’s minimal “trip up” room in TC needing to think of to many data structures.
    Footnotes

    URL Link = https://leetcode.com/problems/serialize-and-deserialize-n-ary-tree/description/

  • OPINE/2 – Will AI Take Over Software Engineering Jobs

    OPINE/2 – Will AI Take Over Software Engineering Jobs

    Should we be scared about AGI : Artificial General Intelligence?

    A Primer

    Hi all,

    I want to delve a bit in AI, and whether I think it’s going to take over our jobs or not.

    So I see a lot of opinion pieces, scare tactics, or LinkedIn articles which state that AI will take over jobs. But, let’s take a step back. Let’s actually investigate the folks face and how they ( probably ) use AI and ML in their day-to-day operations.

    Firstly – how I use ( and don’t use ) AI at work?

    At work, I use AI tools and AI integrations. I’ve used a ChatGPT code integration with Github CoPilot to write the following :

    • Source code – I feed in sufficiently-detailed prompts with enough context and instructions such as “Write me a simplified class to populate three members in Python3”.
    • Unit tests – I feed in source code and ask the tool to write up unit tests. The tools do a good job writing tests for happy path execution and unhappy path execution.
    • Refactor code – I feed in a prompt to ask to clean my changelists or refactor my changelists by targeting specific gauges – human readability, conciseness, and organization.

    I see it more like Microsoft Auto-correct’s capability1 : the AI tools have their imperfections, and frequently, I have to dive in and clean up a couple of things. Seldom do I ship out AI code for immediate code review, because there’s a couple of modifications that I have to make, such as renaming variables or adjusting config values.

    I don’t see AI tools as replacing my work, because there’s a lot of work AI tools can’t do :

    • Design Document Write Ups – AI tools can’t write templated design docs and can’t easily take in feedback from engineers
    • Brainstorming architecture – coding is easy, but figuring out new systems to design or reasoning around distributed system problems is harder.
    • Final Code Reviews – there’s a lot of human audit that still goes on here. Especially for concerns of releasing information that shouldn’t be disclosed in a production environment.

    Secondly – What exactly are companies doing?

    What are the M.A.A.N.G. big tech, finance firms, or start ups even doing with AI and ML? That’s an excellent question, lemme circle back to you!

    AI research has been there since the 80s or the 90s, but the lower cost computer and infrastructure to enable pragmatic use cases have come up in the past few years. Many companies are learning that they can adopt AI or ML to enhance their value proposition – expedite day-to-day operations, build out enhanced capabilities, or tackle previously untackleable problems. From my understandings, here’s what’s going on

    • Big tech companies – focus on researching unknown models or setting up large-scale infrastructure for AI. Big tech companies have dedicated R&D teams, long-term financing, and the ability to take on the risks of releasing the latest SOTA ( state-of-the-art ) ML Models. But before doing so, they need to tackle some pragmatic issues
      • Creating or using the dedicated hardware : think of NVIDIA’s GPU chips.
      • The infrastructure.
      • The testing strategies needed to release reliable models.
    • Non-tech focused companies – extending AI to existing business use cases. Companies outside big tech focus on immediate value delivery : can we directly use off-the-shelf AI models and infrastructure OR incorporate slight customizations to solve a business problem? Business problems encompass :
      • Maximizing advertising campaign revenue.
      • Financial trends predictions.
      • Determining optimal customer segments or cohort groups.
      • Detecting PII or faulty data.
    • These companies usually lack a combination of the resources of the infra needed to create something new; hence, the sense of urgenecy to use what exists.

    Thirdly – why haven’t all companies adopted AI?

    That’s also another good question. I can quickly go over the three big challenges posing impediments to large-scale adoption.

    Challenge #1 : It’s Upstream – we lack the data or the talent

    The biggest issue is upstream – getting the data for AI models. It’s relatively easy to find an AI model and say “Hey we have models, we can start solving problems”. But we need data. Not just any data. High fidelity, high quality, comprehensive, properly-sampled data. And getting data requires solid data engineering talent and software engineering talent – these folks

    a. Determine how to get data from multiple sources into their proper sinks.
    b. Configure multi-stage ETL pipelines to cleanse, validate, and transform data.
    c. Determine if they can source data from a given source

    Getting the right data takes time. It’s not a single month tasks – it can take a quarter to a year to get the right data ( following approvals, legal & compliance, or other factors ).

    Challenge #2 : Do the customers trust AI Model uses?

    Turns out we can’t just throw the latest models all the time to end customers. For legal, compliance, or safety reasons, customers mandate strong understandings of how models arrived to their decisions, which restricts the models to those of high interpretability or high explainability. Deep learning or neural networks get ruled out in such cases in favor of classical models such as decision trees.

    Challenge #3 : Do we know the business problems to solve with AI?

    This is the biggest challenge. Yes, we have incredibly advanced models, but, do we know how to use them? Engineering talent, product owners, and leadership needs to execute feasibility studies, work with minimal POCs [ Proof-of-Concepts ], and establish the following :

    a. Which business problems can be solved using ML and AI?
    b. Even if we can solve them with ML, is ML overkill? Can we bias towards non-ML solutions?

    Challenge #4 – Do we Have the Infrastructure For AI Use Cases?

    It’s true that we’ve tackled major infrastructure-related challenges in the past – think about the distributed systems paradigms which undergird Big Data technologies ( e.g. Hadoop, Spark ) or large data processing systems ( e.g. Spark, Flink ).

    But somehow, we haven’t fully been able to translate years of research and historical efforts from distributed systems or performance research in Operating Systems to the space of Machine Learning.

    I can hypothesize a couple of reasons causing the limitations ( and modern understandings can be limited, maybe 10 years from now, the landscape will change ). Factors encompass :

    • Complexity and size of data sets – Is it because ML models are so complex enough in terms of their sheer CPU, Mem, or Disk Utilization? Is it the size of datasets that need to be crunched?
    • ML Specialized Needs – we have very solid infrastructure to store large data volumes, but only select databases specialized for large feature sets and expedited vectorized processing. Do we also have general capabilities – monitoring & observability – for ML infrastructure?
    • Specialty Hardware Costs – Perhaps a good analogy lies in the the CPU vs. GPUs debate – there’s very few companies that specialize in GPUs, or the expensiveness of Graphics Cards?
    • Talent Imbalance – observations suggest to me that more people are going into the data science part of ML ( working with models ) than the underlying compute and execution parts.

    Footnotes

    1. If I recall correctly, a couple of esteemed AI researchers and professors have also likened AI tools to be more similar to auto-correct or editing software. ↩︎
  • INVESTING – Understanding Home Buying: Lessons for Millenials – Property Ladder and Building Up Over Time

    INVESTING – Understanding Home Buying: Lessons for Millenials – Property Ladder and Building Up Over Time

    Oh My god I’m in my 20’s and I’ll never afford a Single Family Home ( today ) but I can afford a starter-home?

    And how the heck did everyone else get a head start building up real-estate portfolio and a strong long-term equity basis?

    NOTE – do not misconstrue any of this as financial advice, this is my own musings ( and questionable ideas ). Go to investopedia.com or somewhere with credentialing and institutional backing

    But realistically, do you need to start with a SFH ( single-family home ) in your 20’s? First, there’s the debate of renting versus buying. And secondly, maybe not?

    When I first came to San Francisco, I went to websites like Zillow.com ( and a reccomendation from a real estate agent, Zenlist.com – purpoted to be better ) to analyze the FMV ( Fair Market Value ) of prices for real estate – start-home condominiums, SFH ( Single Family Homes ), and MFH ( Multi Family Homes ). And my god, what prices they commanded. I saw averages hovering in these windows :

    • Starter-Home Condominium FMV : [$400K,$800K]
    • Single Family Home FMV : [$1MIL, $2.5MIL ]

    Now don’t get me wrong – I’m young, and I’m not there ( yet as of today ) for such large-scale financial purchases.

    But I have cursory understandings. And it’s how folks get started, and the maneuvers they take to building up their real estate portfolio over time. Let’s begin

    The Ladder

    World 1 : Your 20s or your 30s, 5-10 years, starting with the start-home at $500K

    Let’s suppose a family exists and want a home worth $1.0 USD million – $2.0 million USD in the future ( at the age of 40? 50? 60? It doesn’t matter, just get there for me eventually, ok 🙂 )? Well, they don’t have $2 million today. but they almost paid off their first mortgage, originally taken at a value of $500K, now appreciated and netting a capital gain of $700K ( really good years of home appreciation and well-negotiated closing costs with agents respecting the fiduciary duty )

    World 2 : Your 30’s or 40’s, 10-years later : start-home’s paid off, now to the next 5-10 years at a $1Mil home

    Ok, now they want to go to the $1Mil dollar home. But wait a second, they don’t need to take on a loan of $1Mil. They have $700K from their previous transaction, and if there’s something I recall about 1031c exchanges ( or another rule, don’t quote me ), they can “migrate” the funds over from the previous home to the new home. That leaves about $300K outstanding.

    Oh no $300K in a mortgage loan. That’s a lot to pay … or maybe not, if you do things right and focus on aggressively paying down the debt over a 10-year horizon ( combined with the savings netted from standardized deductions or itemized deductions, depending on current tax laws ), you can knock it out quickly.

    But even better, you’re in a home worth $1M, not $500K. That’s not just $500K of a delta in the principal. That’s also with respect to long-term capital appreciation. There’s a world of a difference with $500K*(1.06)^10 in theoretical appreciation. Let’s do math!

    500K*(1.04)^10 = 740,122.142459
    1M*(1.04)^10 = 1,480,244.28492
    … wow a world of a difference

    So you’re telling me that some of those people who live in the $1M dollar home actually took on really low loan values? … YEP! They’re just transferring equity built up from earlier.


    Now again, is this realistic? I don’t know. I’m thinking theoretically, ok. Actual rates capital appreciation rates are far far different; 4% for a good bear market is about right to most folks 🙂

    World 3 : Your 40’s and your 50’s, 10-years later again: paid off 1Mil with appreciation, incomes and portfolios have build up, now to $2M?

    Now to world three. The family wants to move to a higher-valued home of $2 Mil. They apply the same maneuver again, using the net profits from their current home sale, yielding around $1,300,000 ( conservatively ).

    Ok but they also have 10-years of their own investments or other portfolio funds, meaning an additional $100,000 sitting somewhere. That gets them to $1,4000,000 funds available.

    Perfect. Let’s pay of 1.4M out of the 2M home’s FMV, netting us a $600K outstanding loan balance. Ok it’s a huger loan again, but, … let’s think, not only have familial incomes most likely risen up ( with tenure, seniority, and time ), but, the family also originally took on a $500K value loan for their first home 20 years ago! Taking on $600K isn’t that far fetched to do again.

    World three is one of the best world’s, because at this point, theoretically going higher exponentially ( to $4M, $8M ) nearly puts you in a very, very small percentage of the world’s population. There’s some ungodly point to which you can’t keep doing this, BUT, you go doing other things ( cuz you don’t know what else to really be doing, I’ll get back to you when I figure that out, please don’t file chapter 9 or chapter 13 ).

    So … what’s my takeaway for my 20’s? What do I do with this?

    I’m told that you’re supposed to … feel less anxiety and to stop holding yourself to ungodly, unfounded expectations that you have to get somewhere that very few of us actually get to. Or … do? I dunno. There’s also MFH ( Multi-Family Home ) investing, house hacking, and investing in duplexes.

    You start off small for a couple years, BUT, you gradually build up. And if all the stars in the universe align, things should … work?

    The old-construction SFH of ( mostly good ) quality with a FMV of $1MIL at bare minimum in a San Franciscan neighborhood is an immediate stretch goal. You need to procure a Down Payment in the 20% ( $200K ) for most conventional lending ( unless you qualify for exceptions ), 5%-6% in CC ( closing costs ), and then account for PITI ( principal, interest, property taxes, and home insurance ) on a per annum basis. It’s a hard ask, even for a DINK [ Dual-Income No Kids ] situation.

    So, let’s start easier with some lower-hanging fruit. It’s not ideal, but, why not a start-home condominium, equivalent to a 2Br2B setting, hovering around $500K-$600K in up-and-coming neighborhoods with solid appreciation rates ? This option seems cheaper.

    And congrats. You just learned how to become a (non-liquid in net worth) millionaire. Because a lot of us expect to become ( liquid in net worth ) millionaires. Which is a story for another day and time.

  • BEHAVIORAL – How do you handle Work-Life Balance?

    BEHAVIORAL – How do you handle Work-Life Balance?

    This is actually a really valid question!

    Valid, not only because software engineering jobs can be occasionally stressful, but also because work-life balance is a trait that others select more strongly for before joining an org.

    Personally, I like to create and to enforce strong boundaries separating work and home. I strive to respect my team’s working hours during the standard eight-hour work day of 9:00 a.m. – 5:00 p.m., especially the core six working hours of 10:00 a.m. – 4:00 p.m. I’ve learnt from others that it’s ok to communicate to my immediate team when I need to set a hard stop. There are some nights that I work late, but that’s because I may have been out and about in the day managing errands, or because I’m “in the zone” – I feel motivated and energized to knock out my tasks.

    When I step away from work, I actively engage and seek out hobbies that enable screen-free, device-free time – art classes, cooking, exercising, fitness, hiking, and spending quality time with friends. I also seek out sources of fulfillment outside my day-to-day work. Engaging in activities outside work rejuvenates and revitalizes me.
    There are occasional times – usually coinciding with the boom-and-bust-esque cycles of my job – where I have to shill out more work. This happens in a couple of scenarios – I need to take over for folks outside on PTO, there’s a lookimg deadline for a major production release, or I need to fast learn new skills. It’s during these periods where I work more than your standard 40 hours per week. But I’ve never worked a 9-9-6 esque role, nor have I committed more than approximately 50-55 hours per week. I have my limits too. I figure that if I want my co-workers to prioritize good WLB, I myself have to practice it too – it’s a feedback loop, where your co-workers emulate your practices and vice versa.

  • BEHAVIORAL ( Senior Level Example ) – Walk me Through A Time That You Mentored Someone

    BEHAVIORAL ( Senior Level Example ) – Walk me Through A Time That You Mentored Someone

    Ok this one is my favorite story1, and for many reasons. Personally, I aspire to mentorship stories. And these stories strongly showcase seniority and industry experience2. Some interviewers like to follow up and ask you questions like “What improvements did you notice”, “Talk to me about your mentorship style”, or “What cadence or schedule do you follow?”?

    Alright, let’s begin!

    Back at Capital One, I noticed a junior engineer on my team struggling. Seeing my own self in him, I hopped on a brief 5-10 minute slack huddle and I communicated that I’d help him get acquainted with our codebase’s backend.

    I recognized his many positive qualities – his zealous passion, his hunger to learn, and his from-the-get-go excitement to deliver immediate pull requests and make an impact to our team’s codebase. But I also noticed his feelings of anxieties diving into new territory and tackling our application’s codebase – which was slowly transitioning from a monolith to a microservice that needed extensive refactoring to reduce code smells, bugs, and runtime errors.

    I thought back deeply to what Google Technical Leads taught me. I wrote a couple of low-hanging one to two week JIRA tickets focused on refactoring our team’s codebase with ENUMS in place of constants, bolstering test coverage for our team’s microservices, and creating a new microservice to retrieve Enterprise assets. I could have done the work myself quickly, but I delegated the tickets off to him because I thought it’d make for fantastic learning experiences on his behalf. More ever, work delegation freed up my bandwidth to shift focus on more high-impact work, such as onboarding new customers, addressing automation gaps, and architecting cloud infrastructure.

    I set up a cadence – on a weekly basis, over the span of three months, for approximately 15-minutes, I engaged in dedicated 1:1’s with him. We discussed a medley of topics – informal and formal – from best engineering practices to navigating organizations. In our 1:1’s, I taught coding skills such as writing clean code, introducing strong logging postures ahead of time, and thinking about bad inputs ahead of time. I also taught how to write satisfactory unit tests that stressed-tested out source code – tests that executed both succeeding paths and failing paths.

    Over time, I noticed significant performance improvements manifest. His code velocity and frequency of change lists quickly increased. We spent less time in back-and-forth reviews, and the number of review comments I left for him dropped. His confidence grew too – knocking out easy tickets helped build his momentum; eventually, he started taking on more challenging tickets and feature requests.

    1. This is one of the best questions to ask someone – mentorship experiences truly show dedicated engineers who embody people skills and technical skills! ↩︎
    2. I’d argue better than other stories, because these are much harder to fabricate “out of thin air”. ↩︎
  • BEHAVIORAL ( Senior Level Example ) – Overcoming Senior/Staff+ Behavioral Interview Challenges

    BEHAVIORAL ( Senior Level Example ) – Overcoming Senior/Staff+ Behavioral Interview Challenges

    “You get better at what you practice” ( but only what you practice 😛 )

    Behavioral interviewing ( at higher organizational levels ) is harder. Personal learnings have let me identify reasons contributing to their difficulty. Let me share my insights ( from failed and passed behavioral interviews )1!

    What makes them difficult?

    a. Scenario-specific experiences lacking – it’s hard to answer what you haven’t encountered such as “How you handled a crisis” or “How do you handle failure”. How do you answer if your company’s flagship products don’t entail a need for production on-call duties ( there’s no actual crisis )?
    b. Longer-term tenure needs – scenarios usually arise from having created, shipped, and managed features to production environments. It’s harder for someone who works a role for less than a year to answer questions with satisfactory depth.
    c. A literal interpretation of questions – you can be a junior engineer and still “lead” a project. Just because you’re not a manager or a staff+ engineer doesn’t mean you didn’t “take the lead” or “drive the conversation”.
    d. Over-indexed technically – to many engineers over-index their leetcode or system design skills – skills that are easier to practice individually.
    Partial blame rests on the inherent design and the process of company on-sites – leetcode-style coding questions occupy the bulk of their time. Asa the adage goes, “you get better at what you practice”. Let’s also run a calculation – suppose a senior engineer does on-site loops across five companies emulating a five-hour long big-tech esque style interview : a one hour coding phone screen, two hours of coding, one hour of system design, and one hour of behavioral interviews. Multiplying by the factor of five, we end up with 3*5 = 15 coding hours, 1*5 = 5 design hours, and 1*5 = 4 behavioral hours. Our day jobs and our interviews loops emphasize coding over communication.
    e. Under-indexed behaviorally – there’s a need for more leadership, soft skills, or advisory-role emphasis – engineers need to get practice in other role requirements that are harder, such as work delegation, task decomposition, setting meeting agendas, and working on codebases at “higher levels”.
    f. Subjectivity – technical and design interviews are easy to evaluate ; does your code work, did you arrive at a working solution, did you pass all the test cases, did you design to account for failure modes and scalability concerns? They nearly follow a templated structure ( see Alex Xu’s system design books ). Behavioral interviews are more subjective . Your interviewers index for character traits ; some desire an R&D-esque person who’s good at admitting when they don’t know something; some emphasize stronger leadership; others select for mentor-figures to grow junior talent. What helps you pass one doesn’t help you pass another.

    What helped me ( and may help you )?

    I struggled to understand how to answer commonly-encountered scenarios – online posts and other book material was really drab and dull to go through.

    I lucked out on finding the ( at the time, around $25.00 USD ) The Behavioral Interview Deck2. It helped tremendously ! The color-coded, gamified way of preparation got me through. On a quotidian, daily basis, I practiced one to two cards.

    Step #1 : Play the cards that play to your strengths!

    I focused on cards that played to my strengths and my experiences and answering questions that I knew how to better answer. Let me share the example questions ( and parts of responses ) that helped me :-). Some as S.T.A.R.-esque answers; others deviate from the template3.

    • What motivates you?
      – Collaborating with other engineers – my best work has never been individual, it’s been working in collaboration with other engineers
      – Helping someone else “click” their mental lightbulb – seeing someone’s brain light on fire is amazing.

    • Describe a time when you had to troubleshoot a problem.
      Situation : An enterprise event-stream failed to process payloads one day, and my PM noticed anomalies
      Task : I had to investigate and determine the root cause of failures
      Action : I bolstered our logging and debug posture by introducing more statements and making log outputs more verbose. I re-ran experiments in production and discovered that I needed to expand the rules engine for failed payloads to account for an unexpected edge case scenario with stale server-side assets on valid client events.
      Results : I expanded the underlying rules engine and re-deployed our team’s application to production with minor changes. We stopped observing failures for the next couple of days going forward, and felt more confident in our delivery.
      4
    • Can you describe a time you had to learn a new skill?
    • Describe a time you had to adapt to a new situation

    Tackling the easier, low-hanging fruit questions helped me build momentum and helped me build a “mental” response bank for more rigorous questions.

    Figure 1 : “The Behavioral Interview Deck” – a gamified way of learning common behavioral prep questions with practice scripted answers on the reverse side of each card.

    Step #2 : Don’t memorize the scripts. Modify them for your weaker cards.

    I have my weak cards too. And for those, I have to leverage superpowers – creativity and out-of-the-box thinking.

    One question example – “Have you ever manage a crisis”?

    Ok so in reality, I’ve seldom run into a “crisis“. I haven’t been involved SEV-0 incidents where I hopped on an all-hours PagerDuty call, spending hours to resolve production bugs at 2:00 a.m.. Neither did I run into a crisis where I accidentally caused production to metaphorically “go on fire” or generated a net loss of $1 million plus in revenue.

    Do I have the stories5 to present at higher levels, such as taking on the duties of a direct manager or a senior staff engineer who explicitly received customer complaints regarding major data breaches? Did I ever “break the glass” to activate internal data handling mechanisms to contain and to minimize the consequential blast radius? Absolutely not.

    But I did manage my own levels of crises, as a senior. I ran into a situation with recent production releases to meet feature deadlines. I had to learn how to address issues pushing features, which I said would work in production, which clearly did not, because I signed off of them to early.

    And in the process, I had to coordinate with two other developers and my direct manager on pre-production release meetings and production release meetings to push a feature deadline back by a few days. I had to work through a series of steps and review a mental checklist of checks – configuration values, secrets, network connectivity, and rules engine code deltas for unexpected edge case scenario handling . I also coordinated with my dev team to verify if we could deploy the changes as hot fixes on a release branch ( an undesired practice, since this avoids pre-check safety steps such as code reviews ).

    And I learned. I learned a lot.

    I learned that its easy to stay calm and composed by working together with other developers and not solutioning issues alone.

    I learned how to collaborate with other developers in high priority release meetings.

    And I also learned how to address issues collaborative and with efficiency. We delivered a working and more robust version of the feature to production. Basically, we “went-live”.

    Step #3 :

    1. This post was inspired by a friend who asked for help on preparation! ↩︎
    2. I’m not affiliated 🙂 . https://9to5cards.com/product/the-behavioral-interview-deck/ ↩︎
    3. We ( in my honest opinion) over emphasize S.T.A.R-esque responses. Not all behavioral questions need you to follow this format – you can deviate and answer differently, but genuinely ↩︎
    4. (As of the articles date ) 05/10/2025 ↩︎
  • Data Engineering – Four Flavors of ETL Exports Explained: Challenges & Strategies

    Data Engineering – Four Flavors of ETL Exports Explained: Challenges & Strategies

    Hello everyone!!!

    Today, I wanna go over the different types of database scans & exports I’ve executed – or will be executing – in my current workplace. What makes each rigorous and unique? Why were they developed out in their corresponding order? And can we apply to learnings here to case studies or upcoming scenarios? Do the ETLs exports developed here translate to other case studies or data engineering scenarios?

    The four ETL-esque Exports Horsemen

    1. Historical – scanning the set of all records accumulated so far. Analogize to reading every row of an Excel sheet
    2. Automated – can we trigger an ETL/export upon receipt of a new/latest scan?
    3. Incremental/Delta – what if a record, or a table, previous scanned changes? Say someone adds a new subset of columns OR updates a subset of rows? Can we export the delta of the table ( versus the entire table )?
      • Snowflake, DB2, Blaze
    4. Rescan – On a 10-Q quarterly basis ( or other, yet to be determined ), we executed a mandatory export again over all records scanned so far. Analogize it to execute of a historical export again. The rescans are required to meet legal & compliance requirements as per data retention, privacy, and security rules mandated by government entities.

    As of the start of May, I’ve executed work on historical and I am currently executing work on automated. I’ll soon migrate over to incremental/delta and rescan, once the first two wrap up 🙂

    The Data Sources

    My capabilities focus on a subset of databases – relational and non-relational, transactional and analytical.

    • Relational SQL – SQLServer
    • Non-Relational NOSQL – PostgresDB, DB2, Snowflake
    • Upcoming : Oracle

    Fortuity had it that my first four were easy to work with – SQLServer, PostgresDB, DB2, and Snowflake. Each of these databases

    (A) Already had a chunk of their development done.
    (B) Came in only one flavor.

    But the upcoming ones yield more flavors and rigor? Specifically, Oracle and Atlas with three flavors – one on-premises and two on-cloud ( IaaS and PaaS ).

    The adoption of Airflow DAG

    1. Commonly used mainstream tool
    2. Other teams are using them – easy for a team internally in acompany to adopt

    What are the Challenges ( Encountered and Upcoming )?

    Extending the feedback loop mechanism, across existing and upcoming environments, posed multiple challenges. Let me go over a couple :

    1. Tradeoffs : sparse information versus dense information
      • Sensitive Data Elements and PII are usually sparse. Can we make optimizations ahead of time?
    2. Onboarding an additional data source ( or data source flavor )
      • Modifying the underlying rules engine and making features that work for one source work for another.
      • Preventing future changes from breaking pre-existing changes.
    3. Waiting on dependencies and major blockers from other teams.
      • Wait for their data / DDL ( data definition language ) ingestions
      • Wait for their release of API endpoints
      • Wait for their applications to update.
    4. Addressing network/connectivity/permissions/access issues (Azure Key Vaule rotations ) boundaries across devices and environments ( e.g. Windows versus MacOS )

  • PERSONAL – so how should we assess & evaluate engineers? Good question – it’s definitely more than just lines of code [ LOC ].

    PERSONAL – so how should we assess & evaluate engineers? Good question – it’s definitely more than just lines of code [ LOC ].

    Hello everyone !!!!

    Today, I wanna dive into a technical opinion of mine – reasons why I’m against using LOC [ Lines of Code ] or Number of CLs [ Change Lists] as workplace performance indicators. They’re useful proxies, not exact perfect indicators, of leveling or performance. Let’s go into why.

    Reason #1 : Good Engineers Usually Write Code – that Deletes Code? Wait what?

    Conventional patterns typically hold true – the more senior and the more experienced a developer is, the more frequent they write and they release new source code.

    Not exactly!

    Patterns also have their exceptions and anomalies. And I’d also argue that the best engineers write little to no code. Experience teaches me that the best engineers are so good, that their changes tend deletative – they eliminate large chunks of code, configurations, data, and static files. I’ve seen engineers introduce code that entails a couple of extra steps and a dynamic configuration, but in the process, can delete 5000+ LOC or 20+ statically-written out files. These refactor efforts bolster long-running codebase health.

    Reason #2 : LOC makes no sense for design work

    LOC transitions to become a worse metric for more senior levels which naturally involve less coding. Seniors naturally spend more time in mentorship roles or advisory-esque roles – designing systems and solutioning multi-quarter ambiguous problems. What should the evaluation criteria entail for seniors or staff+ folks who spend six hours weekly in the design phase of stories, with minimal code units? What are good indicators or progress? Does delivering a rough-draft design document or a fully-fleshed out design document count? Or user stories and the minimal code for proof-of-concepts ( some of which can be quickly discarded )?

    Reason #3 – Configuration Changes Carry Impact

    We can run into scenarios where a few lines of code yields substantial impact; they are less “visible”, BUT, they carry long-running positive benefits. Let’s imagine engineers discover changes such as :

    1. Better failure hailing – bolsters retry failure mode handling ( e.g. changing the number of retries for exponential backoff )
    2. Deployment changes – Shifting an application’s YAML file from blue-green traffic deployment ( from 0%-100% sudden shift ) towards a canary traffic deployment ( 0%-20%-…-100% shifting ).
    3. Auto-scaling policy change – discovers that they can change an auto-scaling policy’s min-#-instances and max-#-instances and saving on long-term cloud compute usage.
    4. Expediting build-and-releases – pipelines and environments that aren’t up-to-date with the latest configurations slows down developer velocity. It turns out that the work done to execute R&D and set up the right set of configurations, dependencies, and tests for environments can yield tremendous impacts in feature delivery pace.

    Reason #4 – Savings aren’t always in code

    Did a developer figure out that we can save on database storage by updating our underlying schemas? Did they figure out how to leverage pre-existing compression algorithms and strategies?

    Reason #5 – LOC and User Stories are “gameable”

    Turns out smart engineers can easily figure out how to create a series of incredibly small change list and submit their code at a higher frequency, then they would have if they submitted their same units of code, but in slightly larger change lists. An engineer can figure out how to create to many user stories with minimal task sets, versus sufficiently-sized stories with an apropos number of tasks.

    So how should I assess developers?

    This is a really good question, and in my honest opinion, it’s better deserved to be on a case-by-case, set & setting basis. There’s no “one-size-fits-all” answer – and there probably won’t be one – since no single person has actually figured out the perfect approach. Intuition, instinct, holistic feedback ( from management, peers, and stakeholders), and project visibility play their roles in gauging long-term quality.

    There’s a couple of better, but still imperfect measures, such as :

    1. Total business revenue saved
    2. Total profits generation
    3. Number of customers/teams/end users onboarded.
    4. Conversions of initial design documents to production-ready features
    5. Number of direct reports/mentees getting promoted1.
    6. Number of interviews conducted.2
    7. Demonstrates initiatives and takes on the right number of problems at the right level of scope.
    1. This metric is more typical for leadership & management. Last I correctly recall, the success of managers is based on the success of their teams. ↩︎
    2. Also questionable, since there’s hiring seasons and times when folks conduct fewer or more behavioral, coding, and system design interviews. ↩︎

  • TOTW/16 – TESTING : Avoid Development Crunches and Reduce DevOps Stress ; keep above test coverage metrics by meaningful deltas

    TOTW/16 – TESTING : Avoid Development Crunches and Reduce DevOps Stress ; keep above test coverage metrics by meaningful deltas

    And release features to production with a lot less stress and a lot less agony !!!

    The animus – why does this matter?

    Hello everybody!!!

    Today’s post is motivated by scenarios that I’ve run into three times in my career so far across three different companies – bolstering unit test coverage to meet an enterprise coverage metric. The fact that it’s happened to me and other the other engineers I worked with at least three times intimates at its cruciality.

    Corporations typically mandate that committed source code meets an expected percentage of unit test coverage before shipping to production ( e.g. 80% total coverage ). This is done to ensure that developers not only follow and abide by solid TDD practices, but that they also write sufficient tests ( lest bad habits accrue and test coverage precipitously drops to percentages such as 40% or 20% ). In a sense, the coverage metric acts as a “forcing function” – it ensures that industrial codebases remain healthy and up-to-date with solid engineering practices.

    But there’s many times where coverages drop below expectations. This happens when engineers decide to meet their coverage metric exactly at the measure or only a few points above ( e.g. if coverage expectations are 80%, they meet around 80% – 84% ). Yep, meeting the bar technically passes and prevents stalls in the build and the release of artifacts in a DevOps pipeline, but the hack is short-term ; it’s prone to dropping back to a lower value. This can happen if a developer introduces drastic changes to source code or suddenly submits new files as part of a latest feature release.

    But when do I care about test coverage drops?

    Excellent question!!! Let’s suppose a feature needs to make its way to production within a Sprint’s two week scope, and your changes decreased test coverage from 82% to 70%. Due to a dearth and a lack of Github pre-commit checks ( or other safety features), your submission to build and release pipelines takes 15 minutes 1and fails to pass all stages due to failures at the intermediate coverage stages.

    It looks like you have work cut out for you – you can expect a day ( or two ) getting that unit test coverage up again to meet expectations. And in the span of a dedicated sprint scope lasting 10 business days, this leaves 4/5ths of the remaining sprint dedicated to feature work. You’ll feel more stressed our and crunched for delivery. Could we have avoided the delivery stress and let you have all business days of a Sprint dedicated to feature testing and release?

    What’s the better state?

    The better ideal is having teams or organizations enforce stricter, more meaningful coverage levels ( e.g. 90% or 10% above coverage expectations ) all the time2. The drawback is that doing so entails more front-loaded efforts on developers ( which IMHO is a good thing ). But the benefits outweigh them – we circumnavigate the feelings of anxiety and frustration from deadline crunches when a feature suddenly needs to get productionized.

    Now it’s an imperfect fix – we can still expect black swan scenarios where test coverages precipitously drops on major source code changes ( e.g. a massive codebase rehaul ). But we have done the hard, assiduous work lowering the probability of the black swans, meaning that running into the delays goes more unnoticed in one’s career. Thus resulting in a better experiences for the developers, the producer owners, and the leadership of a tech org.

    Footnotes

    1. Ideally, we leverage pre-commit checks instead of a DevOps pipeline stage to catch the discrepancies earlier and save on developer cycles. Could we have spent 2 minute in a local environment instead of 15 minutes building and releasing massive artifacts in a remote environment? ↩︎
    2. There’re many methods to do so – examples encompass github pre-commit checks or introducing new pipeline stages ↩︎

  • TOTW/17 – TESTING – Leverage Unit Testing Stories to Onboard Junior Engineers

    TOTW/17 – TESTING – Leverage Unit Testing Stories to Onboard Junior Engineers

    A primer

    Hi all,

    I briefly want to talk about a strategy 1for onboarding junior engineers; writing out unit tests and engaging in code refactors.

    Usually, I’ll write up JIRA-esque user stories and tasks instructing something along the following :

    The JIRA-esque testing user story

    Subject line : [Testing] <Insert_Feature_Name> Bolster unit test coverage metric to exceed >= <insert_some_enterprise_threshold>
    Descript : Unit tests are either lacking or need to be refactored for a given application. This is needed to not only meet Enterprise coverage requirements, but also expedite the productionization and deployment of our application and avoid delays or stalls in the build stage and the release stage of our team’s deployment pipelines.
    Task #1 : Bolster test coverage for microservice <A>
    Task #2 : Bolster test coverage for microservice <B>
    Task #3 : Bolster test coverage for directory <C>
    Task #4 : Bolster test coverage for file <D>
    Points : Five

    I’ll put the user story in the backlog and then delegate them off to junior engineers or mid-level engineers with bandwidth for the current Sprint or upcoming Sprints.

    Personally, I think it’s both the de facto industry-standard way and an effective way to onboard2 engineers. I used it before when I was a senior engineer ramping up two junior engineers, and I follow this approach when I enter new codebases.

    But why should junior engineers – or for that matter, newcomers – focus on unit tests? Why can’t they immediately dive into long-term feature or project deliverables?

    • Industry codebases != undergraduate code bases – academia is hard, but in its own unique flavors ( it leans more theoretical ). But large-scale, industrial code bases are another leviathan . Navigating a monorepo with 1000+ LOC ( lines of code ) and learning how multiple components in a system move takes time. Even for a senior engineer or a staff engineer, it can take a solid one month ( or two ) to gain familiarity with a foreign codebase.
    • TDD 3is a must-have skill to learn – academic settings emphasize writing code, but they seldom emphasize writing tests. Industry, on the other hand, emphasizes TDD. Getting junior’s acclimatized to writing tests helps them develop the right set of long-term habits.
    • Facilitates codebase comprehension – writing new source code is hard, but writing unit tests in isolation or making cosmetic changes to code is easier. Unit tests can be written at the level of decomposed functions, and developers can leverage mocks or AI chatbots to write out the structure of tests. It’s quick to generate and test out a coverage report locally ( e.g. python3 run coverage report` ) than it is to make changes and test our source code deltas.
    • Fast to deliver – each unit test itself is an isolated unit, as well as its associated file. Changing source code oftentimes introduces breaking changes across an application, but changing tests typically do not. Thus, its apropos for junior engineers, who tasks involve timelines which typically span from a week to a month.
    • Testing helps developers understand what other developers were thinking – how can a developer quickly understand why a piece of source code was written? How can I quickly answer “why did we write what we wrote”? By looking at their associated tests. It turns out that tests quickly inform developers about the behaviors, the user inputs, and the end expectations that original code writers desired seeing.
    1. Credit due to engineering talent at Google who taught me about writing solid unit tests and test fixtures – both explicitly and implicitly. ↩︎
    2. Do correct me if I am mistaken here 🙂 ! ↩︎
    3. TDD – Test Driven Development ↩︎