LLM adoption has reached a critical inflection point in the enterprise. While the majority of organizations rely on managed API providers to accelerate time-to-market, transitioning from experimental prototypes to business-critical production systems reveals a significant hurdle: upstream throughput constraints.

Managed LLM APIs abstract away infrastructure complexity but introduce a “black box” variable—rate limits. When multiple distributed services, background workers, and real-time user requests compete for a shared upstream quota, rate limiting ceases to be a simple client-side error and becomes a core systems-design constraint.

In this post we try and propose some solutions to the rate limiting problem when interacting with LLM providers.

Problem Statement

LLM providers typically enforce multi-dimensional limits to maintain multi-tenant stability. These are usually defined by:

• RPM (Requests Per Minute): Limits the frequency of orchestrating calls.

• TPM (Tokens Per Minute): Limits the actual “compute” or payload volume.

• Concurrency: Limits the number of active HTTP connections.

These limits are generally existing to provide fair and consistent service to all their customers and ensure service stability.

A sample LLM Application functions something like this

In a naive architecture, scaling traffic simply increases the probability of breaching these thresholds. Without sophisticated backpressure mechanisms, a burst in traffic doesn’t just slow the system down—it triggers a cascade of failures that can compromise the user experience and system reliability. The challenge is not handling individual failures but ensuring that the whole system remains resilient

Let us see few patterns which can help ensure the resiliency

Pattern 1: Exponential Backoff

Exponential backoff is the simplest of resilient patterns to handle rate limiting, By rate limiting what the server is telling us is we cannot get any responses right now as we have exhausted the quota by a burst of traffic, So instead of constantly hitting the servers and asking it to serve we add a retry but in exponential time. That is the client waits for a progressively longer period after each subsequent failure.

However, Standard backoff is insufficient for distributed systems. If ten workers hit a rate limit simultaneously and retry on the same schedule, they create "thundering herd" spikes that re-congest the API. Introduce Jitter (randomized delay). This desynchronizes retries, smoothing out the request distribution and increasing the likelihood of hitting an open window in the provider's bucket.

Pattern 2: Multi Model Fallback

This is also quite simple to implement, Instead of just requesting one model which has limits have multiple LLM models from the same provider or different providers, for example if we are choosing claude-opus model and it is responding with 429 Too Many Requests, We should consider requesting a different model for example a Sonnet or Haiku Model or even a lower generation Opus model.

The advantage to such a chage is that traffic is now distributed across models if one model is congested. While it maintains availability, it introduces output variance. A prompt optimized for a flagship model may produce hallucinations or formatting errors when executed on a smaller, faster model.

Pattern 3: Multi Model Fallback with Appropriate prompts

To mitigate the downside of Pattern 2, the system should store versioned, model-specific prompt templates.

• Mechanism: When falling back from a high-reasoning model to a lower-tier one, the application swaps the prompt for one with more explicit few-shot examples or stricter constraints to maintain output parity.

• Challenge: This increases the maintenance surface area. You are no longer managing one LLM integration, but an “N-model” matrix that requires robust evaluation (Evals) to ensure consistent behavior across all fallback paths.

Pattern 4: Event Driven Processing

By moving from a request response model we now ensure that there is no urgency to serve the response, We can control the following things by moving to a queue based model

1. Backpressure Absorption: The queue acts as a shock absorber for traffic spikes.

2. Concurrency Control: You can limit the number of active consumers to exactly match your provider’s Tier limits.

3. Prioritization: You can route “Premium User” requests to a priority lane while background tasks wait for available quota.

Conclusion

External AI providers are distributed systems with shared, limited capacity. To build at scale, we must treat rate limits as a first-class architectural constraint rather than an edge case.

Successful LLM platforms do not just retry; they orchestrate. They distribute traffic across providers, control global concurrency, and ensure prompts are portable enough to survive a provider’s downtime. Retries may recover from failures, but architecture determines whether those failures happen in the first place.

PS: All diagrams have been generated with the use of AI Model (Gemini)

Singleton pattern

Singleton pattern is a design pattern by which there is only a single instance of an object in the entire application. The common usage of Singleton pattern is to load and maintain compute heavy items that needs loading into the memory such as DB connection, config managers or Machine learning models.

Image Credits: https://belatrix.globant.com/us-en/blog/tech-trends/the-singleton-design-pattern/

How is Singleton Pattern used in Model Serving?

In a machine learning inference service, models must be available to process incoming requests efficiently. Typically, models are downloaded at runtime and loaded into CPU or GPU memory before performing inference. If the model object is re-created for every request, this process repeats—incurring unnecessary network transfer and memory allocation overhead. A common solution is to implement a singleton pattern, ensuring the model is downloaded and loaded into memory only once during the service’s lifecycle. Subsequent requests can then reuse the same in-memory model instance to execute methods such as predict(), significantly reducing latency and resource usage.

LRU Cache (Least Recently Used Cache)

Least Recently used Cache is a cache management technique where the recently used objects are stored at the top of the memory. The LRU cache is implemented by using a combination Doubly Linked List and Hash Table data structure. In Python programming language the lru_cache is available through an inbuilt python library called functools

How is LRU cache used in Model serving?

The LRU cache can be used with maxitems=1 to store the model in memory, this ensures that only that object remains in memory and it basically implements a neat version of the SIngleton pattern mentioned above. However LRU cache also offers switching models when there are multiple large models which need to be replaced in memory. By increase the size of the LRU cache we can hold models which are much frequently used in memory and for models which are infrequent in usage can be loaded into memory as and when required.

Usage

• Use Singleton Pattern when we want to serve a single model in a service and it is guaranteed that model will not change for users, The advantage that it offers is minimal memory usage.

• Use LRU cache when there are multiple models and access patterns are not very predictable and less used models need automatic eviction from the memory.

There is a business case where the organization is dependent on an API to fetch the data and luckily the data can be stored and used for downstream analytics later. This API costs a lot and the data also needs to be stored securely. But the idea is to not write a script which is put together of all the API calls panda conversions etc that will fail on the drop of a hat because it is meant as a one time usage

Requirements

The script should

• re runnable

• configurable

• fast velocity of development

• easy to maintain.

Tasks

1. Create a function that fetches the data from the API

2. Convert the API response into a dataframe

3. Store the dataframe in a database

4. Create a pipeline that does this in a sequence.

Solution

We are going to build the entire thing using prefect and python. I tried my hand at airflow but it just seemed so complex with binaries etc that prefect documentation seemed like a breeze.

0. Install prefect

Install this inside a vitual environment

pip install -U prefect

Code snippets

1. Create a function that fetches data from API

Since the API call that I have to make is a post API call, the input data to be sent in the request is stored as a modifiable content in a json and then loaded during the task phase. This would be a task in the prefect universe which is a smaller brick in the larger pipeline represented as a flow.

import requests
import json

def fetch_data(url, data):
    response = requests.post(url, data=json.dumps(data))
    return response.json()

2. Convert the API response into a dataframe

Once the response is obtained as json convert it into a dataframe by using pandas to concat if the dataframe is not empty and if empty just create a dataframe from json

3. Dumpt the Data to a Postgresql database using SQLAlchemy and pandas

Create a engine for dumping the data using SQLAlchemy and then use df.to_sql() method from pandas to just dump the data into the postgres table

Putting the above parts together

steps 1,2 and 3 are individual tasks in the pipeline, now just write a simple python function which will basically execute this in sequence and then add a @flow decorator to the top of the function something like this

from prefect import flow, task

@task
def fetch_data(url, data):
    ...

@task
def convert_to_df(response):
    ...

@task
def dump_to_db(df):
   ...

@flow
def my_etl_pipeline():
    response = fetch_data(url, data)
    df = convert_to_df(response)
    dump_to_db(df)

Now all that needs to be done is to run this script as if running a python script and this accomplished ETL and the localhost at port 4200 gives the dashboard of the task execution etc. Such a simple way to run simple workflows

In this post I will go on a journey where we assume that we know about synchronous client server interaction to various async patterns and how each of them solve the problem of scale.

Client Server Architecture

In the world of application development which involves internet, there are basically two components one is the frontend/Client which is accessible through a browser or app where people can click or enter data and perform some actions, which is then sent across to the server (some form of a computer) that performs the logical operations on it. The data is relayed from client to server through a secure network connection.

As consumer we would want our actions also technically known as requests to be executed as fast as possible which is the right of the customer.

Synchronous Communication

While simple to implement, synchronous communication in client-server systems suffers from several limitations that impact scalability and responsiveness. Each request blocks the server, occupying resources (like RAM) until the entire logic executes. This creates a single thread of execution, meaning the server can only handle one request at a time. In multi-process systems, this forces you to allocate a dedicated machine for each customer, even if their requests take minimal processing time.

This is why Synchronous communication can be disadvantageous for dynamic and scalable systems. This also brings in

• Inefficient use of compute power

• Scalability bottlenecks — There is only so much machines that can be provisioned at a time

Asynchronous Communication

In an asynchronous communication pattern, the server does not immediately return a response. Instead, it initiates the request and allows the client to continue execution without waiting. The processing can then happen concurrently on the server, and the client receives a notification or the processed information at a later date.

The detail lies in the implementation and if incorrectly implemented async await operations can be slower than synchronous communication patterns.

Patterns of Asynchronous Communication

Now that we understand that asynchronous patterns are essential for building a scalable system let us try and understand different patterns of communication.

• Request Reply Pattern

• Publish Subscribe

• Fire and Forget

• Event Driven

• Websockets

Request Reply pattern

The request-reply asynchronous pattern allows a client to send a request to a server without waiting for an immediate response. This is the pattern that is closest to the synchronous request reply pattern. This frees the client to continue execution on other tasks while the server processes the request concurrently. Once processing is complete, the server sends a response back to the client, often through a callback function or message queue. This pattern is particularly beneficial for tasks involving network calls, database interactions, or any scenario where the client doesn’t need to wait for the server’s response to proceed. It improves responsiveness and throughput by allowing the client and server to work independently. However, it introduces additional complexity compared to synchronous communication, as the client needs a mechanism to handle the eventual response.

This pattern is generally implemented using async/await framework and the coroutines in languages such as python.

Publish Subscribe Pattern

The Publish-Subscribe pattern also known as pub sub is an alternative pattern of asynchronous communication. The advantage of pub sub over request reply is that pub sub patterns allows for a loose coupling between publishers and subscribers.

The loose coupling is established by using a message broker component which uses topics to establish connection. The message broker efficiently routes the message to all interested subscribers. This pattern forms the backbone of microservices architecture.

Some common implementations are message queues and event buses. Some of the open source implementations for usage is the RabbitMQ, Apache Kafka.

Fire and Forget Pattern

The fire-and-forget pattern is a simple approach to asynchronous communication where a sender transmits a message without waiting for a response or confirmation. This pattern prioritizes sending the message quickly and efficiently, and the sender does not track its delivery status.

It’s often used for one-way tasks like sending logs, notifications, or initiating background processes. While fire-and-forget offers simplicity and speed, it lacks the guarantee of message delivery.

Common implementations include message queues, email clients, or any system that allows sending messages without requiring feedback. However, it’s crucial to consider the potential for lost messages and design your system accordingly, especially if reliable delivery is critical.

Event Driven Pattern

The event-driven pattern structures applications around the concept of events — significant occurrences within the system. Producers (components that trigger events) publish these events, carrying relevant data about the happenings. Consumers (components interested in these events) subscribe to specific events or categories.

When an event is published, the event broker efficiently routes it to all interested consumers. Consumers then process the event information to perform their designated tasks.

This loose coupling and asynchronous nature make event-driven systems highly scalable, responsive, and adaptable to changes. Common implementations include message brokers, event buses, and pub-sub systems configured to handle event routing and delivery.

The event-driven pattern is well-suited for microservices architectures, real-time applications, and scenarios where components need to react to changes in the system without tight dependencies on each other.

Websockets

WebSockets, the persistent two-way communication channels for web applications, thrive alongside asynchronous programming patterns. Here’s how they work together:

• Persistent Connection: Unlike traditional HTTP requests, WebSockets establish a long-lived connection between the client and server. This eliminates the need for repeated connection setup, improving efficiency.

• Asynchronous Communication: Both client and server can send and receive messages asynchronously, meaning they don’t have to wait for a response before sending the next message. This allows for real-time data exchange without blocking the main application flow.

• Event-Driven Model: WebSockets often leverage an event-driven approach. The server can push messages to the client, triggering event handlers on the client-side to react to the received data. This enables real-time updates and interactive experiences.

Conclusion

While sync patterns are easy to implement async patterns bring real scalability to code that is written to bring software products to users.

If you also want AI systems to deliver business value, you have come to the right place. To deliver proper business value there needs to be a deep understanding of how to create reliable systems that can consistently deliver value. Having a couple of machine learning models that do well on notebooks but can’t be put into production does the opposite.

In this post, we will talk about how to think about a system that can deliver business value using machine learning effectively.

Maturity of Machine Learning

To deliver value there is a need to understand where we stand and what we need to provide. There are multiple models of the Machine Learning Operations cycle that can help us understand this scenario. Out of the different maturity models, the Azure MLops maturity model is the one I could completely relate to in increasing order of complexity and milestones.

Since we are mostly concerned with designing machine learning systems in this post the emphasis from the Azure model is on the Model creation, Model release and application integration phase. The organization aspect of things isn’t covered in this post but is also a main pillar to ensure that there are teams of different strengths coming together to build a reliable system.

Since we are concerned with bringing a machine learning model to production the emphasis will be on Model creation, Model release and integration with the application. There is another aspect of building cross functional teams of various capacity which we will not delve into detail in this blog post as that is a post for another day.

The issues with most teams across the board is that data scientist generally come from a non engineering background and hence are required to learn skills beyond their paycheck and engineers need an understanding of machine learning which again exceeds their role capacity. The end result is that we have data scientist building a model and handing over to engineers to make it into deployable software into production.

While you can refer to the machine learning maturity model here and maybe look at the images below from here. I would go on to talk about stages of deployment.

Stages of Machine Learning deployment

Minimum Deployable Product:

Like in a startup where there is a minimum viable product on which there are iterations performed we should start thinking about Minimum deployable product which can get the customer some value and is a move towards the grand utopia of no manual intervention anywhere in the ML training cycle.

So let us start thinking about how to serve simple machine learning models into production first, the LLM world brings hardware challenges which is probably too much to aspire if the organisation hasn’t deployed even simple machine learning models.

We would have a machine learning model at this point in time this can be stored at a cloud storage location or at the server. The first step is to wrap an API with endpoints that can take requests and can then return an inference value of some meaning.

For example if the problem is a classification problem just returning a vector of 0 and 1 makes no sense to the end user. This API would do the conversion to possible categories and return value to the user that can be either consumed by the frontend client for display or by other services to do some processing.

In addition to the API we should add some testing in the code unit tests to ensure that functions work at an implementation level and integration tests so that we know that it works with the entire system. How to write these tests is not our main focus now. Add a CI pipeline where we can check for builds and tests in an automated fashion whenever we push code into the repository. This should offer some basic software testing reliability to a model that has not seen production.

Automatic Retraining

Now that we have a minimum deployable version which is available as a containerised deployment in a container registry we should probably take a look at how can we achieve the next feat of making the retraining process smooth, so that in case of issues with the deployed model we can automatically get the data, perform pre-processing and then do a training of the model using the newer data.

Let us take a look at some of the components and why they would be needed at this stage.

• Data Pipelines and data contracts: With data pipeline and data contracts we can fetch very reliable data that meets the input requirements of the model and the data contracts can ensure that this happens in a very type safe fashion.

• Data versioning: While we do retraining it is probably a good idea to maintain the version of the data using timestamps and other metadata requirements. Tools like DVC help us maintain versions and this is essential to create a reproducible training environment if there are issues with the training or any

• Training pipelines: Building Machine learning pipelines where it is probably good practice to use orchestration to run the preprocessing, training and testing as a part of single flow. By using orchestration tools we can modularize the code and recognize if there is any issues in a particular module.

• Model registry: Model registry can be thought of as a metadata store for models which can help version models, provide aliases, tag and annotate them, it provides an interface which solves when multiple versions and multiple models are produced by having something unique.

• Machine Learning monitoring: While a full fledged monitoring is not required an observability framework that can monitor the drifts in models, data is required and maybe an alerting system which can notify when the threshold of performance is declining can be present so that there is a manual intervention from the team on whether the models need to be retrained or not.

While these components can add complexity to the existing workflow it is necessary to remove obvious mistake when scale of model creation increases. The best way to approach this would be to have a team that can maintain these components as a ML platform and the data scientists can be their internal customer who can use the platform.

Complete Automatic Machine Learning pipeline

This stage is attained by very few companies where everything is automated and there is minimal manual involvement in making things work together. It is the utopia that people aim for when they start building an MLOps department. while there is minimal manual intervention needed in between components we would still need a team that can add or remove components perform some spring cleaning to keep the platform clean and technical requirements running up to data.

Conclusion

While the blog limits it within the maturity model, there needs to be a deep dive into each component and this can be extremely challenging. While MLOps is still not a matured field each deployment choice can have different effects on the end product. My aim with this article was to bring some thought into what are different stages and how a beginner can approach each stage depending on their specific requirements.

I havent covered the details of each component and that is something for the future. For starters I would recommend this blog series for building a minimum deployable product version and then the journey deepens quite a bit.

I have been building machine learning models for over 5 years now but now that I work with writing software for a huge company the most common thing I notice is that security is considered paramount so I started researching about what are the security risks when we build models what are the things that I can ignore when I just build a model using some dataset that I find doing a cursory google search and try and get people to use it. While my impression was that not much could go wrong, the reality reflects otherwise.

While building a model there are many possible security threats. Some of them that I found on my research are

1. Data Poisoning

2. Model Tampering

3. Adversarial attacks

4. Data Exposure due to privacy loss

In this blog post I would like to talk about each of the security threat what they are and what are some of practices that we can include in our workflow when we go about building a machine learning or deep learning model for our next use case.

Data Poisoning

Data poisoning is a type of security threat where the attacker intentionally manipulates the training data thus giving wrong model predictions or understanding wrong patterns of behaviour.

Data poisoning can occurs in various forms

Inclusion of incorrect or manipulated data

For example training a model which contains negative connotations of language can cause misunderstanding for the model in terms of correct political language.

Flipping the labels of subset of data to create problems in model training

Labels are primary source of ground truth for the input to mean something. If the labels of a subset of data are mixed up then the model while learning will combine features which aren’t meant to be combined.

A simple example will be what if I take MNIST dataset and in the training set rearaange the labels of the numbers calling some samples of 4 and 8 and flipping them 6 and 9 and flipping them. The model might not have a decent understanding of what features represent which number.

data injection

Data Injection is a data poisoning method where attacker can add new malicious data to modify the behavior of model during training causing it to do incorrect predictions in the inference time.

Example of such an attack is adding bias in data towards a certain outcome, one of the real life examples are where there was racial discrimination against a certain race in facial recognition models.

feature manipulation

This is a type of data poisoning where individual features or distribution of features have been modified to change the decision boundary of the classifier. These are slightly harder to detect as the data might be valid and free of errors.

Adding regularization in the model can generally make the model more robust to such feature manipulations. Also performing adversarial training can make the models more robust to feature manipulations.

Practices to avoid data poisoning:

• Preserving or securing data sources to a principal of least privilege as this would solve most issues regarding manipulation of data

• Data validation by checking if the data input is correct and remove any bias if present in the data.

• Model regularization and data augmentation can be good tools to expose machine learning to a variance that can limit the damage caused

• Adversarial training is a method where some adversarial data is included to improve robustness during training process

• Model monitoring to detect data drifts.

How much of Terraform do you use on a day to day basis?

Ask yourself this question, if you are a light user of terraform which means the scripts are already written by someone in the organization and all that needs to be done is to perform a terraform init, terraform plan and terraform apply then allocate about 6–8 weeks of time to understand the tool completely.

If you are a medium user of terraform where you make changes to the existing scripts and understand the concepts a little bit then allocate about 4–6 weeks of time for preparation of the exam.

If you write terraform on a day to day basis and are efficient with the documentation of terraform then this exam is easy and 2 weeks of browsing through the exam guide should help you pass the exam with ease.

Resources:

Resources used to prepare for the exam are

1. A cloud guru lectures — This resource is good enough to understand the basic concepts of terraform and what each concept does, The labs are quite helpful to gain a practical insight into what is to be done and how to use different concepts being taught

2. Udemy Practice exams — This I felt was the most useful, most of the questions are very similar to ones being asked in the exam, though the time limit is a lot lenient it gives a good idea of how to approach the question and if something went wrong in the exam talks about what went wrong and what can be done to correct it.

3. Exam Guide from terraform — One of the very best and free resources to prepare for the exam, this and the udemy practice exams itself will be sufficient enough to get a deeper insight and crack the certification exam.

The Exam:

The certification exam itself is for 60 minutes with 57 question so approximately 1 minute per question, some of them are pretty straight forward that the entire exam may not take that much time, I would suggest reading through the questions deeply as multiple choice correct answers might be tricky at times.

Conclusion:

Everything being said the exam is my first professional certification for a cloud tool and it was relatively smooth process.

In normal machine Learning the data resides in a single source or is aggregated into a single source by certain ETL (extract, load and transform operations) by data engineers from multiple databases and then used to train a machine learning model.

In Federated learning the data doesn't leave the source of origin. There is a central server which is owned by Data scientists and the data sources are considered clients. Machine learning is done by decentralized communication and collecting only relevant model updates in the central server.

Advantages of Federated learning

Federated learning can be used when data are locked due to several factors such as compliance, location, regulation etc. To do Machine learning on such data in a normal setting is not possible as the data cannot leave the source and distributed training with preserving privacy becomes an advantage in the favour of using Machine learning.

Use cases of Federated Learning

This above-mentioned advantages lead us to think about various use-cases in the field of

• Healthcare

• Insurance

• Fintech

• IoT

• Industries for doing predictive maintenance

In all of the above cases there is a need for privacy and getting the data out of the source can be very cumbersome.

Problem with deploying Federated Learning

Federated Learning in Google board is a good example of how it is being deployed in the real world. However, there is a difference between the Google board use-case and the use-cases mentioned above.

The difference lies in the ownership of the source data. In most of the general cases where privacy is required the ownership differs from the person performing machine learning on the data.

This means that a part of the machine learning pipeline which is cleaning of the data and selecting the features needs to be done on the data owner side before allowing the learner to create a model on the data. This involves the separation of responsibility between the two sides.

The above situation is very similar to a front-end engineer and back-end engineer when they don't have a common API specification to talk to each other. In Machine learning feature engineering and exploratory data analysis help in the understanding of the data. With just selected features being available to the learner it becomes very difficult to understand what those features are and how an efficient model can be made.

Problems can also arise during communication and this may cause incorrect data to be sent to learn. The cost is also high compared to normal machine learning as two individual people with specific skills are required to complete the task at hand.

How to solve this issue?

The issue can be solved by both parties coming together and deciding on a general type specification on how the data should look like and what would be the necessary features (Again may be quite unproductive since there is the cost of two people involved). But that is the best possible solution as of now

Use-cases where ownership of data remains the same

However, the above issue will not exist in use-cases where the data collection is being done by an application which is being deployed by the owner of the server. When the application inputs and outputs are defined the data is generally of a certain structure and it is easy to run federated learning and deploy it with ease.

Data Standard specification

The other way of solving the communication issue is setting of a standard specification for different data types (tabular, image, text, audio) imposing requirements of certain fields can solve the communication problem.

The expected skills are using a debugger in the language used and a basic understanding of the features of any language (for loops will be for loops in any language and if conditions will be if conditions, the syntax might change but the basic idea is the same)

Tests

Tests in a codebase are generally a good starting point since they outline what the functionality is and test with an example. The tests usually contain an example input and assert statements confirming the type of the output or the expected output itself. This helps understand the input-output relation for a particular function. Now that is cleared the next step is to read the function to understand what exactly the function does and how it does it.

For this, I prefer using the debugger with the test. It is possible to create breakpoints in the function and understand what transformations are happening to the data while passing through the function.

Now for a scenario that is very common, what if there are no tests in the codebase?

Examples

If there are no tests in the codebase the next thing to do is to check examples of usage of functions, for this you need your nearest mentor but then they can point you to where the module ends up getting used and follow the steps in the previous example with a debugger to get a detailed understanding of the code.

Documentation

Sometimes the documentation is the best place to start, there are many examples of great documented resources, most large python libraries are documented in a great way as also frameworks like docker and Kubernetes. This document contains basic examples of how to use their software. Something similar should exist for internal software and if they do exists is a good place to start.

Pair Program

This is a very fail-safe way since other points mentioned in the post may or may not be available. It is preferable to request some time from the code owner (The guy who maintains or is responsible for the functionality) to show the ropes around the codebase.

If the code base is too big then this can take a lot of time from that person so make sure to get the examples and starting points of usage so that it is easier to get started.

Conclusion

I hope this helps in understanding how to approach a new codebase as a beginner. The debugger (sometimes print statements) is a very useful skill to learn. If you have better ideas on how to go through a codebase please mention them in the comments section.

Business Impact is Important:

As a Junior Engineer at a small startup that is growing, from the get-go, it is important to be learning and implementing stuff that is required for the business rather than going on tangents and pursuing what seems fascinating tech-wise. In a world where distractions are the new norm, it is quite easy to propose new features and ideas which don't affect the business outcome.

The shift in mindset between doing research at the university with multiple ideas and validation for those ideas is not possible in a company as they cost time and resources which a startup especially are quite lean on. Building products at a startup requires a different thinking process and it is important to adapt and move fast.

Visibility:

Even if you are stuck in doing stuff that doesn't matter to business anymore it is important to be visible to the team. This can be done by performing presentations on what is being worked on with a focus on if and how it can affect the product in the short term, long term or the medium term. So if you are doing some research you can present it in a way that shows a new vision or roadmap. Also, organize some activities for the team to do together like a hike, some sports, cooking for each other etc( if you can cook). This can help show that you are helping in fostering the culture of the team and is an important factor.

Be agile and Move fast:

By means of being agile, I don't refer to the software development philosophy of breaking things fast and fixing them fast. What I mean here is that if there is something that is needed to be learnt instead of doing a complete course or a certification it can take time. Just learning what is necessary for the task at hand and building a Proof of concept would do you and the company a world of good since there is not much time to learn stuff in detail. When a POC is established and breaks in production then there is a need to learn and implement the other details, not before.

Done is better than perfect:

In Academia we are taught perfection is the way to go forward in life. However, the real world is completely the opposite. If there is a task it is better completed with imperfections that can be creased out later than incomplete with components being perfect. If there is a feature to be implemented, implement it in a brute force method at first and get something working. There will be a time in the future to make things perfect.

“Premature optimization is the root of all evil” — Sir Tony Hoare popularized by Donald Knuth

As you grow in experience it becomes important to turn these initial POCs into something structurally sensible before writing code, but as a junior engineer, it is important to have something rather than the perfect thing.

Communication is Key:

It is important to understand that time is money, it is better off to ask for help when you are stuck at something for more than 4 hours, 4 hours can be the time to check google and implement the plethora of solutions that google offer if the problem still persists it is important to ask for help and resolve the issue as fast as possible as it becomes awkward as a junior to ask for help when you have been reassuring the supervisor that all is well and progress is being done. Asking for help makes you stronger over a period of time. Also at work, there is no test to check who is accomplishing everything by themselves.

Try and document everything possible:

This is a life hack and not just an early-career hack, Maintain a log wherein you can document the process and the approach without delving too much into the confidential details. In engineering, we can always abstract the problem and frame it in a general sense. This log can even be used when we move on to the next company. I started doing this quite late in my job. A better way to implement this would be to include it as a part of the daily routine. At the end of each day’s work just have a personal notebook or any apps and just note down the day’s challenge, situation and solution.

Request responsibility:

It is always unclear on whose responsibility is what in a small group and everyone is expected to pick up something and run with it. It is important to show dedication and commitment by requesting things not under your purview to demonstrate team ability. As a developer, this means that if a team member is sick or on vacation pick up their work and perform it to the best of our ability. This can be solving immediate issues relating to a component, not under your ownership.

What this would certainly do is improve the understanding of the system and leads to better education and self-improvement.

Conclusion

These are the simple things that I have learnt being in an organization for over 2.75 years starting from a student to a full-time employee. Hopefully, this helps in those transitioning from academia to the industry. Some points may be relevant for bigger companies too.