Should You Use Containers with Serverless Implementations?

Serverless implementations are usually Cloud Deployment-specific, AWS Lambda being the most popular implementation. However, there are some advantages to using containers. Let's look at them:

Dependency Management

Containers allow you to package all dependencies and libraries directly into the image, making it easier to manage complex dependency structures. 

Consistency

All of the necessary provisions and configurations are part of containers, ensuring consistency across deployments.

Larger Function Size

This, I believe, is the biggest benefit. Container-based Lambda functions can be up to 10GB in size, compared to the 250MB limit for zip packages. However, there is a additional cost associated with containers.

Security Posture

Containers can be scanned for vulnerabilities and software composition analyses much more easily than Lambdas.

Looks good, but what's the issue?

Cold Start Performance

For larger containers, cold start times can be significantly impacted. Although AWS has improved in this area, it can still be a concern for time-sensitive applications.

Lambda Layers and Extensions

This is a limitation when using containers. However, for a company that uses mainly Lambdas for a wide variety of projects which use the same programming language, this can be a considerable advantage.

Additional Cost

Using containers usually incurs additional costs, as it requires a different hosting option.

Should You Use Containers or Not?

All that said, I believe containers may not be suitable for most Serverless use cases. Why? Because cloud providers offer tailor-made trigger functions that need to be incorporated into the code, so the advantage of having them as containers, which the added ability for them to be run anywhere, is lost. For Azure, it's the Azure Function wrapper, and for AWS, it's the Lambda Handler. It's often easier to add these wrappers to existing implementations when deploying them to serverless environments. However, this approach undermines some of the key advantages of containers.

If you need to create a custom application for serverless environments, it's generally better to go with the cloud provider's native approach for maximum performance and integration. This allows you to fully leverage the provider-specific features and optimizations, which can lead to better overall efficiency and effectiveness.

As an example, I have used Azure to create and test a container-based Serverless application. You can find it here. It still uses Azure specific triggers to APIs. 

Deploying Spark Applications on Kubernetes: A Practical Guide to spark-submit and Spark Operator

Combining Kubernetes and Apache Spark has become a powerful tool for processing big data; it offers several advantages over traditional approaches. Packaging Spark applications and dependencies in containers simplifies deployment and ensures consistency across environments. Let's quickly compare this approach with the traditional approach before we look at an example. 

Why use Kubernetes? What about YARN?

When it comes to sharing resources, Kubernetes is much more efficient than YARN. Dynamic allocation allows Spark to add executors to and remove executors from a given deployment based on the given workload, and Kubernetes reallocates resources between applications as they finish, which reduces idle time. 

Kubernetes's efficiency also reduces costs, because with Kubernetes, provisioning and de-provisioning is based on the given workload. Kubernetes allows for a cloud-agnostic approach, enabling organizations to build once, but deploy anywhere. 

What about Spark Operator?

Spark Operator introduces custom resource definitions (CRDs) that allow for the declarative management of Spark applications through Kubernetes Manifest. It uses Mutating Admission Webhook, which handles customizations for Spark driver and executor pods based on annotations on the pods added by the controller. Spark operators allow for centralized administration of all Spark-related workloads, which reduces operational overhead. 

But YARN still has benefits, right?

YARN provides built-in support for data locality, which can significantly improve the performance of data-intensive workloads. There are two levels of locality: Node-level locality and Rack-level locality. But while YARN offers some advantages in data locality for Hadoop-centric workloads, using the Spark Operator on Kubernetes provides a more flexible, cost-effective, and easy-to-manage platform for running Spark workloads in modern cloud-native environments.

Spark-submit or Spark Operator?

Both spark-submit and Spark Operator can be used to run Spark applications on Kubernetes, but they differ in their respective approaches and complexities. I have an example here that deploys using both methods. For the following reasons, I would use the Spark operator:

• Handles the entire life cycle of the Spark application, including clean up
• Manages Spark applications more easily than spark-submit 
• Declarative approach to configurations

Rate limiting in Kubernetes using Istio

Rate limiting is one of my favorite sections. I have implemented rate limiting as part of API Gateway and had multiple versions of it tailored to the needs of the downstream applications that were leveraging them in various ways. It's also another use case that is specific to microservices that can be implemented in Kubernetes. It can also be an add-on to existing rate limit implementations that can be based on customer or subscription.

First, let's talk about the security measure, and how its absence or improper implementation can lead to several security problems:

Vulnerability to Attacks

Denial of Service (DoS) and Distributed Denial of Service (DDoS) Attacks

Attackers can overwhelm an API with a flood of requests, exhausting server resources and making the API unresponsive or unavailable to legitimate users. It causes Resource Exhaustion, depleting server resources like CPU, memory, and network bandwidth, potentially causing system crashes or slowdowns.

Brute Force Attacks

Lack of rate limiting makes it easier for attackers to conduct brute force attacks on authentication endpoints, attempting numerous login attempts in rapid succession. Back in 2014, this was one of the reasons that Apple's authentication was compromised as hackers were able to hack into celebrity accounts. Apple has hardened the authentication service since then, and it's not a concern anymore with 2-factor authentication being the standard.

Credential Stuffing

This almost always happens by bots scanning most known URLs for easy access with common and stolen credentials. Without rate limits in place, this only exacerbates the problem.

Increased Attack Surface

Unlimited API access provides more opportunities for attackers to discover and exploit other vulnerabilities in the system.

Business Impact

This has a direct business impact with services having poor response because of resource exhaustion. It may also lead to excessive financial costs and service disruptions.

Implementing proper rate limiting policies is crucial for maintaining API security, ensuring fair resource allocation, and protecting against various forms of attacks and abuse. It's an essential component of a comprehensive API security strategy.

So how can you implement it , If you are already using Kubernetes and service mesh Its just a matter of adding it as part of your configuration . I have provided an example here -> Github 

Without Rate limit I am able to make 20 Request per second

                                      With Rate limit I am only able to make 10 Request per second 
                        

Istio

Istio, a popular service mesh for Kubernetes, provides built-in support for both rate limiting and circuit breaking. It can address almost all the security concerns described above . It may be possible by other service mesh as well , but I am going to use Istio for demonstration .

Protection Against Brute Force

Rate limit with Circuit Breaker help protect your services from being overwhelmed.

Fine-grained Control

You can set different limits and thresholds for different services or endpoints.

Improved Resilience

Circuit breakers prevent cascading failures, while rate limiting controls the flow of requests.

You can also do Rate limit , Load Balancing and routing on your Ingress and will not require a Service Mesh .  But this will be limited to the services accessed through ingress 

Containerization Github List  ->   Github Containers

AKS: Exploring Capabilities and Building Your First Sample Application

Azure offers a fantastic opportunity for developers to explore container services in the cloud with its $200 free credit for new accounts. 

This credit allows you to experiment, learn, and build applications without incurring immediate costs. Let's dive into how you can make the most of this offer and get hands-on experience with Azure Kubernetes Service (AKS).

What can you do with the Free Credit 

When you sign up for an Azure free account, you receive $200 credit to use within 30 days. This credit gives you access to a wide range of Azure services, including AKS. Here's how you can leverage this opportunity:

Explore Azure Portal 

Familiarize yourself with the Azure portal interface, where you'll manage your resources and services.

Create Resource Groups 

Organize your projects by creating resource groups, which act as containers for related resources.

Deploy AKS Clusters

This was my main focus to deploy a simple app in AKS and understand the Kubernetes integrations .

Azure's control plane management offers significant advantages, particularly for smaller organizations without dedicated DevOps teams.Azure handles the control plane

taking care of patches and upgrades, allowing you to focus solely on application deployment. This approach streamlines your workflow and reduces operational overhead.

While using Docker Hub is convenient for testing Azure services, it doesn't seem to be supported . I was able to use ACR ( Azure Container Registry ) , this would be integrated solution for Azure . However most organizations will prefer private container registries for security and compliance reasons.

Horizontal Pod Autoscaling (HPA)

HPA in cloud environments offers more comprehensive scaling capabilities.HPA manages not just pod scaling but also coordinates with cluster autoscaling for efficient node management.

Cloud platforms excel in scenarios requiring rapid and frequent scaling, offering flexibility that's harder to achieve with on-premises solutions.

Admission Controllers and Add-ons

While many Admission Controller use cases can be implemented as add-ons in Azure Kubernetes Service (AKS), it's important to consider:

Native Kubernetes Flexibility: Some operations that are straightforward in native Kubernetes might require additional configuration or external services in managed Kubernetes offerings.

Trade-offs: The convenience of managed services often comes at the cost of some flexibility. Evaluate whether the simplification aligns with your specific requirements and use cases.

My test stops here but you can do more by Deploying your ML workloads , Implement CICD ,etc

Sample Application Deployment in AKS

To help you get started, I've created a sample Java application that can be deployed in AKS.

Github

The application and detailed deployment instructions are available in my GitHub repository. Here's a high-level overview of the steps:

• Clone the Repository. -  Download the sample application code and Kubernetes manifests.
• Build the Docker Image - Package your Java application into a container image.
• Push to Container Registry -  Upload your image to Azure Container Registry (ACR).
• Create AKS Cluster -  Use the Azure portal or CLI to set up your Kubernetes cluster.
• Deploy the Application -  Apply the Kubernetes manifests to deploy your app to AKS.

Access the Application -  Configure ingress or use port forwarding to access your running application.

Containerization Github List  ->   Github Containers

Power of OSCAL in Streamlining Security Assessments for Containers

OSCAL (Open Security Controls Assessment Language) is a standardized framework developed by NIST (National Institute of Standards and Technology) in collaboration with FedRAMP to digitize and automate security authorization processes. It provides a common machine-readable language for expressing security controls, assessments, and related information.

OSCAL includes several interconnected models:

• System Security Plan (SSP): Documents how security controls are implemented in a system.
• Security Assessment Plan (SAP): Outlines the approach for assessing security controls.
• Security Assessment Report (SAR): Presents the results of security control assessments.
• Plan of Action and Milestones (POA&M): Tracks identified vulnerabilities and planned remediation activities

I have talked about SBOM here   . While OSCAL is not currently mandated, it offers significant value for companies doing business with the federal government:

• Alignment with FedRAMP : FedRAMP is actively involved in OSCAL development, indicating its potential future importance in federal compliance.
• Complementary to SBOM: Like Software Bill of Materials (SBOM) for container security, OSCAL provides standardized security assessment information.
• Automation Potential: OSCAL enables automation of security assessment processes, including vulnerability scans and SAR generation.

Vulnerability Scans for Containers can be done in ongoing basis and Security Assessment Report (SAR) can be automated easily . Here is a simple example of this implementation

Implementaion -> Github

Containerization Github List  -> Github Containers

Integrating gRPC for Efficiency

Message brokers like Apache Kafka are excellent tools for building fault-tolerant architectures, particularly for implementing efficient back-end workflows. Kafka's design, which includes a queue system, can introduce a lag in service-to-service communication that cannot provide immediate responses. However, Kafka's ease of use and high reliability often make it preferable to Microservices.

When it comes to service-to-service communication within an application, many developers rely on REST APIs. This preference is likely due to REST's widespread familiarity and understanding, rather than optimal performance. For high-performance service calls within an application, gRPC is a more suitable option. It supports streaming payloads and offers significant performance benefits.

One challenge with using gRPC and Protocol Buffers is the need to define message structures and generate client and server stubs. Despite this overhead, the advantages of implementing gRPC are substantial, especially for performance-critical applications.

Microservices deployed in Kubernetes as pods, configuring gRPC involves changing properties to specify the service name instead of the host. This setup does not require a service mesh, and the initial configuration effort is rewarded with improved performance. For a demonstration of this setup in Kubernetes, you can refer to the following examples: a basic setup (Github main) and a setup with streaming support (Github Streaming Example).

Containerization Github List  ->   Github Containers

Ownership in the Kubernetes Ecosystem

 I was reviewing my presentation on Containers and Kubernetes with my colleague, and one important issue came up that is not widely talked about or discussed. It’s not a technical issue but an ownership issue. Don’t confuse it with the ownership of Kubernetes setup or security. It has to do with the ownership of features like Container Runtime and Service Mesh. This is not a technical article but something to think about as you roll out new features with Kubernetes.

Take Service Mesh, for example. It affects the way pods communicate with each other and offers additional capabilities with respect to metrics, monitoring, and security tooling. There are cases where this may not be the best for applications that look for high performance and don’t want to share the pod’s resources. Also, the application may be exporting the metrics as part of the framework.

Traffic Shaping and Canary Deployment also fall into a similar category. It’s usually the application teams that take ownership of these features. Developers, being fully aware of Kubernetes capabilities, may still choose to do this in microservice-based orchestration rather than relying on Kubernetes, as it may need coordination, approval, and guidance from DevOps.

Most Service Mesh implementations use sidecars. Sidecars consume resources and will affect the HPA. I talked about it here. The burden of successful HPA is not only with developers but also with DevOps, and most of these features will require developers to work closely with DevOps, something that is lacking in most organizations.

What do you think?

Containerization Github List  ->   Github Containers