Using mirrord for Local Development and Debugging

Introduction

As a developer, creating a seamless and efficient workflow is essential. However, one of the biggest challenges you face is ensuring that your local environment matches the staging environment. Differences in configuration, outdated data, and inconsistent dependencies can make debugging a nightmare. mirrord aims to solve this problem by allowing you to mirror your staging environment's traffic and behavior, streamlining your development and debugging processes.

mirrord acts as a direct window into your staging environment, enabling seamless integration with your local machine.

It allows you to mirror and capture incoming traffic, route outgoing traffic through the remote pod, perform DNS queries, read and write files, and access environment variables from the remote pod. These capabilities ensure that your local development and debugging environment closely matches the staging environment, enhancing the accuracy and reliability of your tests.

In this article, we will explore how mirrord integrates with Kubernetes, guide you through the setup process, demonstrate how to run a simple application, and showcase how to debug effectively with mirrord.

Challenges in Local Development and Debugging

Developers often face significant challenges in local development and debugging due to inconsistencies between local and staging environments. These inconsistencies, caused by differences in configuration, software versions, and dependencies, can lead to unexpected behavior and hard-to-reproduce bugs. Additionally, accessing real staging data locally is difficult due to security and privacy concerns, as well as the sheer volume of data. Replicating the complexity of staging environments, which involve multiple interconnected services, databases, and third-party APIs, requires significant configuration and resources, making it a time-consuming and error-prone process.

By understanding these challenges, we can better appreciate the solutions offered by tools like mirrord, which help bridge the gap between local and remote environments, making the development process more streamlined and efficient.

How mirrord Solves These Challenges

mirrord is designed to address these challenges by allowing developers to run their processes in the context of remote staging environments, providing a comprehensive view and interaction with the staging setup.

Here's how mirrord helps:

• Environment Consistency: mirrord ensures that your local development setup can effectively simulate the configuration and behavior of the remote environment by integrating network requests, file operations, and environment variables, reducing discrepancies and making it easier to reproduce and fix bugs.

• Real-Time Interaction: mirrord enables developers to interact with the staging environment securely, facilitating more accurate testing without the need to copy sensitive data locally. It captures and handles incoming traffic, routes outgoing traffic, performs DNS queries, and manages file operations as if they were happening on the staging environment.

• Simplified Development Process: By integrating the remote environment's context, mirrord eliminates the need for complex local setups, saving time and resources. This ensures a smoother and more efficient development and debugging experience, allowing developers to focus on coding and testing rather than environment configuration.

How mirrord Integrates into the Kubernetes Ecosystem

mirrord seamlessly integrates with Kubernetes to provide a powerful local development and debugging experience. Here’s a closer look at how it works:

1. Starting the mirrord Agent:

• When you use mirrord, it starts a pod within your Kubernetes cluster called the mirrord agent.

• This agent pod is launched in the same network namespace as your target application, ensuring it can communicate directly with other services in the cluster.

• The agent pod is temporary and is automatically cleaned up at the end of your debugging session, leaving no resources in your cluster.

2. Injecting mirrord into Your Process:

• mirrord injects itself into your local process, effectively hooking into low-level functions.

• It overrides these functions to intercept system calls and network requests that your application makes.

3. Relaying Requests to the mirrord Agent:

• When your application attempts to perform an operation, such as making a network request or reading a file, mirrord steps in.

• Instead of allowing your local machine to handle the request, mirrord relays it to the mirrord agent running in your Kubernetes cluster.

4. Executing Operations on the Target:

• The mirrord agent executes the intercepted operations within the cluster on behalf of your local process.

• For example, if your application tries to make a network request, the mirrord agent performs that request within the remote target in the cluster and sends the data back to your local process.

5. Returning Results to Your Local Process:

• After the mirrord agent completes the operation, it sends the results back to mirrord on your local machine.

• mirrord then passes these results back to your application as if the operation had been performed locally.

By leveraging this integration, mirrord allows you to debug and test applications locally with the context of your Kubernetes cluster, offering an enhanced experience and accelerating the development flow.

This ensures your local development environment closely mirrors the staging setup, reducing discrepancies and improving efficiency.

---

Setting Up Enviroments

In this section, we will guide you through the steps to set up a Minikube cluster, install kubectl, and configure mirrord on your local development environment. These tools will provide the foundation for using mirrord effectively in this demonstration.

Prerequisites

Before we begin, ensure you have the following installed on your machine:

• Docker

Step 1: Setting Up a Minikube Cluster

Minikube is a tool that allows you to run a Kubernetes cluster locally. It is an excellent choice for development and testing.

1. Install Minikube:

• Follow the instructions for your operating system from the Minikube installation guide.

• For macOS, you can use Homebrew, a package manager:

    brew install minikube
  

2. Start Minikube:

• Start a local Kubernetes cluster with Minikube:

    minikube start
  

• This command will set up a local Kubernetes cluster using the default configuration. You can customize it with various options if needed.

3. Verify Minikube Installation:

• Check the status of your Minikube cluster:

    minikube status
  

Running the command minikube status should give you a status output indicating that Minikube is running and properly configured.

Step 2: Installing kubectl

kubectl is the command-line tool for interacting with Kubernetes clusters. We need to install so we can communicate with our cluster via the CLI.

1. Install kubectl:

• Follow the instructions from the Kubernetes documentation.

• For macOS, you can use Homebrew:

    brew install kubectl
  

2. Verify kubectl Installation:

• Check the version of kubectl to ensure it is installed correctly:

    kubectl version --client
  

3. Configure kubectl to Use Minikube:

• Point kubectl to use your Minikube cluster:

    kubectl config use-context minikube
  

Step 3: Installing and Configuring mirrord

Now that we have Minikube and kubectl set up, we can proceed with installing and configuring mirrord.

- Install mirrord:

• Follow the quick start documentation on mirrord to install it for your operating system. For macOS, you can use Homebrew:

    brew install metalbear-co/mirrord/mirrord- Verify mirrord Installation:
  

• Run a simple command to check if mirrord is correctly installed and configured:

    mirrord --version
  

You should see the version number, confirming that mirrord is installed correctly.

With Minikube, kubectl, and mirrord set up, you are now ready to use mirrord for local development and debugging.

In the next section, we will demonstrate how to build and run a simple Node.js application, deploy it to the cluster, then run it locally with mirrord, highlighting its capabilities and benefits.

---

Building and Deploying a Node.js Application

We will use a minimalistic setup for this demonstration to keep things simple.

1. Create a Simple Node.js Application

• Create a new directory for your project and initialize a Node.js project:

    mkdir my-node-app
    cd my-node-app
    npm init -y
  

• Install Express.js:

    npm install express
  

2. Create a Basic Express Application

Create a file named app.js with the following content:

const mongoose = require('mongoose');
const express = require('express');
const app = express();
const port = 3000;

// Use environment variable for MongoDB connection string
const mongoURL = process.env.MONGO_URL || 'mongodb://localhost:27017/mydatabase';

// Connect to MongoDB
mongoose.connect(mongoURL, { useNewUrlParser: true, useUnifiedTopology: true })
  .then(() => console.log('Connected to MongoDB'))
  .catch(err => console.error('Failed to connect to MongoDB', err));

// Define a simple route
app.get('/', (req, res) => {
  let message = 'Hello World!';
  console.log(message); // Set breakpoint here for debugging
  res.send(message);
});

app.listen(port, () => {
  console.log(`App running on port ${port}`);
});

This code sets up a basic Express.js application that connects to a MongoDB database using Mongoose. It defines a single route (/) that returns "Hello World!" and logs the message to the console.

3. Write a Dockerfile to Build the Application

   Create a Dockerfile with the following content:

# Stage 1: Build
FROM node:alpine AS build

WORKDIR /usr/src/app

# Copy package.json and package-lock.json
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy the rest of the application code
COPY . .

# Stage 2: Production
FROM node:alpine

WORKDIR /usr/src/app

# Copy package.json and package-lock.json
COPY package*.json ./

# Install only production dependencies
RUN npm install --only=production

# Copy the rest of the application code from the build stage
COPY --from=build /usr/src/app .

# Expose the port your app runs on
EXPOSE 3000

# Command to run the application
CMD ["node", "app.js"]

This Dockerfile uses a multi-stage build to create a minimal production image for the Node.js application.

The first stage installs all dependencies, and the second stage installs only the production dependencies and copies the application code.

4. Build and Push the Docker Image
   Build the Docker image and push it to Docker Hub:

docker build -t username/nodejs-app:latest .
docker push username/nodejs-app:latest

5. Deploy the Application to Kubernetes
   Create a deployment.yaml and service.yaml file to deploy the application and MongoDB to the cluster.

Deployment for the Node.js Application and MongoDB:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodejs-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodejs-app
  template:
    metadata:
      labels:
        app: nodejs-app
    spec:
      containers:
      - name: nodejs-app
        image: username/nodejs-app:latest
        ports:
        - containerPort: 3000
        env:
        - name: MONGO_URL
          value: "mongodb://mongo:27017/mydatabase"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
      - name: mongo
        image: mongo:latest
        ports:
        - containerPort: 27017
        volumeMounts:
        - name: mongo-data
          mountPath: /data/db
      volumes:
      - name: mongo-data
        emptyDir: {}

Deployment for the service to Expose the Application and MongoDB:

apiVersion: v1
kind: Service
metadata:
  name: nodejs-app
spec:
  type: NodePort
  ports:
  - port: 3000
    targetPort: 3000
    nodePort: 30001
  selector:
    app: nodejs-app
---
apiVersion: v1
kind: Service
metadata:
  name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  selector:
    app: mongo

6. Apply the Deployment and Service Configurations
   Deploy the application and services to your Kubernetes cluster:

kubectl apply -f deployment.yaml
kubectl apply -f service.yaml

7. Verify Deployment
   Check if the pods are running:

kubectl get pods

You should see your Node.js application pod and MongoDB pod running.

8. Test MongoDB Connection Inside the Pod Since we used a minimal base image, common shell commands might not be available.
   To test the MongoDB connection, create a simple Node.js script directly in your pod.

   Open a shell session in your Node.js application pod:

kubectl exec -it <nodejs-app-pod-name> -- sh

Inside the pod, create a new Node.js script to test the MongoDB connection:

echo "const mongoose = require('mongoose');
mongoose.connect('mongodb://mongo:27018/mydatabase', { useNewUrlParser: true, useUnifiedTopology: true })
  .then(() => console.log('Connected to MongoDB'))
  .catch(err => console.error('Failed to connect to MongoDB', err));" > testMongo.js

Run the script

node testMongo.js

This script will attempt to connect to the MongoDB instance running in your Kubernetes cluster. If the connection is successful, it will log "Connected to MongoDB". If there's an error, it will log "Failed to connect to MongoDB" along with the error message.

In our case, we got the "Connected to MongoDB" Which demonstrate that our application is running perfectly in our cluster and communicating with our database.

In the next section, we would run this application locally using mirrord and you get to see how mirrord handles this seamlessly.

---

Running and Debugging Our Application Locally with mirrord

Run the Application with mirrord
Use mirrord to start your application while mirroring the remote Kubernetes environment:

mirrord exec --target pod/nodejs-app-6cbbfc7949-d545s -- node app.js

This command tells mirrord to run the Node.js application (app.js) while mirroring the environment and data from your Kubernetes cluster.

What Happens Behind the Scenes with mirrord?

When you execute the mirrord command, it performs several actions to create a seamless local development experience:

1. Setup and Start:

• Reading Context: mirrord reads the current context from your kubeconfig file.

• Establishing Connection: It establishes a connection with the Kubernetes cluster.

• Hooking Network Requests: Hooks are set up to intercept and redirect network requests from your local application to the remote Kubernetes cluster.

• Agent Pod Creation: mirrord creates an agent pod within your cluster. You can verify this by opening another terminal and running kubectl get pods, where you'll see a new pod (the mirrord agent) running in your cluster.

  

2. Configuration:

• Kubeconfig Usage: mirrord uses your local Kubernetes configuration file (~/.kube/config) to determine which cluster to connect to, similar to tools like kubectl.

3. Mirroring Traffic:

• Intercepting Requests: As your local application runs, any network requests it makes (e.g., to databases or other services) are intercepted by mirrord.

• Redirecting to Cluster: These intercepted requests are redirected to the corresponding services in the remote Kubernetes cluster. In our case, any database queries from your local Node.js application are redirected to the database service running in your Kubernetes cluster.

• Seamless Interaction: Your local application interacts with the remote environment as if it were running inside the cluster.

Mirroring is a default feature of mirrord. But it does more than just mirror traffic. mirrord enables you to connect a process on your development machine directly to your Kubernetes cluster.

It achieves this by directly injecting itself into the local process without requiring any code changes.

Mirrord intercepts all input and output points of the process including network traffic and configuration.

This setup allows you to run and debug your application locally while it interacts with the Kubernetes environment and resources as if it were deployed there.

By using mirrord exec, you can interact with your application as if it were running locally while still benefiting from the environment and resources provided by your Kubernetes cluster.

Debugging Our Node.js Application Using VS Code

Open Your Project in VS Code

• Launch VS Code and open your Node.js project directory.

Set Breakpoints in Your Code

Open your app.js file. A breakpoint has already been included in our example:

app.get('/', (req, res) => {
    let message = 'Hello World!';
    debugger; // Force breakpoint here
    console.log(message); // Additional breakpoint
    res.send(message);
});

To set additional breakpoints, click in the left margin next to the line number where you want to pause execution.

A red dot should appear indicating the breakpoint.

Install the mirrord Extension:

• Go to the VS Code marketplace and search for the mirrord extension and install it

Configure the Extension:

• You should see a mirrord icon in your Activity Bar post-installation, with an option to enable or disable it.

• Tick the box to enable it.

The easiest way to configure how mirrord manages the context of your application is to provide a configuration file.

When no configuration file is specified, mirrord uses the first one found in the .mirrord directory. If there is none, one is created automatically:

{
    "feature": {
        "network": {
            "incoming": "mirror",
            "outgoing": true
        },
        "fs": "read",
        "env": true
    }
}

This configuration file tells mirrord to:

• Mirror incoming network traffic.

• Allow outgoing network traffic.

• Provide read access to the file system.

• Use the environment variables from the remote Kubernetes environment.

Debugging Session

Once you hit the debugging icon on the left side of your VS Code, you'll be prompted to select which of the pods running in your cluster you want to mirror. In our case, I selected the nodejs pod. The debugging process will then commence.

When a breakpoint is triggered, you gain the ability to inspect variables, step through the code, and view logs, all while seamlessly mirroring the remote environment locally without the need to be physically within our cluster.

Conclusion

mirrord provides a powerful solution for local development and debugging by integrating seamlessly with Kubernetes to mirror remote environments. It addresses common challenges such as environment inconsistencies and data access issues, making the development process more efficient and reliable. By working with popular development tools, mirrord enables developers to test and debug their applications in realistic settings.

We encourage you to try mirrord for your local development and debugging needs to experience its benefits firsthand. For more information, refer to the official mirrord website.