Background

At AWS re:Invent 2024, AWS introduced AWS S3 Tables, built on Apache Iceberg.

I think this feature has a practical solution for Spring Boot Microservices developers and architects.

Especially with AWS S3 Tables features like the Namespaces, developers and architects can design Spring Boot Microservices components to ingest a structured JSON to store, organize in AWS S3 Tables, and query it for performance-quick insights.

For example, if you want to log execution times for your Spring Boot APIs and analyze them over weeks or months, S3 Tables + QuickSight makes it simple, cost-effective, and easy to scale.

When I brought this idea up to a colleague, their immediate response was, "Why? We already have Grafana and ELK. Isn’t this just adding unnecessary complexity?"

Fair question! Grafana and ELK are excellent tools for real-time monitoring and troubleshooting, but they’re not ideal for everything. 

Let me explain why AWS S3 Tables + QuickSight complements these tools rather than replacing them.

How Overlapping Business Processes Can Be Visualized

When managing API Facade Java classes (also called Spring Boot controller classes) in microservices, multiple business sub-processes, invoked within a single API request, often rely on shared APIs. Identifying these overlaps can help you optimize performance and avoid redundancies. Here’s a practical example:

1. Log API calls with metadata: Each API call is logged with metadata about the business process it belongs to. For example:

{
  "timestamp": "2025-01-25T14:32:00Z",
  "api": "/inventory/validate",
  "process_type": ["Order Checkout", "Inventory Validation"],
  "duration_ms": 150
}

2. Store and Query Logs in S3 Tables: Use S3 Tables to organize logs by API and process type. Amazon Athena makes it easy to run queries to identify shared APIs across processes.

3. Visualize with a Venn Diagram in QuickSight: In QuickSight, create a Venn diagram to visualize overlaps:

   • Process A: APIs for "Order Checkout"

   • Process B: APIs for "Inventory Validation"

   • Overlap: /inventory/validate, /payment/process

4. This lets you see shared dependencies between processes and pinpoint opportunities for optimization.

Why S3 Tables + QuickSight Over ELK and Grafana?

Here’s where it gets practical. ELK and Grafana are fantastic for real-time operations, but AWS S3 Tables + QuickSight fills a different gap.

When to Use S3 Tables and QuickSight?

This approach is particularly useful if:

• You want to analyze long-term trends like API execution times over months or years.

• You need cost-effective storage and analytics for large datasets.

• Your business team needs easy-to-understand dashboards with actionable insights.

When to Stick with ELK or Grafana?

ELK and Grafana remain essential for:

• Real-time monitoring and alerts during active operations.

• Troubleshooting live issues in your microservices.

Final Thoughts

When I finished explaining this to my colleague, they said, "Okay, I see the value. This approach makes sense for scaling log analytics and delivering insights to non-engineering teams. Let’s test it on a small project."

AWS S3 Tables + QuickSight doesn’t replace tools like Grafana or ELK; it complements them.

For teams looking to optimize costs, analyze historical trends, and generate actionable insights, it’s a practical and scalable solution.

Want more actionable insights like this? Subscribe to Cloud-Illumination for updates on cloud innovations and microservice strategies!