Off-by-none: Issue #166

November 30, 2021

Only have a few minutes? Check out this week's MOST POPULAR links as chosen by our email subscribers.

My Database Needs Help!

Welcome to Issue #166 of Off-by-none. This issue is sponsored by our friends at Lumigo and Courier.

Last week, pre:Invent overwhelmed us, Cloudflare tried to take over the serverless world, and Vercel raised another huge funding round to keep the innovation coming. This week, we’re in the throes of AWS re:Invent, we’ve checked off some serverless #awswishlist items, and I share my passion for Amazon DynamoDB. Plus, we have a TON of AWS announcements to cover and plenty of great content from the community.

Monitor & Debug Serverless with 30-Second Auto-Instrumentation
Complete observability over your serverless environment with no code changes required. With one-click distributed tracing, Lumigo lets you effortlessly find & fix issues in serverless and microservices environments. Get serverless-specific smart alerts before they impact performance or cost. Sign up for a free account and get up & running in minutes. Sponsored

Does your database need help?

Most of you probably know how much I love DynamoDB, and more specifically, how much I love single table design. Well, I’ve decided to create a DynamoDB Modeling course to share all the tips, tricks, and strategies I’ve used and developed over the years. The course will launch in early 2022, but you can sign up to get updates (and early access) at dynamodbmodeling.com.

And because I like to go over the top, I created a DynamoDB parody song and video to the tune of Help! by The Beatles – My Database Needs Help! I hope you enjoy it!

Serverless News & Announcements 📣

Okay, here are some of the most existing serverless announcements from the past week. And some of them are BIG! Also, use this post to get a regularly update list of all the top announcements from AWS re:Invent 2021.

Announcing Amazon Kinesis Data Streams On-Demand – This one is huge, not just for production environments that need to scale up and down, but also for DEV and sandbox environments. This gets us closer to being able to test everything in the cloud without incurring charges for provisioned Kinesis shards! Marcia Villalba gives some more context here!

Amazon S3 Event Notifications with Amazon EventBridge help you build advanced serverless applications faster – Also huge! Now we can subscribe to events from S3 using EventBridge, allowing us to use all the amazing routing/retrying/filtering capabilities to fan out and trigger other services including pushing this info to API destinations, SaaS providers, and other accounts. More on this from Jeff Barr.

AWS Lambda now supports event filtering for Amazon SQS, Amazon DynamoDB, and Amazon Kinesis as event sources – Bigger than huge! EventBridge stepped in as a workaround for this very problem, but added extra Lambda Functions and more complexity when most of the time it just wasn’t needed. I did confirm with the Lambda PMs that each filtered Lambda requires a separate connection, so if you are filtering on small bits of data off a Kinesis stream for example, you could theoretically hit the Read Throughput limits. Although, with On-Demand Kinesis, who knows? A few people have written some posts showing how to trigger Lambda Functions with event filtering, or how to filter DynamoDB streams before a Lambda Function, and even how to do event filtering for Lambda Functions using AWS CDK.

AWS Lambda now supports partial batch response for SQS as an event source – Love this one! Now instead of failing an entire batch, you can just send back the failed message IDs. It will then just re-queue the failed messages and presumably follow your redrive policies for Dead Letter Queues (DLQs). And the Serverless Framework already has support for it!

Amazon SQS Announces Server-Side Encryption with Amazon SQS-managed encryption keys (SSE-SQS) – Maybe seems small, but nice to have the option to not need to manage your own encryption keys for SQS queues.

Announcing Amazon Redshift Serverless (Preview) – This one has been needed for a long time! So many amazing things that can be done with Redshift, but again, it was way too expensive to be spinning up for serverless development environments. Game changer. Danilo Poccia shares some more insights.

They also introduced Amazon EMR Serverless in preview and Amazon MSK Serverless in public preview. I don’t really use either of these, but for those that do, I’m sure you appreciate these new serverless models.

And… there was an AWS Free Tier data transfer expansion now allowing 100 GB from regions and 1 TB from Amazon CloudFront per month. I guess we should probably thank Cloudflare for this one. 😬

Serverless Concepts 🏗

There are more announcements to get to, but there were some really helpful posts from the community that need to get shared as well.

Brian Foody has a helpful piece that shows you how to (almost) unknowingly create an infinite loop using the AWS SDK and why idempotency is so import in your Lambda Functions.

Eoin Shanaghy shares three ways to read SSM Parameters.

The Dashbird put out an infographic on securing serverless architectures, and I’m a sucker for serverless security.

Daniel Costa explains how to run Lambda Functions faster and cheaper. He’s talking about power tuning, and there is a very handy library for that.

Christian Bonzelet had a great post a few weeks ago that compared performance between Step Functions and Lambda. He followed up with part 2 that revisits the topic with cost comparisons as well.

Harshit Sharma shows how serverless can be a savior, even when using it as a peripheral service to your traditional infrastructure.

Serverless Tutorials 👷‍♀️

Notifications That Don’t Suck! The Product Hunt Newsletter recently covered a story on a company trying to fix notifications. Many apps that we use and download end up spamming us with notifications ruining the user experience. Similar to Wuphf.com created by Ryan on the hit TV show The Office, we are getting hit on every channel. How do we fix it? Sponsored

Richard Rhodes shows you how to use the Serverless Framework to create an event-driven architecture.

Robert Bulmer explains how to implement rate and max depth limiting with AWS AppSync.

Here’s an interesting tutorial on implementing dynamic pipeline orchestration using CodePipeline, CodeBuild and Step Functions.

Emma Moinat shows you how to build an OpenSearch Index from DynamoDB streams.

If you still need to emulate the cloud locally, here’s how you can setup LocalStack for your local environment.

Emre Ayberk Kaymaz does DynamoDB with AWS SDK Go v2 , in two parts.

You can also deploy a containerized Serverless Flask app to AWS Lambda, if you really wanted to.

Kirk Kirkconnell shares a simple trick to identify items in an Amazon DynamoDB table that haven’t been accessed for a time and move them to Amazon S3.

And last but not least, Adrian Hornsby adds another post in his series on injecting custom faults with AWS Fault Injection Simulator. My advice is just to do whatever Adrian tells you. 😉

Serverless Reads 🤓

According to The Serverless Edge, you can deliver sustainable technology through Serverless.

And according to Snyk, cloud security is shifting to ‘dev’ not ‘ops’. Yep, some of use have been saying this for a long time. Maybe nobody was listening? 🤷‍♂️

Matt Asay wrote some follow up commentary on that whole back and forth about the unfulfilled potential/promise of serverless.

AJ Stuyvenberg explains why you should be using DynamoDB Streams and Lambda in your serverless applications.

And Ben Kehoe had a post from a few weeks ago explaining why AWS CodeArtifact should be the place to store AWS code artifacts. Couldn’t agree more!

Podcasts, Videos, and more 🎧

On Serverless Chats Episode #121: Educating Serverless Developers, Rebecca and I chat with Ivonne Roberts about her new role at Bill.com, how she transitioned into tech and became an AWS Serverless Hero, what drew her to serverless in the first place, why making content is so important for your career, and much more.

Marcia Villalba shows you how to do faster serverless deployments and test your apps in the cloud with SAM Accelerate. I love that AWS is doing this!

New from AWS 🆕

Here is a dump of some other AWS announcements that look really interesting:

• Announcing AWS Data Exchange for APIs: Find, Subscribe to, and Use Third-party APIs with Consistent Authentication
• New for AWS Control Tower – Region Deny and Guardrails to Help You Meet Data Residency Requirements
• Announcing Amazon SageMaker Canvas – a Visual, No Code Machine Learning Capability for Business Analysts
• Announcing data tiering for Amazon ElastiCache for Redis
• Customize your AWS Well-Architected Review using Custom Lenses
• Improved, Automated Vulnerability Management for Cloud Workloads with a New Amazon Inspector
• Introducing Amazon CloudWatch Metrics Insights (Preview)
• Introducing Amazon CloudWatch RUM for monitoring applications’ client-side performance
• Announcing AWS Data Exchange for APIs
• AWS Proton Supports Terraform and Git Repositories to Manage Templates
• Announcing AWS Fargate for Amazon ECS Powered by AWS Graviton2 Processors
• AWS WAF adds support for Captcha
• Introducing Amazon CloudWatch Evidently for feature experimentation and safer launches
• Amazon Transcribe now supports automatic language identification for streaming transcriptions
• AWS App Runner supports GitHub Actions to build and deploy applications
• AWS Control Tower introduces Terraform account provisioning and customization
• Announcing Amazon Athena ACID transactions, powered by Apache Iceberg (Preview)
• AWS Amplify expands its Notifications category to include in-app messaging (Developer Preview)
• Amazon S3 adds new S3 Event Notifications for S3 Lifecycle, S3 Intelligent-Tiering, object tags, and object access control lists
• AWS Amplify announces a redesigned, more extensible GraphQL Transformer for creating app backends quickly
• Announcing usability improvements in the navigation bar of the AWS Management Console
• Announcing AWS Graviton2 Support for AWS Fargate – Get up to 40% Better Price-Performance for Your Serverless Containers
• AWS Lake Formation support Governed Tables, storage optimization and row-level security
• AWS Compute Optimizer now offers resource efficiency metrics
• AWS Nitro SSD – High Performance Storage for your I/O-Intensive Applications
• Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) now supports checking for blue/green deployment when making configuration changes.
• Amazon DynamoDB now helps you meet regulatory compliance and business continuity requirements through enhanced backup features in AWS Backup

Final Thoughts 🤔

Okay, sorry for the firehose again. Hopefully there are some things in here that will help you build better, more amazing serverless applications!

I need to get back to re:Invent, but if you’re here and want to say, “Hi!”, DM me on Twitter.

Hope to see you,
Jeremy

I hope you enjoyed this newsletter. We’re always looking for ideas and feedback to make it better and more inclusive, so please feel free to reach out to me via Twitter, LinkedIn, Facebook, or email.